[RFC] Enabling GitLab CI

classic Classic list List threaded Threaded
31 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Re: [RFC] Enabling GitLab CI

Jonas Hahnfeld
Am Donnerstag, den 21.05.2020, 14:29 +0200 schrieb David Kastrup:

> Jonas Hahnfeld <[hidden email]> writes:
> > Am Dienstag, den 19.05.2020, 08:08 -0400 schrieb Dan Eble:
> > > On May 17, 2020, at 15:27, Jonas Hahnfeld <[hidden email]> wrote:
> > > > before merging. As we only allow fast-forward merges, this means each
> > > > MR has received testing in the form it hits master. This would
> > > > effectively replace the current setup of pushing to staging.
> > >
> > > I'm for this.
> >
> > Thanks. What do others (David, Han-Wen, Valentin) think about this?
> > There's certainly room for improvement, but with an initial setup I can
> > start writing documentation.
>
> The "traffic jam" problem could be avoided by retaining the option of
> pushing to staging.  That would occur without CI, but one could
> occasionally trigger the merge with CI on staging to have everything in
> it migrate to master.  Since staging would be used by the more
> experienced people desiring to bunch their work before testing, the
> triggering could also happen manually by whoever thinks he has pushed
> enough stuff to staging to give it a whirl.
That's not really how CI works. With our policy of FF merges, what
happens if some MR get merged directly to master and some sit in
staging? You'd probably rebase staging which triggers another CI
pipeline and doesn't buy you much.

I generally agree that there's a potential for contention. However
we're aiming for ~1h of CI which means we can merge many more patches
than we currently have per countdown. Merging patches doesn't need to
happen the minute James' message hits lilypond-devel, I think doing
this some time until the next countdown should be fine.

I don't mind deciding that we don't want to enable CI right now. The
purpose of bringing this up now is that I didn't want to spend time
documenting something that's going to change the next day. In my opinion, having CI and merging to master feels more "natural" than the staging setup. And finally if contention proves to be a problem, we can still revert to the old setup.

Jonas

signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [RFC] Enabling GitLab CI

David Kastrup
Jonas Hahnfeld <[hidden email]> writes:

> Am Donnerstag, den 21.05.2020, 14:29 +0200 schrieb David Kastrup:
>> Jonas Hahnfeld <[hidden email]> writes:
>> > Am Dienstag, den 19.05.2020, 08:08 -0400 schrieb Dan Eble:
>> > > On May 17, 2020, at 15:27, Jonas Hahnfeld <[hidden email]> wrote:
>> > > > before merging. As we only allow fast-forward merges, this means each
>> > > > MR has received testing in the form it hits master. This would
>> > > > effectively replace the current setup of pushing to staging.
>> > >
>> > > I'm for this.
>> >
>> > Thanks. What do others (David, Han-Wen, Valentin) think about this?
>> > There's certainly room for improvement, but with an initial setup I can
>> > start writing documentation.
>>
>> The "traffic jam" problem could be avoided by retaining the option of
>> pushing to staging.  That would occur without CI, but one could
>> occasionally trigger the merge with CI on staging to have everything in
>> it migrate to master.  Since staging would be used by the more
>> experienced people desiring to bunch their work before testing, the
>> triggering could also happen manually by whoever thinks he has pushed
>> enough stuff to staging to give it a whirl.
>
> That's not really how CI works. With our policy of FF merges, what
> happens if some MR get merged directly to master and some sit in
> staging? You'd probably rebase staging which triggers another CI
> pipeline and doesn't buy you much.

It buys you that several commits are bunched in staging and are treated
in bulk.  At least I think it does.

> I don't mind deciding that we don't want to enable CI right now. The
> purpose of bringing this up now is that I didn't want to spend time
> documenting something that's going to change the next day. In my
> opinion, having CI and merging to master feels more "natural" than the
> staging setup. And finally if contention proves to be a problem, we
> can still revert to the old setup.

I was more going for the mixed bag right now :)

--
David Kastrup

Reply | Threaded
Open this post in threaded view
|

Re: [RFC] Enabling GitLab CI

Jonas Hahnfeld
Am Donnerstag, den 21.05.2020, 15:19 +0200 schrieb David Kastrup:

> Jonas Hahnfeld <[hidden email]> writes:
> > Am Donnerstag, den 21.05.2020, 14:29 +0200 schrieb David Kastrup:
> > > The "traffic jam" problem could be avoided by retaining the option of
> > > pushing to staging.  That would occur without CI, but one could
> > > occasionally trigger the merge with CI on staging to have everything in
> > > it migrate to master.  Since staging would be used by the more
> > > experienced people desiring to bunch their work before testing, the
> > > triggering could also happen manually by whoever thinks he has pushed
> > > enough stuff to staging to give it a whirl.
> >
> > That's not really how CI works. With our policy of FF merges, what
> > happens if some MR get merged directly to master and some sit in
> > staging? You'd probably rebase staging which triggers another CI
> > pipeline and doesn't buy you much.
>
> It buys you that several commits are bunched in staging and are treated
> in bulk.  At least I think it does.
No, it doesn't: The MRs must pass CI individually before it can be
merged. So it only buys you another test when staging progresses to
master.

signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [RFC] Enabling GitLab CI

David Kastrup
Jonas Hahnfeld <[hidden email]> writes:

> Am Donnerstag, den 21.05.2020, 15:19 +0200 schrieb David Kastrup:
>> Jonas Hahnfeld <[hidden email]> writes:
>> > Am Donnerstag, den 21.05.2020, 14:29 +0200 schrieb David Kastrup:
>> > > The "traffic jam" problem could be avoided by retaining the option of
>> > > pushing to staging.  That would occur without CI, but one could
>> > > occasionally trigger the merge with CI on staging to have everything in
>> > > it migrate to master.  Since staging would be used by the more
>> > > experienced people desiring to bunch their work before testing, the
>> > > triggering could also happen manually by whoever thinks he has pushed
>> > > enough stuff to staging to give it a whirl.
>> >
>> > That's not really how CI works. With our policy of FF merges, what
>> > happens if some MR get merged directly to master and some sit in
>> > staging? You'd probably rebase staging which triggers another CI
>> > pipeline and doesn't buy you much.
>>
>> It buys you that several commits are bunched in staging and are treated
>> in bulk.  At least I think it does.
>
> No, it doesn't: The MRs must pass CI individually before it can be
> merged.

I did not propose to have CI on staging.

> So it only buys you another test when staging progresses to master.

That was the first, not another test in my plan.

--
David Kastrup

Reply | Threaded
Open this post in threaded view
|

Re: [RFC] Enabling GitLab CI

Han-Wen Nienhuys-3
In reply to this post by James Lowe-9
On Thu, May 21, 2020 at 1:17 PM James <[hidden email]> wrote:
>
>
> On 21/05/2020 12:02, Han-Wen Nienhuys wrote:
> > so a next step might be making the countdown process more
> > continuous.
>
> What does that mean - even conceptually?

My idea is that patches could enter 'countdown' stage throughout the
day, and when they do a 48 hour period waiting period kicks in. This
means the moment they are mergable will be more spread out throughout
the day.

My main worry is that the CI process (make doc) will be forced to run
on machines with less CPU power, and the whole test/doc cycle would
take 1 or even 2 hours. If CI takes 2 hours, we'd realistically only
be able to merge 8 changes on a day.

--
Han-Wen Nienhuys - [hidden email] - http://www.xs4all.nl/~hanwen

Reply | Threaded
Open this post in threaded view
|

Re: [RFC] Enabling GitLab CI

Jonas Hahnfeld
Am Donnerstag, den 21.05.2020, 17:21 +0200 schrieb Han-Wen Nienhuys:

> On Thu, May 21, 2020 at 1:17 PM James <[hidden email]> wrote:
> > On 21/05/2020 12:02, Han-Wen Nienhuys wrote:
> > > so a next step might be making the countdown process more
> > > continuous.
> >
> > What does that mean - even conceptually?
>
> My idea is that patches could enter 'countdown' stage throughout the
> day, and when they do a 48 hour period waiting period kicks in. This
> means the moment they are mergable will be more spread out throughout
> the day.
>
> My main worry is that the CI process (make doc) will be forced to run
> on machines with less CPU power, and the whole test/doc cycle would
> take 1 or even 2 hours. If CI takes 2 hours, we'd realistically only
> be able to merge 8 changes on a day.
The full pipeline (build & test & doc) takes less than 1 hour on
GitLab's shared runners with a single core. I think it's a safe bet
that it can't get worse than that. If LilyPond itself gets slower by a
factor of 2, I'd call that a regression in general.

Jonas

signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [RFC] Enabling GitLab CI

Jonas Hahnfeld
In reply to this post by David Kastrup
Am Donnerstag, den 21.05.2020, 17:10 +0200 schrieb David Kastrup:

> Jonas Hahnfeld <[hidden email]> writes:
> > Am Donnerstag, den 21.05.2020, 15:19 +0200 schrieb David Kastrup:
> > > Jonas Hahnfeld <[hidden email]> writes:
> > > > Am Donnerstag, den 21.05.2020, 14:29 +0200 schrieb David Kastrup:
> > > > > The "traffic jam" problem could be avoided by retaining the option of
> > > > > pushing to staging.  That would occur without CI, but one could
> > > > > occasionally trigger the merge with CI on staging to have everything in
> > > > > it migrate to master.  Since staging would be used by the more
> > > > > experienced people desiring to bunch their work before testing, the
> > > > > triggering could also happen manually by whoever thinks he has pushed
> > > > > enough stuff to staging to give it a whirl.
> > > >
> > > > That's not really how CI works. With our policy of FF merges, what
> > > > happens if some MR get merged directly to master and some sit in
> > > > staging? You'd probably rebase staging which triggers another CI
> > > > pipeline and doesn't buy you much.
> > >
> > > It buys you that several commits are bunched in staging and are treated
> > > in bulk.  At least I think it does.
> >
> > No, it doesn't: The MRs must pass CI individually before it can be
> > merged.
>
> I did not propose to have CI on staging.
In the current proposal, CI tests those merge requests that target
master. If we allowed others targeting staging without CI, we would be
unable to rely on automated testing. This essentially reduces the net
win of the whole thing to zero.

If we think contention will be a problem, we cannot do the proposal.
There's no sane "mixed bag": As outlined initially, we would 1) require
CI for merge requests, and 2) disable direct pushes to master. This
includes patchy which has no special permissions as far as GitLab is
concerned.

FWIW I don't see much contention at the current rate of development.
See also my other reply to Han-Wen.

signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [RFC] Enabling GitLab CI

David Kastrup
Jonas Hahnfeld <[hidden email]> writes:

> Am Donnerstag, den 21.05.2020, 17:10 +0200 schrieb David Kastrup:
>> Jonas Hahnfeld <[hidden email]> writes:
>> > Am Donnerstag, den 21.05.2020, 15:19 +0200 schrieb David Kastrup:
>> > > Jonas Hahnfeld <[hidden email]> writes:
>> > > > Am Donnerstag, den 21.05.2020, 14:29 +0200 schrieb David Kastrup:
>> > > > > The "traffic jam" problem could be avoided by retaining the option of
>> > > > > pushing to staging.  That would occur without CI, but one could
>> > > > > occasionally trigger the merge with CI on staging to have everything in
>> > > > > it migrate to master.  Since staging would be used by the more
>> > > > > experienced people desiring to bunch their work before testing, the
>> > > > > triggering could also happen manually by whoever thinks he has pushed
>> > > > > enough stuff to staging to give it a whirl.
>> > > >
>> > > > That's not really how CI works. With our policy of FF merges, what
>> > > > happens if some MR get merged directly to master and some sit in
>> > > > staging? You'd probably rebase staging which triggers another CI
>> > > > pipeline and doesn't buy you much.
>> > >
>> > > It buys you that several commits are bunched in staging and are treated
>> > > in bulk.  At least I think it does.
>> >
>> > No, it doesn't: The MRs must pass CI individually before it can be
>> > merged.
>>
>> I did not propose to have CI on staging.
>
> In the current proposal, CI tests those merge requests that target
> master. If we allowed others targeting staging without CI, we would be
> unable to rely on automated testing.

The automated testing would be done upon asking Gitlab to merge staging
into master.

> If we think contention will be a problem, we cannot do the proposal.
> There's no sane "mixed bag": As outlined initially, we would 1)
> require CI for merge requests, and 2) disable direct pushes to
> master. This includes patchy which has no special permissions as far
> as GitLab is concerned.

Sure, it would be the merge request of staging to master that would
trigger the CI then.

> FWIW I don't see much contention at the current rate of development.

Well yes.  And if there were much contention, we'd not likely stay in
the free plan for CI anyway.

> See also my other reply to Han-Wen.

--
David Kastrup

Reply | Threaded
Open this post in threaded view
|

Re: [RFC] Enabling GitLab CI

David Kastrup
In reply to this post by Jonas Hahnfeld
Jonas Hahnfeld <[hidden email]> writes:

> Am Donnerstag, den 21.05.2020, 17:21 +0200 schrieb Han-Wen Nienhuys:
>> On Thu, May 21, 2020 at 1:17 PM James <[hidden email]> wrote:
>> > On 21/05/2020 12:02, Han-Wen Nienhuys wrote:
>> > > so a next step might be making the countdown process more
>> > > continuous.
>> >
>> > What does that mean - even conceptually?
>>
>> My idea is that patches could enter 'countdown' stage throughout the
>> day, and when they do a 48 hour period waiting period kicks in. This
>> means the moment they are mergable will be more spread out throughout
>> the day.
>>
>> My main worry is that the CI process (make doc) will be forced to run
>> on machines with less CPU power, and the whole test/doc cycle would
>> take 1 or even 2 hours. If CI takes 2 hours, we'd realistically only
>> be able to merge 8 changes on a day.
>
> The full pipeline (build & test & doc) takes less than 1 hour on
> GitLab's shared runners with a single core. I think it's a safe bet
> that it can't get worse than that. If LilyPond itself gets slower by a
> factor of 2, I'd call that a regression in general.

If it is by the translation team doubling the number of supported
languages...  Actually, if translations were all kept up to date, we'd
probably need less time for "make doc" since then most of the
translations would work with the same code examples.

--
David Kastrup

Reply | Threaded
Open this post in threaded view
|

Re: [RFC] Enabling GitLab CI

Jonas Hahnfeld
In reply to this post by David Kastrup
Am Donnerstag, den 21.05.2020, 18:25 +0200 schrieb David Kastrup:
> > If we think contention will be a problem, we cannot do the proposal.
> > There's no sane "mixed bag": As outlined initially, we would 1)
> > require CI for merge requests, and 2) disable direct pushes to
> > master. This includes patchy which has no special permissions as far
> > as GitLab is concerned.
>
> Sure, it would be the merge request of staging to master that would
> trigger the CI then.

Interesting approach, I still had patchy in the equation. We'd still
need to target master initially so that James gets the CI results for
the countdown process. But this could be the easiest workaround in case
a developer with many patches hits contention and would be unable to
merge otherwise. I'm for keeping this in mind, but not making it a
rule.

signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [RFC] Enabling GitLab CI

Jonas Hahnfeld
In reply to this post by Han-Wen Nienhuys-3
Am Donnerstag, den 21.05.2020, 13:02 +0200 schrieb Han-Wen Nienhuys:

> On Thu, May 21, 2020 at 12:34 PM Jonas Hahnfeld <[hidden email]> wrote:
> > Am Dienstag, den 19.05.2020, 08:08 -0400 schrieb Dan Eble:
> > > On May 17, 2020, at 15:27, Jonas Hahnfeld <[hidden email]> wrote:
> > > > before merging. As we only allow fast-forward merges, this means each
> > > > MR has received testing in the form it hits master. This would
> > > > effectively replace the current setup of pushing to staging.
> > >
> > > I'm for this.
> >
> > Thanks. What do others (David, Han-Wen, Valentin) think about this?
> > There's certainly room for improvement, but with an initial setup I can
> > start writing documentation.
>
> Let's try it.
I'll do this once all merge requests currently in Patch::push are
safely in master, either later today or tomorrow. I'll try to update
existing merge requests afterwards and / or give instructions.

Jonas

signature.asc (499 bytes) Download Attachment
12