Re: branching stable/2.22?

classic Classic list List threaded Threaded
48 messages Options
123
Reply | Threaded
Open this post in threaded view
|

Re: branching stable/2.22?

Jean ABOU SAMRA
Hi,

(Sorry about the strange reply style.)

On Sun, Aug 23, 2020 at 1:58 PM Jonas Hahnfeld <[hidden email]> wrote:

> I'd like to ask what it would take in principle to branch stable/2.22
> and what others think about this.
>
> Personally I'm convinced that creating the branch and only picking bug
> fixes from master is the only guaranteed way to stabilize. Now you
> might say that there were only few unstable releases (AFAICT there was
> 2.19.65 before branching stable/2.20). However, I think there are
> already quite some user-facing changes and also the switch to Python 3.
> With Python 2.x being EOL since January, it's only a matter of time
> until Linux distributions and BSDs want to drop that, and it would be
> unfortunate if that put a (temporary) end to providing LilyPond for
> their users. If we had release candidates or even a stable version
> until then, it would definitely help.

Maybe we could try to release 2.20.1 with Python 3?

> The same can of course happen with Guile, but that situation has been
> going on for years. Furthermore, it's at least possible to compile and
> use current master with Guile 2.x, even if slower. In my understanding,
> things are getting better but properly integrating byte compilation is
> still a good amount of work that nobody could give a deadline on.
>
> WDYT?

[Han-Wen] I think that both Python 3 support and usable (if imperfect) GUILE 2 support is a strong argument for releasing a new stable version soon.

Why Guile 2? If it's still imperfect and slower, we don't want to make it the default in the binaries at lilypond.org, do we? So how will the situation be different from 2.20? Sorry, I must be missing something obvious here... I don't understand you.

At any rate, Python 3 support is great, but the scripts are fragile at the moment. This is clear from the tracker and the bug-lilypond list. Our Python scripts still need fixes in the way we distribute them, plus encoding-related issues (I'm planning on tackling the latter point in a short period at the end of August, but who knows what that will reveal).

[Han-Wen] I've been working on the build system (obviously), with in the back of my mind having a build that is no longer recursive, but that work could be paused for a bit while we prepare for releasing a 2.22. Is there a list of problems in the current master that have to be resolved?

Problems are basically popping every week (e.g., info installation, translation tools, etc.). You're fixing them every week, which is really great, but before creating a release branch that is devoted to stability, I think we need a couple months to see what new problems appear, don't we?

We have unstable releases to publish new features and get testing. In my opinion, stable releases should really focus on stability, there's no need to rush because of Python 3 and Guile 2.

At least four areas are currently under flux: Python scripts, the build system, Guile 2 support, and fonts (Owen's project), and I don't see that master is coming any close to stability. I think we are better with focusing on these areas as long as they still require substantial attention, so as to get a stable and nice LilyPond 2.22 in a timely manner. You derecursify the build, David completes his stack of rather extensive purportive changes, Owen merges the GSoC branch, etc., and only then it'll be time to care about LilyPond 2.22.

Am Sonntag, den 23.08.2020, 14:59 +0200 schrieb David Kastrup:
> > I'd like to ask what it would take in principle to branch stable/2.22
> > and what others think about this.
> I don't see that this is a good point of time.
> There has been an influx of badly tested changes to the build system and
> directory setup and the web pages that has to stabilise and result in a
> workable version of LilyPond. I don't see the point in branching a
> "stable" branch if there is so much in a destabilised state: you'd have
 > to cherry-pick loads of stuff from the unstable branch as it comes in.

[Jonas] I fully agree

... and so do I (for what my opinion's worth, really) ...

[Jonas] and I should have been more clear that I don't expect the branch to happen in the next week. The point was to find out what it would take because just waiting for some unspoken condition to become true is not exactly going to happen without some effort.

What about scheduling the release?

While I do know that "Grass doesn't grow faster when you pull on it.", I would definitely like having a defined point in time where the stable release is to happen, so that everyone can focus on bug fixes before it happens. Sure, we aren't going to get agreement in a second about the date (even if not more precise than a month), but to me, having this talk now is preferable so as to give LilyPond development a tempo. To say it with other words, we've got a score to play; arguing about the tempo is better than starting the piece with different tempi.

As sort of a shot in the dark, how about planning the 2.22 release for May 2021, for example?

Cheers,
Jean
Reply | Threaded
Open this post in threaded view
|

Re: branching stable/2.22?

Dan Eble
On Aug 23, 2020, at 17:44, Jean Abou Samra <[hidden email]> wrote:
>
> At least four areas are currently under flux: Python scripts, the build system, Guile 2 support, and fonts (Owen's project), and I don't see that master is coming any close to stability. I think we are better with focusing on these areas as long as they still require substantial attention, so as to get a stable and nice LilyPond 2.22 in a timely manner. You derecursify the build, David completes his stack of rather extensive purportive changes, Owen merges the GSoC branch, etc., and only then it'll be time to care about LilyPond 2.22.

... Dan submits his work on Simultaneous Alternatives, of which Unfoldable Lyrics are a special case ...

Dan


Reply | Threaded
Open this post in threaded view
|

Re: branching stable/2.22?

Jonas Hahnfeld
In reply to this post by Jean ABOU SAMRA
Am Sonntag, den 23.08.2020, 23:44 +0200 schrieb Jean Abou Samra:

> Hi,
>
> (Sorry about the strange reply style.)
>
> On Sun, Aug 23, 2020 at 1:58 PM Jonas Hahnfeld <[hidden email]> wrote:
> > I'd like to ask what it would take in principle to branch stable/2.22
> > and what others think about this.
> >
> > Personally I'm convinced that creating the branch and only picking bug
> > fixes from master is the only guaranteed way to stabilize. Now you
> > might say that there were only few unstable releases (AFAICT there was
> > 2.19.65 before branching stable/2.20). However, I think there are
> > already quite some user-facing changes and also the switch to Python 3.
> > With Python 2.x being EOL since January, it's only a matter of time
> > until Linux distributions and BSDs want to drop that, and it would be
> > unfortunate if that put a (temporary) end to providing LilyPond for
> > their users. If we had release candidates or even a stable version
> > until then, it would definitely help.
>
> Maybe we could try to release 2.20.1 with Python 3?
That would mean porting 50+ commits which sounds like a bad idea and
even gets worse because of the reformatting in master. The latter
implies that any bug fix made in master will result in merge conflicts.
Plus I don't understand the proposal if you're at the same time saying
that the scripts are fragile. By that logic, why would we backport such
extensive changes and claim they're stable?

> > The same can of course happen with Guile, but that situation has been
> > going on for years. Furthermore, it's at least possible to compile and
> > use current master with Guile 2.x, even if slower. In my understanding,
> > things are getting better but properly integrating byte compilation is
> > still a good amount of work that nobody could give a deadline on.
> >
> > WDYT?
>
> [Han-Wen] I think that both Python 3 support and usable (if imperfect) GUILE 2 support is a strong argument for releasing a new stable version soon.
>
> Why Guile 2? If it's still imperfect and slower, we don't want to make it the default in the binaries at lilypond.org, do we? So how will the situation be different from 2.20? Sorry, I must be missing something obvious here... I don't understand you.
At least in my book, it's not about changing the default but at least
making it possible for distributions to compile with Guile 2.x instead
of just throwing LilyPond out.


> At any rate, Python 3 support is great, but the scripts are fragile at the moment. This is clear from the tracker and the bug-lilypond list. Our Python scripts still need fixes in the way we distribute them, plus encoding-related issues (I'm planning on tackling the latter point in a short period at the end of August, but who knows what that will reveal).

Could you please point to concrete issues? The distribution of scripts
hasn't changed, so it basically still works the same it did for years
(decades?).

> [Han-Wen] I've been working on the build system (obviously), with in the back of my mind having a build that is no longer recursive, but that work could be paused for a bit while we prepare for releasing a 2.22. Is there a list of problems in the current master that have to be resolved?
>
> Problems are basically popping every week (e.g., info installation, translation tools, etc.). You're fixing them every week, which is really great, but before creating a release branch that is devoted to stability, I think we need a couple months to see what new problems appear, don't we?
>
> We have unstable releases to publish new features and get testing. In my opinion, stable releases should really focus on stability, there's no need to rush because of Python 3 and Guile 2.

I don't think we'll get testing from distributions until we declare a
way to stabilize.

> At least four areas are currently under flux: Python scripts, the build system, Guile 2 support, and fonts (Owen's project), and I don't see that master is coming any close to stability. I think we are better with focusing on these areas as long as they still require substantial attention, so as to get a stable and nice LilyPond 2.22 in a timely manner. You derecursify the build, David completes his stack of rather extensive purportive changes, Owen merges the GSoC branch, etc., and only then it'll be time to care about LilyPond 2.22.

And that's my point: If we just sit and wait, the next stable release
might happen after all major distributions threw LilyPond out - simply
because it cannot be built in a world of Python 3.

> Am Sonntag, den 23.08.2020, 14:59 +0200 schrieb David Kastrup:
> > > I'd like to ask what it would take in principle to branch stable/2.22
> > > and what others think about this.
> > I don't see that this is a good point of time.
> > There has been an influx of badly tested changes to the build system and
> > directory setup and the web pages that has to stabilise and result in a
> > workable version of LilyPond. I don't see the point in branching a
> > "stable" branch if there is so much in a destabilised state: you'd have
>  > to cherry-pick loads of stuff from the unstable branch as it comes in.
>
> [Jonas] I fully agree
>
> ... and so do I (for what my opinion's worth, really) ...
>
> [Jonas] and I should have been more clear that I don't expect the branch to happen in the next week. The point was to find out what it would take because just waiting for some unspoken condition to become true is not exactly going to happen without some effort.
>
> What about scheduling the release?
>
> While I do know that "Grass doesn't grow faster when you pull on it.", I would definitely like having a defined point in time where the stable release is to happen, so that everyone can focus on bug fixes before it happens. Sure, we aren't going to get agreement in a second about the date (even if not more precise than a month), but to me, having this talk now is preferable so as to give LilyPond development a tempo. To say it with other words, we've got a score to play; arguing about the tempo is better than starting the piece with different tempi.
>
> As sort of a shot in the dark, how about planning the 2.22 release for May 2021, for example?
Do you mean branching stable/2.22 or releasing 2.22.0 in May 2021? In
my understanding, the past process includes the release of beta
versions from the branch. That makes it close to impossible to predict
the final date of the stable version, and that's not the purpose of
this thread.

>
> Cheers,
> Jean

signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: branching stable/2.22?

Jean ABOU SAMRA
Hi,

> Le 24 août 2020 à 08:30, Jonas Hahnfeld <[hidden email]> a écrit :
>
> Am Sonntag, den 23.08.2020, 23:44 +0200 schrieb Jean Abou Samra:
>> Hi,
>>
>> (Sorry about the strange reply style.)
>>
>>> On Sun, Aug 23, 2020 at 1:58 PM Jonas Hahnfeld <[hidden email]> wrote:
>>> I'd like to ask what it would take in principle to branch stable/2.22
>>> and what others think about this.
>>>
>>> Personally I'm convinced that creating the branch and only picking bug
>>> fixes from master is the only guaranteed way to stabilize. Now you
>>> might say that there were only few unstable releases (AFAICT there was
>>> 2.19.65 before branching stable/2.20). However, I think there are
>>> already quite some user-facing changes and also the switch to Python 3.
>>> With Python 2.x being EOL since January, it's only a matter of time
>>> until Linux distributions and BSDs want to drop that, and it would be
>>> unfortunate if that put a (temporary) end to providing LilyPond for
>>> their users. If we had release candidates or even a stable version
>>> until then, it would definitely help.
>>
>> Maybe we could try to release 2.20.1 with Python 3?
>
> That would mean porting 50+ commits which sounds like a bad idea and
> even gets worse because of the reformatting in master. The latter
> implies that any bug fix made in master will result in merge conflicts.
> Plus I don't understand the proposal if you're at the same time saying
> that the scripts are fragile. By that logic, why would we backport such
> extensive changes and claim they're stable?

Right, I was oblique: the scripts are fragile at present, so branching release/2.22 now is no good in my opinion, but hopefully we can stabilize them faster than we stabilize LilyPond as a whole, and have that in 2.20.1 or 2.20.2.

Can you explain why porting 50 commits from master to 2.20 is a bad idea? If we port all Python related-commits (including the reformatting), there won't be any merge conflicts, or am I being dense somehow?

I do understand that having LilyPond 2.20.0 support exclusively Python 2 but 2.20.1 be Python 3 only feels weird. However, I value the interest of the average user more than that of packagers. Neither Python 3 nor Guile 2 is a breaking change from the user's point of view. If having LilyPond 2.20.1 (or 2.20.2) support Python 3 is not feasible, I think we should just let distributions drop LilyPond (see below). I'm not happy about it, but this is, in my opinion, preferable to making a stable release, in a hurry, that will contain more bugs and few user-facing changes.

>>> The same can of course happen with Guile, but that situation has been
>>> going on for years. Furthermore, it's at least possible to compile and
>>> use current master with Guile 2.x, even if slower. In my understanding,
>>> things are getting better but properly integrating byte compilation is
>>> still a good amount of work that nobody could give a deadline on.
>>>
>>> WDYT?
>>
>> [Han-Wen] I think that both Python 3 support and usable (if imperfect) GUILE 2 support is a strong argument for releasing a new stable version soon.
>>
>> Why Guile 2? If it's still imperfect and slower, we don't want to make it the default in the binaries at lilypond.org, do we? So how will the situation be different from 2.20? Sorry, I must be missing something obvious here... I don't understand you.
>
> At least in my book, it's not about changing the default but at least
> making it possible for distributions to compile with Guile 2.x instead
> of just throwing LilyPond out.

Does this mean that we'll receive bug reports that won't be reproducible by others because they'll actually be related to Guile 2? In my opinion, the distributions just throwing out LilyPond is better.

Additionally, the sh installers are recommended by the official website over distribution package managers.

>> At any rate, Python 3 support is great, but the scripts are fragile at the moment. This is clear from the tracker and the bug-lilypond list. Our Python scripts still need fixes in the way we distribute them, plus encoding-related issues (I'm planning on tackling the latter point in a short period at the end of August, but who knows what that will reveal).
>
> Could you please point to concrete issues? The distribution of scripts
> hasn't changed, so it basically still works the same it did for years
> (decades?).

I'm writing this with a browser that is too old to view gitlab.com. There is at least
https://www.mail-archive.com/bug-lilypond@.../msg43376.html
which looks serious at first sight.

>> [Han-Wen] I've been working on the build system (obviously), with in the back of my mind having a build that is no longer recursive, but that work could be paused for a bit while we prepare for releasing a 2.22. Is there a list of problems in the current master that have to be resolved?
>>
>> Problems are basically popping every week (e.g., info installation, translation tools, etc.). You're fixing them every week, which is really great, but before creating a release branch that is devoted to stability, I think we need a couple months to see what new problems appear, don't we?
>>
>> We have unstable releases to publish new features and get testing. In my opinion, stable releases should really focus on stability, there's no need to rush because of Python 3 and Guile 2.
>
> I don't think we'll get testing from distributions until we declare a
> way to stabilize.

We're speaking from different points of view here: in my book, our main source of testing is our development and beta releases brought to users through installers. I mean that LilyPond 2.22 should introduce full support for Guile 2 with byte compilation, probably dropping Guile 1 support too, and we get Guile 2 testing from those who try out the betas.

>> At least four areas are currently under flux: Python scripts, the build system, Guile 2 support, and fonts (Owen's project), and I don't see that master is coming any close to stability. I think we are better with focusing on these areas as long as they still require substantial attention, so as to get a stable and nice LilyPond 2.22 in a timely manner. You derecursify the build, David completes his stack of rather extensive purportive changes, Owen merges the GSoC branch, etc., and only then it'll be time to care about LilyPond 2.22.
>
> And that's my point: If we just sit and wait, the next stable release
> might happen after all major distributions threw LilyPond out - simply
> because it cannot be built in a world of Python 3.
>> Am Sonntag, den 23.08.2020, 14:59 +0200 schrieb David Kastrup:
>>>> I'd like to ask what it would take in principle to branch stable/2.22
>>>> and what others think about this.
>>> I don't see that this is a good point of time.
>>> There has been an influx of badly tested changes to the build system and
>>> directory setup and the web pages that has to stabilise and result in a
>>> workable version of LilyPond. I don't see the point in branching a
>>> "stable" branch if there is so much in a destabilised state: you'd have
>>> to cherry-pick loads of stuff from the unstable branch as it comes in.
>>
>> [Jonas] I fully agree
>>
>> ... and so do I (for what my opinion's worth, really) ...
>>
>> [Jonas] and I should have been more clear that I don't expect the branch to happen in the next week. The point was to find out what it would take because just waiting for some unspoken condition to become true is not exactly going to happen without some effort.
>>
>> What about scheduling the release?
>>
>> While I do know that "Grass doesn't grow faster when you pull on it.", I would definitely like having a defined point in time where the stable release is to happen, so that everyone can focus on bug fixes before it happens. Sure, we aren't going to get agreement in a second about the date (even if not more precise than a month), but to me, having this talk now is preferable so as to give LilyPond development a tempo. To say it with other words, we've got a score to play; arguing about the tempo is better than starting the piece with different tempi.
>>
>> As sort of a shot in the dark, how about planning the 2.22 release for May 2021, for example?
>
> Do you mean branching stable/2.22 or releasing 2.22.0 in May 2021? In
> my understanding, the past process includes the release of beta
> versions from the branch. That makes it close to impossible to predict
> the final date of the stable version, and that's not the purpose of
> this thread.

I mean releasing 2.22.0 in May 2021. This is not about predictions, but objectives. I think that instead of planning each small step on the fly with no idea how the future looks like, we should settle an expected date for the release and plan backwards accordingly. Sure, there could be critical bugs that delay the actual release, but setting the expectation enables more resources to focus on the release by the time it is due to happen. In my opinion, this is the way we can avoid things like the 2.14 release that is documented in the CG.

So, an expected release in May 2021 would mean branching release/2.20 around January (?) and beta releases at a monthly cadence until the release is out.

I'm curious about what others think of that. In fact it looks like you already proposed something along these lines:
https://www.mail-archive.com/lilypond-devel@.../msg72997.html
But David was reluctant for reasons that sound sensible. David, what would be your opinion today?

Best,
Jean
Reply | Threaded
Open this post in threaded view
|

Re: branching stable/2.22?

Jonas Hahnfeld
Am Montag, den 24.08.2020, 12:56 +0200 schrieb Jean Abou Samra:

> Hi,
>
> > Le 24 août 2020 à 08:30, Jonas Hahnfeld <[hidden email]> a écrit :
> >
> > Am Sonntag, den 23.08.2020, 23:44 +0200 schrieb Jean Abou Samra:
> > > Hi,
> > >
> > > (Sorry about the strange reply style.)
> > >
> > > > On Sun, Aug 23, 2020 at 1:58 PM Jonas Hahnfeld <[hidden email]> wrote:
> > > > I'd like to ask what it would take in principle to branch stable/2.22
> > > > and what others think about this.
> > > >
> > > > Personally I'm convinced that creating the branch and only picking bug
> > > > fixes from master is the only guaranteed way to stabilize. Now you
> > > > might say that there were only few unstable releases (AFAICT there was
> > > > 2.19.65 before branching stable/2.20). However, I think there are
> > > > already quite some user-facing changes and also the switch to Python 3.
> > > > With Python 2.x being EOL since January, it's only a matter of time
> > > > until Linux distributions and BSDs want to drop that, and it
> > > > would be
> > > > unfortunate if that put a (temporary) end to providing LilyPond for
> > > > their users. If we had release candidates or even a stable version
> > > > until then, it would definitely help.
> > >
> > > Maybe we could try to release 2.20.1 with Python 3?
> >
> > That would mean porting 50+ commits which sounds like a bad idea and
> > even gets worse because of the reformatting in master. The latter
> > implies that any bug fix made in master will result in merge conflicts.
> > Plus I don't understand the proposal if you're at the same time saying
> > that the scripts are fragile. By that logic, why would we backport such
> > extensive changes and claim they're stable?
>
> Right, I was oblique: the scripts are fragile at present, so branching release/2.22 now is no good in my opinion, but hopefully we can stabilize them faster than we stabilize LilyPond as a whole, and have that in 2.20.1 or 2.20.2.
>
> Can you explain why porting 50 commits from master to 2.20 is a bad idea?
I think it's a bad idea because it goes against my basic understanding
that only bug fixes should be ported a stable branch. Here's the total
number of commits in stable/2.20 since branching:
$ git log --oneline release/2.20.0-1...release/2.19.65-1 | wc -l
588

Sounds high at first, but most of that was translations, ie
$ git log --oneline release/2.20.0-1...release/2.19.65-1 -- $(ls | grep -vE "Documentation|po") | wc -l
268
If only focusing on actual code changes:
$ git log --oneline release/2.20.0-1...release/2.19.65-1 -- flower/ lily/ ly/ python/ scm/ scripts/ | wc -l
198
with the majority in "core" LilyPond:
$ git log --oneline release/2.20.0-1...release/2.19.65-1 -- lily/ ly/ scm/ | wc -l
148

> If we port all Python related-commits (including the reformatting), there won't be any merge conflicts, or am I being dense somehow?

Consider:
$ git log --oneline release/2.20.0-1...release/2.19.65-1 -- python/ | wc -l
22
vs
$ git log --oneline HEAD...release/2.19.65-1 -- python/ | wc -l
115

and

$ git log --oneline release/2.20.0-1...release/2.19.65-1 -- scripts/*.py | wc -l
5
vs
$ git log --oneline HEAD...release/2.19.65-1 -- scripts/*.py | wc -l
67

Being the individual with the most commits in there during the cycle, I
strongly advise against taking all of this for a minor stable release.


> I do understand that having LilyPond 2.20.0 support exclusively Python 2 but 2.20.1 be Python 3 only feels weird. However, I value the interest of the average user more than that of packagers. Neither Python 3 nor Guile 2 is a breaking change from the user's point of view. If having LilyPond 2.20.1 (or 2.20.2) support Python 3 is not feasible, I think we should just let distributions drop LilyPond (see below). I'm not happy about it, but this is, in my opinion, preferable to making a stable release, in a hurry, that will contain more bugs and few user-facing changes.
>
> > > > The same can of course happen with Guile, but that situation has been
> > > > going on for years. Furthermore, it's at least possible to compile and
> > > > use current master with Guile 2.x, even if slower. In my understanding,
> > > > things are getting better but properly integrating byte compilation is
> > > > still a good amount of work that nobody could give a deadline on.
> > > >
> > > > WDYT?
> > >
> > > [Han-Wen] I think that both Python 3 support and usable (if imperfect) GUILE 2 support is a strong argument for releasing a new stable version soon.
> > >
> > > Why Guile 2? If it's still imperfect and slower, we don't want to make it the default in the binaries at lilypond.org, do we? So how will the situation be different from 2.20? Sorry, I must be missing something obvious here... I don't understand you.
> >
> > At least in my book, it's not about changing the default but at least
> > making it possible for distributions to compile with Guile 2.x instead
> > of just throwing LilyPond out.
>
> Does this mean that we'll receive bug reports that won't be reproducible by others because they'll actually be related to Guile 2? In my opinion, the distributions just throwing out LilyPond is better.
>
> Additionally, the sh installers are recommended by the official website over distribution package managers.
>
> > > At any rate, Python 3 support is great, but the scripts are fragile at the moment. This is clear from the tracker and the bug-lilypond list. Our Python scripts still need fixes in the way we distribute them, plus encoding-related issues (I'm planning on tackling the latter point in a short period at the end of August, but who knows what that will reveal).
> >
> > Could you please point to concrete issues? The distribution of scripts
> > hasn't changed, so it basically still works the same it did for years
> > (decades?).
>
> I'm writing this with a browser that is too old to view gitlab.com. There is at least
> https://www.mail-archive.com/bug-lilypond@.../msg43376.html
>
> which looks serious at first sight.
>
> > > [Han-Wen] I've been working on the build system (obviously), with in the back of my mind having a build that is no longer recursive, but that work could be paused for a bit while we prepare for releasing a 2.22. Is there a list of problems in the current master that have to be resolved?
> > >
> > > Problems are basically popping every week (e.g., info installation, translation tools, etc.). You're fixing them every week, which is really great, but before creating a release branch that is devoted to stability, I think we need a couple months to see what new problems appear, don't we?
> > >
> > > We have unstable releases to publish new features and get testing. In my opinion, stable releases should really focus on stability, there's no need to rush because of Python 3 and Guile 2.
> >
> > I don't think we'll get testing from distributions until we declare a
> > way to stabilize.
>
> We're speaking from different points of view here: in my book, our main source of testing is our development and beta releases brought to users through installers. I mean that LilyPond 2.22 should introduce full support for Guile 2 with byte compilation, probably dropping Guile 1 support too, and we get Guile 2 testing from those who try out the betas.
I seriously doubt you'll get that for 2.22 next year.

> > > At least four areas are currently under flux: Python scripts, the build system, Guile 2 support, and fonts (Owen's project), and I don't see that master is coming any close to stability. I think we are better with focusing on these areas as long as they still require substantial attention, so as to get a stable and nice LilyPond 2.22 in a timely manner. You derecursify the build, David completes his stack of rather extensive purportive changes, Owen merges the GSoC branch, etc., and only then it'll be time to care about LilyPond 2.22.
> >
> > And that's my point: If we just sit and wait, the next stable release
> > might happen after all major distributions threw LilyPond out - simply
> > because it cannot be built in a world of Python 3.
> > > Am Sonntag, den 23.08.2020, 14:59 +0200 schrieb David Kastrup:
> > > > > I'd like to ask what it would take in principle to branch stable/2.22
> > > > > and what others think about this.
> > > >
> > > > I don't see that this is a good point of time.
> > > > There has been an influx of badly tested changes to the build system and
> > > > directory setup and the web pages that has to stabilise and result in a
> > > > workable version of LilyPond. I don't see the point in branching a
> > > > "stable" branch if there is so much in a destabilised state: you'd have
> > > > to cherry-pick loads of stuff from the unstable branch as it comes in.
> > >
> > > [Jonas] I fully agree
> > >
> > > ... and so do I (for what my opinion's worth, really) ...
> > >
> > > [Jonas] and I should have been more clear that I don't expect the branch to happen in the next week. The point was to find out what it would take because just waiting for some unspoken condition to become true is not exactly going to happen without some effort.
> > >
> > > What about scheduling the release?
> > >
> > > While I do know that "Grass doesn't grow faster when you pull on it.", I would definitely like having a defined point in time where the stable release is to happen, so that everyone can focus on bug fixes before it happens. Sure, we aren't going to get agreement in a second about the date (even if not more precise than a month), but to me, having this talk now is preferable so as to give LilyPond development a tempo. To say it with other words, we've got a score to play; arguing about the tempo is better than starting the piece with different tempi.
> > >
> > > As sort of a shot in the dark, how about planning the 2.22 release for May 2021, for example?
> >
> > Do you mean branching stable/2.22 or releasing 2.22.0 in May 2021? In
> > my understanding, the past process includes the release of beta
> > versions from the branch. That makes it close to impossible to predict
> > the final date of the stable version, and that's not the purpose of
> > this thread.
>
> I mean releasing 2.22.0 in May 2021. This is not about predictions, but objectives. I think that instead of planning each small step on the fly with no idea how the future looks like, we should settle an expected date for the release and plan backwards accordingly. Sure, there could be critical bugs that delay the actual release, but setting the expectation enables more resources to focus on the release by the time it is due to happen. In my opinion, this is the way we can avoid things like the 2.14 release that is documented in the CG.
>
> So, an expected release in May 2021 would mean branching release/2.20 around January (?) and beta releases at a monthly cadence until the release is out.
>
> I'm curious about what others think of that. In fact it looks like you already proposed something along these lines:
> https://www.mail-archive.com/lilypond-devel@.../msg72997.html
And it didn't get much support. Which is why I don't see what's
different today. Asking what it would take to branch is really the only
sensible thing I think we could possibly agree on.

> But David was reluctant for reasons that sound sensible. David, what would be your opinion today?
>
> Best,
> Jean
>

signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: branching stable/2.22?

David Kastrup
Jonas Hahnfeld <[hidden email]> writes:

> Am Montag, den 24.08.2020, 12:56 +0200 schrieb Jean Abou Samra:
>>
>> Right, I was oblique: the scripts are fragile at present, so
>> branching release/2.22 now is no good in my opinion, but hopefully
>> we can stabilize them faster than we stabilize LilyPond as a whole,
>> and have that in 2.20.1 or 2.20.2.
>>
>> Can you explain why porting 50 commits from master to 2.20 is a bad idea?
>
> I think it's a bad idea because it goes against my basic understanding
> that only bug fixes should be ported a stable branch. Here's the total
> number of commits in stable/2.20 since branching:
> $ git log --oneline release/2.20.0-1...release/2.19.65-1 | wc -l
> 588
>
> Sounds high at first, but most of that was translations, ie
> $ git log --oneline release/2.20.0-1...release/2.19.65-1 -- $(ls |
> grep -vE "Documentation|po") | wc -l
> 268
> If only focusing on actual code changes:
> $ git log --oneline release/2.20.0-1...release/2.19.65-1 -- flower/
> lily/ ly/ python/ scm/ scripts/ | wc -l
> 198
> with the majority in "core" LilyPond:
> $ git log --oneline release/2.20.0-1...release/2.19.65-1 -- lily/ ly/ scm/ | wc -l
> 148

It's worth stressing that the 2.20 branch persisted much longer before
the 2.20 release than planned.  So there were also some feature
cherry-picks in order to avoid the situation that the 2.20 release would
be "outdated from the start" with regard to some "must-have" features
that would be expected to be common in suggested code on the mailing
lists.

So 2.20's history in some way reflects how to muddle through in a
situation that became a lot different from planning.  It's not really a
template for how things should work.

--
David Kastrup

Reply | Threaded
Open this post in threaded view
|

Re: branching stable/2.22?

Jean ABOU SAMRA
In reply to this post by Jonas Hahnfeld
Hi,

> Le 24 août 2020 à 13:46, Jonas Hahnfeld <[hidden email]> a écrit :
>
> Am Montag, den 24.08.2020, 12:56 +0200 schrieb Jean Abou Samra:
>> Hi,
>>
>>> Le 24 août 2020 à 08:30, Jonas Hahnfeld <[hidden email]> a écrit :
>>>
>>> Am Sonntag, den 23.08.2020, 23:44 +0200 schrieb Jean Abou Samra:
>>>> Maybe we could try to release 2.20.1 with Python 3?
>>>
>>> That would mean porting 50+ commits which sounds like a bad idea and
>>> even gets worse because of the reformatting in master. The latter
>>> implies that any bug fix made in master will result in merge conflicts.
>>> Plus I don't understand the proposal if you're at the same time saying
>>> that the scripts are fragile. By that logic, why would we backport such
>>> extensive changes and claim they're stable?
>>
>> Right, I was oblique: the scripts are fragile at present, so branching release/2.22 now is no good in my opinion, but hopefully we can stabilize them faster than we stabilize LilyPond as a whole, and have that in 2.20.1 or 2.20.2.
>>
>> Can you explain why porting 50 commits from master to 2.20 is a bad idea?
>
> I think it's a bad idea because it goes against my basic understanding
> that only bug fixes should be ported a stable branch.

This situation is special anyway: we did a release that is from the start outdated with respect to Python support.

All I'm saying is that porting this to 2.20 would be better than releasing 2.22 in a hurry, for the reasons that David and I mentioned. So...

> Here's the total
> number of commits in stable/2.20 since branching:
> [...]
>> If we port all Python related-commits (including the reformatting), there won't be any merge conflicts, or am I being dense somehow?
>
> [...]
> Being the individual with the most commits in there during the cycle, I
> strongly advise against taking all of this for a minor stable release.

... so since that appears unpossible, I guess we'd just let distros drop LilyPond.


>> I do understand that having LilyPond 2.20.0 support exclusively Python 2 but 2.20.1 be Python 3 only feels weird. However, I value the interest of the average user more than that of packagers. Neither Python 3 nor Guile 2 is a breaking change from the user's point of view. If having LilyPond 2.20.1 (or 2.20.2) support Python 3 is not feasible, I think we should just let distributions drop LilyPond (see below). I'm not happy about it, but this is, in my opinion, preferable to making a stable release, in a hurry, that will contain more bugs and few user-facing changes.
>>
>>>> Why Guile 2? If it's still imperfect and slower, we don't want to make it the default in the binaries at lilypond.org, do we? So how will the situation be different from 2.20? Sorry, I must be missing something obvious here... I don't understand you.
>>>
>>> At least in my book, it's not about changing the default but at least
>>> making it possible for distributions to compile with Guile 2.x instead
>>> of just throwing LilyPond out.
>>
>> Does this mean that we'll receive bug reports that won't be reproducible by others because they'll actually be related to Guile 2? In my opinion, the distributions just throwing out LilyPond is better.
>>
>> Additionally, the sh installers are recommended by the official website over distribution package managers.




>>> I don't think we'll get testing from distributions until we declare a
>>> way to stabilize.
>>
>> We're speaking from different points of view here: in my book, our main source of testing is our development and beta releases brought to users through installers. I mean that LilyPond 2.22 should introduce full support for Guile 2 with byte compilation, probably dropping Guile 1 support too, and we get Guile 2 testing from those who try out the betas.
>
> I seriously doubt you'll get that for 2.22 next year.

That date was a shot in the dark intended at starting the discussion.

See also below.

>>>> Am Sonntag, den 23.08.2020, 14:59 +0200 schrieb David Kastrup:
>>>>>> I'd like to ask what it would take in principle to branch stable/2.22
>>>>>> and what others think about this.
>>>>>
>>>>> I don't see that this is a good point of time.
>>>>> There has been an influx of badly tested changes to the build system and
>>>>> directory setup and the web pages that has to stabilise and result in a
>>>>> workable version of LilyPond. I don't see the point in branching a
>>>>> "stable" branch if there is so much in a destabilised state: you'd have
>>>>> to cherry-pick loads of stuff from the unstable branch as it comes in.
>>>>
>>>> [Jonas] I fully agree
>>>>
>>>> ... and so do I (for what my opinion's worth, really) ...
>>>>
>>>> [Jonas] and I should have been more clear that I don't expect the branch to happen in the next week. The point was to find out what it would take because just waiting for some unspoken condition to become true is not exactly going to happen without some effort.
>>>>
>>>> What about scheduling the release?
>>>>
>>>> While I do know that "Grass doesn't grow faster when you pull on it.", I would definitely like having a defined point in time where the stable release is to happen, so that everyone can focus on bug fixes before it happens. Sure, we aren't going to get agreement in a second about the date (even if not more precise than a month), but to me, having this talk now is preferable so as to give LilyPond development a tempo. To say it with other words, we've got a score to play; arguing about the tempo is better than starting the piece with different tempi.
>>>>
>>>> As sort of a shot in the dark, how about planning the 2.22 release for May 2021, for example?
>>>
>>> Do you mean branching stable/2.22 or releasing 2.22.0 in May 2021? In
>>> my understanding, the past process includes the release of beta
>>> versions from the branch. That makes it close to impossible to predict
>>> the final date of the stable version, and that's not the purpose of
>>> this thread.
>>
>> I mean releasing 2.22.0 in May 2021. This is not about predictions, but objectives. I think that instead of planning each small step on the fly with no idea how the future looks like, we should settle an expected date for the release and plan backwards accordingly. Sure, there could be critical bugs that delay the actual release, but setting the expectation enables more resources to focus on the release by the time it is due to happen. In my opinion, this is the way we can avoid things like the 2.14 release that is documented in the CG.
>>
>> So, an expected release in May 2021 would mean branching release/2.20 around January (?) and beta releases at a monthly cadence until the release is out.
>>
>> I'm curious about what others think of that. In fact it looks like you already proposed something along these lines:
>> https://www.mail-archive.com/lilypond-devel@.../msg72997.html
>
> And it didn't get much support. Which is why I don't see what's
> different today. Asking what it would take to branch is really the only
> sensible thing I think we could possibly agree on.

As I see it, you're asking something nobody, apparently, can give you. We need to create the process instead of finding it out: what do you think it should take to branch?

Branching means collectively commiting to creating a stable release a handful of months later, otherwise we get to the situation that David described in his last reply: the stable branch comes so far from master that features need to be ported to it; clearly, that's not a desirable workflow.

Whatever the option, we will need people to manage the release (yes, I could possibly help next summer ... though I'm afraid I'd be NOT_SMART). So, I think the question is essentially wether we try to plan the release now or just wait for the essential features we'd like in it to be implemented, e.g., full Guile 2 support. Personally, I think it's better to plan it so hopefully developers will naturally organize their respective works accordingly. In this perspective, if full Guile 2 support is not implemented by the deadline, it just waits for another release. But that's just my opinion.

Cheers,
Jean

>> But David was reluctant for reasons that sound sensible. David, what would be your opinion today?



Reply | Threaded
Open this post in threaded view
|

Re: branching stable/2.22?

Jonas Hahnfeld
Am Montag, den 24.08.2020, 22:10 +0200 schrieb Jean Abou Samra:

> > > > > As sort of a shot in the dark, how about planning the 2.22 release for May 2021, for example?
> > > >
> > > > Do you mean branching stable/2.22 or releasing 2.22.0 in May 2021? In
> > > > my understanding, the past process includes the release of beta
> > > > versions from the branch. That makes it close to impossible to predict
> > > > the final date of the stable version, and that's not the purpose of
> > > > this thread.
> > >
> > > I mean releasing 2.22.0 in May 2021. This is not about predictions, but objectives. I think that instead of planning each small step on the fly with no idea how the future looks like, we should settle an expected date for the release and plan backwards accordingly. Sure, there could be critical bugs that delay the actual release, but setting the expectation enables more resources to focus on the release by the time it is due to happen. In my opinion, this is the way we can avoid things like the 2.14 release that is documented in the CG.
> > >
> > > So, an expected release in May 2021 would mean branching release/2.20 around January (?) and beta releases at a monthly cadence until the release is out.
> > >
> > > I'm curious about what others think of that. In fact it looks like you already proposed something along these lines:
> > > https://www.mail-archive.com/lilypond-devel@.../msg72997.html
> >
> > And it didn't get much support. Which is why I don't see what's
> > different today. Asking what it would take to branch is really the only
> > sensible thing I think we could possibly agree on.
>
> As I see it, you're asking something nobody, apparently, can give you. We need to create the process instead of finding it out: what do you think it should take to branch?
For me, creating the branch is nothing more than saying "feature
development is over and the current set is worth making stable". Which
I'm arguing is already there with Python 3 and the possibility to use
Guile 2. See my very initial message.

On the administrative side, I think
 * there should be another reformatting for all C++ and Scheme
 * docs should be updated, including authors.itexi
Everything else can and should be picked as needed.

> Branching means collectively commiting to creating a stable release a handful of months later, otherwise we get to the situation that David described in his last reply: the stable branch comes so far from master that features need to be ported to it; clearly, that's not a desirable workflow.

I don't understand why you would want to backport features? IMO that's
got nothing to do with how far the stable branch diverges.

> Whatever the option, we will need people to manage the release (yes, I could possibly help next summer ... though I'm afraid I'd be NOT_SMART).

If it's just missing people to do the work, I obviously volunteer and
am willing to cherry-pick fixes from master as needed.

> So, I think the question is essentially wether we try to plan the release now or just wait for the essential features we'd like in it to be implemented, e.g., full Guile 2 support. Personally, I think it's better to plan it so hopefully developers will naturally organize their respective works accordingly. In this perspective, if full Guile 2 support is not implemented by the deadline, it just waits for another release. But that's just my opinion.

From my experience in the LLVM project, there is no such thing as
"naturally stabilizing code". Either you create a branch and pick fixes
or you have a strict policy that allows only fixes to master before
branching. That's basically the model GCC is using, and I don't think
it fits the community.

Jonas

signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: branching stable/2.22?

Jean ABOU SAMRA


> Le 25 août 2020 à 08:30, Jonas Hahnfeld <[hidden email]> a écrit :
>
> Am Montag, den 24.08.2020, 22:10 +0200 schrieb Jean Abou Samra:
>>>>>> As sort of a shot in the dark, how about planning the 2.22 release for May 2021, for example?
>>>>>
>>>>> Do you mean branching stable/2.22 or releasing 2.22.0 in May 2021? In
>>>>> my understanding, the past process includes the release of beta
>>>>> versions from the branch. That makes it close to impossible to predict
>>>>> the final date of the stable version, and that's not the purpose of
>>>>> this thread.
>>>>
>>>> I mean releasing 2.22.0 in May 2021. This is not about predictions, but objectives. I think that instead of planning each small step on the fly with no idea how the future looks like, we should settle an expected date for the release and plan backwards accordingly. Sure, there could be critical bugs that delay the actual release, but setting the expectation enables more resources to focus on the release by the time it is due to happen. In my opinion, this is the way we can avoid things like the 2.14 release that is documented in the CG.
>>>>
>>>> So, an expected release in May 2021 would mean branching release/2.20 around January (?) and beta releases at a monthly cadence until the release is out.
>>>>
>>>> I'm curious about what others think of that. In fact it looks like you already proposed something along these lines:
>>>> https://www.mail-archive.com/lilypond-devel@.../msg72997.html
>>>
>>> And it didn't get much support. Which is why I don't see what's
>>> different today. Asking what it would take to branch is really the only
>>> sensible thing I think we could possibly agree on.
>>
>> As I see it, you're asking something nobody, apparently, can give you. We need to create the process instead of finding it out: what do you think it should take to branch?
>
> For me, creating the branch is nothing more than saying "feature
> development is over and the current set is worth making stable". Which
> I'm arguing is already there with Python 3 and the possibility to use
> Guile 2. See my very initial message.

At the same time, you're saying that branching is not going to happen next week. Please elaborate on your mind: when should that happen?

> On the administrative side, I think
> * there should be another reformatting for all C++ and Scheme
> * docs should be updated, including authors.itexi
> Everything else can and should be picked as needed.
>
>> Branching means collectively commiting to creating a stable release a handful of months later, otherwise we get to the situation that David described in his last reply: the stable branch comes so far from master that features need to be ported to it; clearly, that's not a desirable workflow.
>
> I don't understand why you would want to backport features? IMO that's
> got nothing to do with how far the stable branch diverges.

I was talking about what happened in 2.20:

> It's worth stressing that the 2.20 branch persisted much longer before
> the 2.20 release than planned.  So there were also some feature
> cherry-picks in order to avoid the situation that the 2.20 release would
> be "outdated from the start" with regard to some "must-have" features
> that would be expected to be common in suggested code on the mailing
> lists.


Now, you might say that if we do more frequent releases, the problem will disappear. This means, however, that we're planning to make releases more often, which is a decision that extends far beyond simply planning the branching time.


>> Whatever the option, we will need people to manage the release (yes, I could possibly help next summer ... though I'm afraid I'd be NOT_SMART).
>
> If it's just missing people to do the work, I obviously volunteer and
> am willing to cherry-pick fixes from master as needed.

Nice!

>> So, I think the question is essentially wether we try to plan the release now or just wait for the essential features we'd like in it to be implemented, e.g., full Guile 2 support. Personally, I think it's better to plan it so hopefully developers will naturally organize their respective works accordingly. In this perspective, if full Guile 2 support is not implemented by the deadline, it just waits for another release. But that's just my opinion.
>
> From my experience in the LLVM project, there is no such thing as
> "naturally stabilizing code". Either you create a branch and pick fixes
> or you have a strict policy that allows only fixes to master before
> branching. That's basically the model GCC is using, and I don't think
> it fits the community.

I was not meaning that the code base would naturally stabilize but instead that developers would be able to plan what features they can implement in the time that runs from now to branching. Predictability makes people happy.

We're having, in fact, similar views. You say that we need to stabilize the code base through branching, which I  entirely agree with -- except that right now is not the right time. What I'm trying to convey here is that postponing decisions on the ground of them being controversial is damaging to team members' morale.

To me, for the above-mentioned reasons, settling a date for branching 2.22 amounts to scheduling the 2.22 release, which is why I think we should explicitely discuss that schedule, instead of making short-term decisions that only have consensus because the consequences weren't discussed, with no longer term perspectives. The contrary would let the community split into small groups of like-minded persons that avoid each other because don't want to go the trouble of convincing each other. Given that you're ready to endorse the release manager role and responsibility, I no longer see any blocker to scheduling 2.20, except getting agreement about the schedule.

So we better start arguing about the schedule.

Cheers,
Jean
Reply | Threaded
Open this post in threaded view
|

Re: branching stable/2.22?

Jonas Hahnfeld
Am Dienstag, den 25.08.2020, 12:06 +0200 schrieb Jean Abou Samra:

> > Le 25 août 2020 à 08:30, Jonas Hahnfeld <[hidden email]> a écrit :
> >
> > Am Montag, den 24.08.2020, 22:10 +0200 schrieb Jean Abou Samra:
> > > > > > > As sort of a shot in the dark, how about planning the 2.22 release for May 2021, for example?
> > > > > >
> > > > > > Do you mean branching stable/2.22 or releasing 2.22.0 in May 2021? In
> > > > > > my understanding, the past process includes the release of beta
> > > > > > versions from the branch. That makes it close to impossible to predict
> > > > > > the final date of the stable version, and that's not the purpose of
> > > > > > this thread.
> > > > >
> > > > > I mean releasing 2.22.0 in May 2021. This is not about predictions, but objectives. I think that instead of planning each small step on the fly with no idea how the future looks like, we should settle an expected date for the release and plan backwards accordingly. Sure, there could be critical bugs that delay the actual release, but setting the expectation enables more resources to focus on the release by the time it is due to happen. In my opinion, this is the way we can avoid things like the 2.14 release that is documented in the CG.
> > > > >
> > > > > So, an expected release in May 2021 would mean branching release/2.20 around January (?) and beta releases at a monthly cadence until the release is out.
> > > > >
> > > > > I'm curious about what others think of that. In fact it looks like you already proposed something along these lines:
> > > > > https://www.mail-archive.com/lilypond-devel@.../msg72997.html
> > > > >
> > > >
> > > > And it didn't get much support. Which is why I don't see what's
> > > > different today. Asking what it would take to branch is really the only
> > > > sensible thing I think we could possibly agree on.
> > >
> > > As I see it, you're asking something nobody, apparently, can give you. We need to create the process instead of finding it out: what do you think it should take to branch?
> >
> > For me, creating the branch is nothing more than saying "feature
> > development is over and the current set is worth making stable". Which
> > I'm arguing is already there with Python 3 and the possibility to use
> > Guile 2. See my very initial message.
>
> At the same time, you're saying that branching is not going to happen next week. Please elaborate on your mind: when should that happen?
After below points have happened and after gathering agreement that
there are no open blockers to branching. IMO that would be something
fundamentally broken which can be expected to hit every user. AFAICT
that's not the case. Other problems can be addressed by picking fixes
into the branch.
(It probably makes sense to branch right after making some future
unstable release, which implies that GUB is mostly happy, but that's
some minor detail I would say.)

> > On the administrative side, I think
> > * there should be another reformatting for all C++ and Scheme
> > * docs should be updated, including authors.itexi
> > Everything else can and should be picked as needed.
>
> [...]
>
> We're having, in fact, similar views. You say that we need to stabilize the code base through branching, which I  entirely agree with -- except that right now is not the right time.

So what objective function would you use to set an agreeable date? Just
time, January 2021 being a shot in the dark?

> What I'm trying to convey here is that postponing decisions on the ground of them being controversial is damaging to team members' morale.
>
> To me, for the above-mentioned reasons, settling a date for branching 2.22 amounts to scheduling the 2.22 release, which is why I think we should explicitely discuss that schedule, instead of making short-term decisions that only have consensus because the consequences weren't discussed, with no longer term perspectives. The contrary would let the community split into small groups of like-minded persons that avoid each other because don't want to go the trouble of convincing each other. Given that you're ready to endorse the release manager role and responsibility, I no longer see any blocker to scheduling 2.20, except getting agreement about the schedule.
>
> So we better start arguing about the schedule.
>
> Cheers,
> Jean
>

signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: branching stable/2.22?

Jonas Hahnfeld
Am Dienstag, den 25.08.2020, 14:43 +0200 schrieb Jean Abou Samra:

> > Le 25 août 2020 à 12:29, Jonas Hahnfeld <[hidden email]> a écrit :
> >
> > Am Dienstag, den 25.08.2020, 12:06 +0200 schrieb Jean Abou Samra:
> > > > Le 25 août 2020 à 08:30, Jonas Hahnfeld <[hidden email]> a écrit :
> > > > For me, creating the branch is nothing more than saying "feature
> > > > development is over and the current set is worth making stable". Which
> > > > I'm arguing is already there with Python 3 and the possibility to use
> > > > Guile 2. See my very initial message.
> > >
> > > At the same time, you're saying that branching is not going to happen next week. Please elaborate on your mind: when should that happen?
> >
> > After below points have happened and after gathering agreement that
> > there are no open blockers to branching. IMO that would be something
> > fundamentally broken which can be expected to hit every user. AFAICT
> > that's not the case. Other problems can be addressed by picking fixes
> > into the branch.
>
> That can happen in a week, can't it?
It could, but discussion takes time. Only because I don't see open
blockers doesn't mean it's true and others agree right away, as
evidenced here.

> I can't follow your mind anymore. You previously agreed with David that the code base was in too much of a destabilized state for branching soon.

I said "in the next week" (note the singular form) which is basically
proven by this discussion. "soon" for me would be anything within the
next month.

> We're talking about bugs that we don't yet know but could pop up in the months to come given this state.
>
> > (It probably makes sense to branch right after making some future
> > unstable release, which implies that GUB is mostly happy, but that's
> > some minor detail I would say.)
> >
> > > > On the administrative side, I think
> > > > * there should be another reformatting for all C++ and Scheme
> > > > * docs should be updated, including authors.itexi
> > > > Everything else can and should be picked as needed.
> > >
> > > [...]
> > >
> > > We're having, in fact, similar views. You say that we need to stabilize the code base through branching, which I  entirely agree with -- except that right now is not the right time.
> >
> > So what objective function would you use to set an agreeable date? Just
> > time,
>
> Yes, that is basically the idea. I think schedules help people work together even if later deviated from.
>
> > January 2021 being a shot in the dark?
>
> Do speak up about what you would consider a more reasonable time.
I know I'll regret it because I still don't know what objective
criteria others have, but as you really insist on a statement:
in the week of 14th of September (this year, 2020, just to be clear)
or put differently: right after 2.21.7, from that very tag
As said before, I wouldn't make any commitment on the date of the final
release.

Now go off tearing me apart.

Jonas

> Also, I value the actual date less than I value agreement on the date.
>
> Best regards.
> Jean

signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: branching stable/2.22?

Carl Sorensen
On Tue, Aug 25, 2020 at 7:13 AM Jonas Hahnfeld <[hidden email]> wrote:

>
> I know I'll regret it because I still don't know what objective
> criteria others have, but as you really insist on a statement:
> in the week of 14th of September (this year, 2020, just to be clear)
> or put differently: right after 2.21.7, from that very tag
> As said before, I wouldn't make any commitment on the date of the final
> release.
>
>
>
Rather than a date, I'd prefer a criterion.

Once we have an unstable release with the build system in good shape (all
the auxiliary scripts work well, the website builds correctly, the MacOS
build is functional at least on MacPorts), I'd be in favor of creating a
pre-release candidate (which in the past has *not* been a stable branch,
but a big bump in the version number; the CG refers to this as an alpha)
and implementing the Build-frozen state listed in the CG.

After we've had some time to get the alpha release tested, we announce a
beta release and branch stable.

I concur with the idea that a properly functioning full conversion to
Python 3 and workable (though not required) Guile 2 constitutes sufficient
change for the next stable version.  No other features are needed.

I recommend against creating a stable branch too soon.  In my experience,
we get much more testing on the unstable branch than on the stable branch.
People are likely to use the most recent unstable release as a matter of
course; they are much less likely to go check out the most recent stable
(but not yet released) branch.

THanks,

Carl
Reply | Threaded
Open this post in threaded view
|

Re: branching stable/2.22?

Jean ABOU SAMRA
In reply to this post by Jonas Hahnfeld

> Now go off tearing me apart.

That one was a blow in my stomach.(In fact, I was about to write something similar in my last message.)

Taking stock on the thread, my reactions were not appropriate, especially the fourth one which is rubbish. I should have anticipated that personal issues wouldn't help either.

Jonas and other readers, I sincerely apologize for the harm caused.

As I currently appear unable to express my views in an actionable manner that is not offensive or annoying to others, I am going to step down from this discussion. I simply shouldn't have joined it in the first place.

Cheers,
Jean

Reply | Threaded
Open this post in threaded view
|

Re: branching stable/2.22?

Dan Eble
In reply to this post by Carl Sorensen
On Aug 25, 2020, at 09:51, Carl Sorensen <[hidden email]> wrote:

>
> Once we have an unstable release with the build system in good shape (all
> the auxiliary scripts work well, the website builds correctly, the MacOS
> build is functional at least on MacPorts), I'd be in favor of creating a
> pre-release candidate (which in the past has *not* been a stable branch,
> but a big bump in the version number; the CG refers to this as an alpha)
> and implementing the Build-frozen state listed in the CG.
>
> After we've had some time to get the alpha release tested, we announce a
> beta release and branch stable.
>
> I concur with the idea that a properly functioning full conversion to
> Python 3 and workable (though not required) Guile 2 constitutes sufficient
> change for the next stable version.  No other features are needed.
>
> I recommend against creating a stable branch too soon.  In my experience,
> we get much more testing on the unstable branch than on the stable branch.
> People are likely to use the most recent unstable release as a matter of
> course; they are much less likely to go check out the most recent stable
> (but not yet released) branch.

I could get behind all of this.

Dan


Reply | Threaded
Open this post in threaded view
|

Re: branching stable/2.22?

Kevin Barry
In reply to this post by Carl Sorensen
> I concur with the idea that a properly functioning full conversion to
> Python 3 and workable (though not required) Guile 2 constitutes sufficient
> change for the next stable version.  No other features are needed.

I agree with this.

Kevin

Reply | Threaded
Open this post in threaded view
|

Re: branching stable/2.22?

Han-Wen Nienhuys-3
In reply to this post by Jonas Hahnfeld
On Tue, Aug 25, 2020 at 8:31 AM Jonas Hahnfeld <[hidden email]> wrote:
> I don't understand why you would want to backport features? IMO that's
> got nothing to do with how far the stable branch diverges.
>
> > Whatever the option, we will need people to manage the release (yes, I could possibly help next summer ... though I'm afraid I'd be NOT_SMART).
>
> If it's just missing people to do the work, I obviously volunteer and
> am willing to cherry-pick fixes from master as needed.

I am happy to help with fixing bugs for the stable release.

> From my experience in the LLVM project, there is no such thing as
> "naturally stabilizing code". Either you create a branch and pick fixes
> or you have a strict policy that allows only fixes to master before
> branching.

>  That's basically the model GCC is using, and I don't think
> it fits the community.

I agree that we have to do something, either branching and
cherry-picking, or holding off on work that destabilizes the
situation.

I don't understand why you think the GCC model doesn't fit, though?

I think the branching model has the problem that someone has to do the
work of backporting, which is probably less fun to do. If in parallel
cowboys like me keep submitting experimental things to master,
backporting the fixes is made all the more difficult.

I think the stabilization effort could be a joint effort by the entire
dev team, by agreeing with the team to hold off on new features and
invasive changes for a period of time (say, 1 to 2 months).

BTW- aside from GUILE 2 and Python 3 support, I think users will also
be happy with the speedups that I've been working on in this cycle.

--
Han-Wen Nienhuys - [hidden email] - http://www.xs4all.nl/~hanwen

Reply | Threaded
Open this post in threaded view
|

Re: branching stable/2.22?

Jonas Hahnfeld
In reply to this post by Carl Sorensen
Am Dienstag, den 25.08.2020, 07:51 -0600 schrieb Carl Sorensen:

> On Tue, Aug 25, 2020 at 7:13 AM Jonas Hahnfeld <[hidden email]> wrote:
> > I know I'll regret it because I still don't know what objective
> > criteria others have, but as you really insist on a statement:
> > in the week of 14th of September (this year, 2020, just to be clear)
> > or put differently: right after 2.21.7, from that very tag
> > As said before, I wouldn't make any commitment on the date of the final
> > release.
>
> Rather than a date, I'd prefer a criterion.
>
> Once we have an unstable release with the build system in good shape (all the auxiliary scripts work well, the website builds correctly, the MacOS build is functional at least on MacPorts), I'd be in favor of creating a pre-release candidate (which in the past has *not* been a stable branch, but a big bump in the version number; the CG refers to this as an alpha) and implementing the Build-frozen state listed in the CG.
Are you sure this is what happened in the past? I mean I didn't
participate, so all I can rely on is git and the mailing list archives.

For 2.18, I see
 - release/2.17.29-1 tagged on 20 October 2013
 -
https://lists.gnu.org/archive/html/lilypond-devel/2013-10/msg00189.html
 on 22 October 2013
 - release/2.17.95-1 tagged on 04 November 2013
 -
https://lists.gnu.org/archive/html/lilypond-devel/2013-11/msg00077.html
 on 05 November 2013
As far as I understand, stable/2.18 was created before 2.17.95 (which
was incorrectly released from master, see above). The merge base
between current master and stable/2.18 points to April 2014 which is
probably due to a partial merge? Certainly after 2.18.0 had been
released in December 2013.

For 2.20, I see
 - considerations in December 2016:
https://lists.gnu.org/archive/html/lilypond-devel/2016-12/msg00348.html
 -
https://lists.gnu.org/archive/html/lilypond-devel/2017-06/msg00030.html
 on 06 June 2017
 - release/2.19.65-1 tagged on 06 August 2017
 - merge-base HEAD stable/2.20: 0712559601, committed 08 August 2017
and merged into stable/2.20 on 16 August 2017
 -
https://lists.gnu.org/archive/html/lilypond-devel/2017-08/msg00068.html
 on 16 August 2017
 - release/2.19.80-1 tagged on 16 October 2017

This looks like the release candidates (coinciding with the big bump)
have (or should have been) released from the stable/ branches?

That said, I'm not opposed to doing a pre-release candidate from master
and git history tells little about possible freezes. Just trying to
understand what might have worked in the past...

> After we've had some time to get the alpha release tested, we announce a beta release and branch stable.
>
> I concur with the idea that a properly functioning full conversion to Python 3 and workable (though not required) Guile 2 constitutes sufficient change for the next stable version.  No other features are needed.
>
> I recommend against creating a stable branch too soon.  In my experience, we get much more testing on the unstable branch than on the stable branch.  People are likely to use the most recent unstable release as a matter of course; they are much less likely to go check out the most recent stable (but not yet released) branch.
>
> THanks,
>
> Carl
>
>

signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: branching stable/2.22?

Jonas Hahnfeld
In reply to this post by Han-Wen Nienhuys-3
Am Dienstag, den 25.08.2020, 22:56 +0200 schrieb Han-Wen Nienhuys:

> On Tue, Aug 25, 2020 at 8:31 AM Jonas Hahnfeld <[hidden email]> wrote:
> > I don't understand why you would want to backport features? IMO that's
> > got nothing to do with how far the stable branch diverges.
> >
> > > Whatever the option, we will need people to manage the release (yes, I could possibly help next summer ... though I'm afraid I'd be NOT_SMART).
> >
> > If it's just missing people to do the work, I obviously volunteer and
> > am willing to cherry-pick fixes from master as needed.
>
> I am happy to help with fixing bugs for the stable release.
>
> > From my experience in the LLVM project, there is no such thing as
> > "naturally stabilizing code". Either you create a branch and pick fixes
> > or you have a strict policy that allows only fixes to master before
> > branching.
> >  That's basically the model GCC is using, and I don't think
> > it fits the community.
>
> I agree that we have to do something, either branching and
> cherry-picking, or holding off on work that destabilizes the
> situation.
>
> I don't understand why you think the GCC model doesn't fit, though?
If followed strictly, it means no feature commits in master during the
freeze which leaves two options: Either features build up in private
and it'll be serious work to integrate them after the freeze ends, or
they're not developed at all which is also bad.

> I think the branching model has the problem that someone has to do the
> work of backporting, which is probably less fun to do. If in parallel
> cowboys like me keep submitting experimental things to master,
> backporting the fixes is made all the more difficult.

Right, that is the trade-off. So really no easy solution to the
problem.

> I think the stabilization effort could be a joint effort by the entire
> dev team, by agreeing with the team to hold off on new features and
> invasive changes for a period of time (say, 1 to 2 months).

My feeling is that we should prioritize on bug fixes, but not actively
block the development of features.

Jonas

> BTW- aside from GUILE 2 and Python 3 support, I think users will also
> be happy with the speedups that I've been working on in this cycle.

signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: branching stable/2.22?

Han-Wen Nienhuys-3
On Tue, Aug 25, 2020 at 11:17 PM Jonas Hahnfeld <[hidden email]> wrote:

> > I think the stabilization effort could be a joint effort by the entire
> > dev team, by agreeing with the team to hold off on new features and
> > invasive changes for a period of time (say, 1 to 2 months).
>
> My feeling is that we should prioritize on bug fixes, but not actively
> block the development of features.

Right - but we should refrain from changes that will make the
backporting process more difficult, so it is still a freeze of sorts.

> Jonas
>
> > BTW- aside from GUILE 2 and Python 3 support, I think users will also
> > be happy with the speedups that I've been working on in this cycle.



--
Han-Wen Nienhuys - [hidden email] - http://www.xs4all.nl/~hanwen

Reply | Threaded
Open this post in threaded view
|

Re: branching stable/2.22?

David Kastrup
In reply to this post by Jonas Hahnfeld
Jonas Hahnfeld <[hidden email]> writes:

> Am Dienstag, den 25.08.2020, 07:51 -0600 schrieb Carl Sorensen:
>> On Tue, Aug 25, 2020 at 7:13 AM Jonas Hahnfeld <[hidden email]> wrote:
>> > I know I'll regret it because I still don't know what objective
>> > criteria others have, but as you really insist on a statement:
>> > in the week of 14th of September (this year, 2020, just to be clear)
>> > or put differently: right after 2.21.7, from that very tag
>> > As said before, I wouldn't make any commitment on the date of the final
>> > release.
>>
>> Rather than a date, I'd prefer a criterion.
>>
>> Once we have an unstable release with the build system in good shape
>> (all the auxiliary scripts work well, the website builds correctly,
>> the MacOS build is functional at least on MacPorts), I'd be in favor
>> of creating a pre-release candidate (which in the past has *not*
>> been a stable branch, but a big bump in the version number; the CG
>> refers to this as an alpha) and implementing the Build-frozen state
>> listed in the CG.
>
> Are you sure this is what happened in the past? I mean I didn't
> participate, so all I can rely on is git and the mailing list archives.
>
> For 2.18, I see
>  - release/2.17.29-1 tagged on 20 October 2013
>  -
> https://lists.gnu.org/archive/html/lilypond-devel/2013-10/msg00189.html
>  on 22 October 2013
>  - release/2.17.95-1 tagged on 04 November 2013
>  -
> https://lists.gnu.org/archive/html/lilypond-devel/2013-11/msg00077.html
>  on 05 November 2013
> As far as I understand, stable/2.18 was created before 2.17.95 (which
> was incorrectly released from master, see above). The merge base
> between current master and stable/2.18 points to April 2014 which is
> probably due to a partial merge? Certainly after 2.18.0 had been
> released in December 2013.
>
> For 2.20, I see
>  - considerations in December 2016:
> https://lists.gnu.org/archive/html/lilypond-devel/2016-12/msg00348.html
>  -
> https://lists.gnu.org/archive/html/lilypond-devel/2017-06/msg00030.html
>  on 06 June 2017
>  - release/2.19.65-1 tagged on 06 August 2017
>  - merge-base HEAD stable/2.20: 0712559601, committed 08 August 2017
> and merged into stable/2.20 on 16 August 2017
>  -
> https://lists.gnu.org/archive/html/lilypond-devel/2017-08/msg00068.html
>  on 16 August 2017
>  - release/2.19.80-1 tagged on 16 October 2017
>
> This looks like the release candidates (coinciding with the big bump)
> have (or should have been) released from the stable/ branches?

Yes.

> That said, I'm not opposed to doing a pre-release candidate from
> master and git history tells little about possible freezes.

The freezes mostly consisted in telling people to _please_ stop making
life harder for the stable release manager, with moderate success.
However, as release stoppers were accumulating and the release process
got stuck up, the "don't do development" appeals stopped making a whole
lot of sense.

> Just trying to understand what might have worked in the past...

I sure wished I had been able to get an understanding about what might
have worked...

--
David Kastrup

123