Patchy email

classic Classic list List threaded Threaded
25 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Re: Patchy email

Knut Petersen
On 29.07.19 17:55, David Kastrup wrote:
> Well, it is rather obvious that you should not be seeing failure with
> unchanged master (assuming that you have some reasonably current Python
> on your system) and I should not be seeing failure with your patch
> (assuming that it tests out on your system: the symptoms are just too
> clear to seem dependent on Python version).  So it's likely that some
> preinstalled part (assuming both of us worked from clean trees) creeps
> into both our operations.  That would be more likely than not a
> preexisting problem not caused by your patch but we would still better
> fix it before it causes more headaches.

In the chapter "Regtest comparison" of our contributor's guide we can read:

    Run |make|with current git master without any of your changes.
    Before making changes to the code, establish a baseline for the comparison by going to the ‘$LILYPOND_GIT/build/’ directory and running:

        make test-baseline

Stracing shows that after 'make all' it is also necessary to execute 'make install' prior to 'make test-baseline' or 'make test-clean'. If 'make install' is omitted, the result depends both on files from the baseline version and on files that belong to the version of lilypond that was last installed
with the same prefix.

David, please rerun the test of my patch against origin/master on your system. On my system the process succeeds now:

       exec*git clean -dfx *in /home/knut/sources/lily ... succeeded after 2 seconds
       exec *git checkout origin/master* in /home/knut/sources/lily ... succeeded after 0 seconds
    Building LILYPOND origin/master (34a114b835b042537f598620ce41912f51dd4fa4) ...
       exec *./autogen.sh --noconfigure* in /home/knut/sources/lily ... succeeded after 0 seconds
       exec *mkdir -p build* in /home/knut/sources/lily ... succeeded after 0 seconds
       exec *cd /home/knut/sources/lily/build* in /home/knut/sources/lily ... succeeded after 0 seconds
       exec *../configure --prefix=/home/knut/sources/lilybuilt --with-urwotf-dir=/home/knut/sources/urw-core35-fonts* in /home/knut/sources/lily/build ... succeeded after 7 seconds
       exec *make -j 11 CPU_COUNT=11 all* in /home/knut/sources/lily/build ... succeeded after 137 seconds
       exec *make -j 11 CPU_COUNT=11 install* in /home/knut/sources/lily/build ... succeeded after 2 seconds
       exec *make -j 11 CPU_COUNT=11 test-baseline* in /home/knut/sources/lily/build ... succeeded after 175 seconds
       exec .*/make-regtest-pngs.sh -j9 -o* in /home/knut/sources/lily/scripts/auxiliar ... succeeded after 71 seconds
       exec *g**it checkout FIX_MUSICXML2LY* in /home/knut/sources/lily ... succeeded after 0 seconds
    Building LILYPOND FIX_MUSICXML2LY (9aae63f0e0adf31eeffe6e0cb11c0e5a22c9eae5) ...
       exec .*/autogen.sh --noconfigure* in /home/knut/sources/lily ... succeeded after 0 seconds
       exec *mkdir -p build *in /home/knut/sources/lily ... succeeded after 0 seconds
       exec *cd /home/knut/sources/lily/build* in /home/knut/sources/lily ... succeeded after 0 seconds
       exec *../configure --prefix=/home/knut/sources/lilybuilt --with-urwotf-dir=/home/knut/sources/urw-core35-fonts* in /home/knut/sources/lily/build ... succeeded after 7 seconds
       exec *make -j 11 CPU_COUNT=11 all* in /home/knut/sources/lily/build ... succeeded after 8 seconds
       exec *make -j 11 CPU_COUNT=11 instal**l* in /home/knut/sources/lily/build ... succeeded after 5 seconds
       exec *make -j 11 CPU_COUNT=11 test-clean* in /home/knut/sources/lily/build ... succeeded after 0 seconds
       exec *make -j 11 CPU_COUNT=11 check* in /home/knut/sources/lily/build ... succeeded after 188 seconds
       exec *./make-regtest-pngs.sh -j9 -n* in /home/knut/sources/lily/scripts/auxiliar ... succeeded after 606 seconds
    Total regression test time: 1208 seconds

If the process also succeeds on your system we would know that patchy needs a fix.

Knut



_______________________________________________
lilypond-devel mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/lilypond-devel
dak
Reply | Threaded
Open this post in threaded view
|

Re: Patchy email

dak
Knut Petersen <[hidden email]> writes:

> On 29.07.19 17:55, David Kastrup wrote:
>> Well, it is rather obvious that you should not be seeing failure with
>> unchanged master (assuming that you have some reasonably current Python
>> on your system) and I should not be seeing failure with your patch
>> (assuming that it tests out on your system: the symptoms are just too
>> clear to seem dependent on Python version).  So it's likely that some
>> preinstalled part (assuming both of us worked from clean trees) creeps
>> into both our operations.  That would be more likely than not a
>> preexisting problem not caused by your patch but we would still better
>> fix it before it causes more headaches.
>
> In the chapter "Regtest comparison" of our contributor's guide we can read:
>
>    Run |make|with current git master without any of your changes.
>    Before making changes to the code, establish a baseline for the comparison by going to the ‘$LILYPOND_GIT/build/’ directory and running:
>
>        make test-baseline
>
> Stracing shows that after 'make all' it is also necessary to execute
> 'make install' prior to 'make test-baseline' or 'make test-clean'.

That's insane.  Having to install LilyPond before being able to test it
makes no sense.

> If 'make install' is omitted, the result depends both on files from
> the baseline version and on files that belong to the version of
> lilypond that was last installed with the same prefix.

Then we need to fix that.

> David, please rerun the test of my patch against origin/master on your
> system. On my system the process succeeds now:

> If the process also succeeds on your system we would know that patchy
> needs a fix.

No, that make test needs a fix.  make test is supposed to test the
version in the work directory, not some weird amalgamate between system
installed version and work directory version.

Any idea what would be required here?

--
David Kastrup

_______________________________________________
lilypond-devel mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/lilypond-devel
Reply | Threaded
Open this post in threaded view
|

Re: Patchy email

Knut Petersen
On 30.07.19 01:39, David Kastrup wrote:

>> Stracing shows that after 'make all' it is also necessary to execute
>> 'make install' prior to 'make test-baseline' or 'make test-clean'.
> That's insane.  Having to install LilyPond before being able to test it
> makes no sense.

During regression testing we use "musicxml2ly --out=- -". A strace log of such a run is in TC.18804.

    grep ^openat TC.18804 | grep -v ENOENT | grep /home/knut/sources | \
    sort | sed -e 's|[^"]*\"/home/knut/sources/lilybuilt|$PREFIX|'  \
    -e 's|[^"]*\"/home/knut/sources/lily/build|$BUILDDIR|' -e 's|\"[^"]*||' | uniq

results in

    $BUILDDIR/scripts/out/musicxml2ly
    $PREFIX/share/lilypond/2.21.0/python/lilylib.pyc
    $PREFIX/share/lilypond/2.21.0/python/lilylib.py
    $PREFIX/share/lilypond/2.21.0/python/musicexp.pyc
    $PREFIX/share/lilypond/2.21.0/python/musicexp.py
    $PREFIX/share/lilypond/2.21.0/python/musicxml2ly_conversion.pyc
    $PREFIX/share/lilypond/2.21.0/python/musicxml2ly_conversion.py
    $PREFIX/share/lilypond/2.21.0/python/musicxml.pyc
    $PREFIX/share/lilypond/2.21.0/python/musicxml.py
    $PREFIX/share/lilypond/2.21.0/python/rational.pyc
    $PREFIX/share/lilypond/2.21.0/python/rational.py
    $PREFIX/share/lilypond/2.21.0/python/utilities.pyc
    $PREFIX/share/lilypond/2.21.0/python/utilities.py

So only the musicxml2ly file is from the version to test, all the other python files are old.

>> If 'make install' is omitted, the result depends both on files from
>> the baseline version and on files that belong to the version of
>> lilypond that was last installed with the same prefix.
> Then we need to fix that.

Yes.


>> If the process also succeeds on your system we would know that patchy
>> needs a fix.
> No, that make test needs a fix.  make test is supposed to test the
> version in the work directory, not some weird amalgamate between system
> installed version and work directory version.
>
> Any idea what would be required here?
The fastest 'fix' would be to compile with a prefix pointing into a subdirectory
of the build directory and installing into that prior to 'make test-baseline'.
But that is an ugly workaround. I'll have a look at the makefiles this night.

Knut
_______________________________________________
lilypond-devel mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/lilypond-devel
Reply | Threaded
Open this post in threaded view
|

Re: Patchy email

Knut Petersen
In reply to this post by dak
On 30.07.19 01:39, David Kastrup wrote:
>
> No, that make test needs a fix.  make test is supposed to test the
> version in the work directory, not some weird amalgamate between system
> installed version and work directory version.
>
> Any idea what would be required here?

Yes.

https://codereview.appspot.com/560840043/
https://sourceforge.net/p/testlilyissues/issues/5544/

Knut

_______________________________________________
lilypond-devel mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/lilypond-devel
dak
Reply | Threaded
Open this post in threaded view
|

Re: Patchy email

dak
In reply to this post by dak
David Kastrup <[hidden email]> writes:

> Knut Petersen <[hidden email]> writes:
>
>> On 26.07.19 20:36, David Kastrup wrote:
>>>
>>>> Unless Knut is also running patchy now that he has commit access (and
>>>> perhaps didn't have a clean master?).
>>>>
>>>> (I don't want to cast aspersions, but it might be a genuine mistake if
>>>> it was Knut). Knut?
>>> I rather doubt that it was Knut since I mentioned the procedures to him
>>> right now and you said that you pushed the patch in question anyway (so
>>> it's very unlikely that he'd run the patchy subsequently responsible for
>>> promoting it to master).
>> I never used patchy. According to my memories and according to my logs
>> pushing the commits to staging was done right and successful.
>>
>> But yes, there is a local branch master on my system. I never thought that
>> there is a clone of the lilypond repository without a local master as
>>
>>    git clone git://git.sv.gnu.org/lilypond.git
>>
>>
>> clones the repository and checks out 'master'. Sorry for the mess.
>>
>> It's clear that 7583351fbf2f08a4d1f3f0b075fe8b691adc0b95 needs
>> to be fixed.
>>
>> What happened to the musicxml2ly patch? It vanished from staging.
>> Shall I push it again?
>
> It didn't pass patchy testing on my computer with failures in the
> musicxml files.  So it appears to have a problem with, uh, Python
> 2.7.16+ (according to python --version on my current Ubuntu).  If you
> need more pointers, I can try getting useful logs.

20 weeks delay for a mail to reach the list must be some sort of
record.  Good thing that I had Knut in Cc.

--
David Kastrup

12