Please test gub

classic Classic list List threaded Threaded
105 messages Options
1234 ... 6
Reply | Threaded
Open this post in threaded view
|

Please test gub

Knut Petersen
Hi everybody!

I created a branch in my gub repository  that contains https://github.com/gperciva plus pull requests 53-60. Therefore it is pretty easy to test if that version of gub succeeds to build current lilypond master on your machine.

All you need to do is to execute the following commands:

    git clone https://github.com/knupero/gub.git -b DevelHead
    cd gub
    mkdir regtests
    cd regtests
    wget http://lilypond.org/downloads/binaries/test-output/lilypond-2.19.82-1.test-output.tar.bz2
    touch ignore
    cd ..
    time make lilypond

Even on a fast computer 'make lilypond' will take some hours to complete.

If downloading of a source archive fails because of some network problem restart 'make lilypond'.

You'll need some free disk space ... about 20 GB is a minimum.

Please report success / fails with os / version / cpu info.

Knut

_______________________________________________
lilypond-devel mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/lilypond-devel
Reply | Threaded
Open this post in threaded view
|

Re: Please test gub

Federico Bruni-2


Il giorno lun 28 gen 2019 alle 13:53, Knut Petersen
<[hidden email]> ha scritto:

> Hi everybody!
>
> I created a branch in my gub repository  that contains
> <https://github.com/gperciva> plus pull requests 53-60. Therefore it
> is pretty easy to test if that version of gub succeeds to build
> current lilypond master on your machine.
>
> All you need to do is to execute the following commands:
>
>    git clone <https://github.com/knupero/gub.git> -b DevelHead
>    cd gub
>    mkdir regtests
>    cd regtests
>    wget
> <http://lilypond.org/downloads/binaries/test-output/lilypond-2.19.82-1.test-output.tar.bz2>
>    touch ignore
>    cd ..
>    time make lilypond
>
> Even on a fast computer 'make lilypond' will take some hours to
> complete.
>
> If downloading of a source archive fails because of some network
> problem restart 'make lilypond'.
>
> You'll need some free disk space ... about 20 GB is a minimum.
>
> Please report success / fails with os / version / cpu info.
>

Hi Knut

I've already tested this setup on Debian 9, Ubuntu 16.04, Fedora 29
(with gcc-7) and reported the errors.

There's anything new in lilypond-git which should encourage me to try
again?
Or should I try another distro and see if I'm luckier?




_______________________________________________
lilypond-devel mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/lilypond-devel
Reply | Threaded
Open this post in threaded view
|

Re: Please test gub

Knut Petersen
Hi Federico!

> I've already tested this setup on Debian 9, Ubuntu 16.04, Fedora 29 (with gcc-7) and reported the errors.

Yes, an additional commit (added yesterday) in pull request #58 should solve the problem with a failing gs ....

Knut


_______________________________________________
lilypond-devel mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/lilypond-devel
Reply | Threaded
Open this post in threaded view
|

Re: Please test gub

pkx166h-3
In reply to this post by Knut Petersen
Hello,

On 28/01/2019 12:53, Knut Petersen wrote:

> Hi everybody!
>
> I created a branch in my gub repository  that contains
> https://github.com/gperciva plus pull requests 53-60. Therefore it is
> pretty easy to test if that version of gub succeeds to build current
> lilypond master on your machine.
>
> All you need to do is to execute the following commands:
>
>    git clone https://github.com/knupero/gub.git -b DevelHead
>    cd gub
>    mkdir regtests
>    cd regtests
>    wget
> http://lilypond.org/downloads/binaries/test-output/lilypond-2.19.82-1.test-output.tar.bz2
>    touch ignore
>    cd ..
>    time make lilypond
>
> Even on a fast computer 'make lilypond' will take some hours to complete.
>
> If downloading of a source archive fails because of some network
> problem restart 'make lilypond'.
>
> You'll need some free disk space ... about 20 GB is a minimum.
>
> Please report success / fails with os / version / cpu info.
>
> Knut
>

I'd like to help here if I can.

However at home I am on metered internet and it appears that after make
lilypond it starts to download files? Can you tell me approximately how
large the data that will be downloaded is - speed of download isn't an
issue, just the amount.

Also, I assume we can use the same make -jX CPU_COUNT=X parameters to
help speed up things?

Finally - in case I am being stupid, I don't follow the GUB threads in
detail - I obviously have all the reqs to build LP and make doc etc. So
apart from this git repo is there anything else I need to have installed?

Thanks

James


_______________________________________________
lilypond-devel mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/lilypond-devel
Reply | Threaded
Open this post in threaded view
|

Re: Please test gub

Knut Petersen
Hi James!
> I'd like to help here if I can.
>
Thanks


> However at home I am on metered internet and it appears that after make lilypond it starts to download files? Can you tell me approximately how large the data that will be downloaded is - speed of download isn't an issue, just the amount.

Gub + downloaded source files:  about 710 MB


> Also, I assume we can use the same make -jX CPU_COUNT=X parameters to help speed up things?

No. But gub itself (repeatedly executed by 'make lilypond') uses -jX to build packages whenever the package specification allows it. On my system (i7-4790K, 55/10 MBit network connection) 'make lilypond' takes about 185 minutes.


> Finally - in case I am being stupid, I don't follow the GUB threads in detail - I obviously have all the reqs to build LP and make doc etc. So apart from this git repo is there anything else I need to have installed?

Nothing unusual. But our python seems to be incompatible to gcc 8.x. gcc-8 might be installed as long as there is also a gcc-7 somewhere in PATH.


    Status of GUB (branch DevelHead of https://github.com/knupero/gub.git == https://github.com/gperciva/gub.git + pull requests 53-60)
    ===================================================================================================================================
    os: linux / distribution: openSuSE TumbleWeed / cpu: i7-4790K / tested by: Knut / status: ok / remarks: gcc-7 must be installed
    os: linux / distribution: ubuntu 14.04        / cpu: i7-4790K / tested by: Knut / status: ok / remarks: none
    os: linux / distribution: ubuntu 18.04        / cpu: i7-4790K / tested by: Knut / status: ok / remarks: none

Knut



_______________________________________________
lilypond-devel mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/lilypond-devel
Reply | Threaded
Open this post in threaded view
|

Re: Please test gub

Karlin High
In reply to this post by Knut Petersen
On 1/28/2019 6:53 AM, Knut Petersen wrote:
> Please report success / fails with os / version / cpu info.

I really like the simple instructions you posted, Knut. I wouldn't be
testing Gub without them. My setup doesn't like the
darwin-ppc::odcctools package for some reason. Mystified why it's
bringing in iPhone stuff. This same thing happened in 2 separate runs; I
had deleted the cloned Git repository and started over.

Windows 7 Pro 64-bit SP1, Intel Core i5-3450
VirtualBox 5.2.22r126460
VM with 1 CPU core, 4GB RAM, 64GB storage

karlin@vbox-ubuntu ~/knut-gub/gub (DevelHead=)$lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 18.04.1 LTS
Release: 18.04
Codename: bionic

karlin@vbox-ubuntu ~/knut-gub/gub (DevelHead=)$uname -a
Linux vbox-ubuntu 4.15.0-43-generic #46-Ubuntu SMP Thu Dec 6 14:45:28
UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

building package: darwin-ppc::odcctools
  *** Stage: download (odcctools, darwin-ppc)
  *** Stage: untar (odcctools, darwin-ppc)
Command barfed: tar -C
/home/karlin/knut-gub/gub/target/darwin-ppc/src/odcctools-278
--strip-component=1 -v -z -xf
/home/karlin/knut-gub/gub/downloads/odcctools/odcctools-iphone-dev-278.tar.gz

Tail of target/darwin-ppc/log/odcctools.log >>>>>>>>
     gzip: stdin: not in gzip format
     tar: Child returned status 1
     tar: Error is not recoverable: exiting now
     Command barfed: tar -C
/home/karlin/knut-gub/gub/target/darwin-ppc/src/odcctools-278
--strip-component=1 -v -z -xf
/home/karlin/knut-gub/gub/downloads/odcctools/odcctools-iphone-dev-278.tar.gz
<<<<<<<< Tail of target/darwin-ppc/log/odcctools.log

*** Failed target: darwin-ppc::odcctools
gub.make:63: recipe for target 'packages' failed
make[1]: *** [packages] Error 1
make[1]: Leaving directory '/home/karlin/knut-gub/gub'
GNUmakefile:26: recipe for target 'lilypond' failed
make: *** [lilypond] Error 2

real 76m18.221s
user 52m44.624s
sys 8m14.693s
--
Karlin High
Missouri, USA

_______________________________________________
lilypond-devel mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/lilypond-devel
Reply | Threaded
Open this post in threaded view
|

Re: Please test gub

Federico Bruni-2


Il giorno mar 29 gen 2019 alle 0:53, Karlin High <[hidden email]>
ha scritto:

>
>
> building package: darwin-ppc::odcctools
>  *** Stage: download (odcctools, darwin-ppc)
>  *** Stage: untar (odcctools, darwin-ppc)
> Command barfed: tar -C
> /home/karlin/knut-gub/gub/target/darwin-ppc/src/odcctools-278
> --strip-component=1 -v -z -xf
> /home/karlin/knut-gub/gub/downloads/odcctools/odcctools-iphone-dev-278.tar.gz
>
> Tail of target/darwin-ppc/log/odcctools.log >>>>>>>>
>     gzip: stdin: not in gzip format
>     tar: Child returned status 1
>     tar: Error is not recoverable: exiting now
>     Command barfed: tar -C
> /home/karlin/knut-gub/gub/target/darwin-ppc/src/odcctools-278
> --strip-component=1 -v -z -xf
> /home/karlin/knut-gub/gub/downloads/odcctools/odcctools-iphone-dev-278.tar.gz
> <<<<<<<< Tail of target/darwin-ppc/log/odcctools.log
>

The output in the terminal is often useless in GUB. You have to open
the log file mentioned there, in this case
target/darwin-ppc/log/odcctools.log
You may see that some binary is not found (you are missing a
dependency). But I'd be surprised if you did not have tar or gzip
installed.

More likely a download went wrong and the tar.gz file is not really a
tar.gz file. You can check with this command:

file
/home/karlin/knut-gub/gub/downloads/odcctools/odcctools-iphone-dev-278.tar.gz

If it's not a compressed file, remove it and relaunch 'make lilypond'.




_______________________________________________
lilypond-devel mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/lilypond-devel
Reply | Threaded
Open this post in threaded view
|

Re: Please test gub

Urs Liska-3
In reply to this post by Knut Petersen
Hi Knut,

thank you for working on this, and - like Karlin - I wouldn't have
tested without your straightforward recipe. But before going to bed I
logged off, then logged in again in a terminal-only session and started
the process.

Am 28.01.19 um 13:53 schrieb Knut Petersen:

> Hi everybody!
>
> I created a branch in my gub repository  that contains
> https://github.com/gperciva plus pull requests 53-60. Therefore it is
> pretty easy to test if that version of gub succeeds to build current
> lilypond master on your machine.
>
> All you need to do is to execute the following commands:
>
>    git clone https://github.com/knupero/gub.git -b DevelHead
>    cd gub
>    mkdir regtests
>    cd regtests
>    wget
> http://lilypond.org/downloads/binaries/test-output/lilypond-2.19.82-1.test-output.tar.bz2
>    touch ignore
>    cd ..
>    time make lilypond
>
> Even on a fast computer 'make lilypond' will take some hours to complete.
Seems the process completed after ~300 minutes ;-)
>
> If downloading of a source archive fails because of some network
> problem restart 'make lilypond'.
>
> You'll need some free disk space ... about 20 GB is a minimum.
>
> Please report success / fails with os / version / cpu info.
>

How do I know whether it/everything worked? All I can see is the last
bit of the terminal output, which gives me mixed signals. The overall
end doesn't report a failure, but the last "rule" shows some problems,
and I don't see how severe they are:

Two stages report things like "cross/gcc conflicts with cross/gcc-core"
and some follow-ups, it always reports that a non-core version of the
item is already installed and skips the request to install it. This
relates to several "gcc" and "glibc" items.

I see lots of log files which I of course can't check to figure out
everything's fine, and the 'target' directory contains about 17GB od files.

So what can I do to check whether make lilypond succeeded or failed?

Best
Urs


> Knut
>
> _______________________________________________
> lilypond-devel mailing list
> [hidden email]
> https://lists.gnu.org/mailman/listinfo/lilypond-devel

_______________________________________________
lilypond-devel mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/lilypond-devel
Reply | Threaded
Open this post in threaded view
|

Re: Please test gub

Urs Liska-3

Am 29.01.19 um 07:41 schrieb Urs Liska:
> The overall end doesn't report a failure, but the last "rule" shows
> some problems,


I should have mentioned this was the "nsis rule"


_______________________________________________
lilypond-devel mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/lilypond-devel
Reply | Threaded
Open this post in threaded view
|

Re: Please test gub

Knut Petersen
In reply to this post by Karlin High
On 29.01.19 00:53, Karlin High wrote:
> On 1/28/2019 6:53 AM, Knut Petersen wrote:
>> Please report success / fails with os / version / cpu info.
>
> I really like the simple instructions you posted, Knut. I wouldn't be testing Gub without them. My setup doesn't like the darwin-ppc::odcctools package for some reason. Mystified why it's bringing in iPhone stuff. This same thing happened in 2 separate runs; I had deleted the cloned Git repository
> and started over.
>
Thank you for testing.


> building package: darwin-ppc::odcctools
>  *** Stage: download (odcctools, darwin-ppc)
>  *** Stage: untar (odcctools, darwin-ppc)
> Command barfed: tar -C /home/karlin/knut-gub/gub/target/darwin-ppc/src/odcctools-278 --strip-component=1 -v -z -xf /home/karlin/knut-gub/gub/downloads/odcctools/odcctools-iphone-dev-278.tar.gz
>
> Tail of target/darwin-ppc/log/odcctools.log >>>>>>>>
>     gzip: stdin: not in gzip format
>     tar: Child returned status 1
>     tar: Error is not recoverable: exiting now
>     Command barfed: tar -C /home/karlin/knut-gub/gub/target/darwin-ppc/src/odcctools-278 --strip-component=1 -v -z -xf /home/karlin/knut-gub/gub/downloads/odcctools/odcctools-iphone-dev-278.tar.gz
> <<<<<<<< Tail of target/darwin-ppc/log/odcctools.log
>
> *** Failed target: darwin-ppc::odcctools
> gub.make:63: recipe for target 'packages' failed
> make[1]: *** [packages] Error 1
> make[1]: Leaving directory '/home/karlin/knut-gub/gub'
> GNUmakefile:26: recipe for target 'lilypond' failed
> make: *** [lilypond] Error 2
>
That means you were able to build all the tools necessary to start compilation of odcctools. But our own gzip fails to decompress the odcc*tar.gz. Weird.

Please verify that you got the correct source file. Executing

    md5sum downloads/odcctools/odcctools-iphone-dev-278.tar.gz

in  /home/karlin/knut-gub/gub should give that result:

    b067f6311e4c3d923e693dd280fab632 downloads/odcctools/odcctools-iphone-dev-278.tar.gz

If this is ok (it really should!) please execute the following commands:

    mkdir -p STRACE
    rm -f STRACE/*
    rm -f target/darwin-ppc/packages/odcctools*
    strace -v -f -ff -s 1024 -o STRACE/TP bin/gub darwin-ppc::odcctools
    grep -o '^exec[^]]*]' STRACE/*  |  grep '/tar"\|/gzip"'

the output of grep should be similar to

    STRACE/TP.7751:execve("/home/gub/gub/target/tools/root/usr/bin/tar", ["tar", "-C", "/home/gub/gub/target/darwin-ppc/src/odcctools-278", "--strip-component=1", "-v", "-z", "-xf", "/home/karlin/knut-gub/gub/downloads/odcctools/odcctools-iphone-dev-278.tar.gz"]
    STRACE/TP.7752:execve("/home/gub/gub/target/tools/root/usr/bin/gzip", ["gzip", "-d"]

Obviously filenames (STRACE/TP<number>) will differ as they indicate the ID of the processes.

Please send me those two files and target/darwin-ppc/log/odcctools.log.

Knut







_______________________________________________
lilypond-devel mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/lilypond-devel
Reply | Threaded
Open this post in threaded view
|

Re: Please test gub

Werner LEMBERG
In reply to this post by Urs Liska-3

>> The overall end doesn't report a failure, but the last "rule" shows
>> some problems,
>
> I should have mentioned this was the "nsis rule"

If you reach this point you can be rather sure that everything was
fine – at least for your build system, since it was used to run
lilypond's tests to generate diff images of the regression suite.

If we get more success reports, the resulting packages should be
uploaded so that other people not running gub can test them.
Developers can then have a look how to add support for 64bit binaries
on MacOS and Windows.  Especially the former is rather urgent...


    Werner
_______________________________________________
lilypond-devel mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/lilypond-devel
Reply | Threaded
Open this post in threaded view
|

Re: Please test gub

Urs Liska-3

Am 29.01.19 um 08:14 schrieb Werner LEMBERG:
>>> The overall end doesn't report a failure, but the last "rule" shows
>>> some problems,
>> I should have mentioned this was the "nsis rule"
> If you reach this point you can be rather sure that everything was
> fine – at least for your build system, since it was used to run
> lilypond's tests to generate diff images of the regression suite.


Great! I've now found that too in uploads/webtest


>
> If we get more success reports, the resulting packages should be
> uploaded so that other people not running gub can test them.
> Developers can then have a look how to add support for 64bit binaries
> on MacOS and Windows.  Especially the former is rather urgent...


I can upload my stuff, but what would that be? I assume the installer
files are in the uploads directory, and I would upload all the files
that look like installers, but not the documentation stuff, correct?

Urs

>
>
>      Werner

_______________________________________________
lilypond-devel mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/lilypond-devel
Reply | Threaded
Open this post in threaded view
|

Re: Please test gub

Werner LEMBERG
>> If we get more success reports, the resulting packages should be
>> uploaded so that other people not running gub can test them.
>> Developers can then have a look how to add support for 64bit
>> binaries on MacOS and Windows.  Especially the former is rather
>> urgent...
>
> I can upload my stuff, but what would that be?  I assume the
> installer files are in the uploads directory, and I would upload all
> the files that look like installers, but not the documentation
> stuff, correct?

Yes.  In other words, the relevant files would be
`*.{sh,exe,tar.bz2}' but perhaps omitting
`lilypond-2.21.0-1.test-output.tar.bz2'.


    Werner

_______________________________________________
lilypond-devel mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/lilypond-devel
Reply | Threaded
Open this post in threaded view
|

Re: Please test gub

Urs Liska-3

Am 29.01.19 um 08:32 schrieb Werner LEMBERG:

>>> If we get more success reports, the resulting packages should be
>>> uploaded so that other people not running gub can test them.
>>> Developers can then have a look how to add support for 64bit
>>> binaries on MacOS and Windows.  Especially the former is rather
>>> urgent...
>> I can upload my stuff, but what would that be?  I assume the
>> installer files are in the uploads directory, and I would upload all
>> the files that look like installers, but not the documentation
>> stuff, correct?
> Yes.  In other words, the relevant files would be
> `*.{sh,exe,tar.bz2}' but perhaps omitting
> `lilypond-2.21.0-1.test-output.tar.bz2'.
>
>
>      Werner


OK, the files will for some time be available from
https://cloud.ursliska.de/s/QPINwLqJNeVslCu

They have been compiled on Linux Mint 19.1, 4.15.0-43-generic #46-Ubuntu
SMP Thu Dec 6 14:45:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

I think this post on this thread is not sufficient to promote the files
for testing, so I suggest either you, Werner, or Knut keep this
information and at some point throw out a more formal and wide-spread
announcement asking for testing (probably better on lilypond-user than
lilypond-devel.

Best
Urs


_______________________________________________
lilypond-devel mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/lilypond-devel
Reply | Threaded
Open this post in threaded view
|

Re: Please test gub

Knut Petersen
In reply to this post by Urs Liska-3
Hi Urs!

> So what can I do to check whether make lilypond succeeded or failed?


The last lines of the terminal output will look like:

    make -f lilypond.make update-versions

    To upload, run:

         make lilypond-upload LILYPOND_BRANCH=master LILYPOND_REPO_URL=git://git.sv.gnu.org/lilypond.git

    **** nsis rule
    python bin/gub tools::nsis
    calculating dependencies
    Checking for iconv ... /usr/bin/iconv
    Checking for g++ ... /usr/bin/g++
    Checking for gcc ... /usr/bin/gcc
    must rebuild[tools]: system::gcc system::g++ system::iconv
      *** Stage: pkg_install (cross/gcc-core, linux-x86)
       cross/gcc conflicts with cross/gcc-core
         non-core cross/gcc already installed
           skipping request to install cross/gcc-core
       cross/gcc-doc conflicts with cross/gcc-core
         non-core cross/gcc already installed
           skipping request to install cross/gcc-core
       cross/gcc-runtime conflicts with cross/gcc-core
         non-core cross/gcc already installed
           skipping request to install cross/gcc-core

      *** Stage: pkg_install (glibc-core, linux-x86)
       glibc conflicts with glibc-core
         non-core glibc already installed
           skipping request to install glibc-core
       glibc-doc conflicts with glibc-core
         non-core glibc already installed
           skipping request to install glibc-core

    done
    **** rest rule
    **** all rule
    **** Default rule
    make[1]: Leaving directory `/home/knut/sources/gub'

if everything worked fine.

In uploads you see the generated installers, the documentation, test results etc:

    drwxr-xr-x 2 knut users      4096 29. Jan 08:56 signatures
    drwxr-xr-x 4 knut users      4096 29. Jan 08:56 localdoc
    drwxr-xr-x 4 knut users      4096 29. Jan 08:56 webdoc
    -rw-r--r-- 1 knut users 137828875 29. Jan 08:56 lilypond-2.21.0-1.webdoc.tar.bz2
    -rw-r--r-- 1 knut users 158089031 29. Jan 08:55 lilypond-2.21.0-1.documentation.tar.bz2
    drwxr-xr-x 4 knut users      4096 29. Jan 08:34 webtest
    -rw-r--r-- 1 knut users  18416327 29. Jan 08:31 lilypond-2.21.0-1.test-output.tar.bz2
    -rw-r--r-- 1 knut users  17226394 29. Jan 08:28 lilypond-2.21.0.tar.gz
    -rwxr-xr-x 1 knut users  31251820 29. Jan 08:27 lilypond-2.21.0-1.freebsd-64.sh
    -rwxr-xr-x 1 knut users  28063129 29. Jan 08:27 lilypond-2.21.0-1.freebsd-x86.sh
    -rwxr-xr-x 1 knut users  30586997 29. Jan 08:27 lilypond-2.21.0-1.linux-ppc.sh
    -rwxr-xr-x 1 knut users  31375293 29. Jan 08:27 lilypond-2.21.0-1.linux-64.sh
    -rw-r--r-- 1 knut users  34665510 29. Jan 08:26 lilypond-2.21.0-1.mingw.exe
    -rw-r--r-- 1 knut users  26746205 29. Jan 08:25 lilypond-2.21.0-1.darwin-x86.tar.bz2
    -rw-r--r-- 1 knut users  26611149 29. Jan 08:25 lilypond-2.21.0-1.darwin-ppc.tar.bz2
    -rwxr-xr-x 1 knut users  31208955 29. Jan 08:24 lilypond-2.21.0-1.linux-x86.sh

If you reached that point everything went fine.

At least we hope that everything went fine.

When we will start to update relevant components like guile we will need a way to test if the generated archives contain a working lilypond or garbage. I see that you already thought about that by providing the generated files on your cloud. Thanks you.

Knut
_______________________________________________
lilypond-devel mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/lilypond-devel
Reply | Threaded
Open this post in threaded view
|

Re: Please test gub

Urs Liska-3
Hi Knut,

Am 29.01.19 um 10:02 schrieb Knut Petersen:

>
> Hi Urs!
>
>> So what can I do to check whether make lilypond succeeded or failed?
>
>
> The last lines of the terminal output will look like:
>
>     make -f lilypond.make update-versions
>
>     To upload, run:
>
>         make lilypond-upload LILYPOND_BRANCH=master
>     LILYPOND_REPO_URL=git://git.sv.gnu.org/lilypond.git
>
>     **** nsis rule
>     python bin/gub tools::nsis
>     calculating dependencies
>     Checking for iconv ... /usr/bin/iconv
>     Checking for g++ ... /usr/bin/g++
>     Checking for gcc ... /usr/bin/gcc
>     must rebuild[tools]: system::gcc system::g++ system::iconv
>      *** Stage: pkg_install (cross/gcc-core, linux-x86)
>       cross/gcc conflicts with cross/gcc-core
>         non-core cross/gcc already installed
>           skipping request to install cross/gcc-core
>       cross/gcc-doc conflicts with cross/gcc-core
>         non-core cross/gcc already installed
>           skipping request to install cross/gcc-core
>       cross/gcc-runtime conflicts with cross/gcc-core
>         non-core cross/gcc already installed
>           skipping request to install cross/gcc-core
>
>      *** Stage: pkg_install (glibc-core, linux-x86)
>       glibc conflicts with glibc-core
>         non-core glibc already installed
>           skipping request to install glibc-core
>       glibc-doc conflicts with glibc-core
>         non-core glibc already installed
>           skipping request to install glibc-core
>
>     done
>     **** rest rule
>     **** all rule
>     **** Default rule
>     make[1]: Leaving directory `/home/knut/sources/gub'
>
> if everything worked fine.
>

Indeed that's exactly how it looks like :-)


> In uploads you see the generated installers, the documentation, test
> results etc:
>
>     drwxr-xr-x 2 knut users      4096 29. Jan 08:56 signatures
>     drwxr-xr-x 4 knut users      4096 29. Jan 08:56 localdoc
>     drwxr-xr-x 4 knut users      4096 29. Jan 08:56 webdoc
>     -rw-r--r-- 1 knut users 137828875 29. Jan 08:56
>     lilypond-2.21.0-1.webdoc.tar.bz2
>     -rw-r--r-- 1 knut users 158089031 29. Jan 08:55
>     lilypond-2.21.0-1.documentation.tar.bz2
>     drwxr-xr-x 4 knut users      4096 29. Jan 08:34 webtest
>     -rw-r--r-- 1 knut users  18416327 29. Jan 08:31
>     lilypond-2.21.0-1.test-output.tar.bz2
>     -rw-r--r-- 1 knut users  17226394 29. Jan 08:28 lilypond-2.21.0.tar.gz
>     -rwxr-xr-x 1 knut users  31251820 29. Jan 08:27
>     lilypond-2.21.0-1.freebsd-64.sh
>     -rwxr-xr-x 1 knut users  28063129 29. Jan 08:27
>     lilypond-2.21.0-1.freebsd-x86.sh
>     -rwxr-xr-x 1 knut users  30586997 29. Jan 08:27
>     lilypond-2.21.0-1.linux-ppc.sh
>     -rwxr-xr-x 1 knut users  31375293 29. Jan 08:27
>     lilypond-2.21.0-1.linux-64.sh
>     -rw-r--r-- 1 knut users  34665510 29. Jan 08:26
>     lilypond-2.21.0-1.mingw.exe
>     -rw-r--r-- 1 knut users  26746205 29. Jan 08:25
>     lilypond-2.21.0-1.darwin-x86.tar.bz2
>     -rw-r--r-- 1 knut users  26611149 29. Jan 08:25
>     lilypond-2.21.0-1.darwin-ppc.tar.bz2
>     -rwxr-xr-x 1 knut users  31208955 29. Jan 08:24
>     lilypond-2.21.0-1.linux-x86.sh
>
> If you reached that point everything went fine.
>
> At least we hope that everything went fine.
>
> When we will start to update relevant components like guile we will
> need a way to test if the generated archives contain a working
> lilypond or garbage. I see that you already thought about that by
> providing the generated files on your cloud. Thanks you.
>

Right now I have the process running on my dedicated server running
Debian 9.7 (so far running without errors, but I'll reattach to the
session in the afternoon). If that works too I'd "donate" that server
capacity for further and maybe automated tests, as long as I don't have
to be involved in complicated set-ups or configuration. If necessary I
could then also give someone (e.g. you) a non-privileged user account.
But I'll report back when I have results from the current build.

Urs

> Knut
_______________________________________________
lilypond-devel mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/lilypond-devel
Reply | Threaded
Open this post in threaded view
|

Re: Please test gub

Federico Bruni-2
In reply to this post by Urs Liska-3
Il giorno mar 29 gen 2019 alle 7:41, Urs Liska <[hidden email]>
ha scritto:
> So what can I do to check whether make lilypond succeeded or failed?

Good question.

On Ubuntu 16.04...
I just found out that I managed to build the packages on 23rd of
January,
despite the errors I reported last week:

$ ls -lh uploads/
total 247M
-rw-rw-r-- 1 dev dev 26M Jan 28 20:23
lilypond-2.21.0-1.darwin-ppc.tar.bz2
-rw-rw-r-- 1 dev dev 26M Jan 23 13:30
lilypond-2.21.0-1.darwin-x86.tar.bz2
-rwxr-xr-x 1 dev dev 30M Jan 23 13:35 lilypond-2.21.0-1.freebsd-64.sh
-rwxr-xr-x 1 dev dev 27M Jan 23 13:35 lilypond-2.21.0-1.freebsd-x86.sh
-rwxr-xr-x 1 dev dev 30M Jan 23 13:33 lilypond-2.21.0-1.linux-64.sh
-rwxr-xr-x 1 dev dev 30M Jan 23 13:34 lilypond-2.21.0-1.linux-ppc.sh
-rwxr-xr-x 1 dev dev 30M Jan 28 20:23 lilypond-2.21.0-1.linux-x86.sh
-rw-rw-r-- 1 dev dev 34M Jan 23 13:32 lilypond-2.21.0-1.mingw.exe
-rw-rw-r-- 1 dev dev 17M Jan 23 13:36 lilypond-2.21.0.tar.gz
drwxrwxr-x 2 dev dev 4.0K Jan 23 13:36 signatures


Yesterday (28th) I ran a new 'make lilypond' after cleaning target/ and
pulling
Knut's branch. It seems that I managed to rebuild linux-x86 and
darwin-ppc, but
darwin-x86 failed.

Find attached log/gub.log

Here's the terminal output tail:

building package: linux-x86::lilypond-installer
 *** Stage: download (lilypond-installer, linux-x86)
 *** Stage: compile (lilypond-installer, linux-x86)
 *** Stage: install (lilypond-installer, linux-x86)
 *** Stage: package (lilypond-installer, linux-x86)

done
calculating dependencies
Checking for mpost ... /usr/bin/mpost
Checking for xelatex ... /usr/bin/xelatex
Checking for g++ ... /usr/bin/g++
Checking for mf ... /usr/bin/mf
Checking for xetex ... /usr/bin/xetex
Checking for gcc ... /usr/bin/gcc
must rebuild[darwin-ppc]: system::gcc system::g++ system::mpost
system::mf system::xelatex system::xetex lilypond-installer
 *** Stage: pkg_install (cross/gcc-core, linux-x86)
  cross/gcc conflicts with cross/gcc-core
    non-core cross/gcc already installed
      skipping request to install cross/gcc-core
  cross/gcc-doc conflicts with cross/gcc-core
    non-core cross/gcc already installed
      skipping request to install cross/gcc-core
  cross/gcc-runtime conflicts with cross/gcc-core
    non-core cross/gcc already installed
      skipping request to install cross/gcc-core

 *** Stage: pkg_install (glibc-core, linux-x86)
  glibc conflicts with glibc-core
    non-core glibc already installed
      skipping request to install glibc-core
  glibc-doc conflicts with glibc-core
    non-core glibc already installed
      skipping request to install glibc-core

building package: darwin-ppc::lilypond-installer
 *** Stage: download (lilypond-installer, darwin-ppc)
 *** Stage: compile (lilypond-installer, darwin-ppc)
 *** Stage: install (lilypond-installer, darwin-ppc)
 *** Stage: package (lilypond-installer, darwin-ppc)

done
calculating dependencies
Checking for mpost ... /usr/bin/mpost
Checking for xelatex ... /usr/bin/xelatex
Checking for mf ... /usr/bin/mf
Checking for g++ ... /usr/bin/g++
Checking for xetex ... /usr/bin/xetex
Checking for gcc ... /usr/bin/gcc
must rebuild[darwin-x86]: system::gcc system::g++ system::mpost
odcctools system::mf system::xelatex system::xetex lilypond-installer
removing outdated[darwin-x86]: odcctools

Tail of log/gub.log >>>>>>>>
    calculating checksums
    must rebuild[darwin-x86]: system::gcc system::g++ system::mpost
odcctools system::mf system::xelatex system::xetex lilypond-installer
    removing outdated[darwin-x86]: odcctools
    uninstalling package: odcctools-doc
<<<<<<<< Tail of log/gub.log

Traceback (most recent call last):
  File "bin/gub", line 233, in exceptional_build
    build (settings, options, files)
  File "bin/gub", line 229, in build
    b.build_source_packages (names)
  File "bin/../gub/buildrunner.py", line 330, in build_source_packages
    self.uninstall_specs (outdated_installed)
  File "bin/../gub/buildrunner.py", line 309, in uninstall_specs
    self.uninstall_spec (self.specs[name])
  File "bin/../gub/buildrunner.py", line 299, in uninstall_spec
    self.manager (pkg.platform ()).uninstall_package (pkg.name ())
  File "bin/../gub/gup.py", line 340, in uninstall_package
    FileManager.uninstall_package (self, name)
  File "bin/../gub/gup.py", line 175, in uninstall_package
    lst = self.package_installed_files (name)
  File "bin/../gub/gup.py", line 81, in package_installed_files
    lst = self._package_file_db[name]
KeyError: 'odcctools-doc'
gub.make:69: recipe for target 'lilypond-installers' failed
make[1]: *** [lilypond-installers] Error 1
make[1]: Leaving directory '/home/dev/gub'
GNUmakefile:26: recipe for target 'lilypond' failed
make: *** [lilypond] Error 2

real 218m19.121s
user 367m57.214s
sys 67m46.505s




_______________________________________________
lilypond-devel mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/lilypond-devel

gub.log (16K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Please test gub

Urs Liska-3
In reply to this post by Knut Petersen
Hi Knut,

as said I ran the GUB build on my Debian server as well.

Am 29.01.19 um 08:11 schrieb Knut Petersen:
> On 29.01.19 00:53, Karlin High wrote:
>> *** Failed target: darwin-ppc::odcctools
>> gub.make:63: recipe for target 'packages' failed
>> make[1]: *** [packages] Error 1
>> make[1]: Leaving directory '/home/karlin/knut-gub/gub'
>> GNUmakefile:26: recipe for target 'lilypond' failed
>> make: *** [lilypond] Error 2
>>

It seems that one choked at exactly the same point.


> That means you were able to build all the tools necessary to start
> compilation of odcctools. But our own gzip fails to decompress the
> odcc*tar.gz. Weird.
>
> Please verify that you got the correct source file. Executing
>
>    md5sum downloads/odcctools/odcctools-iphone-dev-278.tar.gz
>
> in  /home/karlin/knut-gub/gub should give that result:
>
>    b067f6311e4c3d923e693dd280fab632
> downloads/odcctools/odcctools-iphone-dev-278.tar.gz


That's successful.


>
> If this is ok (it really should!) please execute the following commands:
>
>    mkdir -p STRACE
>    rm -f STRACE/*
>    rm -f target/darwin-ppc/packages/odcctools*


It seems I also had to remove target/darwin-ppc/status/odcctools*


> strace -v -f -ff -s 1024 -o STRACE/TP bin/gub darwin-ppc::odcctools
>    grep -o '^exec[^]]*]' STRACE/*  |  grep '/tar"\|/gzip"'
>
> the output of grep should be similar to
>
> STRACE/TP.7751:execve("/home/gub/gub/target/tools/root/usr/bin/tar",
> ["tar", "-C", "/home/gub/gub/target/darwin-ppc/src/odcctools-278",
> "--strip-component=1", "-v", "-z", "-xf",
> "/home/karlin/knut-gub/gub/downloads/odcctools/odcctools-iphone-dev-278.tar.gz"]
> STRACE/TP.7752:execve("/home/gub/gub/target/tools/root/usr/bin/gzip",
> ["gzip", "-d"]
>

It is.


> Obviously filenames (STRACE/TP<number>) will differ as they indicate
> the ID of the processes.
>
> Please send me those two files and target/darwin-ppc/log/odcctools.log.


Which two files, the tar and gzip files in .../usr/bin?

Urs


>
> Knut
>
>
>
>
>
>
>
> _______________________________________________
> lilypond-devel mailing list
> [hidden email]
> https://lists.gnu.org/mailman/listinfo/lilypond-devel

_______________________________________________
lilypond-devel mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/lilypond-devel
Reply | Threaded
Open this post in threaded view
|

Re: Please test gub

Thomas Morley-2
In reply to this post by Karlin High
Am Di., 29. Jan. 2019 um 00:53 Uhr schrieb Karlin High <[hidden email]>:

>
> On 1/28/2019 6:53 AM, Knut Petersen wrote:
> > Please report success / fails with os / version / cpu info.
>
> I really like the simple instructions you posted, Knut. I wouldn't be
> testing Gub without them. My setup doesn't like the
> darwin-ppc::odcctools package for some reason. Mystified why it's
> bringing in iPhone stuff. This same thing happened in 2 separate runs; I
> had deleted the cloned Git repository and started over.
>
> Windows 7 Pro 64-bit SP1, Intel Core i5-3450
> VirtualBox 5.2.22r126460
> VM with 1 CPU core, 4GB RAM, 64GB storage
>
> karlin@vbox-ubuntu ~/knut-gub/gub (DevelHead=)$lsb_release -a
> No LSB modules are available.
> Distributor ID: Ubuntu
> Description:    Ubuntu 18.04.1 LTS
> Release:        18.04
> Codename:       bionic
>
> karlin@vbox-ubuntu ~/knut-gub/gub (DevelHead=)$uname -a
> Linux vbox-ubuntu 4.15.0-43-generic #46-Ubuntu SMP Thu Dec 6 14:45:28
> UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

Looks like 64-bit Ubuntu, like my main machine. With an earlier
gub-version I had similiar problem, because of missing 32-bit
libraries.
I got further after installing:
lib32ncurses5
lib32z1

This former run failed lateron with a python issue, though.

Right now I retry with Knut's gub, still running. I expect results in
the evening, have to run for my regular job now...

Cheers,
  Harm

>
> building package: darwin-ppc::odcctools
>   *** Stage: download (odcctools, darwin-ppc)
>   *** Stage: untar (odcctools, darwin-ppc)
> Command barfed: tar -C
> /home/karlin/knut-gub/gub/target/darwin-ppc/src/odcctools-278
> --strip-component=1 -v -z -xf
> /home/karlin/knut-gub/gub/downloads/odcctools/odcctools-iphone-dev-278.tar.gz
>
> Tail of target/darwin-ppc/log/odcctools.log >>>>>>>>
>      gzip: stdin: not in gzip format
>      tar: Child returned status 1
>      tar: Error is not recoverable: exiting now
>      Command barfed: tar -C
> /home/karlin/knut-gub/gub/target/darwin-ppc/src/odcctools-278
> --strip-component=1 -v -z -xf
> /home/karlin/knut-gub/gub/downloads/odcctools/odcctools-iphone-dev-278.tar.gz
> <<<<<<<< Tail of target/darwin-ppc/log/odcctools.log
>
> *** Failed target: darwin-ppc::odcctools
> gub.make:63: recipe for target 'packages' failed
> make[1]: *** [packages] Error 1
> make[1]: Leaving directory '/home/karlin/knut-gub/gub'
> GNUmakefile:26: recipe for target 'lilypond' failed
> make: *** [lilypond] Error 2
>
> real    76m18.221s
> user    52m44.624s
> sys     8m14.693s
> --
> Karlin High
> Missouri, USA
>
> _______________________________________________
> lilypond-devel mailing list
> [hidden email]
> https://lists.gnu.org/mailman/listinfo/lilypond-devel

_______________________________________________
lilypond-devel mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/lilypond-devel
Reply | Threaded
Open this post in threaded view
|

Re: Please test gub

Knut Petersen
In reply to this post by Urs Liska-3

>
>> Obviously filenames (STRACE/TP<number>) will differ as they indicate the ID of the processes.
>>
>> Please send me those two files and target/darwin-ppc/log/odcctools.log.
>
>
> Which two files, the tar and gzip files in .../usr/bin?


No, target/darwin-ppc/log/odcctools.log and the two files STRACE/TP<number> that were reported by grep ...

Knut


_______________________________________________
lilypond-devel mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/lilypond-devel
1234 ... 6