Issue #5822 aftermath: download sizes are gone from web site

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

Issue #5822 aftermath: download sizes are gone from web site

David Kastrup

I am not particularly surprised, to be honest: it did seem like the
website would employ something other than the online-root/offline-root
targets in the normal Stepmake build procedure.

The sized were inaccurate anyway.  I'll read up in the CG to see whether
I can figure out just what scripts actually do the website
creation/build/installation and see whether I can get the size script
inside.

--
David Kastrup

Reply | Threaded
Open this post in threaded view
|

Re: Issue #5822 aftermath: download sizes are gone from web site

Han-Wen Nienhuys-3
The website uses scripts that aren't directly checked out from
savannah, so you can't directly compromise the webserver through code
commits.

I can update the scripts.

On Wed, Mar 11, 2020 at 11:56 PM David Kastrup <[hidden email]> wrote:

>
>
> I am not particularly surprised, to be honest: it did seem like the
> website would employ something other than the online-root/offline-root
> targets in the normal Stepmake build procedure.
>
> The sized were inaccurate anyway.  I'll read up in the CG to see whether
> I can figure out just what scripts actually do the website
> creation/build/installation and see whether I can get the size script
> inside.
>
> --
> David Kastrup
>


--
Han-Wen Nienhuys - [hidden email] - http://www.xs4all.nl/~hanwen

Reply | Threaded
Open this post in threaded view
|

Re: Issue #5822 aftermath: download sizes are gone from web site

David Kastrup
Han-Wen Nienhuys <[hidden email]> writes:

> On Wed, Mar 11, 2020 at 11:56 PM David Kastrup <[hidden email]> wrote:
>>
>>
>> I am not particularly surprised, to be honest: it did seem like the
>> website would employ something other than the online-root/offline-root
>> targets in the normal Stepmake build procedure.
>>
>> The sized were inaccurate anyway.  I'll read up in the CG to see whether
>> I can figure out just what scripts actually do the website
>> creation/build/installation and see whether I can get the size script
>> inside.

> The website uses scripts that aren't directly checked out from
> savannah, so you can't directly compromise the webserver through code
> commits.
>
> I can update the scripts.

Ok.  It's just that I don't know whether I got the right script in the
right manner here.  The update itself, assuming the respective script
got transferred to the website, should be not more than a one-liner in
the respective Makefile (?).  And since it works from files that have
gotten the required HTML comments, it should work (or fail gracefully)
depending on whether its prerequisites are at the expected places in the
file tree.

But I don't have a real handle on this myself.

--
David Kastrup

Reply | Threaded
Open this post in threaded view
|

Re: Issue #5822 aftermath: download sizes are gone from web site

David Kastrup
In reply to this post by Han-Wen Nienhuys-3
Han-Wen Nienhuys <[hidden email]> writes:

> The website uses scripts that aren't directly checked out from
> savannah, so you can't directly compromise the webserver through code
> commits.
>
> I can update the scripts.

Any chance of giving this a try now?  The stuff is in master's script
now.

Thanks!

--
David Kastrup

Reply | Threaded
Open this post in threaded view
|

Re: Issue #5822 aftermath: download sizes are gone from web site

David Kastrup
David Kastrup <[hidden email]> writes:

> Han-Wen Nienhuys <[hidden email]> writes:
>
>> The website uses scripts that aren't directly checked out from
>> savannah, so you can't directly compromise the webserver through code
>> commits.
>>
>> I can update the scripts.
>
> Any chance of giving this a try now?  The stuff is in master's script
> now.
>
> Thanks!

Ping?

--
David Kastrup

Reply | Threaded
Open this post in threaded view
|

Re: Issue #5822 aftermath: download sizes are gone from web site

Han-Wen Nienhuys-3
On Mon, Mar 23, 2020 at 1:33 PM David Kastrup <[hidden email]> wrote:

>
> David Kastrup <[hidden email]> writes:
>
> > Han-Wen Nienhuys <[hidden email]> writes:
> >
> >> The website uses scripts that aren't directly checked out from
> >> savannah, so you can't directly compromise the webserver through code
> >> commits.
> >>
> >> I can update the scripts.
> >
> > Any chance of giving this a try now?  The stuff is in master's script
> > now.
> >
> > Thanks!
>
> Ping?

I installed the script on Sunday

The website build tries to do incremental builds (which, given the
state of the build system isn't a great idea).

If I remove 'out-website', and build from scratch, I get

Traceback (most recent call last):
  File "/home/graham/lilypond/trusted-scripts/create-weblinks-itexi.py",
line 602, in <module>
    make_download_source("downloadStableSource", VERSION_STABLE, lang)
  File "/home/graham/lilypond/trusted-scripts/create-weblinks-itexi.py",
line 454, in make_download_source
    make_macro(macroLang(name,lang), string)
  File "/home/graham/lilypond/trusted-scripts/create-weblinks-itexi.py",
line 428, in make_macro
    print(string.encode('utf-8'))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position
74: ordinal not in range(128)
website.make:225: recipe for target 'out-website/weblinks.itexi' failed
make: *** [out-website/weblinks.itexi] Error 1

a debug print suggests this comes from


>>> x = u"@uref{http://lilypond.org/download/sources/v2.20/lilypond-2.20.0.tar.gz, Código fuente: lilypond-2.20.0.tar.gz}"
>>> x[74]
'ó'
>>> ord(x[74])
243
>>> x.encode("utf-8")
b'@uref{http://lilypond.org/download/sources/v2.20/lilypond-2.20.0.tar.gz,
C\xc3\xb3digo fuente: lilypond-2.20.0.tar.gz}'

Looks like these strings come from the source code itself.

graham@lilypond-webserver:~/lilypond/build-website$ python --version
Python 2.7.12

make.website should specify python3 rather than python.

Do the string literals with unicode codepoints need to be declared as u" " ?


--
Han-Wen Nienhuys - [hidden email] - http://www.xs4all.nl/~hanweng

Reply | Threaded
Open this post in threaded view
|

Re: Issue #5822 aftermath: download sizes are gone from web site

Francisco Vila
El 24/3/20 a las 10:21, Han-Wen Nienhuys escribió:

> b'@uref{http://lilypond.org/download/sources/v2.20/lilypond-2.20.0.tar.gz,
> C\xc3\xb3digo fuente: lilypond-2.20.0.tar.gz}'
>
> Looks like these strings come from the source code itself.
>
> graham@lilypond-webserver:~/lilypond/build-website$ python --version
> Python 2.7.12
>
> make.website should specify python3 rather than python.
>
> Do the string literals with unicode codepoints need to be declared as u" " ?

Should I save the file with a different encoding?

--
Francisco Vila, Ph.D. - Badajoz (Spain)
paconet.org , lilypond.es

Reply | Threaded
Open this post in threaded view
|

Re: Issue #5822 aftermath: download sizes are gone from web site

Francisco Vila
El 24/3/20 a las 11:19, Francisco Vila escribió:

> El 24/3/20 a las 10:21, Han-Wen Nienhuys escribió:
>> b'@uref{http://lilypond.org/download/sources/v2.20/lilypond-2.20.0.tar.gz,
>>
>> C\xc3\xb3digo fuente: lilypond-2.20.0.tar.gz}'
>>
>> Looks like these strings come from the source code itself.
>>
>> graham@lilypond-webserver:~/lilypond/build-website$ python --version
>> Python 2.7.12
>>
>> make.website should specify python3 rather than python.
>>
>> Do the string literals with unicode codepoints need to be declared as
>> u" " ?
>
> Should I save the file with a different encoding?
>
Uh-oh, now I see macro names, unexpanded, online.

--
Francisco Vila, Ph.D. - Badajoz (Spain)
paconet.org , lilypond.es


Reply | Threaded
Open this post in threaded view
|

Re: Issue #5822 aftermath: download sizes are gone from web site

David Kastrup
Francisco Vila <[hidden email]> writes:

> El 24/3/20 a las 11:19, Francisco Vila escribió:
>> El 24/3/20 a las 10:21, Han-Wen Nienhuys escribió:
>>> b'@uref{http://lilypond.org/download/sources/v2.20/lilypond-2.20.0.tar.gz,
>>> C\xc3\xb3digo fuente: lilypond-2.20.0.tar.gz}'
>>>
>>> Looks like these strings come from the source code itself.
>>>
>>> graham@lilypond-webserver:~/lilypond/build-website$ python --version
>>> Python 2.7.12
>>>
>>> make.website should specify python3 rather than python.
>>>
>>> Do the string literals with unicode codepoints need to be declared
>>> as u" " ?
>>
>> Should I save the file with a different encoding?
>>
> Uh-oh, now I see macro names, unexpanded, online.
At the toplevel web page http://lilypond.org , no less.




--
David Kastrup

Screenshot_2020-03-24_17-29-18.png (554K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Issue #5822 aftermath: download sizes are gone from web site

Han-Wen Nienhuys-3
Thanks, I patched it up.

On Tue, Mar 24, 2020 at 5:31 PM David Kastrup <[hidden email]> wrote:

>
> Francisco Vila <[hidden email]> writes:
>
> > El 24/3/20 a las 11:19, Francisco Vila escribió:
> >> El 24/3/20 a las 10:21, Han-Wen Nienhuys escribió:
> >>> b'@uref{http://lilypond.org/download/sources/v2.20/lilypond-2.20.0.tar.gz,
> >>> C\xc3\xb3digo fuente: lilypond-2.20.0.tar.gz}'
> >>>
> >>> Looks like these strings come from the source code itself.
> >>>
> >>> graham@lilypond-webserver:~/lilypond/build-website$ python --version
> >>> Python 2.7.12
> >>>
> >>> make.website should specify python3 rather than python.
> >>>
> >>> Do the string literals with unicode codepoints need to be declared
> >>> as u" " ?
> >>
> >> Should I save the file with a different encoding?
> >>
> > Uh-oh, now I see macro names, unexpanded, online.
>
> At the toplevel web page http://lilypond.org , no less.
>
>
>
> --
> David Kastrup



--
Han-Wen Nienhuys - [hidden email] - http://www.xs4all.nl/~hanwen

Reply | Threaded
Open this post in threaded view
|

Re: Issue #5822 aftermath: download sizes are gone from web site

David Kastrup
Han-Wen Nienhuys <[hidden email]> writes:

>> On Tue, Mar 24, 2020 at 5:31 PM David Kastrup <[hidden email]> wrote:
>>>
>>> Francisco Vila <[hidden email]> writes:
>>>
>>> > El 24/3/20 a las 11:19, Francisco Vila escribió:
>>> >> El 24/3/20 a las 10:21, Han-Wen Nienhuys escribió:
>>> >>> b'@uref{http://lilypond.org/download/sources/v2.20/lilypond-2.20.0.tar.gz,
>>> >>> C\xc3\xb3digo fuente: lilypond-2.20.0.tar.gz}'
>>> >>>
>>> >>> Looks like these strings come from the source code itself.
>>> >>>
>>> >>> graham@lilypond-webserver:~/lilypond/build-website$ python --version
>>> >>> Python 2.7.12
>>> >>>
>>> >>> make.website should specify python3 rather than python.
>>> >>>
>>> >>> Do the string literals with unicode codepoints need to be declared
>>> >>> as u" " ?
>>> >>
>>> >> Should I save the file with a different encoding?
>>> >>
>>> > Uh-oh, now I see macro names, unexpanded, online.
>>>
>>> At the toplevel web page http://lilypond.org , no less.
>> --
>> David Kastrup
>
>
>
>
> Thanks, I patched it up.
Partly.  Several download links now contain b'...' visibly.



Sorry, I had not imagined that this change would bite us in the Python3.

--
David Kastrup

Screenshot_2020-03-24_21-55-39.png (360K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Issue #5822 aftermath: download sizes are gone from web site

Francisco Vila
El 24/3/20 a las 21:58, David Kastrup escribió:
>> Thanks, I patched it up.
> Partly.  Several download links now contain b'...' visibly.
>
Unrelated. I saw this in my local build before the sizes script.

--
Francisco Vila, Ph.D. - Badajoz (Spain)
paconet.org , lilypond.es

Reply | Threaded
Open this post in threaded view
|

Re: Issue #5822 aftermath: download sizes are gone from web site

David Kastrup
Francisco Vila <[hidden email]> writes:

> El 24/3/20 a las 21:58, David Kastrup escribió:
>>> Thanks, I patched it up.
>> Partly.  Several download links now contain b'...' visibly.
>>
> Unrelated. I saw this in my local build before the sizes script.

Oh, I didn't even consider that the "sizes" script was at fault.  It
would not have affected that area of the files as far as I can tell.
Any update likely would have gone bad without it as far as I can tell,
due to Python problems.

--
David Kastrup