musicxml2ly unsupported unicode range

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

musicxml2ly unsupported unicode range

Lars Henrik Ørn  Musikskolen  Kultur og Fritid  Vejle Kommune
Hi

I have for some time as a musicshool teacher been a busy user of lilypond on my Linux Desktop. Recently my employer have forced me to work on windows 10. I have got a working setup but musicxml2ly gives the following error:


Starting python-windows.exe...
Traceback (most recent call last):
File "C:\Program Files (x86)\LilyPond\usr\bin\musicxml2ly.py", line 3302, in ?
main()
File "C:\Program Files (x86)\LilyPond\usr\bin\musicxml2ly.py", line 3284, in main
basefilename = args[0].decode('utf-8')
File "/usr/lib/python2.4/encodings/utf_8.py", line 16, in decode
UnicodeDecodeError: 'utf8' codec can't decode bytes in position 60-64: unsupported Unicode code range
Exited with return code 1.

This happens with both 2.18.2 and 2.20. And both from commandline and Frescobaldi. And as I told on windows. No problems on linux.

Have a nice day and thank you for your efforts. Highly appreciated
Lars Henrik Ørn
_______________________________________________
bug-lilypond mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/bug-lilypond
Reply | Threaded
Open this post in threaded view
|

Re: musicxml2ly unsupported unicode range

Jean ABOU SAMRA
   > Hi

   > I have for some time as a musicshool teacher been a busy user

   > of lilypond on my Linux Desktop. Recently my employer have forced

   > me to work on windows 10. I have got a working setup but

   > musicxml2ly gives the following error:

   >

   > Starting python-windows.exe...

   > Traceback (most recent call last):

   > File "C:\Program Files (x86)\LilyPond\usr\bin\musicxml2ly.py", line
   3284, in main

   > basefilename = args[0].decode('utf-8')

   > File "/usr/lib/python2.4/encodings/utf_8.py", line 16, in decode

   > UnicodeDecodeError: 'utf8' codec can't decode bytes in position
   60-64: unsupported Unicode code range

   > Exited with return code 1.

   > This happens with both 2.18.2 and 2.20. And both from commandline and
   Frescobaldi.

   > And as I told on windows. No problems on linux. Have a nice day and
   thank you for

   > your efforts. Highly appreciated Lars Henrik Ørn

   Hi,

   You likely have a special character in the name of your file.

   Windows handles this poorly and LilyPond (actually Python,

   behind the scenes) can't do much for this. Try removing

   special characters from the file name, it should solve

   your problem.

   Regards,

   Jean Abou Samra
_______________________________________________
bug-lilypond mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/bug-lilypond
Reply | Threaded
Open this post in threaded view
|

Re: musicxml2ly unsupported unicode range

Torsten Hämmerle
Jean ABOU SAMRA wrote
>    You likely have a special character in the name of your file.
>    Windows handles this poorly and LilyPond (actually Python,
>    behind the scenes) can't do much for this. Try removing
>    special characters from the file name, it should solve
>    your problem.


I can confirm this (Win 10).
And what's even worse: while one can avoid special characters in filenames,
it is not possible to specify languages containing special characters, as in



I just won't work in Windows, neither on the command line nor via
Frescobaldi:

<http://lilypond.1069038.n5.nabble.com/file/t3887/fresco-language-xml.png>

(language dialogue modded for testing purposes)

C'est dommage !
Torsten

fresco-language-xml.png
<http://lilypond.1069038.n5.nabble.com/file/t3887/fresco-language-xml.png>  



--
Sent from: http://lilypond.1069038.n5.nabble.com/Bugs-f58488.html

_______________________________________________
bug-lilypond mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/bug-lilypond
Reply | Threaded
Open this post in threaded view
|

Re: musicxml2ly unsupported unicode range

Lars Henrik Ørn
In reply to this post by Lars Henrik Ørn Musikskolen Kultur og Fritid Vejle Kommune
Hi again


I could indeed solve the issue by changing the filename to another ;-))

Another issue was that musicxml2ly on windows 10 could not open
compressed xml files (zlib module missing). But that it is not a big
problem. Uncompressed worked OK.

So now I can continue my work with an adapted workflow. Many files just
written in Frescobaldi. Bigger arrangements and compositions is made in
musescore and then exported to lilypond (or Frescobladi). And then
exported to Scribus (I am making my own teaching books as well).


Have a nice day and happy hacking ;-))

Lars Henrik Ørn


_______________________________________________
bug-lilypond mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/bug-lilypond
Reply | Threaded
Open this post in threaded view
|

Re: musicxml2ly unsupported unicode range

Aaron Hill
In reply to this post by Torsten Hämmerle
On 2020-03-06 9:13 am, Torsten Hämmerle wrote:

> Jean ABOU SAMRA wrote
>>    You likely have a special character in the name of your file.
>>    Windows handles this poorly and LilyPond (actually Python,
>>    behind the scenes) can't do much for this. Try removing
>>    special characters from the file name, it should solve
>>    your problem.
>
> I can confirm this (Win 10).
> And what's even worse: while one can avoid special characters in
> filenames,
> it is not possible to specify languages containing special characters,
> as in
>
> I just won't work in Windows, neither on the command line nor via
> Frescobaldi:
>
> <http://lilypond.1069038.n5.nabble.com/file/t3887/fresco-language-xml.png>
While I am not an apologist for any operating system, I fear folks are
being far too quick to blame Windows here.

Like any current operating system concerned with internationalization,
Windows and NTFS handles and supports filenames with Unicode characters
just fine.  There are only a few characters and filenames that are
reserved [1] by the file system and operating system, but those are
ASCII-only.

The attached image shows a file with Greek and Cyrillic characters in
its name.  In all shells, I used tab-completion to enter the filename in
the prompt.  The only visual quirks are due to some shells not using the
right codepage, but that is an application-level issue.  For instance,
CMD has its roots in an older era, so I would not expect it to handle
UTF-8 content.  On the other hand, I was surprised by Powershell's
default encoding.

As my experience with Python is very limited, I will make no attempt to
comment on whether the this thread's original issue is due to the
musicxml2ly script, the Frescobaldi program, or the Python
language/runtime itself.  But rest assured, a properly-written program
can most definitely interface with the Windows API and file system to
allow for Unicode characters in file paths.

----

[1]:
https://en.wikipedia.org/wiki/Filename#Comparison_of_filename_limitations


-- Aaron Hill
_______________________________________________
bug-lilypond mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/bug-lilypond

windows-unicode.png (97K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: musicxml2ly unsupported unicode range

Jean ABOU SAMRA
   > While I am not an apologist for any operating system, I fear folks
   are being far too quick to blame Windows here.
   > Like any current operating system concerned with
   internationalization, Windows and NTFS handles and supports filenames
   with Unicode characters just fine.  There are only a few characters and
   filenames that are reserved [1] by the file system and operating
   system, but those are ASCII-only.
   > The attached image shows a file with Greek and Cyrillic characters in
   its name.  In all shells, I used tab-completion to enter the filename
   in the prompt.  The only visual quirks are due to some shells not using
   the right codepage, but that is an application-level issue.  For
   instance, CMD has its roots in an older era, so I would not expect it
   to handle UTF-8 content.  On the other hand, I was surprised by
   Powershell's default encoding.
   > As my experience with Python is very limited, I will make no attempt
   to comment on whether the this thread's original issue is due to the
   musicxml2ly script, the Frescobaldi program, or the Python
   language/runtime itself.  But rest assured, a properly-written program
   can most definitely interface with the Windows API and file system to
   allow for Unicode characters in file paths.

   The opposite here. I don't understand Windows

   at all but I have quite some experience with Python.

   Basically, one of the key goals for the major transition

   to Python 3 was to handle Unicode natively. Despite

   Python 2 being gone for years (and just getting

   officially unsupported), LilyPond kept using Python 2

   until very recently. Development versions now use

   Python 3.

   The traceback clearly indicates  the use of Python 2

   (and even Python 2.4 which is decades old):

   File "/usr/lib/python**2.4**/encodings/utf_8.py", etc.

   So, I'm not surprised by this problem. It should

   disappear with Python 3.

   However, if it does happen with Python 3, then there

   is an issue to tackle. I don't know what version shipped

   in LilyPond 2.20 but if you ever encounter the same

   problem with 'python3' in the traceback, please

   report.

   Regards,

   Jean Abou Samra
_______________________________________________
bug-lilypond mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/bug-lilypond