font problems

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

font problems

Werner LEMBERG

To make good doc builds of LilyPond it would be important IMHO to have
all necessary fonts included in the docker images.

Here's a list of 'foreign' fonts (or font families) that are not
correctly set up currently.  In particular, I think we should avoid
non-free fonts for demonstration purposes, replacing them with other
fonts where necessary.


* Documentation/snippets/changing-the-default-text-font-family.ly:

    Times New Roman
    Luxi Mono

* Documentation/snippets/changing-stanza-fonts:

    DejaVu Sans

* Documentation/snippets/utf-8.ly:

    [The Japanese text example doesn't have a special font setup; it
     tries to use the global setting, which is

       Linux Libertine O,serif   .

     However, 'Libertine' doesn't cover Japanese, thus a
     platform-dependent fallback gets used (on my box, for example, it
     is 'Droid Sans Japanese').]

    Linux Libertine O
    Linux Libertine Mono O


* input/regression/font-name.ly:

    Times New Roman
    Luxi Mono
    Bitstream Vera Sans

* input/regression/metronome-mark-formatter.ly:

    Times New Roman

* input/regression/markup-compound-meter.ly:

    The character 'e' in on of the `\compound-meter` functions doesn't
    exist in 'Emmentaler', thus a platform-dependent fallback gets
    used (on my box, for example, it is 'Roboto-Regular').

* input/regression/musicxml/31a-Directions.xml:

    The characters 'abc' are used in the argument of
    `make-dynamic-script`; they don't exist in Emmentaler, thus a
    platform-dependent fallback gets used.


* Documentation/en/notation/text.itely

    Times New Roman (3×)
    Luxi Mono
    Bitstream Charter
    Bitstream Vera Sans
Reply | Threaded
Open this post in threaded view
|

Re: font problems

Jonas Hahnfeld
Am Sonntag, den 06.09.2020, 17:52 +0200 schrieb Werner LEMBERG:
> To make good doc builds of LilyPond it would be important IMHO to have
> all necessary fonts included in the docker images.

Are these really important to ensure that the documentation is
buildable? The result is used nowhere (except for failure logs), and
the image is as small as possible on purpose.

signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: font problems

Werner LEMBERG

> Am Sonntag, den 06.09.2020, 17:52 +0200 schrieb Werner LEMBERG:
>> To make good doc builds of LilyPond it would be important IMHO to
>> have all necessary fonts included in the docker images.
>
> Are these really important to ensure that the documentation is
> buildable?

No, but I consider it really bad practice to rely on non-deterministic
data.  A long-term goal is to have repeatable builds that produce
identical results every time (maybe except some well-defined
differences).

> The result is used nowhere (except for failure logs), and the image
> is as small as possible on purpose.

It's not clear what you mean with the last comment, please elaborate.

Some problematic cases that I listed are part of the NR, and I see
absolutely no reason to have doc builds on different system use
different fonts.


    Werner

Reply | Threaded
Open this post in threaded view
|

Re: font problems

Jonas Hahnfeld
Am Sonntag, den 06.09.2020, 22:02 +0200 schrieb Werner LEMBERG:

> > Am Sonntag, den 06.09.2020, 17:52 +0200 schrieb Werner LEMBERG:
> >> To make good doc builds of LilyPond it would be important IMHO to
> >> have all necessary fonts included in the docker images.
> >
> > Are these really important to ensure that the documentation is
> > buildable?
>
> No, but I consider it really bad practice to rely on non-deterministic
> data.  A long-term goal is to have repeatable builds that produce
> identical results every time (maybe except some well-defined
> differences).
We're talking about different objectives here: CI is concerned *that*
something can be built, but the *what* is discarded on success. Thus
the installation of fonts doesn't matter at all as long as it builds.
Moreover, things are deterministic because the container image contains
the whole OS tree, and nothing from the outside spills to the build
process.

> > The result is used nowhere (except for failure logs), and the image
> > is as small as possible on purpose.
>
> It's not clear what you mean with the last comment, please elaborate.

CI building on shared runners starts with pulling the image. Right now,
this takes around 30-40 seconds. As this is limited in download size
and decompression, I'd expect the time to increase linearly.
Moreover, the images take up space on our private runners (so that they
don't have to pull every time). So everything added but not needed for
building is just wasting storage.


> Some problematic cases that I listed are part of the NR, and I see
> absolutely no reason to have doc builds on different system use
> different fonts.

That's an entirely different problem and has nothing to do with Docker
and GitLab CI, which you used as the motivation.

Jonas

signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: font problems

Werner LEMBERG

> We're talking about different objectives here: CI is concerned
> *that* something can be built, but the *what* is discarded on
> success.

Understood now, thanks.  However, I still think it simplifies
debugging if the right fonts *are* in the docker image.

> So everything added but not needed for building is just wasting
> storage.

OK.  Could you please send me the list of available fonts in the
docker image?

>> Some problematic cases that I listed are part of the NR, and I see
>> absolutely no reason to have doc builds on different system use
>> different fonts.
>
> That's an entirely different problem and has nothing to do with
> Docker and GitLab CI, which you used as the motivation.

This was bad wording then on my side, sorry.  However, IMHO, it
belongs together, but you are right that for checking a valid build it
is not necessary to have all fonts in the docker image.

BTW, another issue is the use of non-free fonts in the documentation;
I will replace them with free ones.


    Werner

Reply | Threaded
Open this post in threaded view
|

Re: font problems

Jonas Hahnfeld
Am Montag, den 07.09.2020, 10:16 +0200 schrieb Werner LEMBERG:
> > We're talking about different objectives here: CI is concerned
> > *that* something can be built, but the *what* is discarded on
> > success.
>
> Understood now, thanks.  However, I still think it simplifies
> debugging if the right fonts *are* in the docker image.

IMO you should never debug anything other than build failures in the CI
image. I once had the idea of providing development containers (built
on top of the CI one), but LilyDev does a better job at that already.

> > So everything added but not needed for building is just wasting
> > storage.
>
> OK.  Could you please send me the list of available fonts in the
> docker image?

Get it with
 $ docker run --rm -ti registry.gitlab.com/lilypond/lilypond/ci/ubuntu-16.04:20200829 $cmd

Attached is the full output of fc-list, and these are the packages:
 $ dpkg -S /usr/share/fonts /usr/share/texlive/texmf-dist/fonts /usr/share/texmf/fonts
texlive-fonts-recommended, poppler-data, gsfonts, fonts-dejavu-core: /usr/share/fonts
texlive-lang-cyrillic, texlive-fonts-recommended, texlive-base, texlive-latex-base, texlive-generic-recommended, texlive-metapost: /usr/share/texlive/texmf-dist/fonts
fonts-texgyre: /usr/share/texmf/fonts

Please note that urw-base35-fonts are custom the image, see
https://gitlab.com/lilypond/lilypond/-/blob/master/docker/ci/Dockerfile.ubuntu-16.04#L27

Jonas

fc-list.txt (10K) Download Attachment
signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: font problems

Werner LEMBERG

>> OK.  Could you please send me the list of available fonts in the
>> docker image?
>
> Get it with
>  $ docker run --rm -ti registry.gitlab.com/lilypond/lilypond/ci/ubuntu-16.04:20200829 $cmd
>
> Attached is the full output of fc-list,

Thanks a lot!  I think that we can easily adjust the used fonts to
that, with the exception of Japanese.


    Werner

Reply | Threaded
Open this post in threaded view
|

Re: font problems

Werner LEMBERG
In reply to this post by Werner LEMBERG

Folks,


some time ago I reported font problems in the documentation (this is,
using fonts that are either not optimal, or not freely available).

The basic question is which route we should go.

* Reduce the number of extra fonts as much as possible to not increase
  the size of the documentation additionally.  [Not much a problem I
  think since most fonts will be subsetted by ghostscript.]

* Be as diverse as possible.

Here are my suggestions.  Please comment.  Some of the fonts used
originally *are* free; however, they are either no longer maintained
today, or better replacements are available, thus my changes.

After we have settled on the necessary fonts they should be properly
documented as requirements to build the documentation.


* Documentation/snippets/changing-the-default-text-font-family.ly:

    Times New Roman   →  Source Serif Pro?
    Luxi Mono         →  DejaVu Sans Mono?
                         Iosevka?
                         Source Code Pro?

* Documentation/snippets/changing-stanza-fonts:

    DejaVu Sans  →  ok

* Documentation/snippets/utf-8.ly:

    [for Japanese]          →  Source Han Serif

    Linux Libertine O       →  Libertinus Serif?
                               Source Serif Pro?
    Linux Libertine Mono O  →  Libertinus Mono? (very ugly IMHO)
                            →  DejaVu Sans Mono?
                            →  Iosevka?
                            →  Source Code Pro?

* input/regression/font-name.ly:

    Times New Roman      →  Source Serif Pro?
    Luxi Mono            →  Source Code Pro?
    Bitstream Vera Sans  →  Source Sans Pro?
                            Dejau Sans?

* input/regression/metronome-mark-formatter.ly:

    Times New Roman  →  Source Serif Pro?

* input/regression/markup-compound-meter.ly:

    The character 'e' in one of the `\compound-meter` functions
    doesn't exist in 'Emmentaler', thus a platform-dependent fallback
    gets used (on my box, for example, it is 'Roboto-Regular').

    →  not sure what to do

* input/regression/musicxml/31a-Directions.xml:

    The characters 'abc' are used in the argument of
    `make-dynamic-script`; they don't exist in Emmentaler, thus a
    platform-dependent fallback gets used.

    →  not sure what to do

* Documentation/en/notation/text.itely

    Times New Roman (3×)  →  Source Serif Pro?
    Luxi Mono             →  Source Code Pro?
    Bitstream Charter     →  ok
    Bitstream Vera Sans   →  DejaVu Sans


      Werner
Reply | Threaded
Open this post in threaded view
|

Re: font problems

Masamichi HOSODA-2
> * Documentation/snippets/utf-8.ly:
>
>     [for Japanese]          →  Source Han Serif

For Japanese fonts, I suggest HaranoAji.
It is the default Japanese font from TeX Live 2020.

https://www.ctan.org/pkg/haranoaji

Reply | Threaded
Open this post in threaded view
|

Re: font problems

Jonas Hahnfeld
In reply to this post by Werner LEMBERG
Am Samstag, den 17.10.2020, 11:58 +0200 schrieb Werner LEMBERG:

> Folks,
>
> some time ago I reported font problems in the documentation (this is,
> using fonts that are either not optimal, or not freely available).
>
> The basic question is which route we should go.
>
> * Reduce the number of extra fonts as much as possible to not increase
>   the size of the documentation additionally.  [Not much a problem I
>   think since most fonts will be subsetted by ghostscript.]
I'd prefer to focus on fonts that are widely available. DejaVu and
Libertine are, I'm not sure about Iosevka and Libertinus (but maybe the
packages just have different names?). Source Sans and Source Serif are
by Adobe, are there equivalent replacements?

> * Be as diverse as possible.

Jonas

signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: font problems

Werner LEMBERG
In reply to this post by Masamichi HOSODA-2

>>     [for Japanese]          →  Source Han Serif
>
> For Japanese fonts, I suggest HaranoAji.
> It is the default Japanese font from TeX Live 2020.
>
> https://www.ctan.org/pkg/haranoaji

OK, thanks for the suggestion.


    Werner
Reply | Threaded
Open this post in threaded view
|

Re: font problems

Werner LEMBERG
In reply to this post by Jonas Hahnfeld

>> * Reduce the number of extra fonts as much as possible to not
>> increase   the size of the documentation additionally.  [Not much a
>> problem I   think since most fonts will be subsetted by
>> ghostscript.]
>
> I'd prefer to focus on fonts that are widely available.  DejaVu and
> Libertine are, I'm not sure about Iosevka and Libertinus (but maybe
> the packages just have different names?).

Iosevka seems to be an excellent font, but there are good
alternatives.  Libertinus is the (maintained) successor of Libertine,
fixing buglets here and there.

> Source Sans and Source Serif are by Adobe, are there equivalent
> replacements?

What's the problem with those Adobe fonts?  They are completely free
even in the GNU sense...


    Werner
Reply | Threaded
Open this post in threaded view
|

Re: font problems

Jonas Hahnfeld
Am Sonntag, den 18.10.2020, 06:53 +0200 schrieb Werner LEMBERG:

> > > * Reduce the number of extra fonts as much as possible to not
> > > increase   the size of the documentation additionally.  [Not much
> > > a problem I   think since most fonts will be subsetted by
> > > ghostscript.]
>
>
> > I'd prefer to focus on fonts that are widely available.  DejaVu and
> > Libertine are, I'm not sure about Iosevka and Libertinus (but maybe
> > the packages just have different names?).
>
> Iosevka seems to be an excellent font, but there are good
> alternatives.  Libertinus is the (maintained) successor of Libertine,
> fixing buglets here and there.
Okay, but are there packages for the major distributions? Building
LilyPond is already hard (especially the documentation) and I don't
think we should add fonts that users have to download and install
manually.

> > Source Sans and Source Serif are by Adobe, are there equivalent
> > replacements?
>
> What's the problem with those Adobe fonts?  They are completely free
> even in the GNU sense...

Okay.

Jonas

>
>     Werner


signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: font problems

Werner LEMBERG
In reply to this post by Werner LEMBERG

>>>     [for Japanese]          →  Source Han Serif
>>
>> For Japanese fonts, I suggest HaranoAji.
>> It is the default Japanese font from TeX Live 2020.
>>
>> https://www.ctan.org/pkg/haranoaji
>
> OK, thanks for the suggestion.

After some thinking I'm not sure whether HaranoAji is the best
solution.  Given that both LilyPond and ghostscript need the fonts, a
TeXLive package doesn't help much – usually, FontConfig doesn't scan
TeXLive font directories.  Additionally, older GNU/Linux distributions
don't provide a separate package for this font.

I'm not sure how to tackle both the building of the Japanese texinfo
PDFs and the Japanese demo stuff in the LilyPond example files at the
same time.

Suggestions?  Ideas?


    Werner
Reply | Threaded
Open this post in threaded view
|

Re: font problems

Masamichi HOSODA-2
>>> For Japanese fonts, I suggest HaranoAji.
>>> It is the default Japanese font from TeX Live 2020.
>>>
>>> https://www.ctan.org/pkg/haranoaji
>>
>> OK, thanks for the suggestion.
>
> After some thinking I'm not sure whether HaranoAji is the best
> solution.  Given that both LilyPond and ghostscript need the fonts, a
> TeXLive package doesn't help much – usually, FontConfig doesn't scan
> TeXLive font directories.  Additionally, older GNU/Linux distributions
> don't provide a separate package for this font.
>
> I'm not sure how to tackle both the building of the Japanese texinfo
> PDFs and the Japanese demo stuff in the LilyPond example files at the
> same time.
>
> Suggestions?  Ideas?

If I understand correctly, LilyPond needs to find the HaranoAji font,
but Ghostscript does not need to find it.

If LilyPond uses the HaranoAji font,
the font will be embedded in the generated Postscript file.
Even if you use `-dgs-never-embed-fonts`,
you can still pass the font to Ghostscript with `-dfont-ps-resdir`.

In Texinfo PDF building, you don't need Japanese fonts
if only the included figure PDFs use the Japanese fonts.
If you want to use Japanese characters in the text instead of PDF figures
of Texinfo, you need the Japanese font.

Of course, it is required that Fontconfig can find the HaranoAji font.

Most environments do not have Japanese fonts.
How about using generic font names (e.g serif and sans-serif etc.)
for Japanese font fall back?

This selects the generic serif font
if the environment that doesn't have the HaranoAji font.

```
\override #'(font-name . "HaranoAjiMincho-Regular,serif")
```
Reply | Threaded
Open this post in threaded view
|

Re: font problems

Werner LEMBERG

> If I understand correctly, LilyPond needs to find the HaranoAji
> font, but Ghostscript does not need to find it.

OK.

> In Texinfo PDF building, you don't need Japanese fonts if only the
> included figure PDFs use the Japanese fonts.

Yes.

> If you want to use Japanese characters in the text instead of PDF
> figures of Texinfo, you need the Japanese font.

Correct.

> Of course, it is required that Fontconfig can find the HaranoAji
> font.

The thing is that this doesn't happen automatically if you install a
TeXLive package.  On the other hand, if you install a font that is
provided by a GNU/Linux package manager (like `rpm`), it is available
to both LilyPond and xelatex.  Thus I think that maybe Source Han is a
better choice.

> Most environments do not have Japanese fonts.  How about using
> generic font names (e.g serif and sans-serif etc.)  for Japanese
> font fall back?

Exactly this I want to avoid.  The idea is to have reproducible builds
as much as possible, so I want to specify all fonts in the PDF
documentation.

> This selects the generic serif font if the environment that doesn't
> have the HaranoAji font.
>
> ```
> \override #'(font-name . "HaranoAjiMincho-Regular,serif")
> ```

Appending 'serif' is a good idea for people who are going to compile
the LilyPond tarball directly, and who don't mind if the resulting PDF
is slightly different w.r.t. fonts.


    Werner