Complex/Large Lilypond projects and build automation

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

Complex/Large Lilypond projects and build automation

Jim Duke

I have a large Lilypond project for several hymnals.  The project is organized into a subdirectory structure with each hymn in a separate directory.  For each hymn I produce several products: an 8.5x11 PDF, a 6x9 PDF, a set of images formatted for projection at 4x3 and 16x9 aspect ratios, and a set of part dominant MP3’s for use in learning the hymns.  These products are then assembled at a higher level into a comprehensive PDF of the hymnal including front material and index; and a package of the slide images organized to be compatible with a Slide production product used at my church for assembling a song service; as well as assembling metadata files to be uploaded into a website I maintain that provides access to these hymns and provides simple search capabilities and internet access to these products.

 

I use Make (that old workhorse) to automate the process; but that has some distinct limitations.  So I was wondering what tools others may be using to aid them in building larger Lilypond projects.  What, if anything, are you using?  How well does that work for you?

Reply | Threaded
Open this post in threaded view
|

Re: Complex/Large Lilypond projects and build automation

Urs Liska-3
Hi Jim,

7. Dezember 2019 23:12, "Jim Duke" <[hidden email]> schrieb:

> I have a large Lilypond project for several hymnals. The project is organized into a subdirectory
> structure with each hymn in a separate directory. For each hymn I produce several products: an
> 8.5x11 PDF, a 6x9 PDF, a set of images formatted for projection at 4x3 and 16x9 aspect ratios, and
> a set of part dominant MP3’s for use in learning the hymns. These products are then assembled at a
> higher level into a comprehensive PDF of the hymnal including front material and index; and a
> package of the slide images organized to be compatible with a Slide production product used at my
> church for assembling a song service; as well as assembling metadata files to be uploaded into a
> website I maintain that provides access to these hymns and provides simple search capabilities and
> internet access to these products.
>
> I use Make (that old workhorse) to automate the process; but that has some distinct limitations. So
> I was wondering what tools others may be using to aid them in building larger Lilypond projects.
> What, if anything, are you using? How well does that work for you?

Apart from Make (which I also use occasionally) I have gone in two directions for such challenges:
Controlling the process from LuaLaTeX, and adding functionality to Frescobaldi.

1)
One of the characteristics of LuaLaTeX is that it can be used to make LaTeX a document generation
system (on top of being a typesetting system). (Note: This can of course also be done with ConTeXt)
It is sort-of straightforward to create functionality in LuaLaTeX to compose documents from various
materials, including LilyPond scores that can be configured and triggered from the document.
Depending on the needs lyluatex may be already sufficient for that part of the equation.
This may be a natural solution if the end result should be one big PDF with all the materials

2)
For one project I had the task of managing 650 music examples and decided to add a tool panel to
Frescobaldi for that. Of course that's only realistic because I'm a Frescobaldi developer - but I decided from there to create a comprehensive Extension API to Frescobaldi, and now it is rather straightforward for someone with Python knowledge to create project specific functionality on top of everything Frescobaldi can already do. This may also be something to go after.

Apart from that there has been discussion to add more generic support for build tools to Frescobaldi (https://github.com/frescobaldi/frescobaldi/issues/1113), that is of course dependent on developer time.

HTH
Urs

Reply | Threaded
Open this post in threaded view
|

Re: Complex/Large Lilypond projects and build automation

David F.
In reply to this post by Jim Duke

On Dec 7, 2019, at 2:35 PM, Jim Duke <[hidden email]> wrote:

I have a large Lilypond project for several hymnals.  The project is organized into a subdirectory structure with each hymn in a separate directory.  For each hymn I produce several products: an 8.5x11 PDF, a 6x9 PDF, a set of images formatted for projection at 4x3 and 16x9 aspect ratios, and a set of part dominant MP3’s for use in learning the hymns.  These products are then assembled at a higher level into a comprehensive PDF of the hymnal including front material and index; and a package of the slide images organized to be compatible with a Slide production product used at my church for assembling a song service; as well as assembling metadata files to be uploaded into a website I maintain that provides access to these hymns and provides simple search capabilities and internet access to these products.
 
I use Make (that old workhorse) to automate the process; but that has some distinct limitations.  So I was wondering what tools others may be using to aid them in building larger Lilypond projects.  What, if anything, are you using?  How well does that work for you?

Jim, I’m right behind you!  I have a collection of just over 100 hymns that are mostly Spanish language, but about 20 of those also have bilingual versions.  My build system creates proof PDFs, 4x3 and 16x9 slide images and 4x3 and 16x9 PowerPoint slide decks which get synchronized to a shared Dropbox folder.  I started off with make, but switched to gradle because I was already using that at work.  I’ve got about 400 lines of Kotlin/gradle code.  I haven’t built any hymnals yet, but I plan to.

So far I feel like I’ve gotten further with gradle than I could have with make, but lately I’ve found gradle to be more and more annoying.  Both make and gradle struggle with 1-to-many and (especially) many-to-1 type build tasks.  So, for example, taking 8 png files and combining them into one Powerpoint file is doable in gradle, but you’re really fighting the build system and incremental builds start to fail in certain cases (like when a song goes from fitting on 6 slides to taking up 8 slides).  There are other annoyances.

Just a couple of weeks ago, I came across this blog post:
  "The only build system that might someday replace make”

So I intend to dig in to the design of the redo system and see if it can match the functionality provided by gradle without the pain and annoyances.

David

Reply | Threaded
Open this post in threaded view
|

Re: Complex/Large Lilypond projects and build automation

Jacques Menu Muzhic
Hello folks,

I’ll look into those alternative build systems.

Urs, where can I find a MWE of LuaLaTeX use for LilyPond? Didn't find any when I seached the web recently.
I have MacTeX 2019 installed.

JM

Le 8 déc. 2019 à 00:17, David F. <[hidden email]> a écrit :


On Dec 7, 2019, at 2:35 PM, Jim Duke <[hidden email]> wrote:

I have a large Lilypond project for several hymnals.  The project is organized into a subdirectory structure with each hymn in a separate directory.  For each hymn I produce several products: an 8.5x11 PDF, a 6x9 PDF, a set of images formatted for projection at 4x3 and 16x9 aspect ratios, and a set of part dominant MP3’s for use in learning the hymns.  These products are then assembled at a higher level into a comprehensive PDF of the hymnal including front material and index; and a package of the slide images organized to be compatible with a Slide production product used at my church for assembling a song service; as well as assembling metadata files to be uploaded into a website I maintain that provides access to these hymns and provides simple search capabilities and internet access to these products.
 
I use Make (that old workhorse) to automate the process; but that has some distinct limitations.  So I was wondering what tools others may be using to aid them in building larger Lilypond projects.  What, if anything, are you using?  How well does that work for you?

Jim, I’m right behind you!  I have a collection of just over 100 hymns that are mostly Spanish language, but about 20 of those also have bilingual versions.  My build system creates proof PDFs, 4x3 and 16x9 slide images and 4x3 and 16x9 PowerPoint slide decks which get synchronized to a shared Dropbox folder.  I started off with make, but switched to gradle because I was already using that at work.  I’ve got about 400 lines of Kotlin/gradle code.  I haven’t built any hymnals yet, but I plan to.

So far I feel like I’ve gotten further with gradle than I could have with make, but lately I’ve found gradle to be more and more annoying.  Both make and gradle struggle with 1-to-many and (especially) many-to-1 type build tasks.  So, for example, taking 8 png files and combining them into one Powerpoint file is doable in gradle, but you’re really fighting the build system and incremental builds start to fail in certain cases (like when a song goes from fitting on 6 slides to taking up 8 slides).  There are other annoyances.

Just a couple of weeks ago, I came across this blog post:
  "The only build system that might someday replace make”

So I intend to dig in to the design of the redo system and see if it can match the functionality provided by gradle without the pain and annoyances.

David


Reply | Threaded
Open this post in threaded view
|

Re: Complex/Large Lilypond projects and build automation

Urs Liska-3


Am 8. Dezember 2019 06:22:22 MEZ schrieb Jacques Menu <[hidden email]>:
>Hello folks,
>
>I’ll look into those alternative build systems.
>
>Urs, where can I find a MWE of LuaLaTeX use for LilyPond? Didn't find
>any when I seached the web recently.
>I have MacTeX 2019 installed.
>

I only have one real example so far, and this is *far* from minimal - but includes an extensive documentation.
I'll discuss whether it is time to make this freely available now or if I can give you access to the repository.

Urs

(Please ping me if I don't follow through on this)

>JM
>
>> Le 8 déc. 2019 à 00:17, David F. <[hidden email]> a écrit :
>>
>>
>> On Dec 7, 2019, at 2:35 PM, Jim Duke <[hidden email]
><mailto:[hidden email]>> wrote:
>>
>>> I have a large Lilypond project for several hymnals.  The project is
>organized into a subdirectory structure with each hymn in a separate
>directory.  For each hymn I produce several products: an 8.5x11 PDF, a
>6x9 PDF, a set of images formatted for projection at 4x3 and 16x9
>aspect ratios, and a set of part dominant MP3’s for use in learning the
>hymns.  These products are then assembled at a higher level into a
>comprehensive PDF of the hymnal including front material and index; and
>a package of the slide images organized to be compatible with a Slide
>production product used at my church for assembling a song service; as
>well as assembling metadata files to be uploaded into a website I
>maintain that provides access to these hymns and provides simple search
>capabilities and internet access to these products.
>>>  
>>> I use Make (that old workhorse) to automate the process; but that
>has some distinct limitations.  So I was wondering what tools others
>may be using to aid them in building larger Lilypond projects.  What,
>if anything, are you using?  How well does that work for you?
>>
>> Jim, I’m right behind you!  I have a collection of just over 100
>hymns that are mostly Spanish language, but about 20 of those also have
>bilingual versions.  My build system creates proof PDFs, 4x3 and 16x9
>slide images and 4x3 and 16x9 PowerPoint slide decks which get
>synchronized to a shared Dropbox folder.  I started off with make, but
>switched to gradle because I was already using that at work.  I’ve got
>about 400 lines of Kotlin/gradle code.  I haven’t built any hymnals
>yet, but I plan to.
>>
>> So far I feel like I’ve gotten further with gradle than I could have
>with make, but lately I’ve found gradle to be more and more annoying.
>Both make and gradle struggle with 1-to-many and (especially) many-to-1
>type build tasks.  So, for example, taking 8 png files and combining
>them into one Powerpoint file is doable in gradle, but you’re really
>fighting the build system and incremental builds start to fail in
>certain cases (like when a song goes from fitting on 6 slides to taking
>up 8 slides).  There are other annoyances.
>>
>> Just a couple of weeks ago, I came across this blog post:
>>   "The only build system that might someday replace make”
>>   https://apenwarr.ca/log/20101214 <https://apenwarr.ca/log/20101214>
>>
>> So I intend to dig in to the design of the redo system and see if it
>can match the functionality provided by gradle without the pain and
>annoyances.
>>
>> David
>>

--
Diese Nachricht wurde von meinem Android-Gerät mit K-9 Mail gesendet.

Reply | Threaded
Open this post in threaded view
|

Re: Complex/Large Lilypond projects and build automation

Jacques Menu Muzhic
Urs, who is the creator/maintainer of LuaLaTeX? There seems to be a mist cloud around it...

JM

> Le 8 déc. 2019 à 07:43, Urs Liska <[hidden email]> a écrit :
>
>
>
> Am 8. Dezember 2019 06:22:22 MEZ schrieb Jacques Menu <[hidden email]>:
>> Hello folks,
>>
>> I’ll look into those alternative build systems.
>>
>> Urs, where can I find a MWE of LuaLaTeX use for LilyPond? Didn't find
>> any when I seached the web recently.
>> I have MacTeX 2019 installed.
>>
>
> I only have one real example so far, and this is *far* from minimal - but includes an extensive documentation.
> I'll discuss whether it is time to make this freely available now or if I can give you access to the repository.
>


Reply | Threaded
Open this post in threaded view
|

Re: Complex/Large Lilypond projects and build automation

Andrew Bernard
Hi Jacques,

Is this of any use?

https://www.ctan.org/pkg/lualatex-doc

Andrew

On Sun, 8 Dec 2019 at 18:46, Jacques Menu <[hidden email]> wrote:
>
> Urs, who is the creator/maintainer of LuaLaTeX? There seems to be a mist cloud around it...

Reply | Threaded
Open this post in threaded view
|

Re: Complex/Large Lilypond projects and build automation

Rembrandt Wolpert
In reply to this post by Urs Liska-3
On 12/8/19 07:43, Urs Liska wrote:

>
>
> Am 8. Dezember 2019 06:22:22 MEZ schrieb Jacques Menu <[hidden email]>:
>> Hello folks,
>>
>> I’ll look into those alternative build systems.
>>
>> Urs, where can I find a MWE of LuaLaTeX use for LilyPond? Didn't find
>> any when I seached the web recently.
>> I have MacTeX 2019 installed.
>>
>
> I only have one real example so far, and this is *far* from minimal - but includes an extensive documentation.
> I'll discuss whether it is time to make this freely available now or if I can give you access to the repository.
>
> Urs
>
> (Please ping me if I don't follow through on this)
>
>> JM
>>
>>> Le 8 déc. 2019 à 00:17, David F. <[hidden email]> a écrit :
>>>
>>>
>>> On Dec 7, 2019, at 2:35 PM, Jim Duke <[hidden email]
>> <mailto:[hidden email]>> wrote:
>>>
>>>> I have a large Lilypond project for several hymnals.  The project is
>> organized into a subdirectory structure with each hymn in a separate
>> directory.  For each hymn I produce several products: an 8.5x11 PDF, a
>> 6x9 PDF, a set of images formatted for projection at 4x3 and 16x9
>> aspect ratios, and a set of part dominant MP3’s for use in learning the
>> hymns.  These products are then assembled at a higher level into a
>> comprehensive PDF of the hymnal including front material and index; and
>> a package of the slide images organized to be compatible with a Slide
>> production product used at my church for assembling a song service; as
>> well as assembling metadata files to be uploaded into a website I
>> maintain that provides access to these hymns and provides simple search
>> capabilities and internet access to these products.
>>>>  
>>>> I use Make (that old workhorse) to automate the process; but that
>> has some distinct limitations.  So I was wondering what tools others
>> may be using to aid them in building larger Lilypond projects.  What,
>> if anything, are you using?  How well does that work for you?
>>>
>>> Jim, I’m right behind you!  I have a collection of just over 100
>> hymns that are mostly Spanish language, but about 20 of those also have
>> bilingual versions.  My build system creates proof PDFs, 4x3 and 16x9
>> slide images and 4x3 and 16x9 PowerPoint slide decks which get
>> synchronized to a shared Dropbox folder.  I started off with make, but
>> switched to gradle because I was already using that at work.  I’ve got
>> about 400 lines of Kotlin/gradle code.  I haven’t built any hymnals
>> yet, but I plan to.
>>>
>>> So far I feel like I’ve gotten further with gradle than I could have
>> with make, but lately I’ve found gradle to be more and more annoying.
>> Both make and gradle struggle with 1-to-many and (especially) many-to-1
>> type build tasks.  So, for example, taking 8 png files and combining
>> them into one Powerpoint file is doable in gradle, but you’re really
>> fighting the build system and incremental builds start to fail in
>> certain cases (like when a song goes from fitting on 6 slides to taking
>> up 8 slides).  There are other annoyances.
>>>
>>> Just a couple of weeks ago, I came across this blog post:
>>>   "The only build system that might someday replace make”
>>>   https://apenwarr.ca/log/20101214 <https://apenwarr.ca/log/20101214>
>>>
>>> So I intend to dig in to the design of the redo system and see if it
>> can match the functionality provided by gradle without the pain and
>> annoyances.
>>>
>>> David
>>>
>

To follow up on lyluatex: it work(s|ed) perfectly well for even very
large musicological projects with hundreds of small examples. However,
we hit a snag with the coming upgrade of luatex/lualatex to
luahbtex/luahblatex. If you want to make use of the Renderer (eg.
Renderer=Harfbuzz), it throws errors. Without the Renderer it works (but
if you need it for a complex script, as I do) you will have to revert to
lilypond-book and the usual makefile process.

Here's the hickup (probably easily fixed, Urs?) that interrupts
compilation in a small examples:

\documentclass{article}
\usepackage{fontspec}
\setmainfont[Renderer=Harfbuzz]{TeX Gyre Schola} % with problem
%\setmainfont{Tex Gyre Schola}                   % no problem
\usepackage[nofragment, insert=systems]{lyluatex}
\begin{document}
% \nonstopmode % then it throws the errors, but continues and gives
               % result as expected, but it won't stop of course for
               % all other errors I do want to see and stopped for...
\lysetoption{staffsize}{13}
\noindent This compiles fine with lualatex-dev *without a Renderer*
\begin{lilypond}
{
c'4 d' e'8 f'8 g'4.
}
\end{lilypond}
Two sets of errors *with* Renderer.
\end{document}


The errors thrown are:
...al/texlive/2019/texmf-dist/scripts/lyluatex/lyluatex.lua:1335:
attempt to in
dex a nil value (field 'rawdata')
stack traceback:
    ...al/texlive/2019/texmf-dist/scripts/lyluatex/lyluatex.lua:1335: in
function
'lyluatex.lua.get_font_family'
    [\directlua]:1: in main chunk.
\ly@currentfonts ..._font_family(font.current()))}
                                                  \rmfamily \edef
\rmfamilyi...

l.14 \end{lilypond}

...al/texlive/2019/texmf-dist/scripts/lyluatex/lyluatex.lua:1335:
attempt to in
dex a nil value (field 'rawdata')
stack traceback:
    ...al/texlive/2019/texmf-dist/scripts/lyluatex/lyluatex.lua:1335: in
function
'lyluatex.lua.get_font_family'
    ...al/texlive/2019/texmf-dist/scripts/lyluatex/lyluatex.lua:1349: in
function
'lyluatex.lua.set_fonts'
    [\directlua]:1: in main chunk.
\ly@currentfonts ..., \sffamilyid , \ttfamilyid )}
                                                  \endgroup
l.14 \end{lilypond}

If you don't need the Renderer option, lyluatex is already now the way
to go.

Rembrandt

Reply | Threaded
Open this post in threaded view
|

Re: Complex/Large Lilypond projects and build automation

Urs Liska-3
In reply to this post by Jacques Menu Muzhic


Am 08.12.19 um 08:44 schrieb Jacques Menu:
Urs, who is the creator/maintainer of LuaLaTeX? There seems to be a mist cloud around it...


I'm not completely sure about the relation between LuaTeX and LuaLaTeX. The LuaTeX homepage is http://www.luatex.org/, and the LuaTeX manual is basically what one refers to when learning about the Lua capabilities of working with LuaLaTeX.

With regard to the one project I've right now set the repository visibility to public, and it can be accessed at https://git.ursliska.de/bfsc/kayser. The idea has always been to make it public, but the repository is not in a state where we liked anyone to look at it ;-)

On the plus side I can say that the project is extensively documented - because that was part of my paid commission. You can find the docs in the (surprise) documentation directory but I have uploaded the compiled PDFs to https://cloud.ursliska.de/s/A9gbm8gKqrdKefI and https://cloud.ursliska.de/s/oYXktF3wEXDiA9i (where I'll eventually remove them again, this is not a permanent link).

This project has a very complex infrastructure on the LilyPond side as well, and there's a conference paper giving a high-level introduction to it: https://cloud.ursliska.de/s/mErzJdRut7VAcY6

The LaTeX part is of course very different from the use case asked for here, but it is surely an example showing what it means to use the Lua based LaTeX flavor as a document generation system. I'll try to outline as concisely as possible what the project does.

Our edition (complete works of Isfrid Kayser) is organized as a directory hierarchy <workgroup>/<work>/<movement>/<part>.ily. On the LilyPond side I can basically say

  \engrave \with { 
    workgroup = masses
    work = one
    movement = all
    part = violin-one

  }

with "part" shorthands "score" and "choir" (printing the four voices plus basso continuo). Several options can be given to make some decisions about using modern or original clefs, using original line breaks or highlighting annotations.

The LaTeX infrastructure does a similar thing. You can tell it --  through command line options -- *what* volume you want to have created, and it collects the necessary information from the directory tree.

First it checks which score(s) are needed, then uses lyluatex to conditionally compile them (if they are not already cached). Then it uses Pandoc to convert additional texts (preface, editorial comments) from Markdown files - if they are present in the relevant directory (i.e. if an editor has provided a preface to a specific volume it will be included, otherwise it is simply skipped). Finally the critical report is typeset from the data exported from LilyPond using scholarLY.

Note that much of the code is somewhat outdated because in essence this project was the trigger to make me think further. What I realized is that essentially in *every* Lua project I did since I ended up implementing a kind of template system (starting with a copy from a previous project). So I started off generalizing this, and I hope my `luaformatters` package (https://github.com/lualatex-tools/luaformatters) will become a really useful tool, with the potential of extremely simplifying the use of Lua in LaTeX document(s/ packages). (https://cloud.ursliska.de/s/psHzJCMQMtW4aVs)

Urs

JM

Le 8 déc. 2019 à 07:43, Urs Liska [hidden email] a écrit :



Am 8. Dezember 2019 06:22:22 MEZ schrieb Jacques Menu [hidden email]:
Hello folks,

I’ll look into those alternative build systems.

Urs, where can I find a MWE of LuaLaTeX use for LilyPond? Didn't find
any when I seached the web recently.
I have MacTeX 2019 installed.

I only have one real example so far, and this is *far* from minimal - but includes an extensive documentation.
I'll discuss whether it is time to make this freely available now or if I can give you access to the repository.


    
Reply | Threaded
Open this post in threaded view
|

lyluatex and HarfBuzz (was: Complex/Large Lilypond projects and build automation)

Urs Liska-3
In reply to this post by Rembrandt Wolpert

Am 08.12.19 um 09:04 schrieb Rembrandt Wolpert:
Here's the hickup (probably easily fixed, Urs?) that interrupts
compilation in a small examples:

Well, I had replied to you two times, asking for further information. Since that didn't come I didn't look into it so far ...

> However, we hit a snag with the coming upgrade of luatex/lualatex to luahbtex/luahblatex.

What exactly is this, what is the prerequisite to compile your documents (since you're talking about an "coming upgrade").

When I use

\setmainfont[Renderer=Harfbuzz]{TeX Gyre Schola}

I get this error in *my* LuaLaTeX even without loading lyluatex

This is LuaTeX, Version 1.10.0 (TeX Live 2019)  (format=lualatex 2019.8.8)
...
! LaTeX Error: Missing \begin{document}.

So in order to track your issue I *really* need more information about the environment/context this is in.

Urs

Reply | Threaded
Open this post in threaded view
|

Re: lyluatex and HarfBuzz (was: Complex/Large Lilypond projects and build automation)

Jacques Menu Muzhic
Thanks Urs, I don’t think I’ll give LuaTex/LuaLaTex a try then.

A nice day!

JM

Le 8 déc. 2019 à 09:49, Urs Liska <[hidden email]> a écrit :


Am 08.12.19 um 09:04 schrieb Rembrandt Wolpert:
Here's the hickup (probably easily fixed, Urs?) that interrupts
compilation in a small examples:

Well, I had replied to you two times, asking for further information. Since that didn't come I didn't look into it so far ...

> However, we hit a snag with the coming upgrade of luatex/lualatex to luahbtex/luahblatex.

What exactly is this, what is the prerequisite to compile your documents (since you're talking about an "coming upgrade").

When I use

\setmainfont[Renderer=Harfbuzz]{TeX Gyre Schola}

I get this error in *my* LuaLaTeX even without loading lyluatex

This is LuaTeX, Version 1.10.0 (TeX Live 2019)  (format=lualatex 2019.8.8)
...
! LaTeX Error: Missing \begin{document}.

So in order to track your issue I *really* need more information about the environment/context this is in.

Urs



Reply | Threaded
Open this post in threaded view
|

Re: lyluatex and HarfBuzz (was: Complex/Large Lilypond projects and build automation)

Rembrandt Wolpert
In reply to this post by Urs Liska-3
Sorry, I didn't receive your replies, maybe because I seem to have lost my
subscription to this list, but I am back on now...

Here's my environment:

I am on an up-to-date (08-12-1919) texlive system installed on a Mac running
High Sierra.

My LilyPond tells me that it is GNU LilyPond 2.19.83.

"Coming upgrade" should be perhaps update? my misnomer; it refers to the
announcement that from texlive 2020 luatex (and with it lualatex,
luajittex,...) will be luahbtex/luahblatex. Updates in texlive via tlmgr
already perform all the necessary formatting etc.

On Macs luahblatex is called (until texlive 2020 becomes official) as
lualatex-dev. The present version I use is:

This is LuaHBTeX, Version 1.11.2 (TeX Live 2020/dev)  (format=lualatex-dev
2019.12.5)  7 DEC 2019 23:05
 system commands enabled.

there is no surprise that

| This is LuaTeX, Version 1.10.0 (TeX Live 2019)  (format=lualatex 2019.8.8)
...

does not work: the [Renderer=...] option did not yet exist in LuaTeX Version
1.10.0.
 
luaotfload from version 3.1 (TeXLive 2019, 10.11.2019) already supports for
the Renderer options `Harfbuzz`, `OpenType`, `AAT`, and `Graphite` under
luahbtex (not under luatex-without-the-hb)

texlive updates include now the "-dev" versions for the "upcoming upgrade
(to texlive 2020)" (as in latex-dev, pdflatex-tex, lualatex-dev,...)

The reason why I am using the development version is because my book with
many hundreds of musical notations (longer ones and snippets) will
definitely be after texlive 2020 comes out, and my *text* benefits from (in
the case of non-Western scripts, requires...) the use of the "Renderer..."
option.

The "minimal" example was compiled with:

bash>  lualatex-dev --shell-escape lyminimal.tex

setting: \setmainfont[Renderer=Harfbuzz]{TeX Gyre Schola} fails
setting: \setmainfont{TeX Gyre Schola} succeeds.

Using lilypond-book and then compiling with lualatex-dev and
\setmainfont[Renderer=Harfbuzz]{TeX Gyre Schola} succeeds.

It is the "Renderer=..."-option which throws lyluatex, regardless of the
chosen engine. Any other traditional options are fine. (E.g. a convoluted
"\setmainfont[Microtype,ItalicFont={Trinite No2 Italic Cond},
                        BoldFont={Trinite No2 Roman Cond}]{Trinite No2 Roman
Cond}")

Glad to give any other information I can - lyluatex really simplified my
life enormously! Thank you for it! And I'd like to stick with it!

Rembrandt






--
Sent from: http://lilypond.1069038.n5.nabble.com/User-f3.html

Reply | Threaded
Open this post in threaded view
|

Re: lyluatex and HarfBuzz (was: Complex/Large Lilypond projects and build automation)

Urs Liska-3


Am 8. Dezember 2019 12:57:38 MEZ schrieb Rembrandt Wolpert <[hidden email]>:

>Sorry, I didn't receive your replies, maybe because I seem to have lost
>my
>subscription to this list, but I am back on now...
>
>Here's my environment:
>
>I am on an up-to-date (08-12-1919) texlive system installed on a Mac
>running
>High Sierra.
>
>My LilyPond tells me that it is GNU LilyPond 2.19.83.
>
>"Coming upgrade" should be perhaps update? my misnomer; it refers to
>the
>announcement that from texlive 2020 luatex (and with it lualatex,
>luajittex,...) will be luahbtex/luahblatex. Updates in texlive via
>tlmgr
>already perform all the necessary formatting etc.
>
>On Macs luahblatex is called (until texlive 2020 becomes official) as
>lualatex-dev. The present version I use is:
>
>This is LuaHBTeX, Version 1.11.2 (TeX Live 2020/dev)
>(format=lualatex-dev
>2019.12.5)  7 DEC 2019 23:05
> system commands enabled.
>
>there is no surprise that
>
>| This is LuaTeX, Version 1.10.0 (TeX Live 2019)  (format=lualatex
>2019.8.8)
>...
>
>does not work: the [Renderer=...] option did not yet exist in LuaTeX
>Version
>1.10.0.
>
>luaotfload from version 3.1 (TeXLive 2019, 10.11.2019) already supports
>for
>the Renderer options `Harfbuzz`, `OpenType`, `AAT`, and `Graphite`
>under
>luahbtex (not under luatex-without-the-hb)
>
>texlive updates include now the "-dev" versions for the "upcoming
>upgrade
>(to texlive 2020)" (as in latex-dev, pdflatex-tex, lualatex-dev,...)
>
>The reason why I am using the development version is because my book
>with
>many hundreds of musical notations (longer ones and snippets) will
>definitely be after texlive 2020 comes out, and my *text* benefits from
>(in
>the case of non-Western scripts, requires...) the use of the
>"Renderer..."
>option.
>
>The "minimal" example was compiled with:
>
>bash>  lualatex-dev --shell-escape lyminimal.tex
>
>setting: \setmainfont[Renderer=Harfbuzz]{TeX Gyre Schola} fails
>setting: \setmainfont{TeX Gyre Schola} succeeds.
>
>Using lilypond-book and then compiling with lualatex-dev and
>\setmainfont[Renderer=Harfbuzz]{TeX Gyre Schola} succeeds.
>
>It is the "Renderer=..."-option which throws lyluatex, regardless of
>the
>chosen engine. Any other traditional options are fine. (E.g. a
>convoluted
>"\setmainfont[Microtype,ItalicFont={Trinite No2 Italic Cond},
>                   BoldFont={Trinite No2 Roman Cond}]{Trinite No2 Roman
>Cond}")
>
>Glad to give any other information I can - lyluatex really simplified
>my
>life enormously! Thank you for it! And I'd like to stick with it!
>
>Rembrandt
>
>

Ok, with this information I can try to reproduce the issue and start looking at the Lua errors.

Urs

>
>
>
>
>--
>Sent from: http://lilypond.1069038.n5.nabble.com/User-f3.html

--
Diese Nachricht wurde von meinem Android-Gerät mit K-9 Mail gesendet.

Reply | Threaded
Open this post in threaded view
|

RE: Complex/Large Lilypond projects and build automation

Jim Duke
In reply to this post by David F.

David,

 

Thanks for your response.  I’m looking into using Gradle – and I think I’ll be happiest if I invest the time to develop one or more Gradle plugins to define the notion of a Lilypond project.  I have a steep learning curve in front of me since I’m not familiar with Gradle.  So part of my interest in using Gradle here is so I have an excuse to learn Gradle.

Gradle uses a convention-over-configuration approach (their words), and so that leads to the question: what conventions to people use in file system layout, use of templates, include files, etc.  The Gradle plugin that I write will assume a certain conventional layout – so I’ll start with what I’m using.  But I’m interested in what other people are doing.  Also, as part of the plugin I’ll eventually make the attempt to discover the outputs from a particular Lilypond file.  That’s a longer term goal, and very much non trivial.  So I’ll probably start with the user being required to explicitly define the inputs and outputs.  I already need to do that with Make – so not too big a deal for me.

 

Are you able to share any Gradle build files?  I would love to see what you did as I’m learning Gradle.

 

Thanks again,

 

Jim

 

From: David F. <[hidden email]>
Sent: Saturday, December 7, 2019 6:17 PM
To: Jim Duke <[hidden email]>
Cc: Lilypond User ([hidden email]) <[hidden email]>
Subject: Re: Complex/Large Lilypond projects and build automation

 

 

On Dec 7, 2019, at 2:35 PM, Jim Duke <[hidden email]> wrote:



I have a large Lilypond project for several hymnals.  The project is organized into a subdirectory structure with each hymn in a separate directory.  For each hymn I produce several products: an 8.5x11 PDF, a 6x9 PDF, a set of images formatted for projection at 4x3 and 16x9 aspect ratios, and a set of part dominant MP3’s for use in learning the hymns.  These products are then assembled at a higher level into a comprehensive PDF of the hymnal including front material and index; and a package of the slide images organized to be compatible with a Slide production product used at my church for assembling a song service; as well as assembling metadata files to be uploaded into a website I maintain that provides access to these hymns and provides simple search capabilities and internet access to these products.

 

I use Make (that old workhorse) to automate the process; but that has some distinct limitations.  So I was wondering what tools others may be using to aid them in building larger Lilypond projects.  What, if anything, are you using?  How well does that work for you?

 

Jim, I’m right behind you!  I have a collection of just over 100 hymns that are mostly Spanish language, but about 20 of those also have bilingual versions.  My build system creates proof PDFs, 4x3 and 16x9 slide images and 4x3 and 16x9 PowerPoint slide decks which get synchronized to a shared Dropbox folder.  I started off with make, but switched to gradle because I was already using that at work.  I’ve got about 400 lines of Kotlin/gradle code.  I haven’t built any hymnals yet, but I plan to.

 

So far I feel like I’ve gotten further with gradle than I could have with make, but lately I’ve found gradle to be more and more annoying.  Both make and gradle struggle with 1-to-many and (especially) many-to-1 type build tasks.  So, for example, taking 8 png files and combining them into one Powerpoint file is doable in gradle, but you’re really fighting the build system and incremental builds start to fail in certain cases (like when a song goes from fitting on 6 slides to taking up 8 slides).  There are other annoyances.

 

Just a couple of weeks ago, I came across this blog post:

  "The only build system that might someday replace make”

 

So I intend to dig in to the design of the redo system and see if it can match the functionality provided by gradle without the pain and annoyances.

 

David