lyp - a Package Manager for Lilypond

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

lyp - a Package Manager for Lilypond

Sharon Rosner
Hello all,

I'd like to announce lyp - a package manager for Lilypond:

  https://github.com/noteflakes/lyp

I started writing lyp a couple of months ago, as a result of discussions Urs Liska, Matteo Ceccarello and me were having a on how to go about implementing a package manager for Lilypond. Here's an overview:

lyp does two things: install and manage packages of Lilypond code; and install and manage multiple versions of Lilypond. It currently works on Linux and Mac OSX. Support for Windows is coming. lyp offers the following functionality:

- Install and use versioned packages from git repositories (see below).
- Automatically install sub-dependencies.
- Support for version constraints (e.g. >=0.1.2, ~>0.1.0, a.k.a pessimistic constraint)
- Resolve dependency trees of arbitrary depth.
- Automatically install custom fonts included in packages, automatically patch Lilypond versions prior to 2.19.12 in order to support using custom fonts.
- Run package test files.
- Easily install and switch between different versions of Lilypond using a single command.

In addition, there's:

- lyp-index, a semi-official index for packages, mapping canonical names to git URLs.
- an assert package, which provides scheme procedures for assertion which can be used for testing a package, or indeed any Lilypond code (see below).

Currently, of course, there doesn't exist much in the way of actual packages. But you can have a look at two packages which are already usable, *with tests*:

- assert - a package providing assertion functionality for scheme code. This package also includes a test file demonstrating usage of the assert:* procedures.
- roman-numerals - a fork of David Nalesnik's work on roman numerals for harmonic analysis.

(both of these packages also show how lyp can be integrated into a project's build process in order to automatically install Lilypond and run test files - in this case on travis-ci.)

The goal of lyp is to facilitate code sharing and reuse in the Lilypond community, to encourage cooperation on Lilypond-related projects, and above all to make life easier for Lilypond users. It does so by both offering a simple, cross-platform solution for installing Lilypond, and by offering tools for sharing Lilypond code such as testing, publishing, and installing packages.

I believe lyp can make a real difference in how the Lilypond community shares code. I welcome all questions, remarks and suggestions. What follows is a brief technical discussion of the main points which I think demand explanation. Please note that lyp is still under heavy development, so things might change drastically in the near future.

Packages

A package is any git repository that contains at least one file called package.ly, which serves as the entry point for the package. The package can include other Lilypond scheme files, as well as fonts which are installed automatically into any installed version of Lilypond.

Packages can be referenced in input files by using the \require command, which is automatically defined by lyp when invoking Lilypond. Packages can be versioned, and can be referenced using version constraints (e.g. \require "openlilylib>=1.0.0").

Packages can be referenced either using a partially- or fully-qualified git URL, a github id, or a canonical name (provided they are registered in the lyp-index).

Packages can also depend on other packages, creating dependency trees of arbitrary depth. lyp takes care of resolving the dependency tree for a given input file, selecting the correct version to use for each dependency. Packages are versioned using git tags.

Installing a package is as simple as:

    lyp install assert
 
lyp takes care of cloning the package's repository, searching for the correct version to use (in this case the highest version). lyp can also install a package from local files, for development purposes.

Installing Lilypond

lyp provides a command for installing arbitrary versions of lilypond. For example:

    lyp install lilypond@2.19.13
    # or
    lyp install lilypond@stable # install the latest stable version
 
Versions are installed in separate directories under ~/.lyp/lilyponds. For invocation see below.

Package loading

lyp provides its own Lilypond binary, which does the following:

- Select the correct version of Lilypond to use (based on user settings or environment variables).
- Scan the input file for any dependencies (specified using \require), and also recursively scan any include files for dependencies.
- Resolve the dependency tree and calculate the correct version to use for each required package.
- Create a small Lilypond 'shim' file that provides both package loading information, as well as definitions for a scheme interface used for loading package files, then finally loads the input file.
- Compile the shim file using the selected version of Lilypond.

How lyp is made

lyp is written in Ruby. It can be installed either as a Ruby gem, or as a standalone package for those who don't have Ruby on their machines.

lyp is pretty well tested, with over 60 different tests actually installing different versions of Lilypond, installing packages, running Lilypond test files, and testing the lyp package scheme interface using assert, which is itself a lyp package installed by lyp (that's pretty meta right there.)

---

So, I hope this proves interesting to the people here. Again, I welcome any questions, remarks or suggestions you may have.

Happy Lilyponding,
Sharon Rosner
Reply | Threaded
Open this post in threaded view
|

RE: lyp - a Package Manager for Lilypond

Mike Solomon
Congratulations!
This is a wonderful accomplishment and I am looking forward to getting some of my packages in good shale and contributing them. Do you have a contributor's guide yet?

Cheers,
MS



Sent from my Samsung Galaxy smartphone.
-------- Original message --------
From: Sharon Rosner <[hidden email]>
Date: 28/01/2016 11:52 PM (GMT+02:00)
Subject: lyp - a Package Manager for Lilypond

Hello all,

I'd like to announce lyp - a package manager for Lilypond:

   https://github.com/noteflakes/lyp <https://github.com/noteflakes/lyp> 

I started writing lyp a couple of months ago, as a result of discussions Urs
Liska, Matteo Ceccarello and me were having a on how to go about
implementing a package manager for Lilypond. Here's an overview:

lyp does two things: install and manage packages of Lilypond code; and
install and manage multiple versions of Lilypond. It currently works on
Linux and Mac OSX. Support for Windows is coming. lyp offers the following
functionality:

- Install and use versioned packages from git repositories (see below).
- Automatically install sub-dependencies.
- Support for version constraints (e.g. >=0.1.2, ~>0.1.0, a.k.a pessimistic
constraint)
- Resolve dependency trees of arbitrary depth.
- Automatically install custom fonts included in packages, automatically
patch Lilypond versions prior to 2.19.12 in order to support using custom
fonts.
- Run package test files.
- Easily install and switch between different versions of Lilypond using a
single command.

In addition, there's:

-  lyp-index <https://github.com/noteflakes/lyp-index>  , a semi-official
index for packages, mapping canonical names to git URLs.
- an assert package, which provides scheme procedures for assertion which
can be used for testing a package, or indeed any Lilypond code (see below).

Currently, of course, there doesn't exist much in the way of actual
packages. But you can have a look at two packages which are already usable,
*with tests*:

-  assert <https://github.com/noteflakes/lyp-assert>   - a package providing
assertion functionality for scheme code. This package also includes a test
file demonstrating usage of the assert:* procedures.
-  roman-numerals <https://github.com/noteflakes/lyp-roman-numerals>   - a
fork of David Nalesnik's work on roman numerals for harmonic analysis.

(both of these packages also show how lyp can be integrated into a project's
build process in order to automatically install Lilypond and run test files
- in this case on travis-ci.)

The goal of lyp is to facilitate code sharing and reuse in the Lilypond
community, to encourage cooperation on Lilypond-related projects, and above
all to make life easier for Lilypond users. It does so by both offering a
simple, cross-platform solution for installing Lilypond, and by offering
tools for sharing Lilypond code such as testing, publishing, and installing
packages.

I believe lyp can make a real difference in how the Lilypond community
shares code. I welcome all questions, remarks and suggestions. What follows
is a brief technical discussion of the main points which I think demand
explanation. Please note that lyp is still under heavy development, so
things might change drastically in the near future.

*Packages*

A package is any git repository that contains at least one file called
package.ly, which serves as the entry point for the package. The package can
include other Lilypond scheme files, as well as fonts which are installed
automatically into any installed version of Lilypond.

Packages can be referenced in input files by using the \require command,
which is automatically defined by lyp when invoking Lilypond. Packages can
be versioned, and can be referenced using version constraints (e.g. \require
"openlilylib>=1.0.0").

Packages can be referenced either using a partially- or fully-qualified git
URL, a github id, or a canonical name (provided they are registered in the
lyp-index <https://github.com/noteflakes/lyp-index>  ).

Packages can also depend on other packages, creating dependency trees of
arbitrary depth. lyp takes care of resolving the dependency tree for a given
input file, selecting the correct version to use for each dependency.
Packages are versioned using git tags.

Installing a package is as simple as:

    lyp install assert
 
lyp takes care of cloning the package's repository, searching for the
correct version to use (in this case the highest version). lyp can also
install a package from local files, for development purposes.

*Installing Lilypond*

lyp provides a command for installing arbitrary versions of lilypond. For
example:

    lyp install lilypond@2.19.13
    # or
    lyp install lilypond@stable # install the latest stable version
 
Versions are installed in separate directories under ~/.lyp/lilyponds. For
invocation see below.

*Package loading*

lyp provides its own Lilypond binary, which does the following:

- Select the correct version of Lilypond to use (based on user settings or
environment variables).
- Scan the input file for any dependencies (specified using \require), and
also recursively scan any include files for dependencies.
- Resolve the dependency tree and calculate the correct version to use for
each required package.
- Create a small Lilypond 'shim' file that provides both package loading
information, as well as definitions for a scheme interface used for loading
package files, then finally loads the input file.
- Compile the shim file using the selected version of Lilypond.

*How lyp is made*

lyp is written in Ruby. It can be installed either as a Ruby gem, or as a
standalone package for those who don't have Ruby on their machines.

lyp is pretty well tested, with over 60 different tests actually installing
different versions of Lilypond, installing packages, running Lilypond test
files, and testing the lyp package scheme interface using assert, which is
itself a lyp package installed by lyp (that's pretty meta right there.)

---

So, I hope this proves interesting to the people here. Again, I welcome any
questions, remarks or suggestions you may have.

Happy Lilyponding,
Sharon Rosner



--
View this message in context: http://lilypond.1069038.n5.nabble.com/lyp-a-Package-Manager-for-Lilypond-tp186597.html
Sent from the User mailing list archive at Nabble.com.

_______________________________________________
lilypond-user mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/lilypond-user

_______________________________________________
lilypond-user mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/lilypond-user
Reply | Threaded
Open this post in threaded view
|

RE: lyp - a Package Manager for Lilypond

Urs Liska


Am 29. Januar 2016 07:09:56 MEZ, schrieb Mike Solomon <[hidden email]>:
>Congratulations!This is a wonderful accomplishment and I am looking
>forward to getting some of my packages in good shale and contributing
>them. Do you have a contributor's guide yet?

I think one of the nice things in Sharon's approach is that you don't *have* to contribute - you can also simply make your packages available.

Urs

>Cheers,MS
>
>
>Sent from my Samsung Galaxy smartphone.-------- Original message
>--------From: Sharon Rosner <[hidden email]> Date: 28/01/2016  11:52
>PM  (GMT+02:00) To: [hidden email] Subject: lyp - a Package
>Manager for Lilypond
>Hello all,
>
>I'd like to announce lyp - a package manager for Lilypond:
>
>   https://github.com/noteflakes/lyp
><https://github.com/noteflakes/lyp
>
>I started writing lyp a couple of months ago, as a result of
>discussions Urs
>Liska, Matteo Ceccarello and me were having a on how to go about
>implementing a package manager for Lilypond. Here's an overview:
>
>lyp does two things: install and manage packages of Lilypond code; and
>install and manage multiple versions of Lilypond. It currently works on
>Linux and Mac OSX. Support for Windows is coming. lyp offers the
>following
>functionality:
>
>- Install and use versioned packages from git repositories (see below).
>- Automatically install sub-dependencies.
>- Support for version constraints (e.g. >=0.1.2, ~>0.1.0, a.k.a
>pessimistic
>constraint)
>- Resolve dependency trees of arbitrary depth.
>- Automatically install custom fonts included in packages,
>automatically
>patch Lilypond versions prior to 2.19.12 in order to support using
>custom
>fonts.
>- Run package test files.
>- Easily install and switch between different versions of Lilypond
>using a
>single command.
>
>In addition, there's:
>
>-  lyp-index <https://github.com/noteflakes/lyp-index>  , a
>semi-official
>index for packages, mapping canonical names to git URLs.
>- an assert package, which provides scheme procedures for assertion
>which
>can be used for testing a package, or indeed any Lilypond code (see
>below).
>
>Currently, of course, there doesn't exist much in the way of actual
>packages. But you can have a look at two packages which are already
>usable,
>*with tests*:
>
>-  assert <https://github.com/noteflakes/lyp-assert>   - a package
>providing
>assertion functionality for scheme code. This package also includes a
>test
>file demonstrating usage of the assert:* procedures.
>-  roman-numerals <https://github.com/noteflakes/lyp-roman-numerals>  
>- a
>fork of David Nalesnik's work on roman numerals for harmonic analysis.
>
>(both of these packages also show how lyp can be integrated into a
>project's
>build process in order to automatically install Lilypond and run test
>files
>- in this case on travis-ci.)
>
>The goal of lyp is to facilitate code sharing and reuse in the Lilypond
>community, to encourage cooperation on Lilypond-related projects, and
>above
>all to make life easier for Lilypond users. It does so by both offering
>a
>simple, cross-platform solution for installing Lilypond, and by
>offering
>tools for sharing Lilypond code such as testing, publishing, and
>installing
>packages.
>
>I believe lyp can make a real difference in how the Lilypond community
>shares code. I welcome all questions, remarks and suggestions. What
>follows
>is a brief technical discussion of the main points which I think demand
>explanation. Please note that lyp is still under heavy development, so
>things might change drastically in the near future.
>
>*Packages*
>
>A package is any git repository that contains at least one file called
>package.ly, which serves as the entry point for the package. The
>package can
>include other Lilypond scheme files, as well as fonts which are
>installed
>automatically into any installed version of Lilypond.
>
>Packages can be referenced in input files by using the \require
>command,
>which is automatically defined by lyp when invoking Lilypond. Packages
>can
>be versioned, and can be referenced using version constraints (e.g.
>\require
>"openlilylib>=1.0.0").
>
>Packages can be referenced either using a partially- or fully-qualified
>git
>URL, a github id, or a canonical name (provided they are registered in
>the
>lyp-index <https://github.com/noteflakes/lyp-index>  ).
>
>Packages can also depend on other packages, creating dependency trees
>of
>arbitrary depth. lyp takes care of resolving the dependency tree for a
>given
>input file, selecting the correct version to use for each dependency.
>Packages are versioned using git tags.
>
>Installing a package is as simple as:
>
>    lyp install assert

>lyp takes care of cloning the package's repository, searching for the
>correct version to use (in this case the highest version). lyp can also
>install a package from local files, for development purposes.
>
>*Installing Lilypond*
>
>lyp provides a command for installing arbitrary versions of lilypond.
>For
>example:
>
>    lyp install lilypond@2.19.13
>    # or
>    lyp install lilypond@stable # install the latest stable version

>Versions are installed in separate directories under ~/.lyp/lilyponds.
>For
>invocation see below.
>
>*Package loading*
>
>lyp provides its own Lilypond binary, which does the following:
>
>- Select the correct version of Lilypond to use (based on user settings
>or
>environment variables).
>- Scan the input file for any dependencies (specified using \require),
>and
>also recursively scan any include files for dependencies.
>- Resolve the dependency tree and calculate the correct version to use
>for
>each required package.
>- Create a small Lilypond 'shim' file that provides both package
>loading
>information, as well as definitions for a scheme interface used for
>loading
>package files, then finally loads the input file.
>- Compile the shim file using the selected version of Lilypond.
>
>*How lyp is made*
>
>lyp is written in Ruby. It can be installed either as a Ruby gem, or as
>a
>standalone package for those who don't have Ruby on their machines.
>
>lyp is pretty well tested, with over 60 different tests actually
>installing
>different versions of Lilypond, installing packages, running Lilypond
>test
>files, and testing the lyp package scheme interface using assert, which
>is
>itself a lyp package installed by lyp (that's pretty meta right there.)
>
>---
>
>So, I hope this proves interesting to the people here. Again, I welcome
>any
>questions, remarks or suggestions you may have.
>
>Happy Lilyponding,
>Sharon Rosner
>
>
>
>--
>View this message in context:
>http://lilypond.1069038.n5.nabble.com/lyp-a-Package-Manager-for-Lilypond-tp186597.html
>Sent from the User mailing list archive at Nabble.com.
>
>_______________________________________________
>lilypond-user mailing list
>[hidden email]
>https://lists.gnu.org/mailman/listinfo/lilypond-user
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>lilypond-user mailing list
>[hidden email]
>https://lists.gnu.org/mailman/listinfo/lilypond-user

--
Diese Nachricht wurde von meinem Android-Mobiltelefon mit K-9 Mail gesendet.

_______________________________________________
lilypond-user mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/lilypond-user
Reply | Threaded
Open this post in threaded view
|

RE: lyp - a Package Manager for Lilypond

mskala
On Fri, 29 Jan 2016, Urs Liska wrote:
> I think one of the nice things in Sharon's approach is that you don't
> *have* to contribute - you can also simply make your packages available.

...as hosted Git repositories.  I thought that was a dealbreaker, but I
tried to give it a fair chance.  I read the readme as far as the line
about piping the output of curl into bash.  I stopped there.

--
Matthew Skala
[hidden email]                 People before principles.
http://ansuz.sooke.bc.ca/

_______________________________________________
lilypond-user mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/lilypond-user
Reply | Threaded
Open this post in threaded view
|

Re: lyp - a Package Manager for Lilypond

Johan Vromans
On Fri, 29 Jan 2016 01:08:53 -0600 (CST)
[hidden email] wrote:

> ...as hosted Git repositories.  I thought that was a dealbreaker, but I
> tried to give it a fair chance.  I read the readme as far as the line
> about piping the output of curl into bash.  I stopped there.

Just replace

  curl -sSL https://git.io/getlyp | bash

with

  curl -sSL https://git.io/getlyp > tmp.sh
  less tmp.sh    # visual inspection
  bash tmp.sh

and read on.

-- Johan

_______________________________________________
lilypond-user mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/lilypond-user
Reply | Threaded
Open this post in threaded view
|

RE: lyp - a Package Manager for Lilypond

Sharon Rosner
In reply to this post by mskala
> > *have* to contribute - you can also simply make your packages available.
>
>...as hosted Git repositories.  I thought that was a dealbreaker, but I
> tried to give it a fair chance.

Please explain why packages as hosted git repositories is a bad idea. What would be a better solution in your opinion?

> I read the readme as far as the line about piping the output of curl into bash.  I stopped there.

If you actually *had* read up to that point in the readme, you will have noticed that:

1. Nobody's forcing you to install lyp that way. You can also install it as a Ruby gem, and that's the first option given. The sole reason this install script exists is to allow people who don't have Ruby installed on their machine to use lyp.
2. The paragraph explaining how to install lyp as a standalone package actually links to the install script source, which you can review before running the install command, should you have doubts about what it does.
3. You can also download the install script, inspect it (it's not obfuscated and it does nothing fancy), and then run it.
4. You can also manually download the release file, untar it, inspect its content (it's basically lyp source files + ruby binaries), then run 'lyp install self'.

Just to save you some clicks - here's the link to the source of the install script:

  https://raw.githubusercontent.com/noteflakes/lyp/master/bin/install_release.sh

And here's a list of pretty respectable software that's installed using curl|bash:

- Homebrew (Mac OSX Package manager) - pipes to ruby
- rvm (Ruby version manager)
- nvm (node.JS version manager)
- oh-my-zsh (zsh configuration framework)
- Python setuptools (Python package management) - pipes to python
- gitlab community edition (git hosting) - pipes to sudo bash!
- kubernetes (container orchestration platform from Google)
- Google Cloud SDK

In fact there's a pretty popular tumblog dedicated to curl|bash bashing ;-)
 
  http://curlpipesh.tumblr.com/

I understand that there are risks involved in this technique. But if you're concerned about a 50-line bash script from the intertubes, you should be just as concerned about the 2000 lines of Ruby code inside lyp, or for that matter any piece of code installed on your machine.

Sharon Rosner
Reply | Threaded
Open this post in threaded view
|

RE: lyp - a Package Manager for Lilypond

Sharon Rosner
In reply to this post by Mike Solomon
> Do you have a contributor's guide yet?

The documentation could be improved a lot, but the README includes a section about developing packages:

  https://github.com/noteflakes/lyp#developing-packages

Sharon Rosner
Reply | Threaded
Open this post in threaded view
|

RE: lyp - a Package Manager for Lilypond

mskala
In reply to this post by Sharon Rosner
On Fri, 29 Jan 2016, Sharon Rosner wrote:
> >...as hosted Git repositories.  I thought that was a dealbreaker, but I
> > tried to give it a fair chance.
>
> Please explain why packages as hosted git repositories is a bad idea. What
> would be a better solution in your opinion?

Version control systems in general are not well-suited for distribution of
software to users.  They significantly increase the level of skill and of
previously-installed software a user needs just to get in the door.  It's
not a good idea to require a client for a much more complicated protocol
just to serve the function adequately addressed by HTTP or FTP.  Git in
particular is an especially poor substitute for HTTP (as compared to other
version control systems) because when used as directed, it requires
transferring the entire previous history of development to anyone who
just wants a copy of the current version, and those people will
disproportionately be the ones least able to make use of the history.
Further comments are here:
   http://ansuz.sooke.bc.ca/entry/230

That article discusses individual software packages using Git for
distribution.  Building a package manager tied to Git, so that ALL
software it can install must use Git for distribution, compounds the
issues by unnecessarily imposing them on all developers.  Everybody's
required to have public Git repositories, which is a much higher bar than
requiring developers to be able to put files on HTTP servers, an approach
which has served the community well for many years (as did anonymous FTP
before that).  It's not clear what benefit Git provides over much simpler
ways of distributing files; but it has some real disadvantages.

When distributing software requires Git, there's an implicit encouragement
to use Git for the actual development too, and that unnecessarily limits
the tools and organizational structures anyeone can use when building
software that will be installed by this package manager.  Git may be
appropriate for some projects; but a built-in assumption that Git is
appropriate for every project, even in the limited space of LilyPond
development, is harder to justify.

"A public Git repository" is in practice almost the same thing as "a
public repository on Github," and Github in particular has had some recent
issues with both technical reliability and human governance.  They had a
big outage yesterday.  It's not good to put all the community's eggs in
the Github basket.  Requiring everyone who wants to build a package to
build it in a Git repository does not require them to build it on Github,
but does make it a little harder to build elsewhere.

--
Matthew Skala
[hidden email]                 People before principles.
http://ansuz.sooke.bc.ca/

_______________________________________________
lilypond-user mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/lilypond-user
Reply | Threaded
Open this post in threaded view
|

Re: lyp - a Package Manager for Lilypond

Urs Liska
Am 29.01.2016 um 14:43 schrieb [hidden email]:

> On Fri, 29 Jan 2016, Sharon Rosner wrote:
>>> ...as hosted Git repositories.  I thought that was a dealbreaker, but I
>>> tried to give it a fair chance.
>>
>> Please explain why packages as hosted git repositories is a bad idea. What
>> would be a better solution in your opinion?
>
> Version control systems in general are not well-suited for distribution of
> software to users.  They significantly increase the level of skill and of
> previously-installed software a user needs just to get in the door.  It's
> not a good idea to require a client for a much more complicated protocol
> just to serve the function adequately addressed by HTTP or FTP.  Git in
> particular is an especially poor substitute for HTTP (as compared to other
> version control systems) because when used as directed, it requires
> transferring the entire previous history of development to anyone who
> just wants a copy of the current version, and those people will
> disproportionately be the ones least able to make use of the history.

I can't comment in detail, but the idea behind Sharon's approach is that
such Git based packages *do* contain a history that's useful for
everybody, namely all *versions* of the package.
lyp will fetch the repository and can then checkout arbitrary versions
(that it may need for the dependency resolution Sharon is writing
about). LilyPond itself won't load the packages from the repository but
from the checked out directories for individual versions.

With regard to everything else I think nothing speaks against
*extending* lyp to support arbitrary directories as well. The Git
approach isn't absolutely crucial to lyp's internal working (IIUC) but
is the foundation for a number of its features. I'm sure there will be a
suitable way to support other types of packages, with less features.

Urs

> Further comments are here:
>    http://ansuz.sooke.bc.ca/entry/230
>
> That article discusses individual software packages using Git for
> distribution.  Building a package manager tied to Git, so that ALL
> software it can install must use Git for distribution, compounds the
> issues by unnecessarily imposing them on all developers.  Everybody's
> required to have public Git repositories, which is a much higher bar than
> requiring developers to be able to put files on HTTP servers, an approach
> which has served the community well for many years (as did anonymous FTP
> before that).  It's not clear what benefit Git provides over much simpler
> ways of distributing files; but it has some real disadvantages.
>
> When distributing software requires Git, there's an implicit encouragement
> to use Git for the actual development too, and that unnecessarily limits
> the tools and organizational structures anyeone can use when building
> software that will be installed by this package manager.  Git may be
> appropriate for some projects; but a built-in assumption that Git is
> appropriate for every project, even in the limited space of LilyPond
> development, is harder to justify.
>
> "A public Git repository" is in practice almost the same thing as "a
> public repository on Github," and Github in particular has had some recent
> issues with both technical reliability and human governance.  They had a
> big outage yesterday.  It's not good to put all the community's eggs in
> the Github basket.  Requiring everyone who wants to build a package to
> build it in a Git repository does not require them to build it on Github,
> but does make it a little harder to build elsewhere.
>


--
Urs Liska
www.openlilylib.org

_______________________________________________
lilypond-user mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/lilypond-user
Reply | Threaded
Open this post in threaded view
|

RE: lyp - a Package Manager for Lilypond

Sharon Rosner
In reply to this post by mskala
> Version control systems in general are not well-suited for distribution of
> software to users.  They significantly increase the level of skill and of
> previously-installed software a user needs just to get in the door.  It's
> not a good idea to require a client for a much more complicated protocol
> just to serve the function adequately addressed by HTTP or FTP.  Git in
> particular is an especially poor substitute for HTTP (as compared to other
> version control systems) because when used as directed, it requires
> transferring the entire previous history of development to anyone who
> just wants a copy of the current version, and those people will
> disproportionately be the ones least able to make use of the history.

Please note that installing packages using lyp does not require you to manually clone git repositories. In fact, to install packages using you don't even have to install git on your machine.

As Urs has already written, having the history available means packages can be versioned by using git tags.

> That article discusses individual software packages using Git for
> distribution.  Building a package manager tied to Git, so that ALL
> software it can install must use Git for distribution, compounds the
> issues by unnecessarily imposing them on all developers.

Isn't lilypond using a git repository? You're not suggesting we go back to HTTP/FTP for developing lilypond?

> Everybody's
> required to have public Git repositories, which is a much higher bar than
> requiring developers to be able to put files on HTTP servers

There are websites offering free git hosting, just like there are web sites offering free web hosting. If you're not inclined to use github et al, you can roll your own ala gitlab, but how is that any harder than rolling your own apache/nginx server?

> When distributing software requires Git, there's an implicit encouragement
> to use Git for the actual development too, and that unnecessarily limits
> the tools and organizational structures anyeone can use when building
> software that will be installed by this package manager.

Yes, publishing packages for lyp requires using git for development too, but lyp also supports installing packages from local files (more on that in a separate msg).

Please explain how does using git limit "the tools and organizational structures" one can use?

> "A public Git repository" is in practice almost the same thing as "a
> public repository on Github," and Github in particular has had some recent
> issues with both technical reliability and human governance.  They had a
> big outage yesterday.  It's not good to put all the community's eggs in
> the Github basket.  Requiring everyone who wants to build a package to
> build it in a Git repository does not require them to build it on Github,
> but does make it a little harder to build elsewhere.

Yes, there's not much competition in the git hosting space, and github is the clear dominator, but people looking for alternatives will find them.

Every website will have outages. If I'm not mistaken lilypond.org also goes down every once in a while.

> It's not clear what benefit Git provides over much simpler
> ways of distributing files; but it has some real disadvantages.

... and ...

> but a built-in assumption that Git is
> appropriate for every project, even in the limited space of LilyPond
> development, is harder to justify.

You din't reply to my question regarding an alternative to git, and perhaps I should have gone into more detail as to why using git for distribution was chosen. Urs, Matteo and me finally settled on this solution because it solves a lot of problems related to package management. When you design a packaging system, you need to answer (at least) the following questions:

- Where are packages stored?
- How are packages versioned?
- How are packages referenced?
- How are packages published?

So we discussed a few alternatives:

1. Piggyback on top of an existing packaging system, like Rubygems or npm.js. Which is not necessarily a bad idea, but can add a certain level of complexity.
2. Write a package hosting website from scratch, with the ability to upload/download packages, a package registry, user management, and of course find a machine to run it on.
3. Git-based package distribution - which solves hosting (on any git hosting site or on your own machine), publishing and package references (any git URL), and versioning (using git tags).

Considering the alternatives I think this solution we came up with is clearly the winner. If you can think of a better one please let me know.

Sharon Rosner
Reply | Threaded
Open this post in threaded view
|

Re: lyp - a Package Manager for Lilypond

Sharon Rosner
In reply to this post by Urs Liska
> With regard to everything else I think nothing speaks against
> *extending* lyp to support arbitrary directories as well.

lyp already supports installing packages from local files.

  lyp install mypack:~/mypack-repo

This feature can be used also for package development.

> The Git
> approach isn't absolutely crucial to lyp's internal working (IIUC) but
> is the foundation for a number of its features. I'm sure there will be a
> suitable way to support other types of packages, with less features.

Git is only used for the process of installing packages: lyp clones the git repository, checks out the correct version, then copies the files to ~/.lyp/packages. As I wrote in the other message, lyp doesn't even require having git installed on your machine. It uses gitlib2 with ruby bindings.

Sharon Rosner



Reply | Threaded
Open this post in threaded view
|

Re: lyp - a Package Manager for Lilypond

Urs Liska
In reply to this post by Sharon Rosner
Am 29.01.2016 um 16:38 schrieb Sharon Rosner:
> Considering the alternatives I think this solution we came up with is
> clearly the winner. If you can think of a better one please let me know.

One more comment which may help avoid any unnecessary boiling up of
discussion:

I'm not sure if I 100% agree with all of Sharon's design decisions, and
if we had had more discussion I might have tried to convince him of a
few things (starting with the choice of language).
But: it was him who actually started to work on this. And without this
we wouldn't even have anything to nitpick about.

Best
Urs


--
Urs Liska
www.openlilylib.org

_______________________________________________
lilypond-user mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/lilypond-user
Reply | Threaded
Open this post in threaded view
|

Re: lyp - a Package Manager for Lilypond

Johan Vromans
On Fri, 29 Jan 2016 16:49:36 +0100
Urs Liska <[hidden email]> wrote:

> But: it was him who actually started to work on this. And without this
> we wouldn't even have anything to nitpick about.

Exactly.

_______________________________________________
lilypond-user mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/lilypond-user
Reply | Threaded
Open this post in threaded view
|

Re: lyp - a Package Manager for Lilypond

Caio Barros
In reply to this post by Sharon Rosner


On 28-01-2016 19:52, Sharon Rosner wrote:
> Hello all,
>
> I'd like to announce lyp - a package manager for Lilypond:
>
>     https://github.com/noteflakes/lyp <https://github.com/noteflakes/lyp>
>
Great work, Sharon! I'm very excited!

_______________________________________________
lilypond-user mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/lilypond-user