Extracting symbol location

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

Extracting symbol location

Lukas Tuggener
Dear All

To create labeled training data for a machine learning project, I need to extract the location at which symbols are printed. Is it possible to extract coordinates for every symbol? Methods which reveal the position of one symbol at a time would also already help a lot (e.g. control which symbol will be printed at the exact center)

I am aware that similar questions have been asked have been asked in the past. But those are pretty old, so I am hoping that there has been some development in this area.  (http://lists.gnu.org/archive/html/lilypond-user/2007-05/msg00142.html)

Thank you very much.
Regards
Lukas

_______________________________________________
lilypond-user mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/lilypond-user
Reply | Threaded
Open this post in threaded view
|

Re: Extracting symbol location

David Nalesnik
Hi Lukas,

On Fri, Sep 23, 2016 at 5:06 AM, Lukas Tuggener <[hidden email]> wrote:

> Dear All
>
> To create labeled training data for a machine learning project, I need to
> extract the location at which symbols are printed. Is it possible to extract
> coordinates for every symbol? Methods which reveal the position of one
> symbol at a time would also already help a lot (e.g. control which symbol
> will be printed at the exact center)
>
> I am aware that similar questions have been asked have been asked in the
> past. But those are pretty old, so I am hoping that there has been some
> development in this area.
> (http://lists.gnu.org/archive/html/lilypond-user/2007-05/msg00142.html)
>
The link you cite mentions looking at SVG output, and I suppose that
would be the best way.  I can't think of another method to get the
actual print positions of objects.

I can't help with processing the SVG output to get the information you
want (a bounding box for each element?), but the attached file might
help with preparing an interpretable file.

Objects aren't assigned a unique id by default.  The attached tries to
do this.  Hopefully, the information included (grob name, moment,
horizontal and vertical coordinates) is sufficient.  The coordinates
used in the ID aren't the print position -- they are relative to the
system on which the grob is found, for one thing.  I include them to
distinguish between the note heads in a chord, which obviously occur
at the same time (moment).

There's a patch being reviewed which replaces the 'id property with
'output-properties and this will need to be updated accordingly
(https://sourceforge.net/p/testlilyissues/issues/4974/).

Hope this helps,
David

_______________________________________________
lilypond-user mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/lilypond-user

svg-test.ly (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Extracting symbol location

Lukas Tuggener
Hi David

Many thanks, this will help me greatly, when parsing the .svg files. Upcoming patches will not be an issue, because i will have to run this just once.

Thanks again and have a good day
Lukas


David Nalesnik <[hidden email]> schrieb am Mo., 26. Sep. 2016 um 20:13 Uhr:
Hi Lukas,

On Fri, Sep 23, 2016 at 5:06 AM, Lukas Tuggener <[hidden email]> wrote:
> Dear All
>
> To create labeled training data for a machine learning project, I need to
> extract the location at which symbols are printed. Is it possible to extract
> coordinates for every symbol? Methods which reveal the position of one
> symbol at a time would also already help a lot (e.g. control which symbol
> will be printed at the exact center)
>
> I am aware that similar questions have been asked have been asked in the
> past. But those are pretty old, so I am hoping that there has been some
> development in this area.
> (http://lists.gnu.org/archive/html/lilypond-user/2007-05/msg00142.html)
>

The link you cite mentions looking at SVG output, and I suppose that
would be the best way.  I can't think of another method to get the
actual print positions of objects.

I can't help with processing the SVG output to get the information you
want (a bounding box for each element?), but the attached file might
help with preparing an interpretable file.

Objects aren't assigned a unique id by default.  The attached tries to
do this.  Hopefully, the information included (grob name, moment,
horizontal and vertical coordinates) is sufficient.  The coordinates
used in the ID aren't the print position -- they are relative to the
system on which the grob is found, for one thing.  I include them to
distinguish between the note heads in a chord, which obviously occur
at the same time (moment).

There's a patch being reviewed which replaces the 'id property with
'output-properties and this will need to be updated accordingly
(https://sourceforge.net/p/testlilyissues/issues/4974/).

Hope this helps,
David

_______________________________________________
lilypond-user mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/lilypond-user
Reply | Threaded
Open this post in threaded view
|

Re: Extracting symbol location

Urs Liska
In reply to this post by David Nalesnik


Am 26.09.2016 um 20:13 schrieb David Nalesnik:
> The link you cite mentions looking at SVG output, and I suppose that
> would be the best way.  I can't think of another method to get the
> actual print positions of objects.
>

Hm, but at some point LilyPond *has* to know where objects are placed,
isn't it?
Isn't there anything like a parent hierarchy that could be walked up
like a breadcrumb navigation that eventually leads to coordinates
relative to a page corner?

(Of course this is closely related to my question
http://lists.gnu.org/archive/html/lilypond-user/2016-09/msg00655.html)

Urs

_______________________________________________
lilypond-user mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/lilypond-user
Reply | Threaded
Open this post in threaded view
|

Re: Extracting symbol location

Lukas Tuggener
Tanks to this \assignIDs function and the information in the svg its pretty easy to find the correct location of a given symbol. 

The problem im stuck with right now is creating meaningful labels. The way the ID is constructed right now is Name+Momentum?+RelativeLocation. And i am not able to tell eg. bass and treble clefs apart. 

Is it possible to dump more information about the grob this id tag? Such that it is unique to what symbol is printed. 

I am totally new to lilypond, sorry if I spam you with trivial questions. If you could point me to the relevant parts of the doc it would already help.

cheers
Lukas

Urs Liska <[hidden email]> schrieb am Di., 27. Sep. 2016 um 10:59 Uhr:


Am 26.09.2016 um 20:13 schrieb David Nalesnik:
> The link you cite mentions looking at SVG output, and I suppose that
> would be the best way.  I can't think of another method to get the
> actual print positions of objects.
>

Hm, but at some point LilyPond *has* to know where objects are placed,
isn't it?
Isn't there anything like a parent hierarchy that could be walked up
like a breadcrumb navigation that eventually leads to coordinates
relative to a page corner?

(Of course this is closely related to my question
http://lists.gnu.org/archive/html/lilypond-user/2016-09/msg00655.html)

Urs

_______________________________________________
lilypond-user mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/lilypond-user

_______________________________________________
lilypond-user mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/lilypond-user

test-clefs-svg.ly (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Extracting symbol location

David Nalesnik
Hi Lukas,

On Tue, Sep 27, 2016 at 8:21 AM, Lukas Tuggener <[hidden email]> wrote:
> Tanks to this \assignIDs function and the information in the svg its pretty
> easy to find the correct location of a given symbol.
>
> The problem im stuck with right now is creating meaningful labels. The way
> the ID is constructed right now is Name+Momentum?+RelativeLocation.

moment = fraction of a whole note

The moment I've included gives the absolute rhythmic position in the score.

> And i am not able to tell eg. bass and treble clefs apart.
> Is it possible to dump more information about the grob this id tag? Such
> that it is unique to what symbol is printed.

Yes, you have access to a number of properties.  In the attachment I
use ly:grob-properties to get at these.

Note that to display the clef type in a nice way, I had to create a
lookup list based on two properties: 'glyph-name and 'staff-position.
There's no property that returns "alto," "treble," etc.  If you're
wanting to display "whole note," "half note," you'll have to do
something similarly ugly.

>
> I am totally new to lilypond, sorry if I spam you with trivial questions. If
> you could point me to the relevant parts of the doc it would already help.
>

There's no tutorial-type documentation for this stuff.  The best I can
do is refer you to the Internals Reference
(http://lilypond.org/doc/v2.19/Documentation/internals/index.html) and
to the lists when you get stuck...

HTH,

David

_______________________________________________
lilypond-user mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/lilypond-user
Reply | Threaded
Open this post in threaded view
|

Re: Extracting symbol location

David Nalesnik
On Tue, Sep 27, 2016 at 10:33 AM, David Nalesnik
<[hidden email]> wrote:

>  In the attachment...

And now for the file...

David

_______________________________________________
lilypond-user mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/lilypond-user

test-clefs-svg.ly (2K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Extracting symbol location

Lukas Tuggener
Hey David,

thank you so much! I am fairly sure that i'll be able to extend this file in a similar fashion to cope with other types of simbols...

regards
Lukas

David Nalesnik <[hidden email]> schrieb am Di., 27. Sep. 2016 um 17:35 Uhr:
On Tue, Sep 27, 2016 at 10:33 AM, David Nalesnik
<[hidden email]> wrote:

>  In the attachment...

And now for the file...

David

_______________________________________________
lilypond-user mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/lilypond-user
Reply | Threaded
Open this post in threaded view
|

Re: Extracting symbol location

David Nalesnik
In reply to this post by Urs Liska
Hi Urs,

On Tue, Sep 27, 2016 at 3:58 AM, Urs Liska <[hidden email]> wrote:

>
>
> Am 26.09.2016 um 20:13 schrieb David Nalesnik:
>> The link you cite mentions looking at SVG output, and I suppose that
>> would be the best way.  I can't think of another method to get the
>> actual print positions of objects.
>>
>
> Hm, but at some point LilyPond *has* to know where objects are placed,
> isn't it?
> Isn't there anything like a parent hierarchy that could be walked up
> like a breadcrumb navigation that eventually leads to coordinates
> relative to a page corner?

Don't know how helpful this is, but I see the command-line option
-dbackend=scm which dumps the stencil output of a file as a Scheme
expression.  It includes entries for "grob-cause".

I've tried to create a function which takes a score argument and
produces a corresponding stencil expression to work with, but I'm
having no luck.

It would be nice to do something like this:

 \version "2.19.46"

#(use-modules (scm page))

#(define (of paper-book)
   (for-each
    (lambda (page)
      (display (ly:stencil-expr page))
      (newline))
    (map page-stencil (ly:paper-book-pages paper-book))))

but I can't get the Paper_book object needed.

ly:book-process does create a Paper_book, but it isn't returned...

>
> (Of course this is closely related to my question
> http://lists.gnu.org/archive/html/lilypond-user/2016-09/msg00655.html)
>

Seems like a tall order, but who knows!

David

_______________________________________________
lilypond-user mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/lilypond-user
Reply | Threaded
Open this post in threaded view
|

Re: Extracting symbol location

Lukas Tuggener
Hi David

When I tried to use your very useful script on a fresh installation it didnt work anymore. I suspect it is due to the change you already mentioned in your first message:

There's a patch being reviewed which replaces the 'id property with
'output-properties and this will need to be updated accordingly
(https://sourceforge.net/p/testlilyissues/issues/4974/).



I tried to uptdate the script, but didnt get it to work so far. I assume the updated assignIDs function should look something like this:

assignIDs = #(let ((grob-names (map car all-grob-descriptions)))
   #{
     #@(map (lambda (x)#{ \override #(list 'Score x (assoc-get 'id  output-attributes) = #get-unique-id #})
     grob-names)
   #})

For any hint where I went wrong i would be very grateful.

Regards
Lukas

David Nalesnik <[hidden email]> schrieb am Di., 27. Sep. 2016 um 22:33 Uhr:
Hi Urs,

On Tue, Sep 27, 2016 at 3:58 AM, Urs Liska <[hidden email]> wrote:
>
>
> Am 26.09.2016 um 20:13 schrieb David Nalesnik:
>> The link you cite mentions looking at SVG output, and I suppose that
>> would be the best way.  I can't think of another method to get the
>> actual print positions of objects.
>>
>
> Hm, but at some point LilyPond *has* to know where objects are placed,
> isn't it?
> Isn't there anything like a parent hierarchy that could be walked up
> like a breadcrumb navigation that eventually leads to coordinates
> relative to a page corner?

Don't know how helpful this is, but I see the command-line option
-dbackend=scm which dumps the stencil output of a file as a Scheme
expression.  It includes entries for "grob-cause".

I've tried to create a function which takes a score argument and
produces a corresponding stencil expression to work with, but I'm
having no luck.

It would be nice to do something like this:

 \version "2.19.46"

#(use-modules (scm page))

#(define (of paper-book)
   (for-each
    (lambda (page)
      (display (ly:stencil-expr page))
      (newline))
    (map page-stencil (ly:paper-book-pages paper-book))))

but I can't get the Paper_book object needed.

ly:book-process does create a Paper_book, but it isn't returned...

>
> (Of course this is closely related to my question
> http://lists.gnu.org/archive/html/lilypond-user/2016-09/msg00655.html)
>

Seems like a tall order, but who knows!

David

_______________________________________________
lilypond-user mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/lilypond-user

_______________________________________________
lilypond-user mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/lilypond-user
Reply | Threaded
Open this post in threaded view
|

Re: Extracting symbol location

David Nalesnik
On Mon, Jan 9, 2017 at 4:55 AM, Lukas Tuggener <[hidden email]> wrote:

> Hi David
>
> When I tried to use your very useful script on a fresh installation it didnt
> work anymore. I suspect it is due to the change you already mentioned in
> your first message:
>
> There's a patch being reviewed which replaces the 'id property with
> 'output-properties and this will need to be updated accordingly
> (https://sourceforge.net/p/testlilyissues/issues/4974/).
>
>
> I tried to uptdate the script, but didnt get it to work so far. I assume the
> updated assignIDs function should look something like this:
>
> assignIDs = #(let ((grob-names (map car all-grob-descriptions)))
>    #{
>      #@(map (lambda (x)#{ \override #(list 'Score x (assoc-get 'id
> output-attributes) = #get-unique-id #})
>      grob-names)
>    #})
>
> For any hint where I went wrong i would be very grateful.

Before I give a solution, two observations:

(1) you're missing a close parenthesis before the equals.

(2) there is no variable called "output-attributes" which you can
treat as an alist by applying assoc-get: it is a symbol (needing a
single-quote to reference).

The \override command we would model to change an "id" property nested
in an "output-attributes" property is:

\override Score.someGrob.output-attributes.id = #get-unique-id

and that translates to this definition of assignIDs:

assignIDs =
#(let ((grob-names (map car all-grob-descriptions)))
   #{
     #@(map (lambda (x) #{
       \override #(list 'Score x 'output-attributes 'id) = #get-unique-id
       #})
     grob-names)
   #})

%%%

The new output-attibutes property allows you to create multiple
attributes, so why not adapt the labeling routines to return something
like

value of output-attributes = '((name . someGrob) (moment . whenIAm)
(x-pos . xCoordinate) (y-pos . yCoordinate))

Would this be something you could use in your post-processing?

HTH,
David

_______________________________________________
lilypond-user mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/lilypond-user
Reply | Threaded
Open this post in threaded view
|

Re: Extracting symbol location

David Nalesnik
Lukas,

On Mon, Jan 9, 2017 at 8:43 AM, David Nalesnik <[hidden email]> wrote:

> The new output-attibutes property allows you to create multiple
> attributes, so why not adapt the labeling routines to return something
> like
>
> value of output-attributes = '((name . someGrob) (moment . whenIAm)
> (x-pos . xCoordinate) (y-pos . yCoordinate))

A way to do this is attached.

An excerpt from the SVG:

<g name="StaffSymbol" when="[Mom 0]" x-pos="8.53582677165354" y-pos="-7.552">

-David

_______________________________________________
lilypond-user mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/lilypond-user

svg-test2.ly (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Extracting symbol location

Lukas Tuggener
Hi David,
thanks a lot for the update, it really helped me out greatly.

Thanks
Lukas

David Nalesnik <[hidden email]> schrieb am Mo., 9. Jan. 2017 um 16:11 Uhr:
Lukas,

On Mon, Jan 9, 2017 at 8:43 AM, David Nalesnik <[hidden email]> wrote:

> The new output-attibutes property allows you to create multiple
> attributes, so why not adapt the labeling routines to return something
> like
>
> value of output-attributes = '((name . someGrob) (moment . whenIAm)
> (x-pos . xCoordinate) (y-pos . yCoordinate))

A way to do this is attached.

An excerpt from the SVG:

<g name="StaffSymbol" when="[Mom 0]" x-pos="8.53582677165354" y-pos="-7.552">

-David

_______________________________________________
lilypond-user mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/lilypond-user
Reply | Threaded
Open this post in threaded view
|

Re: Extracting symbol location

Lukas Tuggener
Dear David, Dear lilypond users

I have used an adapted version (see grace.ly) of the script David gave me (see test-clefs-svg.ly) to extract the exact location of various symbols. I recently tried to add support for grace notes. 

The problem is, when i look at the output of (ly:grob-properties grob) i see no reliable difference between a normal notehead and a notehead of a grace note. The only thing is the size of the notehead and i assume using this will be very unreliable due to different scalings.

Does anyone have a idea how to work around that?

Thanks a lot, best
Lukas


Output:

Normal Note:

<g id="NoteHead_((id . calculation-in-progress) (extra-spacing-height 0.0 . 0.954996) (stem-attachment 1.0 . 0.341476392833814) (Y-extent -0.545004 . 0.545004) (font . #<Font_metric ("emmentaler-20" . 0.569055118110236)>) (duration-log . 2) (stencil . #<Stencil>) (staff-position . -9) (cause . #<Prob: Stream_event C++: Stream_event((music-cause . #<Prob: Music C++: Music((length . #<Mom 1/16>) (duration . #<Duration 16 >) (pitch . #<Pitch g >) (origin . #<location mordent.ly:114:20>))((display-methods #<procedure #f (note)>) (name . NoteEvent) (iterator-ctor . #<primitive-procedure ly:rhythmic-music-iterator::constructor>) (types event note-event rhythmic-event melodic-event)) >
) (length . #<Mom 1/16>))((class note-event melodic-event rhythmic-event music-event StreamEvent) (length . #<Mom 1/16>) (duration . #<Duration 16 >) (pitch . #<Pitch g >) (origin . #<location mordent.ly:114:20>)) >
) (horizontal-skylines . #<unpure-pure-container #<primitive-procedure ly:grob::simple-horizontal-skylines-from-extents> #<primitive-procedure ly:grob::pure-simple-horizontal-skylines-from-extents> >) (vertical-skylines . #<Skyline_pair>) (X-extent -0.0 . 1.304212))[Duration 16 ]">
<a style="color:inherit;" xlink:href="textedit:///Users/tugg/Documents/DeepScore/Data_Mngment_New/TrainingDataProcessing/render_svg/lily_experiment/mordent.ly:114:19:20">
<path transform="translate(32.6651, 12.1213) scale(0.0040, -0.0040)" d="M218 136c55 0 108 -28 108 -89c0 -71 -55 -121 -102 -149c-35 -21 -75 -34 -116 -34c-55 0 -108 28 -108 89c0 71 55 121 102 149c35 21 75 34 116 34z" fill="currentColor"/>
</a>
</g>



Small Note:

<g id="NoteHead_((id . calculation-in-progress) (extra-spacing-height 0.0 . 0.605960495541901) (stem-attachment 1.0 . 0.290370069546677) (Y-extent -0.394039504458099 . 0.394039504458099) (font . #<Font_metric ("emmentaler-14" . 0.569141064900509)>) (duration-log . 2) (stencil . #<Stencil>) (staff-position . -8) (cause . #<Prob: Stream_event C++: Stream_event((music-cause . #<Prob: Music C++: Music((length . #<Mom 1/8>) (duration . #<Duration 8 >) (pitch . #<Pitch a >) (origin . #<location mordent.ly:114:17>))((display-methods #<procedure #f (note)>) (name . NoteEvent) (iterator-ctor . #<primitive-procedure ly:rhythmic-music-iterator::constructor>) (types event note-event rhythmic-event melodic-event)) >
) (length . #<Mom 1/8>))((class note-event melodic-event rhythmic-event music-event StreamEvent) (length . #<Mom 1/8>) (duration . #<Duration 8 >) (pitch . #<Pitch a >) (origin . #<location mordent.ly:114:17>)) >
) (horizontal-skylines . #<unpure-pure-container #<primitive-procedure ly:grob::simple-horizontal-skylines-from-extents> #<primitive-procedure ly:grob::pure-simple-horizontal-skylines-from-extents> >) (vertical-skylines . #<Skyline_pair>) (X-extent -0.0 . 0.917888611646328))[Duration 8 ]">
<a style="color:inherit;" xlink:href="textedit:///Users/tugg/Documents/DeepScore/Data_Mngment_New/TrainingDataProcessing/render_svg/lily_experiment/mordent.ly:114:16:17">
<path transform="translate(30.6680, 11.6213) scale(0.0028, -0.0028)" d="M208 139c61 0 117 -33 117 -99c0 -71 -52 -119 -99 -147c-34 -20 -71 -32 -110 -32c-61 0 -116 33 -116 99c0 71 51 119 98 147c34 20 71 32 110 32z" fill="currentColor"/>
</a>
</g>

Lukas Tuggener <[hidden email]> schrieb am Mo., 16. Jan. 2017 um 15:36 Uhr:
Hi David,
thanks a lot for the update, it really helped me out greatly.

Thanks
Lukas

David Nalesnik <[hidden email]> schrieb am Mo., 9. Jan. 2017 um 16:11 Uhr:
Lukas,

On Mon, Jan 9, 2017 at 8:43 AM, David Nalesnik <[hidden email]> wrote:

> The new output-attibutes property allows you to create multiple
> attributes, so why not adapt the labeling routines to return something
> like
>
> value of output-attributes = '((name . someGrob) (moment . whenIAm)
> (x-pos . xCoordinate) (y-pos . yCoordinate))

A way to do this is attached.

An excerpt from the SVG:

<g name="StaffSymbol" when="[Mom 0]" x-pos="8.53582677165354" y-pos="-7.552">

-David

_______________________________________________
lilypond-user mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/lilypond-user

test-clefs-svg.ly (1K) Download Attachment
grace.ly (6K) Download Attachment