Introduction to extending

Inserting Scheme inside LilyPond

To start a Scheme expression in a LilyPond file, prefix it with a hash character. The expression is evaluated, and its value is returned to LilyPond.

\version "2.25.8"

myVar = #(+ 2 2) % myVar takes the value 2 + 2 = 4

\repeat unfold #myVar { c } % This note, 4 times.

Output

This example also shows that LilyPond-defined variables are accessible from Scheme. The reverse holds too:

\version "2.25.8"

#(define mySecondVar 42)

\repeat unfold \mySecondVar { c }

Output

The #@ operators inserts all elements from a Scheme list in the LilyPond context. This is called list splicing.

Inside Scheme, one can even switch back to LilyPond syntax by enclosing a piece of LilyPond code inside #{ and #}.

Here is an example using both list splicing and embedded LilyPond:

\version "2.25.8"

notes = #(list #{ c'4 #}
               #{ e'4 #}
               #{ g'4 #}
               #{ c''4 #})

{
  % All notes from the list in a sequential expression.
  #@notes
  % Same, in simultaneous music.
  << #@notes >>
}

Output

Exported functions and naming

Two kinds of functions are available inside LilyPond. Guile defines a large number of general-purpose data utilities to process strings, lists, etc. And LilyPond itself adds a layer on top by providing functions manipulating its own object types like music, contexts or markups.

Many functions exported by LilyPond have their names start with the ly: prefix, to distinguish them from native Scheme functions, for example: ly:music-length. This is done systematically for functions exported from C++. Additionally, some parts of LilyPond’s Scheme code export functions named with leading ly:. Some others do not, however. Hopefully consistency could be brought in the future.

The Scheme sandbox

To experiment with Scheme in the context of LilyPond, run the terminal command:

lilypond scheme-sandbox

This opens an interactive Guile prompt where all functions exported by LilyPond are available.

Music function primer

Many of the building blocks encountered in LilyPond input are music functions: \relative, \transpose, \bar, etc. You can write your own, too. They are declared using define-music-function, and a type predicate has to be picked for every argument.

\version "2.25.8"

simpleAccompaniment =
#(define-music-function (note1 note2 note3)
                        (ly:music? ly:music? ly:music?)
   #{
      \repeat unfold 2 {
        #note1 #note2 #note3 #note2 #note3 #note2
      }
   #})

{
  \time 6/4
  \simpleAccompaniment c'8 g' e''
  \simpleAccompaniment c'8 g' f''
  \simpleAccompaniment d'8 g' f''
  \simpleAccompaniment c'8 g' e''
}

Output

Here are the most common type predicates:

Predicate name	Accepted values
number?	Any numeric value: integer, floating point number, fraction
integer?	An integer
index?	A non-negative integer
string?	A string (sequence of characters, like "Some words")
markup?	A \markup block; this also accepts strings
boolean?	A boolean, true (#t) or false (#f)
ly:music?	A music expression
ly:pitch?	A pitch (like the first two arguments to \transpose c des { ... })
ly:duration?	A duration (like the optional argument to \tuplet: \tuplet 3/2 8. { ... })
color?	A color, given as string or RGB list (documentation)
list?	A list of any elements

Callback primer

Grobs, short for “Graphical Objects”, are pieces of notation – note heads, stems, rests, etc. They are familiar to LilyPonders, who are used to the syntax of \override to modify grob properties.

Instead of setting the value in stone, any grob property can be set to a callback, a function that calculates the property on-the-fly. The function takes the grob as its only argument. Other properties are retrieved using ly:grob-property. Here is a function that draws feathered beams in the direction where they go up: right is an ascending beam, left if a descending beam. The notational effect is that the score indicates to accelerate whenever the melody goes up – probably to the distaste of a music teacher.

\version "2.25.8"

#(define (grow-in-up-direction beam)
   (let* ((Y-positions (ly:grob-property beam 'positions))
          (left-position (car Y-positions))
          (right-position (cdr Y-positions)))
     (cond
       ((< left-position right-position)
        RIGHT)
       ((> left-position right-position)
        LEFT)
       (else
        CENTER))))

\relative c' {
  \override Beam.grow-direction = #grow-in-up-direction
  c'16 d e f g f e d c g c g c e g c
}

Output

Printing values

The standard Scheme procedure display is ill-suited to debugging messages in the context of LilyPond, because it prints on the standard output stream stdout, while LilyPond’s own logging messages go to stderr, which can cause them to appear intermingled. It is handier to use ly:message. Unlike display, its first argument must be a string. Extra arguments are used to format the message according to the standard format specifiers, as with the format function. Another difference is that ly:message prints a new line automatically.

For example, here is how you might print the content of the Y-positions variable in the callback example above:

(ly:message "Y-positions are: ~a" Y-positions)

The most important format specifiers are ~a and ~s. The former prints values in the prettiest way like display, while the latter tries to make them loadable like write. In the case of strings, this means that ~s prints quotes, while ~a does not.

The ~y formatter attempts to print nested data structures in a readable way. It is only available with the standard procedure format:

\version "2.25.8"

{
  \override Slur.after-line-breaking =
    #(lambda (grob)
       (ly:message
         (format #f "~y" (ly:grob-property grob 'control-points))))
  b'1( b'')
}

Output

This example outputs:

((0.732364943841144 . 1.195004)
 (1.76258769418946 . 3.05939544438528)
 (6.83883925432087 . 5.14594080151585)
 (8.88241194297576 . 4.545004))

First steps with the internals

Smobs

“Smob” means “Scheme Object”. Smob types are exported from LilyPond’s C++ core to Scheme, along with functions to operate on them. For example, pitches are Smobs. As with all Smob types, a predicate is provided for them: ly:pitch?. As with many Smob types, you also get a constructor for building pitches: ly:make-pitch. There are various utilities as well: ly:pitch-notename, ly:pitch-transpose, and others.

Probs

“Prob” is short for “Property Object”. In LilyPond, this term encompasses a wide range of objects that hold properties. All Probs are smobs, but the contrary is not true. Pitches and durations, for example, are not Probs – they have no concept of properties. On the other hand, music objects or contexts are Probs.

For all common Prob types, functions are provided to retrieve and set properties. They all follow the same scheme for naming and signatures.

(ly:<xxx>-property object property [default])

Access a property in a Prob. <xxx> here stands for the prob type. There are ly:music-property, ly:event-property, ly:context-property, etc.

The property name is given as a symbol. To whet the reader’s appetite about music objects, the following example can be tested:

#(display (ly:music-property #{ c'8 #} 'pitch))

When the Prob has no such property, default is returned, if provided, or else, the empty list. Note that contrary to most standard Scheme functions, prob property accessors fall back to the empty list, not the boolean false!

(ly:<xxx>-set-property! object property value)

Set property in object to value.

This corresponds to ly:music-set-property!, ly:event-set-property!, etc.

Glossary of important object types

The following objects play a salient role in the grand scheme of LilyPond.

Output definitions

\layout, \midi and \paper blocks. They are the architects of the process from input to output. They contain settings for context defaults and page spacing as well as line breaking parameters: indent, page-breaking, system-system-spacing, \context { \Staff \override ... }.

Books, bookparts and score

\book, \bookpart and \score blocks. Books are the top-level containers, corresponding to one output file. They can contain scores and bookparts. Scores are the unit for music typesetting, they correspond to the different pieces. Bookparts are an optional intermediary level between books and scores.

These objects can contain output definitions. Specifically, \book and \bookpart blocks can contain \paper blocks while \score blocks can contain \layout and/or \midi blocks. There is an inheritance system: settings made in a \layout block inside \score overwrite those made inside the \paper of the enclosing \bookpart, those inside \bookpart overwrite those in \book, and the latter ones have priority over output definitions on the top level.

Music

Music objects are a nested, tree-like representation of the musical content described by the input. For instance, << c'8 \\ d16\p >> is a piece of simultaneous music (SimultaneousMusic) containing two music expressions. The second one is a note (NoteEvent), in turn containing a dynamic (AbsoluteDynamicEvent).

Contexts

Property containers corresponding to different parts of the score. The most common context types are Score, Staff and Voice. The hierarchy of contexts determines which translators react to a certain stream event or grob. Also, the properties contained in contexts are the way for translators to communicate with the outside, both with the user (who can modify context properties using \set), and with other translators.

Iterators

Objects that advance in the music in a certain way, establishing timing. For example, Sequential_iterator is tasked with iterating in sequential music expressions { ... } and advances step by step in its music. On the other hand, Simultaneous_music_iterator iterates simultaneous music expressions << ... >>, doing a “crab-walk” to advance in several music expressions in parallel, in a synchronized fashion.

Events

While certain music objects are containers holding other music objects, most of them are slated to be eventually turned into “stream events”. There are events for notes, rests, dynamics, articulations, etc. An event occurs at a specific point in time. Working on a stream of events rather than music objects directly simplifies LilyPond’s processing.

Translators

Multi-faceted objects tasked with turning the events into graphical objects and setting the relationships between these objects. There are two kinds of translators: engravers, for graphical output, and performers, for outputting MIDI.

Grobs

The short name for “Graphical Objects”. They represent a piece of notation on the page: a note head (NoteHead), a stem (Stem), a dot (Dots), etc.

Callbacks

Grobs have properties. A callback allows a grob property to be computed from other properties of this grob and related grobs (e.g., the stem associated to a note head). A callback is simply a procedure taking the grob as argument, and returning the value for the property. An example is found in Callback primer.

Stencils

Inkings ready to be output. Nearly every grob has a stencil, which ends up on the page as a bit of notation. Stems produce line stencils, for instance.

Markups

Code-like objects that specify graphical elements. The main thing you can do with a markup is to “execute” it, which yields a stencil. The technical term used in LilyPond for this operation is interpreting the markup. Many grobs use markups to prepare their stencil. They may be inserted in the musical input as textual articulations, as in { c'8^\markup \italic { Cresc. poco a poco } }. They can also be placed on the top level.

Overview of LilyPond’s inner workings and how you might hook in them

Translating text input to music scores is an involved task, and LilyPond performs it according to a complex process that has several steps. In the following, each step is given with the log message LilyPond prints when it starts performing it.

Parsing

Parsing...

The first stage is lexical analysis of the input file. The lexer implements the most basic logic. Its action is to turn the file into a stream of tokens, for easier processing by the parser. Tokens are opening or closing brackets, pitches, durations, … The lexer is written in Flex.

The main parser is written in Bison, and works together with the lexer, parsing the tokens as they are produced. It understands the syntax behind the tokens, and produces various kinds of objects for processing by later steps: output definitions, books, bookparts, scores and music expressions as well as markups.

During parsing, music expressions are transformed by music functions, like \relative or \transpose. The ability to define custom music functions is the source of a large part of LilyPond’s extending powers.

Iterating

Interpreting music...

The step of iterating happens for every \score block. The purpose is to translate the stream of music events into a network of grobs. For example, a note event in the input is reflected by a NoteHead, a Stem, maybe a Beam, Dots, etc. Hence this step is also called translation.

This is where iterators, contexts and translators are created and perform their respective roles.

Custom context types can be defined in output definitions. Engravers can be written in Scheme. Iterators are C++-only.

Pure positioning

Preprocessing graphical objects...

What we get from translation is a set of interconnected objects. Their positioning is going to depend heavily on the chosen line breaking. If things were simple, the next step would be to find a line breaking configuration.

But things are not as simple, since the space that an object might take also depends on the line breaking.

LilyPond’s solution to this problem is to do a preliminary positioning pass, called the pure phase. The extents and offsets of every object are estimated, making best guesses, for use by the line breaker and the later unpure phase.

You can write your own pure functions in Scheme.

Page breaking and line breaking

Finding the ideal number of pages... and Fitting music on x pages...

Breaking algorithms try to fit the music to the page as evenly as possible. Some also try to minimize other kinds of constraints, like the page turn breaking algorithm.

This part is not Scheme-accessible.

Unpure positioning

Drawing systems...

When information about the line breaking configuration is available, all objects can be placed precisely, and their stencils (“inkings”) are built. This part may be customized at will from Scheme.

Writing output

Converting to `document.pdf'

In the end, the stencils are translated into PostScript language. Behind the scenes, this is converted by Ghostscript into a PDF file.