Posts

LV2 has been slowly but surely moving towards message-based control to overcome the limitations of static ports. The required low level facilities have been available for a while now, but the lack of a standard way to document message-based parameters and host UI support has been a sticking point. One particularly pressing need is file parameters. The current sample example has a stop-gap UI to enable sample loading, but parameters really should be host controllable and simple plugins should not need a custom UI to be usable. Today I did some work towards resolving that situation.

Here is ConvoLV2 running in Jalv:

Note this is the built-in UI generated by Jalv and not a custom plugin UI. Unlike traditional LADSPA style controls, the impulse parameter is not a port, but is set by sending a message to an event port. The plugin lists parameters with the new patch:writable property, so the host knows which controls to display (or which parameters it can automatically set). Parameters are described as normal RDF properties, so any existing property can be used, or plugins can define them as needed in their data files (don't let the "RDF" scare you, the description of a Property is as simple as LV2 ports). One nice advantage of this over ports is that a project can describe a parameter once and use it in many plugins, without having to describe the parameter every time. This really adds up when you have hundreds of plugins with, say, an identical gain parameter.

These conventions will be established in the next LV2 release (1.4.0), and the sampler example will serve as a test case. When host support becomes established, we can also move towards using messages for numeric parameters, which will finally allow for dynamic parameters, control ramps, and so on.

I have been working on the control protocol aspect of Ingen, towards a few goals:

• Working GUI communicating with the server via a socket
• Decent Python bindings for socket interface
• Ingen as an LV2 plugin

The Python bindings got fleshed out to support a script for loading AMS patches into Ingen I have been working on (very preliminary, but in the repository and installed as 'ingenams'). The socket interface is now sensibly synchronous, e.g. you send a command, and recv on the socket to get the response, which will include a status as well as whatever patch updates happened as a result of the command. I am happy with this route to language bindings, it's much more pleasant than a mess of direct bindings (e.g. via swig) and being protocol controlled has many other advantages. The Python interface is implemented purely in Python, it should be simple to do the same in any language that can use sockets.

The Ingen GUI can communicate with the engine via the same protocol, so you can build patches over the network (e.g. if Ingen is running on a headless machine as part of an audio rig). It can also run in the same process and use direct function calls for maximum performance.

The Ingen protocol uses the LV2 Patch vocabulary to describe changes. For the socket interface, this is represented entirely in Turtle. When running as an LV2 plugin, Ingen uses conceptually the same protocol, except in a binary format using LV2 atoms. This is a nice example of the benefits of having things defined in a good abstract model which can be encoded in many ways. For example, it would be relatively simple to implement control via JSON for a browser-based Ingen UI, which seems like the best way to go for separate control panels, but that's a topic for another day.

Since Ingen control can be expressed as atoms, it can run as an LV2 plugin with a full patcher UI and communicate only via LV2 ports. The UI is embeddable, and there is no memory sharing between engine and UI or hidden communication, everything is host managed. Here is a screenshot of Ingen running in Ardour 3. The MIDI in Ardour is played through an Ingen synth plugin, and the output has been recorded to the audio track. The patch can be manipulated live while running in Ardour, except for top level ports.

I have been working on getting accurate transport information provided to LV2 plugins. Until now, there was the ability to specially designate e.g. BPM control ports, but this has limitations, the most obvious being that it is is limited to conveying simple numeric values at the start of a cycle.

In general, things are headed in an event-based direction, and this is no exception. The LV2 Time extension defines all the properties needed to describe a transport position/speed/tempo/meter and the LV2 Atom extension defines a dictionary type that can be used to send several properties at once in a single event. The host sends updates whenever the transport has changed in a way the plugin can not anticipate (anything but rolling along at the same speed/tempo/meter). Since events are time stamped, this makes it possible to give plugins a sample accurate tempo map, even when there are changes in the middle of a cycle.

There is a new example plugin in the LV2 repository, eg-metro, which is a simple metronome that follows this scheme. If the host implements the above, the metronome will click in sync with the host transport.

I have implemented this in Jalv (which uses Jack transport), Ardour 3, and Ingen (via a new internal "Time" block). Ardour will send an update at the start of the cycle if a relocate has happened, and one on the exact sample of any meter or tempo change within the cycle. Due to the limitations of Jack transport, Jalv and Ingen will only send updates at the start of a cycle. The metronome should tick correctly when loaded in Jalv or Ingen when there is a Jack transport master active (I have tested with Ardour), and when loaded directly in Ardour.

This functionality is pretty important for many musical plugins, so hopefully other host authors implement it as well. Depending on feedback (and available time) I may document the above rules as a feature and add it to the time extension, and release LV2 1.2.2. I think a well-defined way to achieve this is worthy of a point release, and it will get the news out there to implementers.

A new utility has been added to Lilv for benchmarking LV2 plugins. As you might expect, it is called lv2bench.

Simple, but useful, lv2bench runs all discovered plugins for a specified number of frames split into a specified block size, and records the time each plugin took to execute. It prints a simple table like this:

This makes it easy to get a good overall feel for how expensive plugins on the system are relative to each other. Of course, a more expressive view of this information with several parameters would be useful... Robin Gareus to the rescue! Robin made a script currently called lilvplot which does several runs with various parameters and plots the data, which makes the relative performance more clear and shows the variance of a plugin's run time with error bars. This is important information because run-time variance is a good indicator of how real-time appropriate a plugin is.

Here is an example of lilvplot output generated from lv2bench data (as a massive SVG, you may want to open this image in a separate window and zoom in to see the plugin labels):

Both of these tools are very fresh and there are certainly improvements to be made (normalizing the y-axis so it showed where real-time is would be more informative, and lv2bench currently supports very few features), but they already make a useful addition to the LV2 developer's toolkit. I found them useful for evaluating the improvements had by vectorizing the arithmetic plugins in my soon-to-be-released port of blop. It will be interesting to see how things stack up in more thorough runs that include more plugins.

I have been working on Suil support for wrapping native Windows plugin UIs in Gtk. Here is a plugin with a Pugl UI embedded in the Gtk version of Jalv:

Very fresh work, but the basics (display, and mouse/keyboard input) are working well. This work is not actually OpenGL specific, any native Windows UI will work equally well. Apologies for the boring screenshot, this is just my test GL plugin that does nothing. However, it is a working example of a plugin with a portable custom UI; hopefully we'll soon see production plugins with UIs that work equally well across toolkits and platforms.

LV2 allows plugins to implement UIs in any toolkit. This has led to UIs being implemented in several (which is a Good Thing, and works fine in all hosts via Suil), but mostly Gtk.

Unfortunately, Gtk is not really suitable for use in plugins on platforms where a "system Gtk" can't be relied on. Some toolkits are suitable for static linking, but personally, I am somewhat disillusioned with "toolkits" lately, and massive libraries in general. Sometimes all you want or need is a straightforward standard graphics API and some keyboard/mouse events.

When it comes to standard cross-platform graphics APIs, the undisputed heavyweight king is OpenGL. While not perfect (what is?), no other API is already there on almost any platform you'd care about (heck, most modern phones have hardware accelerated OpenGL). Unfortunately, OpenGL deals only with rendering, and not user input or windowing issues. What is needed is a minimal framework to get an OpenGL view to draw to, and receive keyboard and mouse events. Enter Pugl.

Pugl handles all the platform specific business behind an API very similar to GLUT, but much smaller and appropriate for plugins (which GLUT unfortunately is not). In terms of size, this is a few hundred lines of C per platform (on a personal note, this fits in well with my ever-increasing distaste in dealing with bloated junk with tons of dependencies... give me a Couple Hundred Lines of C™ any day). The breakdown on Pugl is on its homepage, but suffice to say after a few days' work it does the job it was designed to do on X11, Mac OS X, and Windows. Since embedding X11 works in LV2 land, that means an OpenGL plugin UI can be embedded in the host, and I have the pretty pictures to prove it:

This is a simple test plugin (complete with gratuitous use of 3D) embedded in Ingen and Ardour. Both are Gtk based programs, but this works in Qt as well. Testing so far on other platforms has only been top-level since I have no programs to embed in, but the bulk of the work is done. This includes full keyboard and mouse support, with significantly more complete keyboard support than GLUT.

Naturally I can't predict the future, so it remains to be seen how much OpenGL UI action we'll see for plugins. Being just a low level drawing API and not a set of boxed widgets, it's a bit open ended with a bit of a learning curve, but on the other hand there is lots of existing OpenGL code out there. Perhaps someone will throw together a library of audio appropriate widgets, if one doesn't already exist. Either way, I think an easy to use API for writing truly portable LV2 plugin UIs is a very good thing, which hopefully eliminates a barrier for some plugin developers and helps LV2 invade the territory of its proprietary adversaries... or, at the very least, makes for a really cool 3D panner GUI :)

I have been working on full round-trip serialisation for everything in the LV2 Atom extension (which also applies for implementing state). I am doing this as a small library with a simple API, with the goal that it be simple to include in any project.

svn co http://svn.drobilla.net/lad/trunk/sratom

Currently this only writes (using Serd), I still need to move the reading stuff into it (which will probably use a model and thus require using Sord).

The Atom extension defines a simple data model for use in LV2 plugins and hosts (or elsewhere). The general Big Idea here is to have a smallish set of primitive types, a few collections, and out of that one can build arbitrarily complex structures. Everything (including containers) is a simple and compact chunk of POD data, but serialises to/from (a subset of) RDF, so it can nicely be described in a plugin's Turtle file, among other advantages.

An easy to adopt serialisation API is important to making these advantages a reality for many implementations, so I have decided to provide one before marking these extensions stable. It also serves as a nice test case with complete coverage. Here is an example of an Atom that contains every currently defined Atom type, as well as MIDI, serialised to Turtle by sratom:

[]
    rdf:value [
        a eg:Object ;
        eg:one "1"^^xsd:int ;
        eg:two "2"^^xsd:long ;
        eg:three "3.0"^^xsd:float ;
        eg:four "4.0"^^xsd:double ;
        eg:true true ;
        eg:false false ;
        eg:path </foo/bar> ;
        eg:uri eg:value ;
        eg:urid eg:value ;
        eg:string "hello" ;
        eg:langlit "bonjour"@fra ;
        eg:typelit "value"^^eg:Type ;
        eg:blank [
            a eg:Object ;
        ] ;
        eg:tuple [
            a atom:Tuple ;
            rdf:value (
                "foo"
                true
            ) ;
        ] ;
        eg:vector [
            a atom:Vector ;
            rdf:value (
                "1"^^xsd:int
                "2"^^xsd:int
                "3"^^xsd:int
                "4"^^xsd:int
            ) ;
        ] ;
        eg:seq [
            a atom:Sequence ;
            rdf:value (
                [
                    atom:frameTime 1 ;
                    rdf:value "901A01"^^midi:MidiEvent ;
                ] [
                    atom:frameTime 3 ;
                    rdf:value "902B02"^^midi:MidiEvent ;
                ]
            ) ;
        ] ;
    ] .

I anticipate/intend for all plugin control to happen via such messages, since this approach has a few important qualities:

1. Typically no need to define new binary formats for things (and be held back waiting for others to implement them).
2. Everything has a portable serialization for free (meaning network transparency, saving/loading, and for developers or power users the ability to dump any message to see what is going on).
3. The convention is to use "objects" (resources, i.e. things with properties) as messages, which are inherently extensible. No "oops I needed to add a parameter so now compatibility is broken".
4. Easy to bind to other languages or syntaxes, so e.g. Python or browser-based UI frameworks should be possible.
5. Any RDF vocabulary can be used, meaning millions of well-defined and documented predicates ("keys") are available right now (though it is perfectly okay to create one-off objects - compatibility with RDF is a benefit, not a burden).

The atom extension includes an API that makes it relatively simple to build such objects in C, so plugins can write them directly to an output port or a ring buffer. See the "forge" API in the Atom extension for details. There are also iterators for all the collections and a "get" function for objects to make reading data simple.

Just in case it's not crystal clear, the above is only the external representation of the corresponding atom. At run-time, an atom (i.e. what plugins work with) is just a blob of data with an integer type and size header. 100% of the API provided for reading and writing atoms is real-time safe and suitable for use in an audio processing thread.

For an example, see the LV2 sampler example, which has a UI that loads samples via such messages. It currently works in Jalv, Ardour support is coming soon. This is the way forward for more powerful LV2 plugin control, and hopefully will end the worrying practice of abusing direct instance access to provide such functionality.

This work isn't finished yet, but the important parts are done and not likely to change significantly. I am interested in hearing any developer feedback, feel free to comment on this post or at the LV2 mailing list.

I have released my new stack of libraries for implementing LV2 in hosts:

• Serd, a fast, lightweight, dependency-free Turtle syntax library
• Sord, an in-memory RDF quad store
• Lilv, an LV2 host library (replaces SLV2)
• Suil, an LV2 UI loading/embedding library

These libraries collectively replace SLV2, and have no dependencies except amongst themselves, and Glib (by Lilv and Sord, but this dependency will likely be removed in the near future). Serd and Sord replace Redland, making for a dramatically smaller implementation more appropriate for audio software and embedded applications.

Overall, Lilv is dramatically faster and leaner than SLV2, enough that the improvement should be quite noticeable from a user point of view (typically in a lag when the host first loads all LV2 data). Anyone using SLV2 is highly encouraged to migrate to Lilv.

These libraries are well tested, each (except Suil) with a test suite covering over 90% of the code, which runs without memory errors or leaks. They are new, however, so (as always) there may be problems; feedback is most welcome.

(See below for important information for anyone following my SVN repository)

I've decided to replace SLV2 with a new library (named "Lilv") instead of breaking the API which has been causing a lot of hassle, and would only cause more when it gets released and packaged. Among other things, this gives me the opportunity to rework the installation to support parallel installs of different major versions to prevent this kind of problem in the future. It also gives me a chance to improve the API and remove some cruft that has accumulated to avoid breaking the API (SLV2 has been around for many years now, and LV2 (and myself) have come a long way since then, it's a good time to do some laundry).

Anyone who has an SVN version of SLV2 installed, please uninstall it immediately and pretend it never existed. SLV2 0.6.6 is the last version of SLV2. The first Lilv release will be out shortly.

I've reworked the LV2 documentation generation system to generate all code documentation for lv2plug.in in a single Doxygen run. This means the core and all extensions' documentation are hyperlinked between each other, and there are handy global indices of everything.

See http://lv2plug.in/ns/doc/html/ for an index.

As before, you can go directly to the URI of any lv2plug.in extension in your browser for documentation, which links to the code documentation for that particular extension.

Hopefully this reference proves useful for implementers.