Posts

LV2 has had a "units" extension since the beginning, which allows plugins to specify units like Hz or dB for their controls. To date this information has not been used very widely by hosts, so I've done some work in Ardour to use this information for better UI generation and intelligent plugin control.

Units can specify a format string which describes how to print a value in that unit. This is now used to draw the text on control sliders:

The same controls are used in automation lane headers. If the control is a note number, then right-clicking will present a menu with an option to pop up a note selector dialog where the value can be chosen on a piano keyboard:

Similarly, numeric frequency controls have a context menu which can set the value to a specific number of beats in the current tempo, if the frequency range is low:

If the frequency range is high, then numeric frequency ports can be set with the note selector dialog just like note numbers:

In the future, it would be nice to have this idea applied more extensively so automation lanes can "pretend" a port is in the desired unit, for example allowing the user to automate an LFO frequency in beats, or a cutoff frequency in notes. Until then, being able to at least easily set controls to musically sensible values makes many production tasks easier, particularly in electronic music where it's often desirable to set plugin controls based on key or tempo.

Up next is setting time ports based on tempo, for cases like setting delay lines to a certain number of beats, but many plugins are missing the unit information required to make this possible. Hopefully better and more widespread host support will provide some incentive for plugin authors to specify the units of their ports. It is very simple to do so, see the LV2 units documentation for examples.

A while ago, I implemented MIDI patch, controller, and note name support for Ardour.

There was one thing missing from that work: labelled controller values. This is particularly useful for controllers that don't represent a continuous numeric parameter, but instead have a set of specific values. For example, the Moog Minitaur supports CC #86 to control the LFO sync clock division, but it's not obvious how a 0-127 number maps to a clock division. From the manual, we can learn that 61-67 is "1/4", 68-73 is "1/8 Dot", and so on.

Translate that information into standard MIDINameDocument format, and a few hours of hacking later... voila, Ardour displays something sensible when hovering over an automation point instead of a meaningless number.

If you want to add support for your MIDI device or program, my .midnam documents for the Moog Minitaur, MF-104M, and MF-108M, included in the Ardour distribution all contain examples of labelled values. It should be relatively obvious how to copy and modify these for other devices.

Ardour has supported displaying MIDI patch names loaded from a MIDINameDocument (or "midnam" file) for a while, though only patches, and even that was pretty flaky. Since this has become an itch for me and it's time for Ardour MIDI to reach release quality, I took some time to give it a serious overhaul. In this case, a picture or two is indeed worth a thousand words:

The note names are mainly useful for doing percussion. Several of the bundled midnam files have note names defined, and I have added one for General MIDI drums, which more or less corresponds to a lot of instruments, including Hydrogen.

Controller names, to my surprise, were not present in any of the existing midnam files at all. I wanted to sequence a hardware synth (the Moog Minitaur) without constantly referring to the manual and trying to remember which CC is which, so I wrote a new midnam file for the device and implemented support in Ardour. I am very disappointed to learn that there is no ability to group controllers, which really is the best way to do things, so I may embrace and extend (in a completely backwards compatible way) the format in the future to provide a better interface.

The quality of midnam files scattered around the 'net is atrocious. The MMA actually hosts a DTD for it, but most files use the wrong DOCTYPE (usually with a broken link), among other problems. I fixed and cleaned up all the bundled ones in Ardour, validated them against the DTD, and left a README in that directory about how to do so. Hopefully this, along with incentive added by these features, encourages the community to grow a nice set of quality midnam files.

So, MIDI name support in Ardour is now in pretty good shape. Now we just need to add the corresponding LV2 support...

I have been working on the control protocol aspect of Ingen, towards a few goals:

• Working GUI communicating with the server via a socket
• Decent Python bindings for socket interface
• Ingen as an LV2 plugin

The Python bindings got fleshed out to support a script for loading AMS patches into Ingen I have been working on (very preliminary, but in the repository and installed as 'ingenams'). The socket interface is now sensibly synchronous, e.g. you send a command, and recv on the socket to get the response, which will include a status as well as whatever patch updates happened as a result of the command. I am happy with this route to language bindings, it's much more pleasant than a mess of direct bindings (e.g. via swig) and being protocol controlled has many other advantages. The Python interface is implemented purely in Python, it should be simple to do the same in any language that can use sockets.

The Ingen GUI can communicate with the engine via the same protocol, so you can build patches over the network (e.g. if Ingen is running on a headless machine as part of an audio rig). It can also run in the same process and use direct function calls for maximum performance.

The Ingen protocol uses the LV2 Patch vocabulary to describe changes. For the socket interface, this is represented entirely in Turtle. When running as an LV2 plugin, Ingen uses conceptually the same protocol, except in a binary format using LV2 atoms. This is a nice example of the benefits of having things defined in a good abstract model which can be encoded in many ways. For example, it would be relatively simple to implement control via JSON for a browser-based Ingen UI, which seems like the best way to go for separate control panels, but that's a topic for another day.

Since Ingen control can be expressed as atoms, it can run as an LV2 plugin with a full patcher UI and communicate only via LV2 ports. The UI is embeddable, and there is no memory sharing between engine and UI or hidden communication, everything is host managed. Here is a screenshot of Ingen running in Ardour 3. The MIDI in Ardour is played through an Ingen synth plugin, and the output has been recorded to the audio track. The patch can be manipulated live while running in Ardour, except for top level ports.

I have been working on full round-trip serialisation for everything in the LV2 Atom extension (which also applies for implementing state). I am doing this as a small library with a simple API, with the goal that it be simple to include in any project.

svn co http://svn.drobilla.net/lad/trunk/sratom

Currently this only writes (using Serd), I still need to move the reading stuff into it (which will probably use a model and thus require using Sord).

The Atom extension defines a simple data model for use in LV2 plugins and hosts (or elsewhere). The general Big Idea here is to have a smallish set of primitive types, a few collections, and out of that one can build arbitrarily complex structures. Everything (including containers) is a simple and compact chunk of POD data, but serialises to/from (a subset of) RDF, so it can nicely be described in a plugin's Turtle file, among other advantages.

An easy to adopt serialisation API is important to making these advantages a reality for many implementations, so I have decided to provide one before marking these extensions stable. It also serves as a nice test case with complete coverage. Here is an example of an Atom that contains every currently defined Atom type, as well as MIDI, serialised to Turtle by sratom:

[]
    rdf:value [
        a eg:Object ;
        eg:one "1"^^xsd:int ;
        eg:two "2"^^xsd:long ;
        eg:three "3.0"^^xsd:float ;
        eg:four "4.0"^^xsd:double ;
        eg:true true ;
        eg:false false ;
        eg:path </foo/bar> ;
        eg:uri eg:value ;
        eg:urid eg:value ;
        eg:string "hello" ;
        eg:langlit "bonjour"@fra ;
        eg:typelit "value"^^eg:Type ;
        eg:blank [
            a eg:Object ;
        ] ;
        eg:tuple [
            a atom:Tuple ;
            rdf:value (
                "foo"
                true
            ) ;
        ] ;
        eg:vector [
            a atom:Vector ;
            rdf:value (
                "1"^^xsd:int
                "2"^^xsd:int
                "3"^^xsd:int
                "4"^^xsd:int
            ) ;
        ] ;
        eg:seq [
            a atom:Sequence ;
            rdf:value (
                [
                    atom:frameTime 1 ;
                    rdf:value "901A01"^^midi:MidiEvent ;
                ] [
                    atom:frameTime 3 ;
                    rdf:value "902B02"^^midi:MidiEvent ;
                ]
            ) ;
        ] ;
    ] .

I anticipate/intend for all plugin control to happen via such messages, since this approach has a few important qualities:

1. Typically no need to define new binary formats for things (and be held back waiting for others to implement them).
2. Everything has a portable serialization for free (meaning network transparency, saving/loading, and for developers or power users the ability to dump any message to see what is going on).
3. The convention is to use "objects" (resources, i.e. things with properties) as messages, which are inherently extensible. No "oops I needed to add a parameter so now compatibility is broken".
4. Easy to bind to other languages or syntaxes, so e.g. Python or browser-based UI frameworks should be possible.
5. Any RDF vocabulary can be used, meaning millions of well-defined and documented predicates ("keys") are available right now (though it is perfectly okay to create one-off objects - compatibility with RDF is a benefit, not a burden).

The atom extension includes an API that makes it relatively simple to build such objects in C, so plugins can write them directly to an output port or a ring buffer. See the "forge" API in the Atom extension for details. There are also iterators for all the collections and a "get" function for objects to make reading data simple.

Just in case it's not crystal clear, the above is only the external representation of the corresponding atom. At run-time, an atom (i.e. what plugins work with) is just a blob of data with an integer type and size header. 100% of the API provided for reading and writing atoms is real-time safe and suitable for use in an audio processing thread.

For an example, see the LV2 sampler example, which has a UI that loads samples via such messages. It currently works in Jalv, Ardour support is coming soon. This is the way forward for more powerful LV2 plugin control, and hopefully will end the worrying practice of abusing direct instance access to provide such functionality.

This work isn't finished yet, but the important parts are done and not likely to change significantly. I am interested in hearing any developer feedback, feel free to comment on this post or at the LV2 mailing list.

Well, I've been working on Ardour but not directly on my SoC project.. finally got around to fixing that whole mixer-strip-element thing that's been on the table since last year. Things are more open to extensibility now (and a unified bus implementation should appear soon, so we'll have MIDI busses or even MIDI/audio (instrument) busses). Maybe we'll see things like this actually happen some time soonish...

It would be nice to figure out a really good MIDI meter (something more clever than just an audio peak meter abused), and a MIDI fader that can work in various ways, or even instrument plugin support, but it's probably time to get on the actual piano roll editing part of the project and leave that stuff until afterwards.

Kinda ruins my flow though, you know?

Well.. finally fully moved; I guess it's time to stop screwing around and get back into hacking. Specifically Ardour, since I'm being payed to do so and all.

Looks like we don't have a fancy new cairo canvas to play with. Maybe it will be worth my time to do that first, but I'm going to do some fiddling with displaying MIDI data with gnomecanvas first to get a grasp on things. Wouldn't hurt to have some visually obvious clue that I am, in fact, actually doing something. This summer's project should be more rewarding than last in that sense; most of what I need to do is visual stuff which tends to be more fun since you have something nice and tangible at the end of the day.

Of course, displaying data is one thing. One relatively easy thing. Actually editing it on the other hand.......... that's where the "fun" (i.e. hard) part comes in.

I think a top-down (GUI->implementation) strategy is best here. The Grand Battle Plan(TM) goes something like this:

• Get MIDI data displayed in regions, notes visible as one canvas item (rect) per note
• Attach event signals from note canvas items (move, click, etc) to a set of stub methods that encompass all the editing operations
• Now there's a nice centralized area where all the editing operations need to be implemented
• Figure out how the hell to implement them (ie ???????????)
• Implement them (ie Profit!)