LV2 atom and state serialisation

I have been working on full round-trip serialisation for everything in the LV2 Atom extension (which also applies for implementing state). I am doing this as a small library with a simple API, with the goal that it be simple to include in any project.

svn co http://svn.drobilla.net/lad/trunk/sratom

Currently this only writes (using Serd), I still need to move the reading stuff into it (which will probably use a model and thus require using Sord).

The Atom extension defines a simple data model for use in LV2 plugins and hosts (or elsewhere). The general Big Idea here is to have a smallish set of primitive types, a few collections, and out of that one can build arbitrarily complex structures. Everything (including containers) is a simple and compact chunk of POD data, but serialises to/from (a subset of) RDF, so it can nicely be described in a plugin's Turtle file, among other advantages.

An easy to adopt serialisation API is important to making these advantages a reality for many implementations, so I have decided to provide one before marking these extensions stable. It also serves as a nice test case with complete coverage. Here is an example of an Atom that contains every currently defined Atom type, as well as MIDI, serialised to Turtle by sratom:

[]
    rdf:value [
        a eg:Object ;
        eg:one "1"^^xsd:int ;
        eg:two "2"^^xsd:long ;
        eg:three "3.0"^^xsd:float ;
        eg:four "4.0"^^xsd:double ;
        eg:true true ;
        eg:false false ;
        eg:path </foo/bar> ;
        eg:uri eg:value ;
        eg:urid eg:value ;
        eg:string "hello" ;
        eg:langlit "bonjour"@fra ;
        eg:typelit "value"^^eg:Type ;
        eg:blank [
            a eg:Object ;
        ] ;
        eg:tuple [
            a atom:Tuple ;
            rdf:value (
                "foo"
                true
            ) ;
        ] ;
        eg:vector [
            a atom:Vector ;
            rdf:value (
                "1"^^xsd:int
                "2"^^xsd:int
                "3"^^xsd:int
                "4"^^xsd:int
            ) ;
        ] ;
        eg:seq [
            a atom:Sequence ;
            rdf:value (
                [
                    atom:frameTime 1 ;
                    rdf:value "901A01"^^midi:MidiEvent ;
                ] [
                    atom:frameTime 3 ;
                    rdf:value "902B02"^^midi:MidiEvent ;
                ]
            ) ;
        ] ;
    ] .

I anticipate/intend for all plugin control to happen via such messages, since this approach has a few important qualities:

1. Typically no need to define new binary formats for things (and be held back waiting for others to implement them).
2. Everything has a portable serialization for free (meaning network transparency, saving/loading, and for developers or power users the ability to dump any message to see what is going on).
3. The convention is to use "objects" (resources, i.e. things with properties) as messages, which are inherently extensible. No "oops I needed to add a parameter so now compatibility is broken".
4. Easy to bind to other languages or syntaxes, so e.g. Python or browser-based UI frameworks should be possible.
5. Any RDF vocabulary can be used, meaning millions of well-defined and documented predicates ("keys") are available right now (though it is perfectly okay to create one-off objects - compatibility with RDF is a benefit, not a burden).

The atom extension includes an API that makes it relatively simple to build such objects in C, so plugins can write them directly to an output port or a ring buffer. See the "forge" API in the Atom extension for details. There are also iterators for all the collections and a "get" function for objects to make reading data simple.

Just in case it's not crystal clear, the above is only the external representation of the corresponding atom. At run-time, an atom (i.e. what plugins work with) is just a blob of data with an integer type and size header. 100% of the API provided for reading and writing atoms is real-time safe and suitable for use in an audio processing thread.

For an example, see the LV2 sampler example, which has a UI that loads samples via such messages. It currently works in Jalv, Ardour support is coming soon. This is the way forward for more powerful LV2 plugin control, and hopefully will end the worrying practice of abusing direct instance access to provide such functionality.

This work isn't finished yet, but the important parts are done and not likely to change significantly. I am interested in hearing any developer feedback, feel free to comment on this post or at the LV2 mailing list.