Apologies for not blogging along the way for those who are interested in
this project. I'm more into actually doing things rather than blogging
about doing things ;)
So, LV2 in Gst, where are we? Well, I've had LV2 plugins working with
the same functionality as LADSPA plugins for a while. The main downside
was shared by LADSPA: no support for multi-channel streams.
Some background for those unfamiliar with LADSPA or LV2: LADSPA and LV2
plugins are very simple in terms of I/O. A plugin has a collection of
"ports" which contain either a single floating point value (control) or
an array of floating point values (audio). LV2 extends this to any types
but that's not important here. This is simple but causes problems in the
face of stereo and other multi-channel streams: if a plugin has 3 audio
inputs and 2 audio outputs, for example, what is what? Some kind of
3-channel audio input with stereo output? 5 completely unrelated ports?
A stereo input and stereo output with a "sidechain" (common with e.g.
compressors)? This information is not available in LADSPA, the best you
can do is guess (which is practice means it just doesn't work).
GStreamer works with multi-channel streams as a single interleaved
stream, so this is a problem. Thankfully, LV2 makes it trivial to add
whatever information you like about plugins without having to touch a
line of code since plugins are described in RDF (see the LV2
site for more information). All that needs to be
decided is how to actually model that information. A specification for
this is called an "ontology" in general/theory, and an "extension" in
the LV2 community.
So, the problem is we need(ed) a good multi-channel ontology for LV2
plugins to work well inside Gst, since most things in Gst are at least
stereo. The difficult thing with creating ontologies is making sure
anything anyone might want to describe in the future is accounted for.
Here's my best shot at this so far: LV2 Port
based largely on earlier work by Lars Luthman and some input from the
LV2 community. This extension isn't final, but expresses all the
information needed by Gst for multi-channel (and more). (I also wrote
the documentation generator that created the aforementioned
specification page in the hopes that more user friendly documentation
will encourage adoption by plugin and host authors).
I've created patches for the popular SWH and
Calf plugin packages to add this
information. When the extension goes final they will be included in
these projects, but in the mean time are included in my git branch of
This is, as far as I know, the first time coherent multi-channel
information has been available about plugins from the "LAD" community
(e.g. LADSPA, DSSI, LV2). Coincidentally, this information is required
for recent work on Ardour, among other things. In
hindsight, this was a pretty glaring hole in the general state of LAD
plugins, but back to GStreamer...
I've rewritten quite a bit of the GstSignalProcessor class (used by the
LADSPA and now LV2 wrapper elements) to support multi-channel plugins.
In code terms this means creating a set of buffers for non-interleaved
data used by the plugin, and interleaving/deinterleaving buffers to/from
In summary: many LV2 plugins now exist with useful multi-channel
information, and stereo LV2 plugins now work correctly in GStreamer. It
should now be simple to add support for other audio plugin interfaces
(VST? AudioUnits?) that also works with multi-channel streams.
- Get the "role" information from the plugin data and use it to
support surround streams correctly
- Finalize and publish the LV2 Port Groups extension, and contribute
patches for all major plugin collections
Where to go from there is pretty open-ended. Unlike LADSPA, LV2 can
theoretically support any kind of data, or any feature (non-realtime and
non-audio things included). Extensions just need to be made to bridge
the gap. What sort of functionality would you like to see bridge the