drobilla.net - Hacking

A More Modern Gtk3 Jalv Frontend

2025-11-26T21:36:00-05:00

My simple single-plugin LV2 host, Jalv, isn't quite sure whether it's a developer utility or polished user program, but in any case, it had become stale in the past few years and needed an update.

Most of those changes are internal and only interesting for those who use it as a basis for larger systems. The internals have been largely rewritten to support various things, but this post isn't about that. This post is about a more obviously stale thing: the Gtk2 interface.

In keeping with the free desktop tradition of constant breakage with reduced functionality, that toolkit is now EOLed, and soon the ability to embed GUIs whatsoever will probably go away. Luckily though, we're not quite there yet, and it's still possible/feasible to embed GUIs in Gtk3 (at least on X11), so things can continue roughly as they were for a while. Gtk2 is EOLed though, which is a problem for distributions, and I have no interest in maintaining code for a dead toolkit, so that frontend is gone entirely in the latest release. This does mean that some plugin GUIs written in Gtk2 will no longer work, but that's inherent to the situation (and why general plugin GUIs shouldn't use Gtk).

This seemed like a good time to update the UI to be a bit more “modern”, particularly since a menu bar has never really made much sense here anyway. I replaced this with a header bar, which I think does suit plugins better. For example, here's the custom GUI for the LSP Compressor:

As always, there's also generic controls, with a few refinements but still using the same boring stock widgets:

All of the menu items have been moved into a single menu button, which is a pattern I'm sceptical of in general, but it works fine for a very simple application like this. The preset menu can be unwieldy, but that's a whole topic unto itself that I hope to tackle more comprehensively later.

Code-wise, it's long been a problem that the rudimentary (lack of) architecture couldn't easily support the more advanced features people wanted from it. So, I've reworked everything into a more serious application, with a more explicit architecture and communication patterns that make adding new features much easier. As far as the Gtk frontend goes, I've also switched to using more modern APIs like GtkApplication, GAction, and so on. To be fair, these parts are quite nice. Actions are a pretty good model for building accessible GUI applications, and these new APIs encourage doing the right thing. There's still some areas that need work, but jalv.gtk3 (the version which has a .desktop file and all that) is much closer to being a proper application that integrates with the desktop environment now, and smells less like a hackey program that developers just use to check if their plugin works.

That aside, Jalv is still frequently used from the command-line, and there's a major QoL improvement there as well: the positional argument now accepts files and directories, not just plugin URIs. The code will try to figure out what to do automatically, for example, if a bundle or data file only describes a single plugin, then that plugin is loaded. Presets can also be passed (by path or by URI), which will load the appropriate plugin with that preset initially applied. In short, it's more like the “do what I mean” interface many people expect.

It's been entirely too long since the last release, but now that the host libraries and Jalv are up to date with most issues resolved, I'm going to try to do some broader cross-project efforts to address a few things that are a mess across the LV2 ecosystem as a whole, with Jalv serving as a sort of reference implementation. For now, though, it's just a much better implementation of the same old features.

Lilv “Object Index” Option Revised

2025-11-12T13:58:00-05:00

My previous post, “Improving LV2 Discovery Performance”, described adding an option to lilv that controls whether an additional index is maintained for loaded LV2 data. While testing those changes for release, I discovered a few things (like lv2lint) that break without the index, which convinced me that disabling it by default wasn't the right decision.

So, I'm instead enabling it by default, which avoids this “break by default” situation. The downside is that hosts will need to add code in order to enjoy the performance improvements. To make this change as simple as possible, I also made lilv_world_set_option accept a null value to disable boolean options, so most hosts can simply add this line:

lilv_world_set_option(world, LILV_OPTION_OBJECT_INDEX, NULL);

This requires the new version, but the same change can be made without bumping the required version by avoiding the niceties:

LilvNode* false_node = lilv_new_bool(world, false);

lilv_world_set_option(
  world,
  "http://drobilla.net/ns/lilv#object-index",
  false_node);

lilv_node_free(false_node);

This will log a warning about the unsupported option on older versions, but otherwise have no effect.

Improving LV2 Discovery Performance

2025-09-29T10:25:00-04:00

LV2 plugins (and other data, like presets) are discovered by parsing Turtle files installed in various bundle directories. If the host only needs to know which things are installed, then only manifest.ttl files need to be loaded. That takes only a fraction of a second (60 ms on this machine, which has 897 plugins installed), but if the host needs more information (like the plugin's label or what ports it has), then the data files need to be loaded as well, which can take several seconds. The parser (from serd) is very fast, so the overwhelming majority of this time is spent inserting data into a model.

I was recently doing some work on lilv related to discovery, and got sidetracked into investigating how much of this overhead could be eliminated. Quite a bit, as it turns out, but before explaining how, I'll give a brief summary of the relevant fundamentals so people who don't spend their days in the triple mines can understand what I'm talking about.

LV2 data is written in the Turtle syntax, which can have a lot of structure, but is ultimately just a shorthand for data that is simply a “flat” series of triples. For example, the abbreviated Turtle

<http://example.org/amp>
    a lv2:Plugin ;
    doap:name "Amplifier" .

represents the two triples

<http://example.org/amp> rdf:type  lv2:Plugin .
<http://example.org/amp> doap:name "Amplifier" .

which describe a plugin with the name “Amplifier”.

Two implications of this format are important to understand here:

All data is a set of triples which can be stored in some lexicographical order to be quickly searchable (for example, a set ordered subject-first can be used to quickly find triples that start with a given subject).
All data can be streamed via a simple “flat” interface (for example a function with three parameters), and is trivial to inspect and/or filter on the fly (much like line-based text in POSIX pipelines).

The data may not be stored as a “literal” ordered set of triples (they're actually quads in memory for one thing), but this simplified way of thinking about it is good enough for this explanation.

Which order is best depends on the query. For example (borrowing a bit of syntax from SPARQL where ?name represents a query variable), if the host asks something like “what's the name of this plugin” or

<http://example.org/amp> doap:name ?name .

then it needs to find triples that start with the given subject and predicate (<http://example.org/amp> and doap:name) to see what object (?name) they have. So, a subject-predicate-object or “SPO” index will work well. If, however, the host asks something like “which things are plugins” or

?plugin rdf:type lv2:Plugin .

then an SPO index doesn't help because there's no known subject to search for. Most queries are like this, with either a subject or object wildcard (querying relatively “fixed” predicates is rare), so lilv has always had both an SPO and OPS index to support them.

This is convenient, but twice the indexing means roughly twice the overhead. The SPO index is the natural order used in the syntax, and supports most code (which mainly looks up properties of known things), but the OPS index isn't used so much. Can it be removed entirely to speed things up? It is used for many important things, but conveniently, there's only a few fixed cases of those (like finding plugins as above). So, we can take advantage of the streaming nature of the data to record this information while it's being read, instead of inserting it into an index only to query it out later.

To do this in the implementation, I introduced the concept of a “skimmer”. A skimmer inserts triples into a model as usual, but first “skims” them and records items of interest for later use. For example, a skimmer checks whether a triple has rdf:type lv2:Plugin, and if so, records the subject in a result set that stores only plugin URIs. Some cases are a bit trickier, and actually pulling this off cleanly took quite an overhaul, but details aside, it turns out that this approach can be used to eliminate the OPS index entirely without losing any functionality.

How much of an improvement does this make? On this machine, using lv2ls -n as a crude benchmark, the time goes from 3.11 to 1.72 seconds, or about a 45% improvement. The memory consumption goes down a bit as well, from (even more crudely) a max-RSS of about 138 to 112 MiB, or about a 19% improvement (everything here is an average of three runs). The improvement will vary dramatically based on what's installed, for example in some earlier tests with a different configuration I saw drops from around 7 to 4 seconds, but in general, not bad!

Are there any downsides to this? Yes, primarily two:

As with any significant change, there is the possibility of regression. Discovery is a little more restricted than before, so although I've done my best to test things, there is a possibility of things not being discovered anymore. Usually the fix for this is adding information to the manifest that should have been there anyway.
Though the implementation itself doesn't make queries that require the OPS index, hosts themselves can since lilv provides generic query functions. To support this, there's a new boolean option, LILV_OPTION_OBJECT_INDEX, which can be used to enable or disable the OPS index. Since most hosts don't need it, I decided to disable this by default. That will be a regression for hosts that do, however, who need to explicitly opt in to the old behaviour by enabling this option. A warning is printed if a query will be slow because of the missing index, so it should at least be obvious if this happens.

These changes are now in the main branch in git, which will be released soon, probably as version 0.26.0.

Software Pages Removed

2025-02-16T11:32:00-05:00

I haven't been sure what to do about the software pages here for quite a while. Most of them were essentially just stale versions of the README files from their projects, and for better and worse, most of the projects I maintain are libraries that don't have as much of a need for a homepage as user-facing software. It's easy to just ignore things that don't really matter in day-to-day work, but the embarrassingly bad state of things became really clear when I sat down to actually poke through this site.

Since it's much more important for me right now to streamline maintenance duties and eliminate as much overhead as possible, I've simply removed all of the software pages, and redirected those addresses to the corresponding Gitlab projects where possible. I might bring them back at some point, but for now, no pages are better than stale pages that really only serve to make things look bad. I don't have traffic metrics here, but I seriously doubt anyone will either notice or care.

I'm not sure about the utility of the software release posts or tarballs either. Ideally, the effort required to make a release could be reduced to simply pushing a git tag, and cross-domain posting hugely complicates that. Besides, the tarballs are made manually on my personal machine, so they're absolutely less trustworthy than the signed tags in git anyway, and, I assume, not reproducible. At the same time, for many reasons I'm wary of fully investing in some git forge or another, the automatic tarballs provided by all of them leave much to be desired (the silly “v” names for example), and I don't want to disrupt things for packagers. We'll see, but for now I'll leave the mechanics of actual releases as they are.

Ultimately, pages and posts are largely a waste of time for libraries and similar things that only support other projects anyway. So, a more radical simplification of the release process would be a good idea, but for now I'll just take out the trash and reduce the amount of things I need to consider in that process.

Intermission

2025-02-04T14:40:00-05:00

I don't suppose inactivity from me will be terribly surprising to anyone after the past several years. Still, since I was working on emptying my bug-fix and maintenance queue, making releases, and finally getting to some significant forward progress again, an update:

Unfortunately I got violently ill last week with some horrible flu-like thing, the worst I've ever had (and I'm a COVID casualty, so that's saying quite a lot). Somehow, it's still going strong. So, aside from maybe a few minutes a day of idle tinkering, everything I was doing is on pause for a while. Hopefully a short while, but so far so not good, so we'll see.

As an added bonus, Google just bricked my phone with a botched forced update, so I'm locked out of 2FA and many other things besides (I suppose I had to learn the hard way that I've gotten lazy and too dependent on that horrible device). So, yeah, things aren't going great, to put it lightly.

On the bright side, I do have some exciting things in the queue, but since I don't do vapourware or hype (to a fault, really), and have a huge amount of “infrastructure” work to do first anyway, you'll have to stay tuned for that. Assuming I don't die first, anyway.

... if I do, it would be pretty funny that this was my last post though, so I've got that going for me, which is nice.

[Edited on 2025-02-16 to remove broken link]

Meson Features and Packaging

2022-08-16T15:42:00-04:00

Recently, I have been porting all of the software I maintain to meson. For the most part, this has been a nice improvement. Meson strives to be a generally sensible and coherent build system that addresses the needs of both upstream developers and packagers, and it shows. It isn't perfect (like all software), but it's a pretty big improvement over what came before in many ways.

One disagreement has come up, however, and I think it - at best - betrays a misunderstanding of meson feature options, and - at worst - threatens to effectively destroy their utility entirely for no good reason. This disagreement originates from arch-meson, which adds --auto-features=enabled by default. This isn't technically wrong, but it's problematic at scale. To explain why, I first need to explain how meson feature options work.

Meson feature options are a better solution to the --enable-this and --disable-that style options of autoconf and similar. They make all the possible states of a feature clear and explicit. Those states are:

disabled, which unconditionally disables the feature.
auto, which enables the feature if possible (typically if dependencies are found).
enabled, which unconditionally enables the feature.

The default value for an option can be any of the three, and is given by the developer in the option definition. This is a better system because there are three states with clear meanings. They allow the builder to:

Ensure that a feature is definitely not included.
Ask that a feature be included if possible.
Ensure that a feature is definitely included (failing to configure otherwise).

Each of these states (and desires) are distinct and, importantly, their definitions are not conditional. There is no “unless” in the definition of disabled or enabled, and adding one would destroy their utility. It's not “enabled unless someone feels like it's fine to disable”, it's just “enabled”.

Concretely, for enabled in particular, this means that, as a builder of whatever meson project, you can:

meson setup build -Dsomething=enabled

Then, you know that if configuration succeeded, the something feature is enabled. This is important. It is also the only way to reliably/mechanically ensure that a build contains a feature. “Destroying” it (as something meson users can generally expect) would therefore mean that there there is no way to ensure that a build contains a feature in general. Surely everyone - especially packagers - can agree that's bad.

Hopefully that's a decent enough explanation of why meson feature options are a good, coherent, and reliable system.

What's the problem then? Well, there is that convenience option, --auto-features=enabled, which enables all features for convenience. This allows the builder to override the developer-chosen defaults, and force the build script to be explicit about all non-enabled features. This is, again, not technically wrong, and is indeed convenient at times.

The problem is more of a social one. Because Arch chose to make this the default (unlike most other distributions, as far as I can tell), people have apparently adopted the mentality that --auto-features=enabled not building is an upstream bug. It isn't. The shorthand --auto-features is just that - a shorthand for enabling all features. It is equivalent to explicitly enabling every feature independently, so doing so is quite literally asking for the build to fail if any feature is not present.

Where this gets weird is when features seem unlikely, or even impossible, to build on a given system. The bug filed against my project was about Cocoa on Linux. I get it, Cocoa is mostly a MacOS thing, and is almost certainly unavailable elsewhere, but my argument is that this doesn't matter at all. Even if it were literally impossible for Cocoa to exist on top of, say, FreeBSD somewhere (which it isn't, and cross-compiling is a thing anyway), it still wouldn't matter. A similar case can be made for optional dependencies that are not publicly available at all (something I have done in the past). Feature options working as expected in general is much more important than any of this subjective case-specific thinking.

“This doesn't work for me, so the package should not build when this feature is enabled” is not a good argument, regardless of why it doesn't work. It doesn't matter if you imagine a feature “can't” be built on some system (anyone who has been in this game long enough should be able to see where that slippery slope leads...). The power of feature options depend on the states meaning just what they say, and making enabled sometimes effectively equivalent to auto destroys that power. There's not really any point in using features at that point, you might as well just use a boolean flag.

Put another way, the proposed “fix” here would make it so that:

meson setup build -Dsomething=enabled

Can successfully configure, but without the something feature being included. Et voila, we've made the feature states effectively meaningless. That would be a bug, and letting this mentality take over would resign us back to manually wading through logs to see if what we asked for actually happened.

It's also worth noting that feature options are deeply integrated all over the place in meson, for example, you can pass them to as “required” parameters and the right thing will happen. Diluting the meaning of enabled would cause deeper problems than the builder-visible ones explained here. What does it even mean for a feature to be enabled, except not actually enabled, when that feature status is being used to dictate that another dependency is required? Everything falls apart. The feature option system, when used as intended, is designed well, but messing around with the meaning of enabled like this ruins it. I don't think it's hyperbole to say it “destroys” the whole thing: it turns a sensible and coherent system into some confused boolean, which is both objectively less powerful, and, well... confusing.

This may seem trivial when you're zoomed in to a single project or handful of options, but globally it is a much bigger deal than it might seem at first. Even in my relatively tidy third-party source directory, there's nearly a thousand meson options. I can't imagine how many there are globally. The basic way that features work can't depend on the builder understanding details about every single option, that would be a complete disaster. It is crucial that disabled and enabled mean exactly what they say, always.

I have never seen this problem come up from a user configuring the project themselves. Presumably, people realize that explicitly requesting a potentially failed build, then complaining that it did what they ask, is... rather silly. This gets obscured when a tool (arch-meson) does it automatically, but remains just as silly. It makes no difference from the perspective of a meson package or its maintainer: by enabling all features, the builder has explicitly requested ignoring the defaults and failing to build if any feature was not present.

Yet, here we are, with upstream developers getting “bug” reports for feature options working exactly how they are supposed to work. This, unfortunately, means that Arch is putting toxic pressure on the meson ecosystem, pushing upstream developers to make things objectively worse.

I don't know what the solution to that is, and I'm not an Arch developer (though I'm typing this on an Arch-derived distribution and have nothing in particular against it). Breaking all the existing packages that use arch-meson is probably not going to happen, but perhaps improving the documentation would be good enough.

All I know is that --auto-features=enabled not building is very definitely not a bug. Quite the opposite, that is literally the whole point of feature options. If there's something out there that's leading people to thinking that enabled features not building is a bug, then it is misleading people in a bad way, and needs to be fixed.

Beautiful C and C++ Documentation with Sphinx

2020-11-26T17:12:00-05:00

Like many, I've long suffered under the antiquated and inflexible HTML documentation generated by Doxygen. Having recently worked on some Python documentation using Sphinx, though, I found it powerful and pleasant enough to use. It also has a way of encouraging actually writing documentation, rather than just generating a dump of glorified comments, which is a good thing. Though I'm not at all a fan of ReStructuredText syntax (which at times seems like it's trying to be cryptic on purpose), Sphinx is undeniably powerful, and I like the “assemble a bunch of plainish text files” approach in general. The support for multiple languages is also very appealing, though not without its problems, as we'll get to.

So, is it possible to use Sphinx to generate documentation for C and C++ libraries? Yes! As explained somewhat recently in a post by Sy Brand, there is a project called Breathe that integrates Doxygen (for extracting documentation) with Sphinx (for generating output). That sounded promising, so I attempted to migrate a library to using Breathe instead of Doxygen's HTML support. Unfortunately, though, I encountered quite a few roadblocks where I couldn't quite get output that I was happy with. Worse, the project itself is very complicated, and as I poked around in swaths of originally generated but manually modified code, I decided that Breathe was not for me. That would feel like just exchanging one inflexible and unhackable system for another.

What, then, to do? Though I realize that deep integration via modules like Breathe is usually the way things are done with Sphinx, I am a KISS sort of person, so I like to think of it as something more like a Static Site Generator: it reads a bunch of plainish text input files, and outputs HTML (or whatever other presentation format). How do we describe C and C++ things in Sphinx? It turns out that recent versions have built-in support for these “domains” now, which define markup for describing everything in these languages. This means that everything to do with nicely formatting and cross-referencing C and C++ is already dealt with out of the box. Excellent.

So, taking a step back and assessing the situation: we have some XML files that describe the documentation, and we have a tool that reads text files and produces nice documentation. This strikes me as a relatively straightforward task for a nice and simple “files in, files out” script, not somewhere a Goldbergian contraption that mashes Doxygen into Sphinx is required. So, after investigating any other promising options (no such luck), I resigned myself to trying to write such a thing, at the very least to see if it's feasible. I certainly have no time or interest in writing and maintaining a Documentation System, but a self-contained script to convert one thing to another seems reasonable enough.

As it turns out, I wouldn't call it trivial, but it's certainly feasible. I ended up with a ~700 line Python script that does everything I need (though this is of course not the same as everything possible). It's a bit “gluey” and makes some assumptions about the structure and so on, but it does the job and is something I feel I can maintain as necessary. I won't be publishing or supporting this as an independent project any time soon, and make no claims about it being general purpose, but feel free to steal it if any of this sounds appealing.

With this, I was able to get around some long-standing gripes I have with Doxygen, and easily make whatever I wanted to happen a reality, so I'm pretty happy with this approach. Everything is nicely decoupled, so I don't feel over-invested in any of the tools involved. If, for example, someone finally writes a good clang-based extractor that gains traction (JSON please, I did not enjoy this revisitation of the horrors of XML at all), I should be able to switch to using that easily enough. I've actually found this somewhat crude and UNIXey approach quite convenient: you can simply look at the ReST files to understand what is happening, or tweak them a bit and run Sphinx to test what you're aiming for, and so on. Text files are good.

So, after however many years, I think I've found an approach to documentation I'm actually quite happy with, that can support all of the languages that I use, and in general doesn't seem to get in my way. Hooray. For starters, I did my window system portability layer, Pugl. The generated documentation for the C API can be seen at https://lv2.gitlab.io/pugl/c/singlehtml/, and the C++ at https://lv2.gitlab.io/pugl/cpp/singlehtml/. This is more or less the standard Alabaster theme with a few tweaks, which I'm not sure feels appropriate for API documentation (and is much more bloated with a bunch of Javascript than I'd like), but it's pretty enough, at least. I'll tinker with themes later when I feel like jumping down that rabbit hole.

The slightly cumbersome links are an artifact of the one problem I encountered using Sphinx domains: you can't really document C and C++ APIs nicely in the same documentation set. If you use the cpp domain everywhere, you get name mangling in links even for C symbols, which is really unfortunate, and you can't really mix them. To take a contrived example, if you have a struct MylibThing in C, then a type alias in C++ like using Thing = MylibThing, Sphinx isn't clever enough to figure out that MylibThing is from C, and will generate warnings and not link correctly. Perhaps someday it will, which would be nice, but for now I opted to simply generate completely separate documentation sets. This means the C documentation is duplicated in the C++ documentation so that things can be hyperlinked, which isn't ideal, but I can live with it. A certain amount of redundancy is inherent in multi-language documentation anyway.

As I add Python bindings to most libraries, having a unified documentation system for all of these languages will be very nice. There is one additional thing I'll need at some point for the LV2 documentation in particular: a domain for RDF properties and classes. The LV2 documentation really suffers from an unnatural code (via Doxygen) and data (via lv2specgen) documentation split, and my hope is that Sphinx can provide a nice environment for writing documentation that refers to both worlds freely. That, unfortunately, will be much more work, but hopefully writing a custom Sphinx domain isn't too hard...

Git migration

2015-11-14T16:46:00-05:00

I have finally migrated all of my software to git. This was not a very fun process due to the nested nature of my “drobillad” repository, but now all projects live in their own git repositories with history and tags from SVN preserved.

It is still possible to build all my audio software in one step, the top-level repository http://git.drobilla.net/drobillad.git is now a skeleton with git submodules for each project. Anyone using SVN should switch immediately, the SVN repositories will remain in their current state for the foreseeable future but all development activity will move to git.

All the repositories are available in cgit.

[Edited on 2025-02-16 to remove broken links]

LV2 Plugin Control Units in Ardour

2014-11-03T20:09:00-05:00

LV2 has had a “units” extension since the beginning, which allows plugins to specify units like Hz or dB for their controls. To date this information has not been used very widely by hosts, so I've done some work in Ardour to use this information for better UI generation and intelligent plugin control.

Units can specify a format string which describes how to print a value in that unit. This is now used to draw the text on control sliders:

The same controls are used in automation lane headers. If the control is a note number, then right-clicking will present a menu with an option to pop up a note selector dialog where the value can be chosen on a piano keyboard:

Similarly, numeric frequency controls have a context menu which can set the value to a specific number of beats in the current tempo, if the frequency range is low:

If the frequency range is high, then numeric frequency ports can be set with the note selector dialog just like note numbers:

In the future, it would be nice to have this idea applied more extensively so automation lanes can “pretend” a port is in the desired unit, for example allowing the user to automate an LFO frequency in beats, or a cutoff frequency in notes. Until then, being able to at least easily set controls to musically sensible values makes many production tasks easier, particularly in electronic music where it's often desirable to set plugin controls based on key or tempo.

Up next is setting time ports based on tempo, for cases like setting delay lines to a certain number of beats, but many plugins are missing the unit information required to make this possible. Hopefully better and more widespread host support will provide some incentive for plugin authors to specify the units of their ports. It is very simple to do so, see the LV2 units documentation for examples.

Pretty names in Patchage via Jack Metadata

2014-04-09T21:26:00-04:00

The much-awaited (by me, at least) Jack metadata API has arrived. This will allow us to easily achieve many new things with minimal/nonexistent API friction. One of the simplest and most obvious is pretty names for Jack clients and ports, so I've chosen this as the first thing to tackle (as part of a drive to get Patchage polished up for a much overdue release).

Unfortunately, as far as I can tell, there is a chicken & egg scenario here since nothing is setting this metadata yet. So, I've made Jalv and Ingen both set pretty names for their ports. In the case of Jalv, the “pretty name” is the label given in the plugin data (distinct from the LV2 “symbol” which is restricted and unique).

Firing up Jalv with the Tal Dub III plugin (LV2 port courtesy KXStudio), we can see the port symbols, which are a bit awkward for end users with their prefixes and underscores. Conflating strong identifiers with human-readable labels is a serious design error I learned of the hard way, but that's a discussion for another time...

Enable “Human Names” in the view menu, or press C-h, and voilà, we see the pretty names set in Jack metadata (if present) instead.

The metadata API is very simple to use for ports, though there seems to be a hole in the API which makes it difficult to get the UUID for your client to set metadata (I want a simple jack_client_uuid, like jack_port_uuid, but it seems you have to get a string UUID and parse it to a jack_uuid_t, clunky enough that I just didn't bother). In any case, I am happy to see a low-friction mechanism in Jack which apps can use to share metadata towards making a better user experience.

It will be interesting to see what sort of information proves useful and becomes established/standard. For those of us of a mad scientist bent who live in a nest of patch cables, a CV tag seems like another obvious simple step, but for everyone, a big step is finally having meaningful port grouping and channel roles. I have always liked to joke that Jack (like LADSPA) doesn't really even do stereo, but with metadata, we can mark up stereo, 5.1, Ambisonics, etc., and other clients will be able to make sense of the channel assignments without resorting to dirty kludges based on guessing from names. All without changing the ABI one bit. Good stuff.

Labeled MIDI controller values in Ardour

2014-01-06T17:09:00-05:00

A while ago, I implemented MIDI patch, controller, and note name support for Ardour.

There was one thing missing from that work: labelled controller values. This is particularly useful for controllers that don't represent a continuous numeric parameter, but instead have a set of specific values. For example, the Moog Minitaur supports CC #86 to control the LFO sync clock division, but it's not obvious how a 0-127 number maps to a clock division. From the manual, we can learn that 61-67 is “1/4”, 68-73 is “1/8 Dot”, and so on.

Translate that information into standard MIDINameDocument format, and a few hours of hacking later... voila, Ardour displays something sensible when hovering over an automation point instead of a meaningless number.

If you want to add support for your MIDI device or program, my .midnam documents for the Moog Minitaur, MF-104M, and MF-108M, included in the Ardour distribution all contain examples of labelled values. It should be relatively obvious how to copy and modify these for other devices.

Interactive Force-Directed Graph Layout in Ingen

2013-12-20T00:51:00-05:00

GraphViz has, unfortunately, broken reverse compatibility of their library API, which has been nagging as a maintenance headache for FlowCanvas and its successor Ganv for a while now. It would be nice to just get rid of that dependency altogether...

Meanwhile, I had to write some force-directed graph layout (FDGL) code for another project...

... You can probably see where this is going. In the spirit of it theoretically being a holiday (there are no holidays in practice in grad school), I spent some time implementing live FDGL in Ganv, and recorded a video of interactive layout in Ingen to show progress.

One nice thing about using live FDGL in a program like Ingen is that the user can customize the layout in an intuitive tactile way, since objects behave physical laws. This video demonstrates some manual rearranging of a patch, and a total reconfiguration by switching the signal flow direction. Manual tweaking of the layout like this was not possible with the previous GraphViz-based layout code.

It took quite a bit of tweaking to get the physics working well, so layouts looked good but the patch didn't explode if there are disconnected components. It's still not quite ideal, but usable.

This is implemented at the Ganv level and thus works in Ingen (as shown), Patchage, and Machina. I'm still not confident enough in this implementation to enable it by default and/or drop Graphviz quite yet, but the adventurous who follow svn, ./waf configure --debug --fdgl. If you're hit by the graphviz breakage (and I still haven't fixed the check...), add --no-graphviz to work around the problem.

[Edited on 2025-02-16 to remove broken link]

Better Stock LV2 Plugin UIs

2013-02-10T22:11:00-05:00

Today I implemented better UI generation for LV2 plugin controls in Jalv, particularly grouping controls under headings which makes a big difference. Unfortunately few plugins group their controls currently, but hopefully more host support will provide the incentive to do so.

Other improvements:

Added a spinbutton to slider controls for precisely setting values numerically
Much more efficient layout to pack more controls on the screen at once
Support for named scale points (“ticks”) on non-enumeration ports

Here is the UI generated for Amsynth (after I added the necessary metadata):

Still pretty utilitarian, but much more usable, which is the important thing. Of course, this plugin is quite large, and has a pretty good custom UI, but I happened to be polishing up its metadata anyway and the controls group nicely so I used it as my test case.

Maybe some day this code will get smarter and evolve into a library that can be used by other Gtk hosts. Better group layout is the obvious “next level” advancement, a flat linear list of controls is pretty limited. Unfortunately there's no stock Gtk container which can do text-like reflow, which would be ideal. Perhaps a simpler scheme based on a maximum reasonable width for a controller would do, where the table will expand to have more columns if the window is wide enough for several.

Most of the metadata required to generate a good UI is also useful for other purposes, for example groups make building a decent menu of many controls feasible (e.g. to add automation lanes in Ardour), and some types of host UIs like a modular patcher inherently need to generate their own “UI” of sorts.

Good metadata is not at odds with custom UIs, they each have their uses, but it is important to remove the burden of custom UIs for simple plugins with just a few controls that really don't need them. It's a waste of that most precious of all resources - developer time - to have to code UIs for a few sliders and toggles. Hopefully better host generated UIs and support for more advanced controls like file selectors free up developers to spend more time making useful plugins and less time wrestling with GUI toolkits.

It is, after all, all about the sound.

Ardour MIDI Patch, Controller, and Note Names

2013-01-20T00:50:00-05:00

Ardour has supported displaying MIDI patch names loaded from a MIDINameDocument (or “midnam” file) for a while, though only patches, and even that was pretty flaky. Since this has become an itch for me and it's time for Ardour MIDI to reach release quality, I took some time to give it a serious overhaul. In this case, a picture or two is indeed worth a thousand words:

The note names are mainly useful for doing percussion. Several of the bundled midnam files have note names defined, and I have added one for General MIDI drums, which more or less corresponds to a lot of instruments, including Hydrogen.

Controller names, to my surprise, were not present in any of the existing midnam files at all. I wanted to sequence a hardware synth (the Moog Minitaur) without constantly referring to the manual and trying to remember which CC is which, so I wrote a new midnam file for the device and implemented support in Ardour. I am very disappointed to learn that there is no ability to group controllers, which really is the best way to do things, so I may embrace and extend (in a completely backwards compatible way) the format in the future to provide a better interface.

The quality of midnam files scattered around the 'net is atrocious. The MMA actually hosts a DTD for it, but most files use the wrong DOCTYPE (usually with a broken link), among other problems. I fixed and cleaned up all the bundled ones in Ardour, validated them against the DTD, and left a README in that directory about how to do so. Hopefully this, along with incentive added by these features, encourages the community to grow a nice set of quality midnam files.

So, MIDI name support in Ardour is now in pretty good shape. Now we just need to add the corresponding LV2 support...

Reviving Machina

2013-01-14T03:26:00-05:00

In the past few years of hammering on my repository while moulding Ingen into the modular of my dreams, my poor little generative sequencer Machina was often left to rot. That's a shame, because it's a fun tool to play with if you're into generative gadgets and pretty graphs.

So, I spent the weekend working on turning Machina back into a usable app. It's come a very long way in two days, from a mess that doesn't even compile to most functionality working pretty well:

Aside from rewriting a lot of the code, testing, and fixing things that were broken, I've changed the conceptual model a bit. There is now always just one start node for a machine. This tends to create more manageable structures, especially when recording several takes into the same machine.

Step recording has also been significantly improved. You can build machines entirely in the UI, but this is pretty tedious. It's much more effective to record, or step record if you want regular timing and no delays in-between notes. The above pictured machine was created by step recording MIDI, though I have tweaked the edge probabilities with the mouse. You can also import a MIDI file, which is an easy way to get a big machine to work with.

Hopefully I can find the time to turn this into an LV2 plugin and release it, but for now it is an unreleased Jack application. Give it a try if you're feeling adventurous, I would appreciate any feedback.

Host-Generated LV2 File Choosers

2013-01-10T00:29:00-05:00

LV2 has been slowly but surely moving towards message-based control to overcome the limitations of static ports. The required low level facilities have been available for a while now, but the lack of a standard way to document message-based parameters and host UI support has been a sticking point. One particularly pressing need is file parameters. The current sample example has a stop-gap UI to enable sample loading, but parameters really should be host controllable and simple plugins should not need a custom UI to be usable. Today I did some work towards resolving that situation.

Here is ConvoLV2 running in Jalv:

Note this is the built-in UI generated by Jalv and not a custom plugin UI. Unlike traditional LADSPA style controls, the impulse parameter is not a port, but is set by sending a message to an event port. The plugin lists parameters with the new patch:writable property, so the host knows which controls to display (or which parameters it can automatically set). Parameters are described as normal RDF properties, so any existing property can be used, or plugins can define them as needed in their data files (don't let the “RDF” scare you, the description of a Property is as simple as LV2 ports). One nice advantage of this over ports is that a project can describe a parameter once and use it in many plugins, without having to describe the parameter every time. This really adds up when you have hundreds of plugins with, say, an identical gain parameter.

These conventions will be established in the next LV2 release (1.4.0), and the sampler example will serve as a test case. When host support becomes established, we can also move towards using messages for numeric parameters, which will finally allow for dynamic parameters, control ramps, and so on.

Ingen as Remote Server or LV2 Plugin

2012-12-22T13:52:00-05:00

I have been working on the control protocol aspect of Ingen, towards a few goals:

Working GUI communicating with the server via a socket
Decent Python bindings for socket interface
Ingen as an LV2 plugin

The Python bindings got fleshed out to support a script for loading AMS patches into Ingen I have been working on (very preliminary, but in the repository and installed as ‘ingenams’). The socket interface is now sensibly synchronous, e.g. you send a command, and recv on the socket to get the response, which will include a status as well as whatever patch updates happened as a result of the command. I am happy with this route to language bindings, it's much more pleasant than a mess of direct bindings (e.g. via swig) and being protocol controlled has many other advantages. The Python interface is implemented purely in Python, it should be simple to do the same in any language that can use sockets.

The Ingen GUI can communicate with the engine via the same protocol, so you can build patches over the network (e.g. if Ingen is running on a headless machine as part of an audio rig). It can also run in the same process and use direct function calls for maximum performance.

The Ingen protocol uses the LV2 Patch vocabulary to describe changes. For the socket interface, this is represented entirely in Turtle. When running as an LV2 plugin, Ingen uses conceptually the same protocol, except in a binary format using LV2 atoms. This is a nice example of the benefits of having things defined in a good abstract model which can be encoded in many ways. For example, it would be relatively simple to implement control via JSON for a browser-based Ingen UI, which seems like the best way to go for separate control panels, but that's a topic for another day.

Since Ingen control can be expressed as atoms, it can run as an LV2 plugin with a full patcher UI and communicate only via LV2 ports. The UI is embeddable, and there is no memory sharing between engine and UI or hidden communication, everything is host managed. Here is a screenshot of Ingen running in Ardour 3. The MIDI in Ardour is played through an Ingen synth plugin, and the output has been recorded to the audio track. The patch can be manipulated live while running in Ardour, except for top level ports.

LV2 Plugin Transport Synchronisation

2012-11-17T01:20:00-05:00

I have been working on getting accurate transport information provided to LV2 plugins. Until now, there was the ability to specially designate e.g. BPM control ports, but this has limitations, the most obvious being that it is is limited to conveying simple numeric values at the start of a cycle.

In general, things are headed in an event-based direction, and this is no exception. The LV2 Time extension defines all the properties needed to describe a transport position/speed/tempo/meter and the LV2 Atom extension defines a dictionary type that can be used to send several properties at once in a single event. The host sends updates whenever the transport has changed in a way the plugin can not anticipate (anything but rolling along at the same speed/tempo/meter). Since events are time stamped, this makes it possible to give plugins a sample accurate tempo map, even when there are changes in the middle of a cycle.

There is a new example plugin in the LV2 repository, eg-metro, which is a simple metronome that follows this scheme. If the host implements the above, the metronome will click in sync with the host transport.

I have implemented this in Jalv (which uses Jack transport), Ardour 3, and Ingen (via a new internal "Time" block). Ardour will send an update at the start of the cycle if a relocate has happened, and one on the exact sample of any meter or tempo change within the cycle. Due to the limitations of Jack transport, Jalv and Ingen will only send updates at the start of a cycle. The metronome should tick correctly when loaded in Jalv or Ingen when there is a Jack transport master active (I have tested with Ardour), and when loaded directly in Ardour.

This functionality is pretty important for many musical plugins, so hopefully other host authors implement it as well. Depending on feedback (and available time) I may document the above rules as a feature and add it to the time extension, and release LV2 1.2.2. I think a well-defined way to achieve this is worthy of a point release, and it will get the news out there to implementers.

Benchmarking LV2 plugins

2012-08-22T23:29:00-04:00

A new utility has been added to Lilv for benchmarking LV2 plugins. As you might expect, it is called lv2bench.

Simple, but useful, lv2bench runs all discovered plugins for a specified number of frames split into a specified block size, and records the time each plugin took to execute. It prints a simple table like this:

Block	Samples	Time	Plugin
512	524288	0.005872	http://calf.sourceforge.net/plugins/Filter
512	524288	0.006461	http://drobilla.net/plugins/blop/amp
512	524288	0.008772	http://calf.sourceforge.net/plugins/Equalizer8Band
512	524288	0.008959	http://drobilla.net/plugins/fomp/autowah

This makes it easy to get a good overall feel for how expensive plugins on the system are relative to each other. Of course, a more expressive view of this information with several parameters would be useful... Robin Gareus to the rescue! Robin made a script currently called lilvplot which does several runs with various parameters and plots the data, which makes the relative performance more clear and shows the variance of a plugin's run time with error bars. This is important information because run-time variance is a good indicator of how real-time appropriate a plugin is.

Here is an example of lilvplot output generated from lv2bench data (as a massive SVG, you may want to open this image in a separate window and zoom in to see the plugin labels):

Both of these tools are very fresh and there are certainly improvements to be made (normalizing the y-axis so it showed where real-time is would be more informative, and lv2bench currently supports very few features), but they already make a useful addition to the LV2 developer's toolkit. I found them useful for evaluating the improvements had by vectorizing the arithmetic plugins in my soon-to-be-released port of blop. It will be interesting to see how things stack up in more thorough runs that include more plugins.

LV2 OpenGL UIs on Windows

2012-08-02T21:54:00-04:00

I have been working on Suil support for wrapping native Windows plugin UIs in Gtk. Here is a plugin with a Pugl UI embedded in the Gtk version of Jalv:

Very fresh work, but the basics (display, and mouse/keyboard input) are working well. This work is not actually OpenGL specific, any native Windows UI will work equally well. Apologies for the boring screenshot, this is just my test GL plugin that does nothing. However, it is a working example of a plugin with a portable custom UI; hopefully we'll soon see production plugins with UIs that work equally well across toolkits and platforms.

Portable OpenGL Plugin UIs

2012-04-30T22:40:00-04:00

LV2 allows plugins to implement UIs in any toolkit. This has led to UIs being implemented in several (which is a Good Thing, and works fine in all hosts via Suil), but mostly Gtk.

Unfortunately, Gtk is not really suitable for use in plugins on platforms where a “system Gtk” can't be relied on. Some toolkits are suitable for static linking, but personally, I am somewhat disillusioned with “toolkits” lately, and massive libraries in general. Sometimes all you want or need is a straightforward standard graphics API and some keyboard/mouse events.

When it comes to standard cross-platform graphics APIs, the undisputed heavyweight king is OpenGL. While not perfect (what is?), no other API is already there on almost any platform you'd care about (heck, most modern phones have hardware accelerated OpenGL). Unfortunately, OpenGL deals only with rendering, and not user input or windowing issues. What is needed is a minimal framework to get an OpenGL view to draw to, and receive keyboard and mouse events. Enter Pugl.

Pugl handles all the platform specific business behind an API very similar to GLUT, but much smaller and appropriate for plugins (which GLUT unfortunately is not). In terms of size, this is a few hundred lines of C per platform (on a personal note, this fits in well with my ever-increasing distaste in dealing with bloated junk with tons of dependencies... give me a Couple Hundred Lines of C™ any day). The breakdown on Pugl is on its homepage, but suffice to say after a few days' work it does the job it was designed to do on X11, Mac OS X, and Windows. Since embedding X11 works in LV2 land, that means an OpenGL plugin UI can be embedded in the host, and I have the pretty pictures to prove it:

This is a simple test plugin (complete with gratuitous use of 3D) embedded in Ingen and Ardour. Both are Gtk based programs, but this works in Qt as well. Testing so far on other platforms has only been top-level since I have no programs to embed in, but the bulk of the work is done. This includes full keyboard and mouse support, with significantly more complete keyboard support than GLUT.

Naturally I can't predict the future, so it remains to be seen how much OpenGL UI action we'll see for plugins. Being just a low level drawing API and not a set of boxed widgets, it's a bit open ended with a bit of a learning curve, but on the other hand there is lots of existing OpenGL code out there. Perhaps someone will throw together a library of audio appropriate widgets, if one doesn't already exist. Either way, I think an easy to use API for writing truly portable LV2 plugin UIs is a very good thing, which hopefully eliminates a barrier for some plugin developers and helps LV2 invade the territory of its proprietary adversaries... or, at the very least, makes for a really cool 3D panner GUI :)

LV2 Atom and State Serialisation

2012-02-23T01:42:00-05:00

I have been working on full round-trip serialisation for everything in the LV2 Atom extension (which also applies for implementing state). I am doing this as a small library with a simple API, with the goal that it be simple to include in any project.

Currently this only writes (using Serd), I still need to move the reading stuff into it (which will probably use a model and thus require using Sord).

The Atom extension defines a simple data model for use in LV2 plugins and hosts (or elsewhere). The general Big Idea here is to have a smallish set of primitive types, a few collections, and out of that one can build arbitrarily complex structures. Everything (including containers) is a simple and compact chunk of POD data, but serialises to/from (a subset of) RDF, so it can nicely be described in a plugin's Turtle file, among other advantages.

An easy to adopt serialisation API is important to making these advantages a reality for many implementations, so I have decided to provide one before marking these extensions stable. It also serves as a nice test case with complete coverage. Here is an example of an Atom that contains every currently defined Atom type, as well as MIDI, serialised to Turtle by sratom:

[]
    rdf:value [
        a eg:Object ;
        eg:one "1"^^xsd:int ;
        eg:two "2"^^xsd:long ;
        eg:three "3.0"^^xsd:float ;
        eg:four "4.0"^^xsd:double ;
        eg:true true ;
        eg:false false ;
        eg:path </foo/bar> ;
        eg:uri eg:value ;
        eg:urid eg:value ;
        eg:string "hello" ;
        eg:langlit "bonjour"@fra ;
        eg:typelit "value"^^eg:Type ;
        eg:blank [
            a eg:Object ;
        ] ;
        eg:tuple [
            a atom:Tuple ;
            rdf:value (
                "foo"
                true
            ) ;
        ] ;
        eg:vector [
            a atom:Vector ;
            rdf:value (
                "1"^^xsd:int
                "2"^^xsd:int
                "3"^^xsd:int
                "4"^^xsd:int
            ) ;
        ] ;
        eg:seq [
            a atom:Sequence ;
            rdf:value (
                [
                    atom:frameTime 1 ;
                    rdf:value "901A01"^^midi:MidiEvent ;
                ] [
                    atom:frameTime 3 ;
                    rdf:value "902B02"^^midi:MidiEvent ;
                ]
            ) ;
        ] ;
    ] .

I anticipate/intend for all plugin control to happen via such messages, since this approach has a few important qualities:

Typically no need to define new binary formats for things (and be held back waiting for others to implement them).
Everything has a portable serialization for free (meaning network transparency, saving/loading, and for developers or power users the ability to dump any message to see what is going on).
The convention is to use “objects” (resources, i.e. things with properties) as messages, which are inherently extensible. No “oops I needed to add a parameter so now compatibility is broken”.
Easy to bind to other languages or syntaxes, so e.g. Python or browser-based UI frameworks should be possible.
Any RDF vocabulary can be used, meaning millions of well-defined and documented predicates (“keys”) are available right now (though it is perfectly okay to create one-off objects - compatibility with RDF is a benefit, not a burden).

The atom extension includes an API that makes it relatively simple to build such objects in C, so plugins can write them directly to an output port or a ring buffer. See the “forge” API in the Atom extension for details. There are also iterators for all the collections and a “get” function for objects to make reading data simple.

Just in case it's not crystal clear, the above is only the external representation of the corresponding atom. At run-time, an atom (i.e. what plugins work with) is just a blob of data with an integer type and size header. 100% of the API provided for reading and writing atoms is real-time safe and suitable for use in an audio processing thread.

For an example, see the LV2 sampler example, which has a UI that loads samples via such messages. It currently works in Jalv, Ardour support is coming soon. This is the way forward for more powerful LV2 plugin control, and hopefully will end the worrying practice of abusing direct instance access to provide such functionality.

This work isn't finished yet, but the important parts are done and not likely to change significantly. I am interested in hearing any developer feedback, feel free to comment on this post or at the LV2 mailing list.

[Edited on 2025-02-16 to update broken links]

New LV2 Host Implementation Stack

2011-05-25T00:40:00-04:00

I have released my new stack of libraries for implementing LV2 in hosts:

Serd, a fast, lightweight, dependency-free Turtle syntax library
Sord, an in-memory RDF quad store
Lilv, an LV2 host library (replaces SLV2)
Suil, an LV2 UI loading/embedding library

These libraries collectively replace SLV2, and have no dependencies except amongst themselves, and Glib (by Lilv and Sord, but this dependency will likely be removed in the near future). Serd and Sord replace Redland, making for a dramatically smaller implementation more appropriate for audio software and embedded applications.

Overall, Lilv is dramatically faster and leaner than SLV2, enough that the improvement should be quite noticeable from a user point of view (typically in a lag when the host first loads all LV2 data). Anyone using SLV2 is highly encouraged to migrate to Lilv.

These libraries are well tested, each (except Suil) with a test suite covering over 90% of the code, which runs without memory errors or leaks. They are new, however, so (as always) there may be problems; feedback is most welcome.

[Edited on 2025-02-16 to remove broken links]

SLV2 Is Dead, Long Live SLV2

2011-04-29T14:22:00-04:00

(See below for important information for anyone following my SVN repository)

I've decided to replace SLV2 with a new library (named “Lilv”) instead of breaking the API which has been causing a lot of hassle, and would only cause more when it gets released and packaged. Among other things, this gives me the opportunity to rework the installation to support parallel installs of different major versions to prevent this kind of problem in the future. It also gives me a chance to improve the API and remove some cruft that has accumulated to avoid breaking the API (SLV2 has been around for many years now, and LV2 (and myself) have come a long way since then, it's a good time to do some laundry).

Anyone who has an SVN version of SLV2 installed, please uninstall it immediately and pretend it never existed. SLV2 0.6.6 is the last version of SLV2. The first Lilv release will be out shortly.

Unified LV2 Core/Extensions Code Documentation

2010-09-22T15:37:00-04:00

I've reworked the LV2 documentation generation system to generate all code documentation for lv2plug.in in a single Doxygen run. This means the core and all extensions' documentation are hyperlinked between each other, and there are handy global indices of everything.

See http://lv2plug.in/ns/doc/html/ for an index.

As before, you can go directly to the URI of any lv2plug.in extension in your browser for documentation, which links to the code documentation for that particular extension.

Hopefully this reference proves useful for implementers.

GStreamer LV2 Surround

2009-08-12T15:06:00-04:00

My last post was mostly about adding multi-channel support to GStreamer.

That was only partially implemented though - many elements in GStreamer (and pretty much all of the ones that support more than stereo) require each channel to be explicitly labeled with its position. This information wasn't bridged until now.

The two sides of this situation are:

The LV2 Port Groups extension: http://lv2plug.in/ns/dev/port-groups.lv2/port-groups.html
GStreamer multi-channel: http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-base-libs/html/gst-plugins-base-libs-gstmultichannel.html

LV2 “roles” correspond to GstAudioChannelPosition. Unfortunately, the mapping was a bit strained with the Port Groups extension as it was at midterm time. Fortunately, that extension is not finalized yet, and in the process I found some other problems with it, so I decided to bring it closer to the way GStreamer does things, which seems to be common in other software that does this sort of stuff. Formerly, the port groups extension used the channel names from the relevant surround spec (see the Wikipedia page on Surround Sound). This resulted in a lot of roles that were really the same thing, e.g. the “Surround Left” channel in 5.1 is really the same as the “Left Back” channel in 7.1. This was pretty confusing in practice. GStreamer instead always uses positional names like “Left Back”, which is unambiguous regardless of context (you don't have to know if it's 17.2 or whatever to know what “Left Back” probably means...). So, I changed the LV2 port groups extension to be similar (though not identical; munging an LV2 extension to match GStreamer API wouldn't be appropriate).

This information is now queried out from LV2 plugins that have it, mapped to GStreamer channel positions, and set as capabilities on mulit-channel pads. This means multi-channel surround LV2 plugins should now work fine with surround sound GStreamer set-ups, for example playing a Theora video through some audio processing and out through Alsa.

I updated swh-plugins for the minor changes, and added port group information to Invada Studio Plugins.

More to come...

GStreamer SoC Midterm Summary

2009-07-05T19:30:00-04:00

Apologies for not blogging along the way for those who are interested in this project. I'm more into actually doing things rather than blogging about doing things ;)

So, LV2 in Gst, where are we? Well, I've had LV2 plugins working with the same functionality as LADSPA plugins for a while. The main downside was shared by LADSPA: no support for multi-channel streams.

Some background for those unfamiliar with LADSPA or LV2: LADSPA and LV2 plugins are very simple in terms of I/O. A plugin has a collection of “ports” which contain either a single floating point value (control) or an array of floating point values (audio). LV2 extends this to any types but that's not important here. This is simple but causes problems in the face of stereo and other multi-channel streams: if a plugin has 3 audio inputs and 2 audio outputs, for example, what is what? Some kind of 3-channel audio input with stereo output? 5 completely unrelated ports? A stereo input and stereo output with a “sidechain” (common with e.g. compressors)? This information is not available in LADSPA, the best you can do is guess (which is practice means it just doesn't work).

GStreamer works with multi-channel streams as a single interleaved stream, so this is a problem. Thankfully, LV2 makes it trivial to add whatever information you like about plugins without having to touch a line of code since plugins are described in RDF (see the LV2 site for more information). All that needs to be decided is how to actually model that information. A specification for this is called an “ontology” in general/theory, and an “extension” in the LV2 community.

So, the problem is we need(ed) a good multi-channel ontology for LV2 plugins to work well inside Gst, since most things in Gst are at least stereo. The difficult thing with creating ontologies is making sure anything anyone might want to describe in the future is accounted for. Here's my best shot at this so far: LV2 Port Groups, based largely on earlier work by Lars Luthman and some input from the LV2 community. This extension isn't final, but expresses all the information needed by Gst for multi-channel (and more). (I also wrote the documentation generator that created the aforementioned specification page in the hopes that more user friendly documentation will encourage adoption by plugin and host authors).

I've created patches for the popular SWH and Calf plugin packages to add this information. When the extension goes final they will be included in these projects, but in the mean time are included in my git branch of gst-plugins-bad (see http://git.drobilla.net).

This is, as far as I know, the first time coherent multi-channel information has been available about plugins from the “LAD” community (e.g. LADSPA, DSSI, LV2). Coincidentally, this information is required for recent work on Ardour, among other things. In hindsight, this was a pretty glaring hole in the general state of LAD plugins, but back to GStreamer...

I've rewritten quite a bit of the GstSignalProcessor class (used by the LADSPA and now LV2 wrapper elements) to support multi-channel plugins. In code terms this means creating a set of buffers for non-interleaved data used by the plugin, and interleaving/deinterleaving buffers to/from Gst, respectively.

In summary: many LV2 plugins now exist with useful multi-channel information, and stereo LV2 plugins now work correctly in GStreamer. It should now be simple to add support for other audio plugin interfaces (VST? AudioUnits?) that also works with multi-channel streams.

Next up:

Get the “role” information from the plugin data and use it to support surround streams correctly
Finalize and publish the LV2 Port Groups extension, and contribute patches for all major plugin collections

Where to go from there is pretty open-ended. Unlike LADSPA, LV2 can theoretically support any kind of data, or any feature (non-realtime and non-audio things included). Extensions just need to be made to bridge the gap. What sort of functionality would you like to see bridge the GStreamer/LV2 gap?

DOAP replacing AUTHORS/MAINTAINERS/etc

2008-01-18T12:25:00-05:00

There's been a bit of talk in the GNOME camp lately about using DOAP instead of the unstructured text files that are the current norm for source packages. On the one hand, people want the benefits of having machine readable data in projects, OTOH, RDF/XML is a nightmare (“I'll never maintain such bloat!” - “That is one hell of an ugly file.”).

This is how RDF/XML hurts RDF. The original loathed file:

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl"?>
<rdf:RDF xml:lang="en"
         xmlns="http://usefulinc.com/ns/doap#" 
         xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" 
         xmlns:asfext="http://projects.apache.org/ns/asfext#">
  <Project rdf:about="http://ant.apache.org/">
    <created>2006-02-17</created>
    <license rdf:resource="http://usefulinc.com/doap/licenses/asl20" />
    <name>Apache Ant</name>
    <homepage rdf:resource="http://ant.apache.org" />
    <asfext:pmc rdf:resource="http://ant.apache.org" />
    <shortdesc>Java-based build tool</shortdesc>
    <description>Apache Ant is a Java-based build tool. In theory, it is kind of like Make, but without Make\'s wrinkles.</description>
    <bug-database rdf:resource="http://issues.apache.org/bugzilla/buglist.cgi?product=Ant" />
    <mailing-list rdf:resource="http://ant.apache.org/mail.html" />
    <download-page rdf:resource="http://ant.apache.org/bindownload.cgi" />
    <programming-language>Java</programming-language>
    <category rdf:resource="http://projects.apache.org/category/build-management" />
    <release>
      <Version>
        <name>Apache Ant 1.7.0</name>
        <created>2006-12-13</created>
        <revision>1.7.0</revision>
      </Version>
    </release>
    <repository>
      <SVNRepository>
        <location rdf:resource="http://svn.apache.org/repos/asf/ant"/>
        <browse rdf:resource="http://svn.apache.org/viewcvs.cgi/ant"/>
      </SVNRepository>
    </repository>
  </Project>
</rdf:RDF>

and the equivalent in Turtle (a subset of N3) (automatically generated with rapper -o turtle doap_Ant.rdf):

@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix : <http://usefulinc.com/ns/doap#> .
@prefix asfext: <http://projects.apache.org/ns/asfext#> .


    a :Project;
    :created "2006-02-17"@en;
    :license ;
    :name "Apache Ant"@en;
    :homepage <http://ant.apache.org>;
    asfext:pmc <http://ant.apache.org>;
    :shortdesc "Java-based build tool"@en;
    :description "Apache Ant is a Java-based build tool. In theory, it is kind of like Make, but without Make's wrinkles."@en;
    :bug-database <http://issues.apache.org/bugzilla/buglist.cgi?product=Ant>;
    :mailing-list <http://ant.apache.org/mail.html>;
    :download-page <http://ant.apache.org/bindownload.cgi>;
    :programming-language "Java"@en;
    :category <http://projects.apache.org/category/build-management>;
    :release [
        a :Version;
        :name "Apache Ant 1.7.0"@en;
        :created "2006-12-13"@en;
        :revision "1.7.0"@en
    ];
    :repository [
        a :SVNRepository;
        :location <http://svn.apache.org/repos/asf/ant>
        :browse <http://svn.apache.org/viewcvs.cgi/ant>
    ] .

I wouldn't want to hand maintain for RDF/XML version either, but the Turtle version? Sure. It's the exact same information, far more human readable, and about as terse as it could be while representing the same information.

The best thing about a syntax independent model like RDF is.. well, it's syntax independent. Choose one that doesn't suck :)

Ardour: Back On Track. Maybe.

2007-06-24T12:00:00-04:00

Well, I've been working on Ardour but not directly on my SoC project.. finally got around to fixing that whole mixer-strip-element thing that's been on the table since last year. Things are more open to extensibility now (and a unified bus implementation should appear soon, so we'll have MIDI busses or even MIDI/audio (instrument) busses). Maybe we'll see things like this actually happen some time soonish...

It would be nice to figure out a really good MIDI meter (something more clever than just an audio peak meter abused), and a MIDI fader that can work in various ways, or even instrument plugin support, but it's probably time to get on the actual piano roll editing part of the project and leave that stuff until afterwards.

Kinda ruins my flow though, you know?

Back to the Hack

2007-05-29T13:58:00-04:00

Well.. finally fully moved; I guess it's time to stop screwing around and get back into hacking. Specifically Ardour, since I'm being payed to do so and all.

Looks like we don't have a fancy new cairo canvas to play with. Maybe it will be worth my time to do that first, but I'm going to do some fiddling with displaying MIDI data with gnomecanvas first to get a grasp on things. Wouldn't hurt to have some visually obvious clue that I am, in fact, actually doing something. This summer's project should be more rewarding than last in that sense; most of what I need to do is visual stuff which tends to be more fun since you have something nice and tangible at the end of the day.

Of course, displaying data is one thing. One relatively easy thing. Actually editing it on the other hand.......... that's where the “fun” (i.e. hard) part comes in.

I think a top-down (GUI->implementation) strategy is best here. The Grand Battle Plan(TM) goes something like this:

Get MIDI data displayed in regions, notes visible as one canvas item (rect) per note
Attach event signals from note canvas items (move, click, etc) to a set of stub methods that encompass all the editing operations
Now there's a nice centralized area where all the editing operations need to be implemented
Figure out how the hell to implement them (ie ???????????)
Implement them (ie Profit!)

Unix Modular Synth

2007-04-28T17:09:00-04:00

I've decided to use my blog to archive random ideas until I figure something better out (on that note, anyone know of any good RDF based mind-map software, or something generic that can easily be used as one?)

Until then:

<drobilla> crazy thought of the day: write an LV2 extension for text
stream input/output ports, and you can write a trivial unix cmd line
utility wrapper plugin
<drobilla> anyone want to convert midi to text then run it through
sed and grep and convert back to midi in a modular? :)
<gordonjcp> ahahaha
<gordonjcp> you sick sick man
<drobilla> unix modular, best thing ever

Ingix? Really wouldn't be that hard. Could even make it parallelize forkey pipelines, if the LV2 contexts extension is powerful enough. More on that later...