Cleaning Up the LV2 Extension Mess

After reading my last post, and watching a few old talks around LV2 and so on, I got to thinking about the extension mess problem I mentioned, and it occured to me that there might be some commonality here with the "staging" or "contrib" area question as well.

This is all based on some ideas that have been bouncing around in my head for ages, but that I haven't really developed and certainly not written down, so I'm going to try and sketch out a proposal for how to handle these things without breaking anything.

Concretely, there are two problems here: one is that the spec is just a mess. For example, the Data Access and Instance Access extensions are really just parts of the same thing and should live together, nobody cares about Morph and it's not in a state that really belongs in the "recommended standard" list (sorry, flagrant abuse of power on my part there), and so on.

The other problem is that there are sometimes contributions which solve a problem, and are a reasonable enough pragmatic step, but also not really up to par. Maybe they aren't portable, aren't defined well enough, could do more harm than good if they're presented as recommendations, and so on. People, for whatever reason, want them "in LV2". Yet, nobody has the time to spend to develop them into a more proper specification yet, and nobody is happy when things don't get merged.

It seems there is a common factor to these problems, and it's moving things without breaking anything. To clean up the current mess, we can move extensions to the contrib area. When a previously half-baked contribution is developed further, we can move it from the contrib area. This is an obvious coarse-grained use case; I think there is also a case for finer-grained URI migration, but I'll focus on the easy and most useful case for now.

How might we do this? Though moving instance-access to contrib is not a goal, it's about as simple as an extension gets, so I'll pretend we want to do that for the sake of a simple example. At the very least, it will be a nice little fantasy for me to pretend that the curse of crappy plugin UIs that mess with DSP guts has finally been vanquished for good :) This is just about the mechanism, what we should actually do to clean things up is a question for another time.

So, what's instance-access? It's a handful of URIs, and a feature. The feature is extremely simple, the payload is just some pointer. Can those URIs be moved without breaking anything? For at least this simple case, I think so:

I can't think of a reason this wouldn't work, and it doesn't even require any host changes. It's a bit bloated, but not in a way that matters, and would need a significant (but not too bad) amount of code specifically to deal with this in lilv, but such is my lot in life.

In the more general case, there is also the issue of URID mappings. Let's pretend that is mapped to a URID both by the host and the plugin, and that URID is sent between them. Though this isn't really an intended use-case for this particular extension, it's a perfectly valid thing to do:

... that's it, actually. Regardless of which "version" either host or plugin know about, the URID is identical. This requires hosts to actually implement something though, or for a URI map to be added to lilv, so it's not as easy. It can't just be done in LV2 and would take some time to get established.

There is one remaining snag: extension_data. This one is a bit trickier, because we need to assume the hosts uses lilv_instance_get_extension_data which is just a trivial wrapper, and probably not used by everyone. That's an easy enough fix to make, though. Then, lilv just needs to call the plugin method for the new URI, return that if it isn't NULL, and fallback to calling it with the old URI.

All of this requires a map between old and new to exist, of course, but this would be written down in the specs themselves and it's easy enough to load such a thing inside lilv.

I'm sure there are other places where URIs as strings are used in the API that would need thinking about, and I'll have to scan through the spec to see, but I suspect the above is at least 90% of what matters.

So... am I missing something? Do send me (or lv2-dev) an email if so, but now that I write it down this seems more viable than I assumed it would be. There will definitely be corner cases, since plugins and hosts can use these strings for anything everywhere, but as far as the actual interface is concerned it seems possible to make this happen without too much pain. What could we do with this?

That would leave a specification list like this (assuming parameters and port-properties move to "control"):

Not everything left is immaculate, and from a user-facing documentation point of view other things like putting the data-only vocabularies in a separate section might help even more, but I think this would be a big improvement. More importantly, it would of course give us an attic to put slightly more sketchy things. Looking at LV2 as a Specification™, that feels wrong, but looking at it as a project, it seems really necessary.