drobilla

… done and done. It is so much nicer to right click a canvas and hop through a few class (”category”) menus to find the plugin you want than search a big flat list in a dialog. Short of a couple of semantic wars over classes (I already had to change the schema a bit to make the menu less crap), that’s done. As usual, need to go over the API and make sure there’s no potential future binary incompatibility issues, but I think it’s pretty good. SLV2 is quite a bit more of an abstraction layer than I initially planned, but the API is more solid and flexible, as far as being able to change things underneath without breaking things.

Look at me, finally learning that compatibility is important.

So what’s left now is the hard part: exposing full(ish) RDF querying. I don’t want to expose Redland (gone this far without) at least for normal uses, so I guess I will have to make a query results ‘class’ etc.

First is the bigger issue though: at the moment there’s no typed literals in SLV2 at all. This is not good…

Well, I did the Redland modularization thing to tear those dependencies out. Luckily for me, Dave Becket wrote the storage module stuff in anticipation of modularization, so it was actually really easy (what’s with people named Dave and being absolutely brilliant eh?) So, pending whatever autohell stuff needs doing to make it possible/easy to package into separate bits, that problem’s solved.

Armed with my new anti-whining-user ammo of future dependency dissapearance, I pretty much rewrote all of SLV2 to use one big model of all LV2 related RDF on the system. I figured with an in memory model, querying the whole thing should be quite fast (it’s not that big), and there’s some fun to be had with a big single web of data containing everything plugin/extension/whatever related. Happily, going through and parsing all the files is nearly instantaneous (Raptor has to be the fastest RDF parser in the west by a long shot)…

… and non-trivial (but necessary) queries take so long and chew 100% CPU that they’re unusable.

Shit.

Long story short, debugging and/or rewriting half of Rasqal isn’t exactly high on my Things I Feel Like Doing Right Now list, so I rewrote SLV2 again. Now each plugin has it’s own in-memory model, which works quite well. Lost that big web of data, which I thought was a pretty cool idea, but hopefully whatever’s wrong with Redland can get sorted out by the time anyone has a real use for that anyway. The plugin model can still load plugin extension bundles etc. into the same model, and that’s the important part anyway (which makes LV2 plugins extendable without having to modify the plugin’s bundle itself). Not really sure how the main ontology and LV2 extensions like categories and other system stuff fit in to this just yet, but that’s a train of thought for another day.

The performance problems Zynjacku was having are much better now. The loading guage no longer pauses on plugins with large numbers of ports, it zips along at a constant speed and is quite a bit faster overall.

It could be faster still if I added an API to get plugins with a custom query, but that will probably have to wait until the Redland query lock-up thing is solved. Anyway, it’s much, much better now, and the API has been redone in a better more abstractioney way that should allow any kind of redesign behind the scenes (as far as cacheing models and whatnot) without breaking the API. Also the API is now reentrant (well, if Redland is anyway?) which I figure is probably going to be an issue at some point once some plugin and it’s host both use SLV2.

Now to just get the thing fully working again…

Blissfully ignoring the fact that I have an exam next wednesday that’s probably going to be pretty hard and involves mathematics and whatnot…. free time!

So, now is the time to get some damn releases out and off my back, starting with stuff that other people depend on (though my toys are frankly much more fun to work on. You all owe me a beer). Particularly before the Summer of Code starts, which (27 hour a day jokes aside) is going to chew time like crazy. Hopefully I manage to get outside and see the sun once or twice this summer, but that’s another topic.

Step 1 is SLV2. LV2 really needs to get out there, yesterday. The main problem right now is exposing categories and performance issues with querying lots of plugins with large numbers of ports.

Categories is a bit tough to do right. The easy way out is to just assume the categories are as in in ontology, but that’s crap and very un-LV2ey. The way bundles work you can just drop in extensions with more categories and they should be able to magically show up in all the user’s apps without the apps having to change at all. The hard part is dealing with weird category heirarchies and how to expose it all in an API. The weird heirarchies is solved easily enough by making a few assumptions. Exposing via an API I don’t know. I suppose having separate methods to get the category heirarchy (as URIs and names, probably), then just allow getting the (many? a?) category for a plugin and the host can do whatever it wants from there. That’s easy enough, seems like it will work. How to return the heirarchy is a question, I suppose the structure in memory can be a tree that directly maps to the heirarchy somehow. This kind of thing is much more annoying in C (especially for a little OO weenie yungun’ like myself), but it can work. One thing that I should probably keep in mind is that clamping things to a strict category heirarchy is not a good idea (kids these days and their “tags”), even though virtually all apps will use categories as a (intrinsically heirarchial) menu. Both need to work.

Performance is a bit more ominous. I think maybe depending on full Redland might end up being necessary. I have avoided that in all my software so far because of dependencies (mostly mySQL) which people would complain about. Having mySQL as a dependency to use LV2 plugins is definitely not acceptable, but not using Redland (i.e. just using Raptor and Rasqal) is really, really, really limiting, and slow, and annoying, and just not good. They simply weren’t designed for the kind of higher level stuff I’m doing, that’s what Redland is for.

So, fiddling with Redland to split the stores etc. into dynamically loaded modules so they can be packages up separately is The Solution here. Unfortunately adding yet another project to the stack is exactly the opposite of what I need right now… it would be nice to be able to spend a bit more time on Redland though. My little vendetta getting RDF/Turtle on the desktop depends on having an RDF toolkit that’s lightweight/appropriate enough for installing on user machines, and Redland is clearly the one for the job. I know this is good, powerful technology, and the things it will allow us to do are concrete, tangible, awesomeness; despite what any (usually just plain ignorant) naysayers may think. Having a proper in-memory RDF store (Redland) is necessary for most of these things, and the API is much nicer regardless (plus there’s Python bindings which are, IMNSHO, the single greatest thing ever).

On a related note, I am so incredibly sick of school interfering with my time and ability to work on this stuff it’s not funny. You know your life is fucked up when you do more ‘work’ in your free time than most people do at work. My brilliant solution to this problem? Subject myself to even more of this utter waste of time and go for a Comp Sci Master’s! GOOD PLAN. I’ll be exactly where I am right now, except 2 years more bitter, stressed to death, and old (which at this particular point in life is a really, really relevant factor. Do I really want to waste what tiny shred of youth I have left slaving away at pointless work I despise? No, no I do not.)

I’m not a masochist; just weak. Weak and stupid.