Writing an LV2 Book

It has been clear for a while that some kind of higher level documentation is needed for LV2. The reference documentation is much better than it used to be, but this isn’t very good at giving an overview, and is very imposing to new developers since LV2 is a bit different than, say, a library API due to the code/data split.

Overview documentation can’t really go in the extension documentation, since many tasks plugin developers want to achieve involve several extensions (or no extensions at all). I have been wondering for a while what form this documentation should take: FAQ? Wiki pages? Just examples that illustrate specific things? For whatever reason I had never thought of a book format, but now I think that is the best approach. Too many LV2 ideas live in my head alone, especially the umbrella ideas, which is not good. A book seems like the best way to convey an overall understanding of LV2. This is important – LV2 is what I call a “click” technology (like LISP or RDF), it seems insane until it “clicks” and you get it. You really need that little enlightenment experience to be able to work with it well (and especially to be able to work on it). I need to convey that to more people. Of course it would also serve as a simple “How To” manual for plugin developers, which is the primary goal.

By “book” I don’t necessarily mean paper based. While that might be nice a web version definitely should exist. I mean a largish document with several chapters and sections, with a clear overall arc that starts with the most basic plugin possible, and each section is about a specific type of functionality, like “receiving MIDI events” or “transport synchronization”. That way, when a developer has a broad question like “how do I implement state, and dynamically control it?” there is a nice book chapter on exactly that topic with explanatory text.

The other thing that is definitely good is examples. We still have only a few official examples in the LV2 tree, more would be nice. Making this book very example centric would be a good idea, solving several problems at once. Perhaps a Literate Programming like approach, where each chapter is really just the comments of a working plugin that demonstrates the relevant functionality. This way, even people who won’t read the documentation (i.e. most) still have working examples to build from, and these examples would also serve as nice test cases for hosts.

The early sections could also serve as a more proper specification for things like discovery, module loading, and so on, which thus far we haven’t documented so well, but honestly I don’t see this as a very high priority. It’s a nice idea to have an authoritative LV2 document, but in practice hosts should just use a library (like Lilv) for that stuff anyway, and most do.

In terms of publishing tech, I am not sure. I would like the examples to be actual C source code without duplicating everything, hence the Literate Programming idea. Literate Programming in philosophy only, not actually using a weird format like CWEB. I’d just parse out the comments and render those as text, and pretty-print the code in-between. This is simple, and with Pygments it can easily output both HTML and LaTeX for both an online and a typeset version. This way, for “just show me the code” types, the example source is just normal C code, but heavily documented with explanatory prose. As it happens I banged out a little Python script to do just that this evening. The tricky part is things like links, references, and so on, that require special syntax. I’d really rather not have to invent and implement a new comment syntax. However, perhaps something so directly tied to code can just avoid the need for any of that anyway, and the master document can just include all the generated bits. Might have to manually make a separate HTML and LaTeX master document but that’s not too bad. This would all be trivial if I only use LaTeX, but since this is documentation, being able to link to a chapter online and such is important. Anyway, this is beside the point, the important thing is the general idea of writing an LV2 book.

I think this is a really good idea, and though I don’t exactly have all the time in the world to start a book-level project, I might start building it in the LV2 repository so it is part of the official distribution. As the spec grows, the book can grow along with it. I envision future LV2 releases adding specific functionality (like transport sync, which will be the upcoming one), along with a new book chapter on the topic. On a personal note, this meshes well with desire to stick to non-realtime communication (e.g. email), since I can’t have the constant distraction of IRC in my life any more. I’ve spewed enough text on to the Internet to fill volumes, might as well put that into something of lasting value.

To me it seems clear that this is the best way to better document LV2 and make it more accessible to newbies, but I’m notoriously bad at predicting what others want, so any feedback would be appreciated. If I wrote an LV2 book, would you read it, or at least use it as a reference?

Is it worth my time to do this, or should I just stick to hacking when I have time to do LAD stuff? If there’s a good chance that this will increase the Bus Factor of LV2, I think it’s probably worth it.

(Flattr this)

16 thoughts on “Writing an LV2 Book

  1. Chances are not that great that I end up writing LV2 plugins anytime soon, but I think literate programming examples would be the best possible documentation. The audience might stay small, though.

    While XML has its issues and is overused, it works well for documentation with several targets. A Docbook source could make it relatively straightforward to transform to HTML and PDF or even Latex as intermediate. (I recently used a Docbook subset and wrote custom XSLT to get HTML5 out of it.)

    • While my hatred of XML is well-known, for documents it’s decent enough. The abuse of XML for data, for which it is crap, is the real problem.

      I’ll have to look in to the quality of Docbook output for HTML and PDF.

      The audience is always going to be relatively small, but if such an effort would increase it, or be helpful to the few new people a year who do show up and do some work, it’s worth it.

    • Yeah, Docbook should do. However, LaTeX+links is possible: check the hyperref package (http://www.tug.org/applications/hyperref/).

      Anyway, I am one of those who reads documentation and I already “clicked” long time ago. If it’s going to be done, dependending on my mood during these Xmas holidays, I might even decide to give a hand.

      • Sure, but I think it’s a pretty safe bet that people would rather have web documentation than a PDF. Being immersed in academic land lately, naturally my instinct is to make a PDF, and I would like something that can produce a decent looking document, but not having a proper web version seems unwise. There are things like latex2html which do a decent job, though they choke on things as advanced as what Pygments outputs for LaTeX.

  2. this is a fantastic idea and sorely needed.

    personally i think lv2 is a fantastic API and i can see myself doing a lot with it, but there’s a lot that i just don’t understand. i find it largely impossible to relate the RDF side of things to C. for example, the “patch” extension — I have no idea how or what the hell it does. and the “time” extension, until eg-metro came out, was similarly impenetrable. control ports? they are still mentioned nowhere in the time documentation. i presume these are how “patch” works as well?

    a good start for the book would be example code in C for all of the extensions. a header file that defines some constants is absolutely worthless to someone unfamiliar with RDF or LV2 as a whole. what we really need, honestly, is “LV2 for VST developers” or something like that. we’re all used to writing C/C++, and without good documentation about how the RDF relates to the code itself, LV2 is much harder than it needs to be.

    • Yes, these are the kind of concepts that don’t really fit in to extension documentation.

      The patch extension is pretty much just a vocabulary for describing changes to key/value data. It doesn’t really have much of a relation to ports, and certainly not lv2:ControlPort. The main case for its existence is sending messages (i.e. events) to manipulate plugin state. The sampler example works this way. The idea is that you only need to define one key, say, foo:mySampleFile, then you can use it both dynamically at runtime to control things (e.g. from the GUI), and also use the exact same key to store your state with the state extension. This avoids having to both define some message interface (e.g. OSC style /set_my_sample_file) and keys for saving state. You just figure out how to describe your state in a key/value way and you get both for free. The atom extension is heavily involved in actually being able to realize such things.

      I would not target such a book to VST developers specifically. Its purpose would be to simply explain all the concepts well so anybody could understand it. Making every other sentence a strained analogy to some VST concept that doesn’t really match up at all would likely just make it less clear. Comparison to other APIs is probably best left in a short chapter dedicated to that topic, but I don’t think it’s a particularly high priority.

      • w.r.t. “VST developers”, yeah, I definitely wouldn’t prioritize easing people in from other plugin APIs specifically. i should have been clearer in saying that your target audience should be “what the hell is RDF? why do I suddenly need this? can’t I do all this in C/C++? why not?” my point here really is that I *want* to use LV2 but generally it doesn’t take much to make me feel like I’m in over my head.

        patch makes a bit more sense now. it really seems like the whole concept of URIs is the same as defining magic numbers as macros or in enums but it’s one step removed and these URLs must be “resolved” to magic numbers with urid. this is confusing but it’s making sense.

        the biggest thing LV2 needs right now (imho) is example code for extensions. eg-metro was the first thing that started to make the concept of control ports/events make sense and i still feel quite a ways away from a thorough understanding. time.h was (still is, tbh) the most frustrating thing, man. there is *no* indication of how to use it.

        certainly some of these concepts aren’t best explained in extension documentation but they need to be explained somewhere. RDF, dude. there is a ton of power in unrealized (as of yet) extensions but you need to explain this to people before they can build them.

        • Understand that time and patch are deliberately written as general vocabularies. The time extension isn’t a plugin API, it’s just a bunch of properties for describing time. It’s completely usable in a ton of contexts, both in an LV2 context and in general. This is a good thing, but yes, if you don’t “get” properties and are looking for an API reference, it will be confusing.

          This might sound crazy at first, but the important thing to realize is that you can use any vocabulary at all in exactly the same way. You don’t need a special LV2 one. There are thousands of these things all over the web designed to describe this and that. You can also just make one up as you go along to do whatever you need to do. Having the vocabulary and the mechanism (like sending events or function calls or whatever) separate is actually a really nice thing.

          This is pretty much exactly why I am toying with this book idea – it’s not that the time/patch/etc extensions are poorly documented, they’re just a set of properties that are documented well enough. They just aren’t directly tied to a logistical mechanism.

          In short, you’re right that documentation is lacking right now (that’s what the book idea is about); but it’s not really that e.g. the time extension is confusing, it’s that there’s not really any documentation that fully describes how to communicate with the host in that style at all. Since that cross-cuts many extensions, I think such documentation can only be done separately. It is “higher level documentation”, so to speak.

          All that said… with sufficient examples people can get things done without having to “get” it anyway. Copy/paste the metronome example and voila, there’s your tempo information. Ideally doing common things shouldn’t really require deep knowledge of why things are the way they are.

          • yeah i agree completely. it’s all making more sense as i see examples, but RDF is definitely somewhat alien to me, especially as it applies to the design decisions made with LV2.

            outside of LV2 specifically, i’d be interested in reading a sort of “RDF for C guys” type post, perhaps detailing some of the design decisions that led you to use it in LV2. i am intrigued by this concept of using URIs to represent things, but have no idea, well…what I can do with it. do you have any resources about introductions to RDF, specifically in this usecase?

  3. today i was thinking about how i could help lv2 docs and i found this post.

    i’m a lv2 newbie, started working on a host and porting some plugins last year, and now i’m really getting it. I think a book/guide for reference would really help me to get it easily/faster. the docs are helpful but it always can get better :)

    as I said, i’ll be glad to help with any documentation project

    • Great, any help would be appreciated. The preliminary work on the book has started in the LV2 SVN repository, if you configure with –docs (and have asciidoc) the book will be compiled. It is still at an incredibly early stage, though, so how the “chapters” will progress is not clear. If you have a good idea for a specific example though, feel free (it is completely example based).

      Others good ways of contributing to the documentation are contributing to the wiki, and providing feedback on any pat of the extension documentation that you find confusing.

  4. Funny thing, that it was actually a search for readable and beginner friendly LV2 documentation that led me to this post. So yes, I would definitely read and make use of the kind of document you propose…

  5. as another LV2 newbie who happens to be a tech doc author in real life, I’d be happy to contribute to the documentation. Perhaps if I keep tracks on my initital efforts then write up later as a newbie guide, with lessons learned

    • If you don’t want to write examples / book chapters, the best way is probably to contribute to the wiki… but “write up later” usually turns in to “write up never” ;)

  6. Absolutely, more documentation to ease a developer into using lv2 would be beneficial to many – and expanding the use of LV2 would benefit all!

    As a software engineer but not a plugin developer, LV2 does not seem to have an accessable “starting point” and “tutorial” really available – sure, for those of you who understand and know the various parts in depth, it all just makes sense, but the rest of us need a helping hand into the realm of the knowing!

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>