Benchmarking LV2 plugins

A new utility has been added to Lilv for benchmarking LV2 plugins. As you might expect, it is called lv2bench.

Simple, but useful, lv2bench runs all discovered plugins for a specified number of frames split into a specified block size, and records the time each plugin took to execute. It prints a simple table like this:

# Block Samples Time Plugin
512 524288 0.005872 http://calf.sourceforge.net/plugins/Filter
512 524288 0.006461 http://drobilla.net/plugins/blop/amp
512 524288 0.008772 http://calf.sourceforge.net/plugins/Equalizer8Band
512 524288 0.008959 http://drobilla.net/plugins/fomp/autowah

This makes it easy to get a good overall feel for how expensive plugins on the system are relative to each other. Of course, a more expressive view of this information with several parameters would be useful… Robin Gareus to the rescue! Robin made a script currently called lilvplot which does several runs with various parameters and plots the data, which makes the relative performance more clear and shows the variance of a plugin’s run time with error bars. This is important information because run-time variance is a good indicator of how real-time appropriate a plugin is.

Here is an example of lilvplot output generated from lv2bench data (as a massive SVG, you may want to open this image in a separate window and zoom in to see the plugin labels):

Both of these tools are very fresh and there are certainly improvements to be made (normalizing the y-axis so it showed where real-time is would be more informative, and lv2bench currently supports very few features), but they already make a useful addition to the LV2 developer’s toolkit. I found them useful for evaluating the improvements had by vectorizing the arithmetic plugins in my soon-to-be-released port of blop. It will be interesting to see how things stack up in more thorough runs that include more plugins.

(Flattr this)

3 thoughts on “Benchmarking LV2 plugins

    • Ah, my mistake. I suppose I should have done the math first :)

      By ‘normalizing’, I was thinking of perhaps making that the 0, though I guess as long as the limit is clear on the plot it makes little difference.

  1. Is it miliseconds or microsencods?
    If it is miliseconds somethings seems strange because I use vocproc with and rt-kernel and it runs inside a 128 frame JACK window.
    Is there something I am missing?
    Cheers and thanks for posting this much useful tool.
    Gian

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>