Benchmarking LV2 plugins

A new utility has been added to Lilv for benchmarking LV2 plugins. As you might expect, it is called lv2bench.

Simple, but useful, lv2bench runs all discovered plugins for a specified number of frames split into a specified block size, and records the time each plugin took to execute. It prints a simple table like this:

This makes it easy to get a good overall feel for how expensive plugins on the system are relative to each other. Of course, a more expressive view of this information with several parameters would be useful... Robin Gareus to the rescue! Robin made a script currently called lilvplot which does several runs with various parameters and plots the data, which makes the relative performance more clear and shows the variance of a plugin's run time with error bars. This is important information because run-time variance is a good indicator of how real-time appropriate a plugin is.

Here is an example of lilvplot output generated from lv2bench data (as a massive SVG, you may want to open this image in a separate window and zoom in to see the plugin labels):

Both of these tools are very fresh and there are certainly improvements to be made (normalizing the y-axis so it showed where real-time is would be more informative, and lv2bench currently supports very few features), but they already make a useful addition to the LV2 developer's toolkit. I found them useful for evaluating the improvements had by vectorizing the arithmetic plugins in my soon-to-be-released port of blop. It will be interesting to see how things stack up in more thorough runs that include more plugins.