A new utility has been added to Lilv for benchmarking LV2 plugins. As you might expect, it is called
Simple, but useful,
lv2bench runs all discovered plugins for a specified number of frames split into a specified block size, and records the time each plugin took to execute. It prints a simple table like this:
# Block Samples Time Plugin 512 524288 0.005872 http://calf.sourceforge.net/plugins/Filter 512 524288 0.006461 http://drobilla.net/plugins/blop/amp 512 524288 0.008772 http://calf.sourceforge.net/plugins/Equalizer8Band 512 524288 0.008959 http://drobilla.net/plugins/fomp/autowah
This makes it easy to get a good overall feel for how expensive plugins on the system are relative to each other. Of course, a more expressive view of this information with several parameters would be useful… Robin Gareus to the rescue! Robin made a script currently called lilvplot which does several runs with various parameters and plots the data, which makes the relative performance more clear and shows the variance of a plugin’s run time with error bars. This is important information because run-time variance is a good indicator of how real-time appropriate a plugin is.
Here is an example of lilvplot output generated from lv2bench data (as a massive SVG, you may want to open this image in a separate window and zoom in to see the plugin labels):
Both of these tools are very fresh and there are certainly improvements to be made (normalizing the y-axis so it showed where real-time is would be more informative, and lv2bench currently supports very few features), but they already make a useful addition to the LV2 developer’s toolkit. I found them useful for evaluating the improvements had by vectorizing the arithmetic plugins in my soon-to-be-released port of blop. It will be interesting to see how things stack up in more thorough runs that include more plugins.