Benchmarking LV2 plugins
drobilla.net ⋅A new utility has been added to
Lilv for benchmarking LV2 plugins.
As you might expect, it is called lv2bench.
Simple, but useful, lv2bench runs all discovered plugins for a
specified number of frames split into a specified block size, and
records the time each plugin took to execute. It prints a simple table
like this:
| Block | Samples | Time | Plugin |
|---|---|---|---|
| 512 | 524288 | 0.005872 | http://calf.sourceforge.net/plugins/Filter |
| 512 | 524288 | 0.006461 | http://drobilla.net/plugins/blop/amp |
| 512 | 524288 | 0.008772 | http://calf.sourceforge.net/plugins/Equalizer8Band |
| 512 | 524288 | 0.008959 | http://drobilla.net/plugins/fomp/autowah |
This makes it easy to get a good overall feel for how expensive plugins on the system are relative to each other. Of course, a more expressive view of this information with several parameters would be useful... Robin Gareus to the rescue! Robin made a script currently called lilvplot which does several runs with various parameters and plots the data, which makes the relative performance more clear and shows the variance of a plugin's run time with error bars. This is important information because run-time variance is a good indicator of how real-time appropriate a plugin is.
Here is an example of lilvplot output generated from lv2bench data (as a
massive SVG, you may want to open this image in a separate window and
zoom in to see the plugin labels):
Both of these tools are very fresh and there are certainly improvements to be made (normalizing the y-axis so it showed where real-time is would be more informative, and lv2bench currently supports very few features), but they already make a useful addition to the LV2 developer's toolkit. I found them useful for evaluating the improvements had by vectorizing the arithmetic plugins in my soon-to-be-released port of blop. It will be interesting to see how things stack up in more thorough runs that include more plugins.