15 Jan 2014
Two months have passed since I posted an initial benchmark of the Rust and C++ implementations of Cap’n Proto. Enough has changed in that time to make it worth presenting updated results.
One change is that I got a faster computer. Unfortunately, that means that results from the two benchmarks will not be directly comparable.
A more pertinent change is that I (partially) implemented scratch-space reuse, an optional feature that can reduce allocations. The addition of this feature doubles the number of benchmark configurations, as we can run each communication mode with or without “reuse” turned on.
Otherwise, the benchmark itself is the same as before.
First, the “carsales” case, heavy on numbers.
Recall that in November’s benchmark, capnproto-rust was slightly faster than capnproto-c++ in “object” mode. This is no longer true. I believe that capnproto-c++ was previously disadvantaged because it was providing extra thread safety—in particular, the ability for multiple threads to share a mutable MessageBuilder. That feature was dropped in this commit.
Next, the “catrank” case, heavy on strings.
In November, capnproto-rust was hampered here by its lack of support for direct writing of string fields. That has been remedied. However, capnproto-rust has another disadvantage here; cpu profiling reveals that it spends roughly ten percent of its time verifying that strings are valid UTF-8, while capnproto-c++ does not bother with any such verification. Note that the Cap’n Proto encoding spec requires that strings be valid UTF-8, but says nothing about whether the receiver of a non-UTF-8 string should report an error.
Finally, the “eval” case, heavy on pointer indirections.
In contrast to November’s results, the relative performance of capnproto-rust now does not significantly degrade when it must perform I/O in the “pipe” communication mode. The main reason for the improvement is that the benchmark now uses libnative, Rust’s 1:1 threading runtime, whereas in November it used libgreen, Rust’s M:N threading runtime built on libuv. Only recently has it become convenient to swap between these two runtimes. If I run the “eval” case in “pipe” mode with libgreen today, Rust takes approximately twice as long as it does with libnative.
