Automated benchmarking system that tests Vitess's performance on a nightly basis. Providing adopters and maintainers a clear vision of how Vitess is performing throughout different releases is the goal of arewefastyet. To achieve this, the system runs a variety of benchmarks every night. The default branch, pull requests impacting performance, release branches, and tags are all automatically tested when needed.
Our benchmarks are divided in two category: micro and macro. Microbenchmarks are responsible for a small portion of the codebase, usually a single function. We use golang's testing library to write and execute these benchmarks. On the other hand, macrobenchmarks can be referred as end to end benchmarks, they test the entire codebase. We use sysbench to enable our macrobenchmarks.
Our benchmarks are executed on m2.xlarge.x86 baremetal servers provided by Equinix Metal. The server's specs can be found here.
To improve our results' reliability, we provision a new server for each new benchmark and tear it down once the benchmark is completed.
This website and arewefastyet code can all be found on GitHub (vitessio/arewefastyet).