Running the benchmarks: July 16th, 2 days

✅ reserve 32 machine for the Read Storage + 2 machines for the Write Storage for 60 hours starting July 16th 7pm
✅ July 16th 7pm, install the latest version of the benchmark software #3149 (closed) and prepare the run
✅ Populate the global index with 10 billions entries
✅ July 17th 7pm, run the benchmark with 20TB
✅ July 19th, analyze the results https://git.easter-eggs.org/biceps/biceps/-/tags/v2021-06-25

The results are consistent with previous runs. There is a degradation of read speed (MB/s) over time. The longer run scheduled August 6th, 2021 will provide a confirmation and help analyze the root cause. It may be cause by the increasing number of images: they are all mapped and mounted (200 towards the end) even though only five of them are used for reading. Thousands of images (smaller images) were mapped on a single machine and did not cause problem (that was a few months ago) but the focus was not on read speed at the time.

Bytes write   108.0 MB/s
Objects write 5.3 Kobject/s
Bytes read    99.4 MB/s
Objects read  24.3 Kobject/s
353739 random reads take longer than 100ms (1.7432793812449536%)

Migrated from T3421 (view on Phabricator)

Edited Jan 07, 2023 by Phabricator Migration user