Module: workloads/bestbuy_agg_query_comparison

Measure performance of aggregation and corresponding find/count/distinct commands against the BestBuy Developer API data.

Owning-team

mongodb/product-query

Keywords

• bestbuy
• aggregate
• throughput
• distinct
• find
• count

Pre-requisite

The dataset must be installed on the target cluster before running the test. The data can be downloaded from here and installed using mongorestore (mongorestore --gzip --archive=bestbuyproducts.bson.gz)

Setup

None

Test

There are two types of tests: those that attempt to measure performance of an aggregate vs. other equivalent read commands, and those which are simply testing the throughput of aggregation.

Agg/Query Comparison Tests

Each test has an aggregation (useAgg) and non-aggregation (noAgg) variant. The tests cover: distinct, find, count commands against the Best Buy dataset.

Results are reported as ops / second.

The test takes an optional 'thread_levels' parameters: an array specifying the number of threads to use.

Exclusively Aggregation Tests

These tests use a simple for loop to repeatedly run a specific pipeline, reporting both the throughput in documents processed per second and the performance variability, measured as the percentage difference in latency between the 99th percentile and the 50th percentile.

Notes

1. The tests vary greatly in execution time. Tests that nominally take less than 50 ms are run

• using benchRun, while those that take more than 50 ms are run in a simple loop for at least
• TEST_SECONDS.