Test performance of a change stream pipeline.
Test
This workload initializes the database with 'numDocs' inserts (defined below) into a test collection, drops the test collection, quiesces the database, and then executes change stream pipelines that return the events recorded by the initial insert and drop commands.
Change streams use a resume token from the beginning of the test in order to observe the insert and drop commands from the initialization phase, which allows them to execute without any other concurrent activity. The drop command invalidates change stream cursors as soon as they reach the end of initialization activity, preventing them from spending time waiting for future events.
There are three test pipelines, one with just a $changeStream, one with a $match stage that includes 30% of events in the result set, and one with a $match stage that includes 60% of events in the result set. The throughput of a change stream pipeline is computed as the number of events it returns per second.
Setup
Supports replica sets and sharded clusters but not standalone instances.
Owning-team
mongodb/replication
Methods
(inner) insertDocs()
Initialize the oplog by inserting 'numDocs' documents into a test collection and then dropping the collection. Note that the data itself is not important; we only need the commands to be in the oplog. That being said, we have found that using a trivially small document size leads to noise in the perf numbers, so we pad the documents to be roughly 1 KiB. This size means that trivial changes to the change stream output format do not significantly affect the throughput.
Returns a resume token that can be used to start a change stream from just before the insert commands generated by this function.
(inner) runQueriesAndReportThroughput()
Launches 'nThreads' benchmark threads, waits for all of them to finish, and reports the total throughput from all the threads.
(inner) runQueriesInThread()
The entry point for each benchmark thread: executes a test change stream 'numTrials' times and returns the average throughput from all of the trials. The caller can specify additional stages in the 'pipeline' parameter to append to the change stream pipeline.