Test performance of database load in the presence of change stream listeners.
Setup
None.
Test
The test applies a mixed CRUD workload to a collection in the cluster while a number of threads read change streams for the collection. The measured throughputs are the throughputs of the CRUD workload. The reading threads simply loop over the change stream cursor, increasing a counter without saving or looking at the data.
The test takes two main optional parameters: an array specifying the number of CRUD threads 'thread_levels' and an array specifying the number of change stream listeners 'listener_levels'. The test then runs all the combinations of number of threads and number of listeners specified. Other parameters exist to specify different change stream behaviors, the number of collections to target or sharding specific options.
Results are reported as ops/sec.
Notes
- The test runs for 5 minutes.
Owning-team
mongodb/replication
Members
(inner) change_stream_read_preference
Determines the read preference for the change stream. Defaults to primary.
(inner) doc_sizes
The size in bytes of the documents to insert in each collection during the workload. Defaults to [100] document size.
(inner) full_document
Possible modes for the 'fullDocument' parameter of the $changeStream stage. Defaults to undefined.
(inner) full_document_before_change
Possible modes for the 'fullDocumentBeforeChange' parameter of the $changeStream stage. Defaults to undefined.
(inner) listener_levels
The number of listeners to run in parallel. The default is [1, 10, 100, 1000].
(inner) mongos_hosts
The mongos hosts assigned IP addresses. The expected format is a list of dictionaries containing a "private_ip" field. This parameter is used to connect through multiple mongoses in a sharded cluster. If left unassigned, the workload and listeners will connect to the default mongos. Only applies if the cluster is sharded.
(inner) nb_collections
The number of collections to run the workload and listeners against. Defaults to [1].
(inner) nb_docs
The number of documents to insert in each collection before starting the workload. Defaults to undefined. If 'nb_docs' is undefined, the value is computed dynamically: nb_docs = 10_000_000 / doc_size.
(inner) pre_images_enabled
Represents the collection option of pre-image recording being enabled for all collections. Defaults to false.
(inner) selective_change
Whether the change stream pipeline should include an additional filtering stage. Defaults to false.
(inner) shard_collections
If the collections should be sharded when running against a sharded cluster. Defaults to true. Only applies if the cluster is sharded.
(inner) test_duration_secs
The test duration in seconds per thread level and listener level combination. Defaults to 300 (5 minutes). Note: stalls come in 60 sec cycles.
(inner) thread_levels
The number of threads to run in parallel. The default is [4, 64]. Note: Thread level should be divisible by 4 * nb_collections * mongos_hosts.length.