Test the latency performance of change stream listeners under a write workload.
Setup
None.
Test
The test applies a write workload to a collection in the cluster while a number of threads read change streams for the collection. The measured throughputs are the average latency and maximum latency of the change streams. Results are reported as seconds. The latency is approximated by taking the difference between the clusterTime of a change document and the clusterTime of the getmore command it came from. The reading threads simply loop over the change stream cursors, keeping track of the latency measurements.
The test takes two main optional parameters: an array specifying the number of write threads 'thread_levels' and an array specifying the number of change stream listeners 'listener_levels'. The test then runs all the combinations of number of threads and number of listeners specified. Other parameters exist to specify different change stream behaviors, the number of collections to target or sharding specific options.
Notes
- The test runs for 5 minutes
Owning-team
mongodb/replication
Members
(inner) change_stream_read_preference
Determines the read preference for the change stream. Defaults to primary.
(inner) doc_sizes
The size in bytes of the documents to insert in each collection during the workload. Defaults to [100] document size.
(inner) full_document
Possible modes for the 'fullDocument' parameter of the $changeStream stage. Defaults to undefined.
(inner) full_document_before_change
Possible modes for the 'fullDocumentBeforeChange' parameter of the $changeStream stage. Defaults to undefined.
(inner) listener_levels
The number of listeners to run in parallel. The default is [1].
(inner) mongos_hosts
The mongos hosts assigned IP addresses. The expected format is a list of dictionaries containing a "private_ip" field. This parameter is used to connect through multiple mongoses in a sharded cluster. If left unassigned, the workload and listeners will connect to the default mongos. Only applies if the cluster is sharded.
(inner) nb_collections
The number of collections to run the workload and listeners against. Defaults to [1].
(inner) nb_docs
The number of documents to insert in each collection before starting the workload. Defaults to undefined. If 'nb_docs' is undefined, the value is computed dynamically: nb_docs = 10_000_000 / doc_size.
(inner) pre_images_enabled
Represents the collection option of pre-image recording being enabled for all collections. Defaults to false.
(inner) selective_change
Whether the change stream pipeline should include an additional filtering stage. Defaults to false.
(inner) shard_collections
If the collections should be sharded when running against a sharded cluster. Defaults to true. Only applies if the cluster is sharded.
(inner) target_host
The target host on which change stream will be opened. Defaults to undefined.
(inner) test_duration_secs
The test duration in seconds per thread level and listener level combination. Defaults to 300 (5 minutes). Note: stalls come in 60 sec cycles.
(inner) thread_levels
The number of threads to run in parallel. The default is [24]. Note: Thread level should be divisible by 3 * nb_collections * mongos_hosts.length.