Module: workloads/change_streams_latency

Test the latency performance of change stream listeners under a write workload.

Setup

None.

Test

The test applies a write workload to a collection in the cluster while a number of threads read change streams for the collection. The measured throughputs are the average latency and maximum latency of the change streams. Results are reported as seconds. The latency is approximated by taking the difference between the clusterTime of a change document and the clusterTime of the getmore command it came from. The reading threads simply loop over the change stream cursors, keeping track of the latency measurements.

The test takes two main optional parameters: an array specifying the number of write threads 'thread_levels' and an array specifying the number of change stream listeners 'listener_levels'. The test then runs all the combinations of number of threads and number of listeners specified. Other parameters exist to specify different change stream behaviors, the number of collections to target or sharding specific options.

Notes

  • The test runs for 5 minutes

Owning-team

mongodb/replication

Source:

Members

(inner) change_stream_read_preference

Determines the read preference for the change stream. Defaults to primary.

Source:

(inner) doc_sizes

The size in bytes of the documents to insert in each collection during the workload. Defaults to [100] document size.

Source:

(inner) full_document

Possible modes for the 'fullDocument' parameter of the $changeStream stage. Defaults to undefined.

Source:

(inner) full_document_before_change

Possible modes for the 'fullDocumentBeforeChange' parameter of the $changeStream stage. Defaults to undefined.

Source:

(inner) listener_levels

The number of listeners to run in parallel. The default is [1].

Source:

(inner) mongos_hosts

The mongos hosts assigned IP addresses. The expected format is a list of dictionaries containing a "private_ip" field. This parameter is used to connect through multiple mongoses in a sharded cluster. If left unassigned, the workload and listeners will connect to the default mongos. Only applies if the cluster is sharded.

Source:

(inner) nb_collections

The number of collections to run the workload and listeners against. Defaults to [1].

Source:

(inner) nb_docs

The number of documents to insert in each collection before starting the workload. Defaults to undefined. If 'nb_docs' is undefined, the value is computed dynamically: nb_docs = 10_000_000 / doc_size.

Source:

(inner) pre_images_enabled

Represents the collection option of pre-image recording being enabled for all collections. Defaults to false.

Source:

(inner) selective_change

Whether the change stream pipeline should include an additional filtering stage. Defaults to false.

Source:

(inner) shard_collections

If the collections should be sharded when running against a sharded cluster. Defaults to true. Only applies if the cluster is sharded.

Source:

(inner) target_host

The target host on which change stream will be opened. Defaults to undefined.

Source:

(inner) test_duration_secs

The test duration in seconds per thread level and listener level combination. Defaults to 300 (5 minutes). Note: stalls come in 60 sec cycles.

Source:

(inner) thread_levels

The number of threads to run in parallel. The default is [24]. Note: Thread level should be divisible by 3 * nb_collections * mongos_hosts.length.

Source: