Module: workloads/change_streams_crud_throughput

Test performance of database load in the presence of change stream listeners.

Setup

None.

Test

The test applies a mixed CRUD workload to a collection in the cluster while a number of threads read change streams for the collection. The measured throughputs are the throughputs of the CRUD workload. The reading threads simply loop over the change stream cursor, increasing a counter without saving or looking at the data.

The test takes two main optional parameters: an array specifying the number of CRUD threads 'thread_levels' and an array specifying the number of change stream listeners 'listener_levels'. The test then runs all the combinations of number of threads and number of listeners specified. Other parameters exist to specify different change stream behaviors, the number of collections to target or sharding specific options.

Results are reported as ops/sec.

Notes

  • The test runs for 5 minutes.

Owning-team

mongodb/replication

Source:

Members

(inner) change_stream_read_preference

Determines the read preference for the change stream. Defaults to primary.

Source:

(inner) doc_sizes

The size in bytes of the documents to insert in each collection during the workload. Defaults to [100] document size.

Source:

(inner) full_document

Possible modes for the 'fullDocument' parameter of the $changeStream stage. Defaults to undefined.

Source:

(inner) full_document_before_change

Possible modes for the 'fullDocumentBeforeChange' parameter of the $changeStream stage. Defaults to undefined.

Source:

(inner) listener_levels

The number of listeners to run in parallel. The default is [1, 10, 100, 1000].

Source:

(inner) mongos_hosts

The mongos hosts assigned IP addresses. The expected format is a list of dictionaries containing a "private_ip" field. This parameter is used to connect through multiple mongoses in a sharded cluster. If left unassigned, the workload and listeners will connect to the default mongos. Only applies if the cluster is sharded.

Source:

(inner) nb_collections

The number of collections to run the workload and listeners against. Defaults to [1].

Source:

(inner) nb_docs

The number of documents to insert in each collection before starting the workload. Defaults to undefined. If 'nb_docs' is undefined, the value is computed dynamically: nb_docs = 10_000_000 / doc_size.

Source:

(inner) pre_images_enabled

Represents the collection option of pre-image recording being enabled for all collections. Defaults to false.

Source:

(inner) selective_change

Whether the change stream pipeline should include an additional filtering stage. Defaults to false.

Source:

(inner) shard_collections

If the collections should be sharded when running against a sharded cluster. Defaults to true. Only applies if the cluster is sharded.

Source:

(inner) test_duration_secs

The test duration in seconds per thread level and listener level combination. Defaults to 300 (5 minutes). Note: stalls come in 60 sec cycles.

Source:

(inner) thread_levels

The number of threads to run in parallel. The default is [4, 64]. Note: Thread level should be divisible by 4 * nb_collections * mongos_hosts.length.

Source: