abq test
Starts a worker for an ABQ test suite run.
abq test can be used either to parallelize a test suite on a single machine ("locally" or "local mode") or to parallelize a test suite across multiple machines ("distributed" or "distributed mode"). When used for a distributed test suite run, you'll need to connect to a distributed queue instance.
Usage
abq test [OPTIONS] -- <command>
<command> can be any executable command which implements the ABQ protocol.
Examples
Run a test suite locally with a single runner:
abq test -- npm test
Run a test suite locally with three runners:
abq test -n 3 -- npm test
Run a single runner for a distributed test suite run with a worker named worker 1:
abq test --worker 1 --run-id my-test-1 -- npm test
Options
--access-token <ACCESS_TOKEN>
The RWX access token to use when fetching queue configuration information from a managed RWX queue. This is only used for distributed mode.
You can also specify the access token by using abq login or by setting the RWX_ACCESS_TOKEN environment variable.
Example: abq test --run-id my-test-1 --access-token rwx_org_eyJhbGci -- npm test
--batch-size <BATCH_SIZE>
The number of tests for the worker to pull from the queue at a time. The purpose of batching is to decrease the amount of time spent communicating with the queue. If the batch size is too low, your test suite may waste time communicating with the queue. If the batch size is too high, your tests may be unevenly distributed across workers.
If your tests are primarily slow, use a low value for --batch-size. If they are primarily fast, use a large value for --batch-size.
The default value is 7. The minimum batch size is 1.
Example: abq test --run-id my-test-1 --batch-size 10 -- npm test
--color <VAL>
Whether to use color when writing to stdout.
The following values are available:
- auto: Colors will be used if possible. Some of the heuristics used are: whether the output channel is detected as a TTY; whether (on Windows) the console is available; whether the
NO_COLORenvironment variable is set; whether theTERMenvironment variable is set todumb. - never: Colors will not be used.
The default value is auto.
Example: abq test --color never -- npm test
--inactivity-timeout-seconds <INACTIVITY_TIMEOUT_SECONDS>
The number of seconds after which to cancel a test run if inactivity is detected. The only way inactivity is currently detected is if a test takes longer than <INACTIVITY_TIMEOUT_SECONDS> to complete.
When setting an inactivity timeout, we recommend over-estimating based on historical test runtimes. Hitting a timeout should be indicative of a failure in a test suite's setup or configuration.
The default value is 3600.
Example: abq test --inactivity-timeout-seconds 7200 -- npm test
--local
When present, ABQ will run in local mode with an ephemeral queue rather than using an external managed queue, even if you specify a RWX_ACCESS_TOKEN.
Example: abq test --local -- npm test
--num <NUM>
The number of runners to start on this worker.
Use the special value cpu-cores to use all but one of the available (physical) CPU cores.
The default value is 1.
You can also specify this option with the shorthand -n
Examples:
abq test --num 2 -- npm testabq test -n 2 -- npm testabq test --num cpu-cores -- npm test
--queue-addr <QUEUE_ADDR>
The address of the queue to connect to when running in distributed mode.
When using a managed RWX queue, use
--access-token instead, which will
automatically fetch the queue address from a managed RWX queue. A queue
address is only needed if you are self-hosting a queue.
Example: abq test --run-id my-test-1 --queue-addr 10.0.0.1:9000 -- npm test
--reporter <REPORTER>
A reporter to use for outputting results from the tests. The following reporters are currently available:
- dot: Print a dot for each successful test and a letter for non-successful test (e.g.
Ffor failed tests).
....P......F....
- line: Print a line with the test name and status for each test.
Module#method does something: ok
Module#method does something else: FAILED
- progress: Print an automatically-updating message with the current progress of the test run.
> ABQ status
> [10 seconds] 48 tests run, 46 passed, 2 failing
- junit-xml[=path/to/results.xml]: Output a JUnit-compatible XML file to the specified path. If the path is not specified, the file will be written to
./abq-test-results.xml. - rwx-v1-json[=path/to/results.json]: Output a RWX-V1-compatible JSON file to the specified path. If the path is not specified, the file will be written to
./abq-test-results.json.
When there is exactly one runner in this worker (--num 1), the default is to pass through output from the native test framework as output is received.
When there is more than one runner in this worker, the default value is dot.
To use multiple reporters, specify --reporter multiple times.
Example: abq test --reporter dot --reporter rwx-v1-json=/tmp/results.json -- npm test
--retries <RETRIES>
The number of times to retry failed tests. If a test fails, it will be attempted up to <RETRIES> additional times. If it succeeds on any attempt, the test will be reported as a success and will not be retried again. If it fails in all of those attempts, it will be reported as failed.
The default value is 0, i.e. no tests will be retried.
Example: abq test --retries 1 -- npm test
--run-id <RUN_ID>
The distributed run ID for workers to connect to. When not specified, abq test will run in local mode. When it is specified, abq test will run tests for the distributed test suite run for <RUN_ID>.
In CI environments, abq test will attempt to infer the run ID from CI environment variables. See the CI platforms documentation.
You can also specify the run ID by setting the ABQ_RUN_ID environment variable.
Example: abq test --run-id my-test-1 -- npm test
--startup-timeout-seconds <SECONDS>
The maximum number of seconds to wait for a test command to start up.
ABQ waits for the test executable to start up before starting a test run. If the test command does not start up within the timeout, the test run will be cancelled.
The default value is 60. For test suites with particularly slow boot up times, you can nudge this higher.
Example: abq test --startup-timeout-seconds 100 -- npm test
--test-strategy <TEST_STRATEGY>
How ABQ will distribute the tests.
- by-test (default): distribute the next test to any worker.
- by-file: distribute all tests in a file to the same worker. This ensures that expensive per-file shared setups or teardowns will run only once on one worker, however it may cause tests to be less evenly distributed.
Jest and Playwright always run with a by-file strategy regardless of the value of this flag.
Example: abq test --test-strategy by-file -- bundle exec rspec
--tls-cert <TLS_CERT>
The path to the TLS certificate to use when exchanging messages with the queue.
When using a managed RWX queue, use
--access-token instead, which will
automatically fetch the TLS certificate from a managed RWX queue. A TLS
certificate is only needed if you are self-hosting a
queue.
Example: abq test --run-id my-test-1 --queue-addr 10.0.0.1:9000 --tls-cert ./cert.pem -- npm test
--tls-key <TLS_KEY>
The path to the TLS key the queue should use. This is only useful when running with TLS in local mode.
When --tls-key is supplied, --tls-cert must also be supplied.
Example: abq test --tls-cert ./cert.pem --tls-key ./cert.key -- npm test
--token <TOKEN>
The token to use for authorizing messages sent to the queue. This is typically the user token that is used when running abq start.
When using a managed RWX queue, use
--access-token instead, which will
automatically fetch the token from a managed RWX queue. A token is only needed
if you are self-hosting a queue.
Example: abq test --run-id my-test-1 --queue-addr 10.0.0.1:9000 --token abqs_OSRMcU9W6YJkLyVDkzhJ4HCkrWHzEq -- npm test
--worker <WORKER>
The identity of the test worker connecting to a distributed test suite run. <WORKER> must be a whole number.
You cannot have a duplicate worker identity in an ABQ test suite run. Only one worker can connect to the queue with a particular <WORKER> number for a particular test suite run.
The default value is 0. For local test suite runs, this option can always be omitted since there can only be one worker for a local test suite run.
Example: abq test -run-id my-test-1 --worker 1 -- npm test
--working-dir <WORKING_DIR>
The working directory for the work to use.
The default value is the current working directory.
Example: abq test --working-dir ./app/tests -- npm test
--help
Print detailed help information about the abq test command.
Example: abq test --help
-h
Print summarized help information about the abq test command.
Example: abq test -h