Karma

To use Captain with Karma, you need to configure your test suite to output test results to a file and then tell Captain where to find those test results.

Getting started

Karma does not natively produce a test results artifact that Captain can parse. To integrate, you'll need to install and configure the karma-json-reporter. Once installed, you can modify your karma.conf.js to output a JSON artifact:

reporters: ["progress", "json"],
jsonReporter: {
  stdout: false,
  outputFile: "tmp/karma.json",
},

Configure Captain by creating a .captain/config.yaml file in the root directory of your repository:

test-suites:
  your-project-karma:
    command: npx karma start --single-run
    results:
      path: tmp/karma.json

You can change your-project-karma to any name you like, but we typically recommend using the name of your project followed by a dash followed by karma. The command is the command you already use to run your test suite. Captain will invoke this command to run your tests. The example above shows what you might use if you use npx karma start --single-run and want to store test results in tmp/karma.json.

Once Captain is configured, you can run captain run your-project-karma --print-summary. If you see your typical test output following by a captain block like this:

--------------------------------------------------------------------------------
----------------------------------- Captain ------------------------------------
--------------------------------------------------------------------------------

then you've configured everything correctly! You can now supercharge your test framework's capabilities. See below for configuring each of Captain's features.

Quarantining Tests

Traditionally, you might mark a test as pending or skipped to triage flaky or failing tests. With Captain, you can quarantine them instead. When only quarantined tests fail, Captain will still report your build as successful and exit with a 0 exit code. Unlike skipped tests, quarantined tests will continue to run, so you can still view their failure messages and see how frequently they are failing.

You can quarantine tests in OSS mode with captain add quarantine like so:

captain add quarantine your-project-karma \
  --browserName "Headless Chrome Linux" \
  --description "Some test is a passing test"

See the identifying tests section of this page for more information on finding the browserName and description, and see the OSS quarantining guide for more information on managing quarantined tests in OSS mode.

Retrying Tests

Unfortunately, due to limitations in Karma, retrying tests is not supported without some manual workarounds. If you're interested in this feature, please let us know.

Partitioning

Unfortunately, due to limitations in Karma, partitioning is not supported without some manual workarounds. If you're interested in this feature, please let us know.

Identifying Tests

Captain uses framework specific "identity recipes" to identify the tests in your suite. These recipes are order dependent components extracted from native test framework output.

We use this identity to track the executions of a test over the course of their lifetime in your suite. This enables us to do things like flake detection, quarantining, and retries.

For Karma, Captain constructs the identity by parsing out the browserName and description attributes.