Report Formats

Check Run Reporter supports the common report formats below as well as tool-specific formats.

Checkstyle

Nearly every code-style-checker out there supports Checkstyle. Check Run Reporter's Checkstyle support is tested against reports created by the following tools.

• The Checkstyle project's fixtures
• Swiftlint
• ESLint

JUnit

JUnit is the lowest common denominator of test reporting. Nearly every tool out there can produce some variant of JUnit, so it's a great fallback if there isn't specific support for your tool. That being said, JUnit is poorly documented and very inconsistently implemented. If there is specific support for the tools you're using, you'll almost certainly have a better experience avoiding JUnit.

Check Run Reporter's JUnit support is tested against reports created by the following tools.

• Dredd
• ESLint
• Jest
• Mocha (the JavaScript one)
• Webdriver.io

JUnit Best Practices

JUnit is a poorly defined but oft-implemented standard. These guidelines should help you tweak your JUnit reporter to get the best results out of Check Run Reporter.

JUnit has two relatively well-known specifications, though neither one tends to be followed exactly.

• windyroad/JUnit-Schema
• junit-team/junit5

Filenames

The single most important thing you can do to make JUnit work better with Check Run Reporter is to get filenames into your reports. Technically, neither of the well-known JUnit specifications recommend filenames, but many reporters automatically put the filename on the <testsuite/>'s file attribute. Some reporters don't do so automatically, but can be configured to do so. Jest Junit Reporter

If you're using the jest-junit-reporter, you can use either an environment variable or a config item to add filenames.

JEST_JUNIT_ADD_FILE_ATTRIBUTE=true

or

addFileAttribute: "true"

Note that "true" must be a string, not a boolean.

Ancestors

If your test runner allows you to nest suites within suites like the following, your test reporter might nest suites or it might combine titles. In the latter case, you may want to configure the separator for your reporter so Check Run Reporter can split them accordingly.

For example:

describe('MyClass', () => {
  describe('#someMethod()', () => {
    it('does the thing');
  });
});

Could produce

<testsuite name="MyClass">
  <testsuite name="#someMethod()">
    <testcase name="does the thing">
  </testsuite>
</testsuite>

or, it could produce something like

<testsuite name="MyClass › #someMethod()">
  <testcase name="does the thing">
</testsuite>

Note that the above examples dropped most attributes in order to highlight just the various behaviors of name.

Check Run Reporter uses › as the ancestor separator. It's both vaguely human-readable and otherwise unlikely to appear in a test title, so it should have few, if any, false matches.

Capturing Logs

Both JUnit specifications allow capturing stdout and stderr, though they recommended emitting them in different places. Check Run Reporter only reads stdout and stderr when they are part of a <testcase /> node; they'll be discarded when part of a <testsuite /> node.