Report Formats

Check Run Reporter supports the common report formats below as well as tool-specific formats.

JUnit

JUnit is the lowest common denominator of test reporting. Nearly every tool out there can produce some variant of JUnit, so it's a great fallback if we don't have specific support for your tool. That being said, JUnit is poorly document and very inconsistently implemented. If we specifically support the tools you're using, you'll almost certainly have a better experience avoiding JUnit.

We test Check Run Reporter's JUnit support against reports created by the following tools.

JUnit Best Practices

JUnit is a poorly defined but oft-implemented standard. These guidelines should help you tweak your JUnit reporter to get the best results out of Check Run Reporter.

JUnit has two relatively well-known specifications, though neither one tends to be followed exactly.

Filenames

The single most important thing you can do to make JUnit work better with Check Run Reporter is to get filenames into your reports. Technically, neither of the well-known JUnit specifications recommend filenames, but many reporters automatically put the filename on the <testsuite/>'s file attribute. Some reporters don't do so automatically, but can be configured to do so.

Ancestors

If your test runner allows you to nest suites within suites like the following, your test reporter might nest suites or it might combine titles. In the latter case, you may want to configure the separater for your reporter to Check Run Reporter can split them accordingly.

For example:

describe('MyClass', () => {
  describe('#someMethod()', () => {
    it('does the thing');
  });
});

Could produce

<testsuite name="MyClass">
  <testsuite name="#someMethod()">
    <testcase name="does the thing">
  </testsuite>
</testsuite>

or, it could produce something like

<testsuite name="MyClass › #someMethod()">
  <testcase name="does the thing">
</testsuite>

We recommend using › as your ancestor separator. It's both vaguely human-reable and otherwise unlikely to appear in a test title, so it shouldn't have any false matches.

Note that the above examples dropped most attributes in order to highly just the various behaviors of name.

Capturing Logs

Both JUnit specs allow capturing stdout and stderr, though they recommended emitting them in different places. Check Run Reporter only reads stdout and stderr when they are parrt of a <testcase /> node; they'll be discard when part of a <testsuite /> node.