Report Formats
Check Run Reporter supports the common report formats below as well as tool-specific formats.
Checkstyle
Nearly every code-style-checker out there supports Checkstyle. We test Check Run Reporter's JUnit support against reports created by the following tools.
JUnit
JUnit is the lowest common denominator of test reporting. Nearly every tool out there can produce some variant of JUnit, so it's a great fallback if we don't have specific support for your tool. That being said, JUnit is poorly documented and very inconsistently implemented. If we specifically support the tools you're using, you'll almost certainly have a better experience avoiding JUnit.
We test Check Run Reporter's JUnit support against reports created by the following tools.
JUnit Best Practices
JUnit is a poorly defined but oft-implemented standard. These guidelines should help you tweak your JUnit reporter to get the best results out of Check Run Reporter.
JUnit has two relatively well-known specifications, though neither one tends to be followed exactly.
Filenames
The single most important thing you can do to make JUnit work better with Check Run Reporter is to get filenames into your reports. Technically, neither of the well-known JUnit specifications recommend filenames, but many reporters automatically put the filename on the <testsuite/>
's file
attribute. Some reporters don't do so automatically, but can be configured to do so.
Ancestors
If your test runner allows you to nest suites within suites like the following, your test reporter might nest suites or it might combine titles. In the latter case, you may want to configure the separater for your reporter to Check Run Reporter can split them accordingly.
For example:
describe('MyClass', () => {
describe('#someMethod()', () => {
it('does the thing');
});
});
Could produce<testsuite name="MyClass">
<testsuite name="#someMethod()">
<testcase name="does the thing">
</testsuite>
</testsuite>
or, it could produce something like<testsuite name="MyClass › #someMethod()">
<testcase name="does the thing">
</testsuite>
We recommend using ›
as your ancestor separator. It's both vaguely human-reable and otherwise unlikely to appear in a test title, so it shouldn't have any false matches.Note that the above examples dropped most attributes in order to highly just the various behaviors of name
.
Capturing Logs
Both JUnit specs allow capturing stdout
and stderr
, though they recommended emitting them in different places. Check Run Reporter only reads stdout
and stderr
when they are parrt of a <testcase />
node; they'll be discard when part of a <testsuite />
node.