Fetch the report
data.summary. Reports are owner scoped and retained for your
plan’s retention window, which GET /api/v1/usage reports.
The three verdicts
A run counts three things: checks that passed, checks that failed, and checks that warned. The verdict is derived from those counts and nothing else.Why a warning is not a failure
The OCPI specification says two different kinds of thing. It says what you must do (SHALL, MUST), and it says what you should do (SHOULD, “it is advised”). These
are not the same, and the difference is not a matter of emphasis. It is the difference
between breaking the protocol and declining a suggestion.
A partner who ignores a SHOULD has broken nothing.
So we will never fail you for one. We will never call you “partially compliant” either,
because that phrase would send you to go and fix code that already works. If your report
says Compliant with three warnings, you are compliant. The warnings are there because
you might want to know, not because you owe anyone anything.
This is the single most important sentence in this documentation, so it is worth being
blunt about the alternative. A tester that fails you for declining a recommendation is
not being strict, it is being wrong: it fails a compliant partner and sends a
working integration back for repair. That is the worst thing a conformance tool can do,
worse than missing a real bug, and it is the failure this product is built to avoid.
data.summary.failed, never on warned.
Why a red result can be trusted
Here is the question nobody asks a test suite, and everybody should:If your check were broken, would it still show green?A check that can never fail certifies everybody. It looks exactly like a check that works, it costs nothing to write, and it will happily approve a partner who is completely broken. A test suite full of those is not a weak test suite. It is a decorative one. So we test our own tests. Alongside the tester we run a set of mutant peers: copies of a correct OCPI server, each deliberately broken in exactly one way. One sends no pagination headers. One returns an HTTP error where the spec requires a 2xxx rejection to ride HTTP 200. One accepts any credentials token you give it. One emits timestamps in a format the spec forbids. Each mutant breaks one rule, and the check that guards that rule must turn red. If it does not, we have found a check that cannot fail, and we either fix it or delete it. Every OCPI 2.2.1 conformance check has been proven capable of failing, and that is enforced in our build rather than asserted in a blog post. A check that stops being falsifiable breaks our own test suite before it ever reaches you.
The limits of that claim, stated plainly
The mutant peers cover OCPI 2.2.1 conformance checks. They do not yet cover 2.1.1, 2.2 or 2.3.0, and they do not cover the guided scenario suites. Those checks are not known to be vacuous. They are simply not yet proven not to be, and we are not going to describe that as the same thing. If you are relying on this property for a version other than 2.2.1, know that we have not earned that claim yet.The exchange behind every check
Every check in the report carries the HTTP request that was sent and the response that came back, along with the clause of the specification it is testing. This is deliberate, and it is the feature that ends arguments. When you forward a report to a partner, they do not have to trust our verdict. They can read the exchange and see for themselves. A conformance report that says “failed” without showing you what was sent is asking for faith, and no engineer under deadline pressure is going to give it.What to do next
Share a report
Send the evidence to your partner.
Troubleshooting
Common failures and what they usually mean.