> ## Documentation Index
> Fetch the complete documentation index at: https://evsim.synergyboat.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.
# Read your report
> The three verdicts a run can return, why a warning is never a failure, and why a red result can be trusted to name the exact clause that broke.
This is the chapter that decides whether you believe anything else here.
A conformance report is only worth reading if you know two things about it: what its
verdict actually licenses you to do, and whether a green tick could have gone red. Most
tools answer neither. Here are both.
## Fetch the report
```bash theme={null}
curl -fsS "https://evsim.synergyboat.com/api/v1/testing/runs/$REPORT_ID" \
-H "Authorization: Bearer $EVRT_KEY"
```
Returns the full report: the per-check verdicts, the expected versus actual outcome, the
validation messages, and the recorded request and response exchange behind every check.
The summary counts live in `data.summary`. Reports are owner scoped and retained for your
plan's retention window, which `GET /api/v1/usage` reports.
## The three verdicts
A run counts three things: checks that **passed**, checks that **failed**, and checks that
**warned**. The verdict is derived from those counts and nothing else.
| Verdict | What happened | What it licenses you to do |
| ------------------- | --------------------------------------------------- | ----------------------------------------------------------------------------------------------------------- |
| **Fully compliant** | Nothing failed. Nothing was declined. | Ship. Every rule the spec requires is met, and every recommendation was taken. |
| **Compliant** | Nothing failed. Some recommendations were declined. | **Ship.** This is a pass. See below, because this row is the one people misread. |
| **Not compliant** | At least one required rule is broken. | Fix the failures. Nothing outranks a failure: a run with a hundred passes and one failure is not compliant. |
## Why a warning is not a failure
The OCPI specification says two different kinds of thing. It says what you **must** do
(`SHALL`, `MUST`), and it says what you **should** do (`SHOULD`, "it is advised"). These
are not the same, and the difference is not a matter of emphasis. It is the difference
between breaking the protocol and declining a suggestion.
**A partner who ignores a `SHOULD` has broken nothing.**
So we will never fail you for one. We will never call you "partially compliant" either,
because that phrase would send you to go and fix code that already works. If your report
says **Compliant** with three warnings, you are compliant. The warnings are there because
you might want to know, not because you owe anyone anything.
This is the single most important sentence in this documentation, so it is worth being
blunt about the alternative. A tester that fails you for declining a recommendation is
not being strict, it is being **wrong**: it fails a compliant partner and sends a
working integration back for repair. That is the worst thing a conformance tool can do,
worse than missing a real bug, and it is the failure this product is built to avoid.
The verdict word carries the pass. Only the count carries the amber. In CI, that means
gate on `data.summary.failed`, never on `warned`.
## Why a red result can be trusted
Here is the question nobody asks a test suite, and everybody should:
> If your check were broken, would it still show green?
A check that can never fail certifies everybody. It looks exactly like a check that
works, it costs nothing to write, and it will happily approve a partner who is completely
broken. A test suite full of those is not a weak test suite. It is a **decorative** one.
So we test our own tests. Alongside the tester we run a set of **mutant peers**: copies
of a correct OCPI server, each deliberately broken in exactly one way. One sends no
pagination headers. One returns an HTTP error where the spec requires a 2xxx rejection to
ride HTTP 200. One accepts any credentials token you give it. One emits timestamps in a
format the spec forbids.
Each mutant breaks one rule, and the check that guards that rule **must** turn red. If it
does not, we have found a check that cannot fail, and we either fix it or delete it.
**Every OCPI 2.2.1 conformance check has been proven capable of failing**, and that is
enforced in our build rather than asserted in a blog post. A check that stops being
falsifiable breaks our own test suite before it ever reaches you.
### The limits of that claim, stated plainly
The mutant peers cover **OCPI 2.2.1 conformance checks**. They do not yet cover 2.1.1,
2.2 or 2.3.0, and they do not cover the guided scenario suites. Those checks are not
known to be vacuous. They are simply not yet *proven* not to be, and we are not going to
describe that as the same thing.
If you are relying on this property for a version other than 2.2.1, know that we have not
earned that claim yet.
## The exchange behind every check
Every check in the report carries the HTTP request that was sent and the response that
came back, along with the clause of the specification it is testing.
This is deliberate, and it is the feature that ends arguments. When you forward a report
to a partner, they do not have to trust our verdict. They can read the exchange and see
for themselves. A conformance report that says "failed" without showing you what was sent
is asking for faith, and no engineer under deadline pressure is going to give it.
## What to do next
Send the evidence to your partner.
Common failures and what they usually mean.