Running Without an Account
Xping works with no API key, no signup, and no network access. Add the SDK to your test project, run your tests a few times, and xping report will tell you which of your tests are unreliable.
This is local-only mode. Everything stays on your machine.
What You'll Learn
- How to collect test history with no Xping account
- How to read the local flakiness report
- Where local data is stored and how to remove it
- What local-only mode can and cannot tell you
Prerequisites
- An existing test project using NUnit, xUnit, or MSTest
- The .NET SDK
No account. No API key.
Step 1: Add the SDK
Install the adapter for your test framework:
dotnet add package Xping.Sdk.NUnit # or Xping.Sdk.XUnit (xUnit v2) / Xping.Sdk.XUnit.V3 (xUnit v3) / Xping.Sdk.MSTest
Wire it up exactly as the framework quickstart describes (NUnit, xUnit, MSTest) — but skip the API key step.
With no ApiKey configured, the SDK starts in local-only mode:
[Xping] Initialized in local-only mode. No API key configured - results stay on this machine.
Step 2: Run your tests a few times
dotnet test
Each run is recorded to a local store. Cross-run flakiness detection needs at least three runs, because two runs cannot distinguish a flaky test from a test that just broke.
Step 3: Install the CLI
Requires the .NET 10 SDK or runtime. The tool targets
net10.0and will not launch on older runtimes. The Xping SDK packages themselves targetnetstandard2.0and are unaffected — your test projects can stay on any supported .NET version.
dotnet tool install -g Xping.Cli
This puts an xping command on your PATH, usable from any repository. If your shell answers command not found: xping, add ~/.dotnet/tools to your PATH and reopen the terminal.
To pin the CLI version alongside your repo instead, install it into a tool manifest — see Installation in the CLI reference.
Step 4: Read the report
xping report
Xping · Checkout.Tests · 20 runs · 2026-08-05 → 2026-08-19 · main@a3f9c2e
412 tests · 409 healthy · 3 flagged · 41 awaiting more runs · 6 not significant
◷ nothing to measure: concurrency 108 · slower 32
```
NEEDS ATTENTION (1 high, 1 medium, 1 low) most severe first
────────────────────────────────────────────────────────────────────────
1. HIGH flaky
SummaryTests.GenerateMonthlySummary
failed 7 of 20 executions (35%) in 5 of 20 runs, 3 failure modes
evidence moderate | -env-cluster | f_2a91c0de | ...ummaryTests.cs:88
2. MED slower
FlowTests.CheckoutFlow_Completes
3.51x slower (95% CI 1.94-5.87x), 340ms -> 1.2s on the clock
evidence high | -env | f_8c04b71a | tests/Checkout/FlowTests.cs:214
3. LOW stopped running
LegacyImport.Roundtrip
ran in 12 of 17 earlier runs, absent from the last 3
evidence moderate | -partial | f_1d77e3f5 | .../ImportTests.cs:41
```
rates: the marker on each finding says which runs its percentage was
counted out of; compare two only where the markers match.
https://docs.xping.io/cli/command-reference.html#the-population-marker
Detail of row 1: xping report --id f_2a91c0de --assembly Checkout.Tests
How to read it
Each finding is four things: a severity, a kind, the test, and the evidence behind the claim.
| Part | Means |
|---|---|
HIGH / MED / LOW |
Impact ranking. Findings are ordered most severe first, so the top of the block is the part worth reading. |
flaky, slower, stopped running |
The kind — what the finding claims. There are sixteen; the CLI reference lists them all. |
| The counts line | The measurement the claim rests on, in plain numbers. |
evidence low\|moderate\|high |
How many runs stand behind this claim — which is not always every run the test appeared in, since a finding can only count the runs it could measure. A low-evidence finding is a lead, not a verdict. |
f_2a91c0de |
A stable id for that finding, so you can refer to it in a ticket or diff two reports. |
The report body sits inside a fenced code block and stays under 72 columns, so pasting it into
Slack, a PR, or a ticket keeps its columns. Only the top ten findings show by default —
--all shows the rest.
Separating flaky from always failing is deliberate. A test that never passes is not unreliable, it is broken, and the fix is completely different.
What gets flagged
The report distinguishes sixteen finding kinds. The ones you meet first:
| Printed as | --kind value |
Claims |
|---|---|---|
| masked by retry | RetryMasked |
The test failed and passed on retry, never reaching the run's outcome. The strongest flakiness signal, and the only one available from a single run. |
| flaky | Flaky |
The test both passes and fails, or fails in varying ways. |
| always failing | AlwaysFailing |
The test fails almost always, in one dominant way — a likely real bug, not flake. |
| timing out | TimingOut |
The test is mostly killed for overrunning its timeout rather than failing. |
| slower | DurationRegression |
The test's recent runs are slower than its earlier ones by more than the variation it already had. The headline states the factor and the interval around it, and the evidence states each side's spread — see known limitations. |
| stopped running | Vanished |
The test ran habitually through the baseline and has stopped. |
The rest — order dependence, concurrency sensitivity, broken fixtures, shared failures, time-of-day
and network clustering — are in the CLI reference.
Restrict the report to one or more with --kind:
xping report --kind Flaky --kind AlwaysFailing
Skipped and not-executed tests are ignored; they carry no reliability signal.
Multiple test projects
Every test project in a solution shares one store. When it holds more than one, xping report lists them, one row each, most severe first:
Xping · 3 assemblies
Checkout.Tests 4 findings (1 high, 3 medium)
412 tests · 4 flagged · 20 runs · last 2026-11-08 16:45
Billing.Tests no findings
96 tests · 0 flagged · 20 runs · last 2026-11-08 16:44
Search.Tests no findings
57 tests · 0 flagged · 6 runs · last 2026-11-02 09:12
Details: xping report --assembly Checkout.Tests
To see one suite's findings:
xping report --assembly Billing.Tests
--fail-on judges every suite, so a CI step fails when any of them has a finding at the threshold.
Each assembly is analysed against its own run history, so a suite that has run 3 times is never described against another suite's 12-run window. --all is not the way to see every suite — it shows every finding, and every row of the latest run, instead of the first ten of each.
Where your data lives
xping where
/Users/you/src/my-repo/.xping
36 runs · 1.2 MB on disk
Checkout.Tests 12 runs last 2026-08-08 09:14 UTC
Billing.Tests 12 runs last 2026-08-08 09:13 UTC
The store lives in a .xping folder at your repository root and ignores itself — it contains a .gitignore that hides the whole directory, so nothing appears in git status and your own .gitignore is never modified.
It keeps the last 50 runs of each test project and prunes automatically. See Local Store for retention settings and how to relocate it.
To delete history:
xping clear # prompts before deleting
xping clear --assembly Billing.Tests # only one suite
What local-only mode cannot tell you
Local history contains only your runs, on your machine. It structurally cannot include:
- CI runs
- Your teammates' runs
- Behaviour differences across operating systems or environments
- More history than your retention window holds
This is why the local report does not show a confidence score. The confidence score weighs six factors, several of which — environment consistency in particular — need data that does not exist on one developer's machine. Showing a number here that looked like Xping Cloud's but disagreed with it would be worse than showing none.
Local-only mode answers "what is unstable on my machine, in my last N runs". Xping Cloud answers "what is unstable across CI, every branch, and everyone's machines, over months".
Connecting later
Nothing is lost by starting local. Add an API key whenever you want — that is the only variable you need, and no test code changes:
export XPING_APIKEY="your-key"
You do not name a project. Xping derives one per test assembly, created on first upload.
XPING_PROJECTID is optional and only pins several assemblies into a single project — see
ProjectId.
Keep the key out of your repository: export it from your shell or a secret store, never
appsettings.json, which is committed and copied into build output.
The SDK switches to Cloud mode and uploads as normal. It keeps writing the local store, so xping report continues to work offline and on your own machine.
To see Cloud's view in the same report, sign in once on your machine:
xping login
xping report then adds Xping Cloud's confidence score to the rows it already prints, labelled so
it is never mistaken for a local number. Offline, it prints the local report as before. See
Cloud data.
To keep the SDK from uploading even with credentials present, set the mode explicitly:
export XPING_MODE=LocalOnly
XPING_MODE is an SDK setting; the CLI does not read it. Without a sign-in, xping report tries
the XPING_APIKEY in your shell, and an upload key answers that it cannot read: the report stays
local and, at a terminal, says so in one line. xping login replaces that line with Cloud data.
See Also
- Xping CLI Reference — every command and option
- Local Store — location, retention, and settings
- Configuration Reference — all SDK settings
- Identifying Flaky Tests