Table of Contents

Running Without an Account

Xping works with no API key, no signup, and no network access. Add the SDK to your test project, run your tests a few times, and xping report will tell you which of your tests are unreliable.

This is local-only mode. Everything stays on your machine.


What You'll Learn

  • How to collect test history with no Xping account
  • How to read the local flakiness report
  • Where local data is stored and how to remove it
  • What local-only mode can and cannot tell you

Prerequisites

  • An existing test project using NUnit, xUnit, or MSTest
  • The .NET SDK

No account. No API key.


Step 1: Add the SDK

Install the adapter for your test framework:

dotnet add package Xping.Sdk.NUnit     # or Xping.Sdk.XUnit (xUnit v2) / Xping.Sdk.XUnit.V3 (xUnit v3) / Xping.Sdk.MSTest

Wire it up exactly as the framework quickstart describes (NUnit, xUnit, MSTest) — but skip the API key step.

With no ApiKey configured, the SDK starts in local-only mode:

[Xping] Initialized in local-only mode. No API key configured - results stay on this machine.

Step 2: Run your tests a few times

dotnet test

Each run is recorded to a local store. Cross-run flakiness detection needs at least three runs, because two runs cannot distinguish a flaky test from a test that just broke.


Step 3: Install the CLI

Requires the .NET 10 SDK or runtime. The tool targets net10.0 and will not launch on older runtimes. The Xping SDK packages themselves target netstandard2.0 and are unaffected — your test projects can stay on any supported .NET version.

dotnet tool install -g Xping.Cli

This puts an xping command on your PATH, usable from any repository. If your shell answers command not found: xping, add ~/.dotnet/tools to your PATH and reopen the terminal.

To pin the CLI version alongside your repo instead, install it into a tool manifest — see Installation in the CLI reference.


Step 4: Read the report

xping report
Xping · Checkout.Tests · 20 runs · 2026-08-05 → 2026-08-19 · main@a3f9c2e
412 tests · 409 healthy · 3 flagged · 41 awaiting more runs · 6 not significant
◷ nothing to measure: concurrency 108 · slower 32

```
NEEDS ATTENTION (1 high, 1 medium, 1 low)              most severe first
────────────────────────────────────────────────────────────────────────

1.  HIGH  flaky
    SummaryTests.GenerateMonthlySummary
    failed 7 of 20 executions (35%) in 5 of 20 runs, 3 failure modes
    evidence moderate | -env-cluster | f_2a91c0de | ...ummaryTests.cs:88

2.  MED   slower
    FlowTests.CheckoutFlow_Completes
    3.51x slower (95% CI 1.94-5.87x), 340ms -> 1.2s on the clock
    evidence high | -env | f_8c04b71a | tests/Checkout/FlowTests.cs:214

3.  LOW   stopped running
    LegacyImport.Roundtrip
    ran in 12 of 17 earlier runs, absent from the last 3
    evidence moderate | -partial | f_1d77e3f5 | .../ImportTests.cs:41
```

rates: the marker on each finding says which runs its percentage was
counted out of; compare two only where the markers match.
https://docs.xping.io/cli/command-reference.html#the-population-marker

Detail of row 1: xping report --id f_2a91c0de --assembly Checkout.Tests

How to read it

Each finding is four things: a severity, a kind, the test, and the evidence behind the claim.

Part Means
HIGH / MED / LOW Impact ranking. Findings are ordered most severe first, so the top of the block is the part worth reading.
flaky, slower, stopped running The kind — what the finding claims. There are sixteen; the CLI reference lists them all.
The counts line The measurement the claim rests on, in plain numbers.
evidence low\|moderate\|high How many runs stand behind this claim — which is not always every run the test appeared in, since a finding can only count the runs it could measure. A low-evidence finding is a lead, not a verdict.
f_2a91c0de A stable id for that finding, so you can refer to it in a ticket or diff two reports.

The report body sits inside a fenced code block and stays under 72 columns, so pasting it into Slack, a PR, or a ticket keeps its columns. Only the top ten findings show by default — --all shows the rest.

Separating flaky from always failing is deliberate. A test that never passes is not unreliable, it is broken, and the fix is completely different.


What gets flagged

The report distinguishes sixteen finding kinds. The ones you meet first:

Printed as --kind value Claims
masked by retry RetryMasked The test failed and passed on retry, never reaching the run's outcome. The strongest flakiness signal, and the only one available from a single run.
flaky Flaky The test both passes and fails, or fails in varying ways.
always failing AlwaysFailing The test fails almost always, in one dominant way — a likely real bug, not flake.
timing out TimingOut The test is mostly killed for overrunning its timeout rather than failing.
slower DurationRegression The test's recent runs are slower than its earlier ones by more than the variation it already had. The headline states the factor and the interval around it, and the evidence states each side's spread — see known limitations.
stopped running Vanished The test ran habitually through the baseline and has stopped.

The rest — order dependence, concurrency sensitivity, broken fixtures, shared failures, time-of-day and network clustering — are in the CLI reference. Restrict the report to one or more with --kind:

xping report --kind Flaky --kind AlwaysFailing

Skipped and not-executed tests are ignored; they carry no reliability signal.


Multiple test projects

Every test project in a solution shares one store. When it holds more than one, xping report lists them, one row each, most severe first:

Xping · 3 assemblies

Checkout.Tests                               4 findings (1 high, 3 medium)
  412 tests · 4 flagged · 20 runs · last 2026-11-08 16:45
Billing.Tests                                                  no findings
  96 tests · 0 flagged · 20 runs · last 2026-11-08 16:44
Search.Tests                                                   no findings
  57 tests · 0 flagged · 6 runs · last 2026-11-02 09:12

Details: xping report --assembly Checkout.Tests

To see one suite's findings:

xping report --assembly Billing.Tests

--fail-on judges every suite, so a CI step fails when any of them has a finding at the threshold.

Each assembly is analysed against its own run history, so a suite that has run 3 times is never described against another suite's 12-run window. --all is not the way to see every suite — it shows every finding, and every row of the latest run, instead of the first ten of each.


Where your data lives

xping where
/Users/you/src/my-repo/.xping
  36 runs · 1.2 MB on disk
  Checkout.Tests                              12 runs  last 2026-08-08 09:14 UTC
  Billing.Tests                               12 runs  last 2026-08-08 09:13 UTC

The store lives in a .xping folder at your repository root and ignores itself — it contains a .gitignore that hides the whole directory, so nothing appears in git status and your own .gitignore is never modified.

It keeps the last 50 runs of each test project and prunes automatically. See Local Store for retention settings and how to relocate it.

To delete history:

xping clear                              # prompts before deleting
xping clear --assembly Billing.Tests     # only one suite

What local-only mode cannot tell you

Local history contains only your runs, on your machine. It structurally cannot include:

  • CI runs
  • Your teammates' runs
  • Behaviour differences across operating systems or environments
  • More history than your retention window holds

This is why the local report does not show a confidence score. The confidence score weighs six factors, several of which — environment consistency in particular — need data that does not exist on one developer's machine. Showing a number here that looked like Xping Cloud's but disagreed with it would be worse than showing none.

Local-only mode answers "what is unstable on my machine, in my last N runs". Xping Cloud answers "what is unstable across CI, every branch, and everyone's machines, over months".


Connecting later

Nothing is lost by starting local. Add an API key whenever you want — that is the only variable you need, and no test code changes:

export XPING_APIKEY="your-key"

You do not name a project. Xping derives one per test assembly, created on first upload. XPING_PROJECTID is optional and only pins several assemblies into a single project — see ProjectId.

Keep the key out of your repository: export it from your shell or a secret store, never appsettings.json, which is committed and copied into build output.

The SDK switches to Cloud mode and uploads as normal. It keeps writing the local store, so xping report continues to work offline and on your own machine.

To see Cloud's view in the same report, sign in once on your machine:

xping login

xping report then adds Xping Cloud's confidence score to the rows it already prints, labelled so it is never mistaken for a local number. Offline, it prints the local report as before. See Cloud data.

To keep the SDK from uploading even with credentials present, set the mode explicitly:

export XPING_MODE=LocalOnly

XPING_MODE is an SDK setting; the CLI does not read it. Without a sign-in, xping report tries the XPING_APIKEY in your shell, and an upload key answers that it cannot read: the report stays local and, at a terminal, says so in one line. xping login replaces that line with Cloud data.


See Also