Delta Report¶

The -diff flag compares two MethodAtlas scan outputs (CSV files) and reports which test methods were added, removed, or modified between the two runs. It is the primary mechanism for change-tracking, audit evidence, and CI gating on security-test coverage.

A delta report answers the regulator's recurring question: did the security-test surface of this release change relative to the previous one, and if so, where? The CSV emitted by a single scan is a snapshot; the delta turns two snapshots into a change log, which is what most compliance frameworks actually require for a release-to-release audit.

A delta is also the cheapest way to enforce a non-regression rule in CI. Rather than scanning twice from scratch and comparing in some external tool, you run a single methodatlas -diff before.csv after.csv, post- process the textual output with a one-line shell pattern (see CI gating), and either fail the build or annotate the pull request inline.

When to use this mode¶

There are four recurring situations where -diff is the right answer:

You want to prove to an auditor that security-test coverage did not regress between two releases.
You want to gate a CI pipeline on whether any security-relevant test was removed or reclassified.
You want a change log of which security tests were added during a sprint or on a pull request.
You want to detect source edits to existing security tests (requires both scans to use -content-hash).

For the first three scenarios a regular scan with no extra flags is sufficient on both sides. The fourth — detecting whether a test body was rewritten while keeping its name and signature — is the only one that requires content hashing on both inputs; without it the delta will still detect added or removed methods but cannot tell whether an existing test was silently weakened.

How the comparison works¶

sequenceDiagram
    participant A as Before scan<br/>(e.g. v1.2-scan.csv)
    participant D as Delta engine<br/>(MethodAtlas -diff)
    participant B as After scan<br/>(e.g. v1.3-scan.csv)

    A->>D: load all rows<br/>(fqcn + method = key)
    B->>D: load all rows<br/>(fqcn + method = key)
    D->>D: key in A only → mark as REMOVED (-)
    D->>D: key in B only → mark as ADDED (+)
    D->>D: key in both → compare shared fields<br/>loc, tags, display_name, content_hash, security, ai_tags
    D->>D: fields differ → mark as MODIFIED (~)
    D-->>D: emit summary with counts

The two CSV files are compared using the (fqcn, method) pair as the key. Fields that are present in both files are compared; fields absent from either file (because the two scans used different flag sets) are skipped silently.

Basic usage¶

./methodatlas -diff scan-before.csv scan-after.csv

Both files must be CSV outputs produced by MethodAtlas. All other flags are ignored when -diff is present.

Example output¶

MethodAtlas delta report
  before: scan-2026-04-10.csv  (scanned: 2026-04-10T09:00:00Z · 45 methods · 5 security-relevant)
  after:  scan-2026-04-24.csv  (scanned: 2026-04-24T14:30:00Z · 47 methods · 7 security-relevant)

+ com.acme.auth.Oauth2FlowTest  test_authCode
+ com.acme.auth.Oauth2FlowTest  test_tokenRefresh
- com.acme.auth.LegacyAuthTest  test_basicAuth
~ com.acme.crypto.AesGcmTest    roundTrip_encryptDecrypt  [source; security: false → true]

2 added  ·  1 removed  ·  1 modified  ·  42 unchanged
security-relevant: 5 → 7  (+2)

Change-type indicators¶

Symbol	Meaning
`+`	Method is new in the after scan — added since the before run
`-`	Method is absent from the after scan — removed or renamed since the before run
`~`	Method is present in both scans but one or more fields changed

What fields are compared¶

For each method present in both scans:

Field	Compared when
`loc`	Always (lines of code in the method declaration)
`tags`	Always (JUnit `@Tag` annotations; order-independent)
`display_name`	Both files contain the `display_name` column; detects `@DisplayName` annotations added, removed, or renamed
`source`	Both files were produced with `-content-hash`; a difference means the enclosing class source was edited
`security`	Both files were produced with `-ai`; a flip of `ai_security_relevant`
`ai_tags`	Both files were produced with `-ai`; the AI taxonomy tag set changed

Fields absent from either file (produced with different flag sets) are silently skipped. You can compare a scan with -content-hash against one without it — only the fields common to both files are compared.

Change detail notation¶

A ~ (modified) line means the method is present in both the before and after scan, but at least one field changed. The bracketed summary after the method name lists every field that changed and, where possible, the old and new values.

Reading the example from above:

~ com.acme.crypto.AesGcmTest    roundTrip_encryptDecrypt  [source; security: false → true]

This means: the content_hash changed (the source was edited — indicated by source) and the AI security classification changed from false to true. Both changes happened in the same method between the two scans.

The full set of possible bracket tokens:

Token	Meaning
`source`	`content_hash` differs — the class source was edited since the before scan
`loc: 5 → 8`	Lines of code in the method declaration changed (5 in before, 8 in after)
`tags`	The JUnit `@Tag` set changed (tags were added, removed, or renamed)
`display_name`	The `@DisplayName` annotation was added, removed, or its text changed
`security: false → true`	AI classification flipped from not-security-relevant to security-relevant
`security: true → false`	AI classification flipped from security-relevant to not-security-relevant (regression)
`ai_tags`	The AI taxonomy tag set changed (e.g. `auth` was added or removed)

Multiple tokens are separated by semicolons. A method may appear with several changed fields at once.

Why the delta report matters in the SDLC¶

Sprint close / PR review¶

Run MethodAtlas at the start and end of a sprint (or on the base branch and the feature branch), then diff the two outputs to answer "did this sprint / this PR add, remove, or reclassify security-relevant tests?":

git stash
./methodatlas -ai -content-hash -emit-metadata src/test/java > before.csv
git stash pop
./methodatlas -ai -content-hash -emit-metadata src/test/java > after.csv
./methodatlas -diff before.csv after.csv

Regulatory evidence¶

Many security standards (PCI DSS, ISO 27001, SOC 2) require evidence that security controls are tested and that the test coverage does not regress between releases. A delta report generated at each release boundary provides a machine-readable change log of the security-test layer:

+ lines prove new security tests were added alongside new features
- lines must be reviewed and justified (was the test replaced or deleted?)
~ lines with security: true → false require particular scrutiny — a test that was previously considered security-relevant was reclassified

Store the delta output alongside the scan CSV in your artifact repository or compliance folder.

CI gating — detecting regressions¶

A simple shell gate that fails the build if any security-relevant test is removed or reclassified away from security:

./methodatlas -diff before.csv after.csv > delta.txt
cat delta.txt

# Fail if any security-relevant test was removed
if grep -qE "^- " delta.txt; then
  echo "ERROR: test methods were removed — review delta.txt"
  exit 1
fi

# Fail if a security-relevant method was reclassified to non-security
if grep -qE "security: true → false" delta.txt; then
  echo "ERROR: security classification regressed — review delta.txt"
  exit 1
fi

Nightly scan with archiving¶

A typical CI setup stores one scan output per day and computes the delta on each run:

# .github/workflows/security-delta.yml (excerpt)
- name: Run today's scan
  run: |
    ./methodatlas -ai -content-hash -emit-metadata \
      -ai-api-key-env ANTHROPIC_API_KEY \
      src/test/java > scans/$(date +%F).csv

- name: Compute delta from yesterday
  run: |
    YESTERDAY=$(date -d "yesterday" +%F)
    if [ -f "scans/$YESTERDAY.csv" ]; then
      ./methodatlas -diff "scans/$YESTERDAY.csv" scans/$(date +%F).csv \
        | tee delta-$(date +%F).txt
    else
      echo "No previous scan found; skipping delta"
    fi

- name: Archive scan and delta
  uses: actions/upload-artifact@v4
  with:
    name: security-scan-$(date +%F)
    path: |
      scans/$(date +%F).csv
      delta-$(date +%F).txt

Combining with scan options¶

The -diff mode only consumes CSV files — it does not run a scan itself. To produce the input files you can use any combination of scan options. For the most informative delta, include both -content-hash (detects source edits) and -ai (detects classification changes) in both scan runs:

# First scan (e.g. at release tag v1.2)
./methodatlas -ai -content-hash -emit-metadata \
  -ai-provider anthropic \
  -ai-api-key-env ANTHROPIC_API_KEY \
  src/test/java > v1.2-scan.csv

# Second scan (e.g. at release tag v1.3)
./methodatlas -ai -content-hash -emit-metadata \
  -ai-provider anthropic \
  -ai-api-key-env ANTHROPIC_API_KEY \
  src/test/java > v1.3-scan.csv

# Delta report
./methodatlas -diff v1.2-scan.csv v1.3-scan.csv

If you omit -content-hash, source changes cannot be detected; if you omit -ai, only structural changes (method added, removed, LOC changed) will appear in the delta.

Limitations¶

The delta engine is deliberately conservative: it reports facts it can prove from two CSV files and does not infer intent. Three limits follow from that design choice.

Rename tracking — when a class or method is renamed, it appears as one REMOVED entry and one ADDED entry. MethodAtlas does not attempt to infer renames from content-hash similarity, because content-similarity inference produces false positives that would weaken the audit signal. If your release process renames classes routinely, post-process the delta with a similarity matcher of your own.
Plain / SARIF inputs — only CSV files produced by MethodAtlas are accepted. Plain-text and SARIF outputs cannot be diffed, because they omit fields (such as the per-method content hash and the AI taxonomy tag set) that the comparison depends on.
Exit code — the command always exits with code 0. Use shell post-processing (see the CI gating example above) to gate on specific change types. The reason for the design is that different teams care about different change types: a test-engineering team may treat any REMOVED as a regression, while a refactoring team may accept removals provided no security: true → false reclassification is present.

Reproducibility and audit retention¶

Because the inputs to -diff are plain CSV files, a delta is fully reproducible: archive the two scan CSVs alongside the delta report and any auditor can re-run the comparison years later without needing the original codebase or even the MethodAtlas binary that produced them. In regulated environments this property — being able to verify a historic claim about test coverage from durable artefacts — is the reason the delta report exists in the first place. See the Audit trail chapter for the recommended retention layout.

See CLI reference — -diff for the flag description.