Package org.egothor.methodatlas.eval

Class AccuracyMetrics

java.lang.Object

org.egothor.methodatlas.eval.AccuracyMetrics

public final class AccuracyMetrics
extends Object

Pure, deterministic computation of AI-classification quality metrics from (prediction, truth) pairs.

This class performs no I/O and calls no AI provider: given the same AccuracyMetrics.Sample list it always returns the same AccuracyMetrics.Result, so the metric math is unit-testable offline. Reading labels and predictions from files is the job of AccuracyEvaluator.

Two evaluated dimensions

• securityRelevant — a binary decision per method, scored as a confusion matrix (TP/FP/FN/TN) where "positive" means security-relevant. Only samples whose truth label carries a relevance decision take part.
• taxonomy tags — a multi-label decision per method, scored one-vs-rest per tag (a tag is a TP when predicted and labelled, an FP when predicted only, an FN when labelled only) plus micro- and macro-averages. Only samples whose truth label carries a tag set take part.

Undefined values

Precision is TP/(TP+FP), recall is TP/(TP+FN), F1 is 2TP/(2TP+FP+FN) and accuracy is (TP+TN)/total. When a denominator is zero the metric is mathematically undefined, and it is reported as OptionalDouble.empty() rather than silently as 0 or 1: "no evidence" must not read as "perfect" or "terrible" in an audit report.

Tag normalisation

Tags are trimmed, lower-cased with Locale.ROOT and de-duplicated, and blank tags are dropped, so "Crypto " and "crypto" are the same taxonomy tag. The taxonomy is a closed lower-case vocabulary, so this never merges distinct tags.

This class is stateless and thread-safe.

Since:

6.1.0

See Also:

• AccuracyEvaluator

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static final record	AccuracyMetrics.Counts	Confusion counts for one binary decision.
static final record	AccuracyMetrics.Result	The complete metric set.
static final record	AccuracyMetrics.Sample	One method for which a prediction is compared with a human label.
static final record	AccuracyMetrics.TagResult	Metrics of a single taxonomy tag.

Method Summary

Modifier and Type	Method	Description
static AccuracyMetrics.Result	compute(Collection<AccuracyMetrics.Sample> samples)	Computes all metrics for the given samples.
static Set<String>	normalise(Collection<String> tags)	Normalises a tag collection (trim, lower-case with Locale.ROOT, drop blanks, de-duplicate).

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

compute

public static AccuracyMetrics.Result compute(Collection<AccuracyMetrics.Sample> samples)

Computes all metrics for the given samples.

Parameters:

samples - prediction/label pairs; must not be null; may be empty, which yields all-zero counts and undefined ratios

Returns:

the metrics; never null; per-tag list sorted by tag

normalise

public static Set<String> normalise(Collection<String> tags)

Normalises a tag collection (trim, lower-case with Locale.ROOT, drop blanks, de-duplicate).

Parameters:

tags - tags to normalise; null is treated as empty

Returns:

sorted, normalised tag set; never null