Package org.egothor.methodatlas.command

Class EvalAccuracyCommand

java.lang.Object

org.egothor.methodatlas.command.EvalAccuracyCommand

All Implemented Interfaces:

Command

public final class EvalAccuracyCommand
extends Object
implements Command

Utility CLI mode (-eval-accuracy) that measures how accurate the AI classification is against a human-reviewed labelled set.

Synopsis: methodatlas -eval-accuracy <labels.yaml> <predictions.csv> [-eval-format markdown|json] [-eval-min-f1 <0..1>]

• labels.yaml — the ground truth, in the -override-file format (the recommended source: those entries are already human-reviewed);
• predictions.csv — a scan CSV produced with -ai and without -override-file, so the AI columns hold genuine AI output;
• -eval-format — markdown (default) or json;
• -eval-min-f1 — optional gate: exit 1 when the F1 of the securityRelevant decision or the micro-averaged tag F1 is below the threshold, or when neither could be computed (no labelled data — a gate must not pass on no evidence).

No scan runs and no AI provider is called: the command only reads the two files, so it is deterministic and offline. The report is written with \n line endings regardless of platform.

Since:

6.1.0

See Also:

• AccuracyEvaluator
• AccuracyMetrics

Field Summary

Fields

Modifier and Type	Field	Description
static final String	FLAG_EVAL_ACCURACY	Flag that selects this utility mode; takes two path arguments.
static final String	FLAG_EVAL_FORMAT	Flag selecting the report format.
static final String	FLAG_EVAL_MIN_F1	Flag setting the minimum acceptable F1.

Constructor Summary

Constructors

Constructor	Description
EvalAccuracyCommand(Path labels, Path predictions, boolean json, OptionalDouble minF1)	Creates a command.

Method Summary

Modifier and Type	Method	Description
int	execute(PrintWriter out)	Evaluates the predictions against the labels and writes the report.
static EvalAccuracyCommand	fromArgs(String... args)	Builds the command from raw command-line arguments.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

FLAG_EVAL_ACCURACY

public static final String FLAG_EVAL_ACCURACY

Flag that selects this utility mode; takes two path arguments.

See Also:

• Constant Field Values

FLAG_EVAL_FORMAT

public static final String FLAG_EVAL_FORMAT

Flag selecting the report format.

See Also:

• Constant Field Values

FLAG_EVAL_MIN_F1

public static final String FLAG_EVAL_MIN_F1

Flag setting the minimum acceptable F1.

See Also:

• Constant Field Values

Constructor Details

EvalAccuracyCommand

public EvalAccuracyCommand(Path labels,
 Path predictions,
 boolean json,
 OptionalDouble minF1)

Creates a command.

Parameters:

labels - override-format YAML with the human-reviewed labels

predictions - scan CSV produced with -ai

json - true for JSON output, false for Markdown

minF1 - minimum acceptable F1 in [0,1], or empty for no gate

Method Details

fromArgs

public static EvalAccuracyCommand fromArgs(String... args)

Builds the command from raw command-line arguments.

Parameters:

args - raw arguments; must contain -eval-accuracy followed by the labels path and the predictions path

Returns:

a configured command; never null

Throws:

IllegalArgumentException - if the paths are missing, the format is neither markdown nor json, or the threshold is not a number in [0,1]

execute

public int execute(PrintWriter out)
            throws IOException

Evaluates the predictions against the labels and writes the report.

Specified by:

execute in interface Command

Parameters:

out - writer that receives the report; never null

Returns:

0 on success or when no gate is set; 1 when the -eval-min-f1 gate fails

Throws:

IOException - if either input file cannot be read or parsed