5. Model Evaluation

A trained model is only as trustworthy as the number you judge it by. The wrong number hides real failures. Plain accuracy looks excellent on a fraud dataset that is 99% negative, even when the model catches nothing. This chapter covers RustyML’s metrics module. These are the array-to-scalar scoring functions that turn raw predictions into the diagnostics you report. Everything here lives under rustyml::metrics, gated by the metrics feature, which full turns on. The module is re-exported flat, so mean_squared_error is reachable as both metrics::mean_squared_error and metrics::regression::mean_squared_error. Pull the whole set into scope with use rustyml::prelude::metrics::*;.

Two conventions run through the entire module. First, arguments follow the order (y_true, y_pred), ground truth first. That order does not matter for the symmetric scores (MSE, MAE, accuracy). It does change the result for r2_score, ConfusionMatrix::new, and roc_auc, so get the order right by habit. Second, unlike the estimators in Classical Machine Learning, which return the crate’s Error, these functions panic on a precondition violation. A mismatched length or an empty input always triggers a panic, instead of returning a Result. A NaN score also triggers a panic in most functions, but r2_score and explained_variance_score treat a NaN as data instead, as Regression Metrics explains. The module is a lightweight leaf of pure functions. It mirrors ndarray’s own dimension-mismatch behavior. See Error Handling for how that choice differs from the rest of the crate.

use ndarray::array;
use rustyml::metrics::{accuracy, mean_squared_error, r2_score};

fn main() {
    // Regression: (y_true, y_pred), ground truth first
    let y_true = array![3.0, -0.5, 2.0, 7.0];
    let y_pred = array![2.5, 0.0, 2.0, 8.0];
    println!("MSE = {:.4}", mean_squared_error(&y_true, &y_pred));
    println!("R^2 = {:.4}", r2_score(&y_true, &y_pred));

    // Classification: exact-match accuracy over integer labels stored as f64
    let labels = array![0.0, 1.0, 1.0, 0.0];
    let preds = array![0.0, 1.0, 0.0, 0.0];
    println!("accuracy = {:.4}", accuracy(&labels, &preds));
}

The Regression Metrics section covers the continuous-target scores: mean_squared_error and its root root_mean_squared_error, mean_absolute_error, the outlier-resistant median_absolute_error, mean_absolute_percentage_error, and the 2 variance-explained scores r2_score and explained_variance_score. Choose between them with care. r2_score lets a NaN propagate, so corrupt data surfaces loudly. explained_variance_score instead skips non-finite samples silently, and ignores a constant prediction bias. That behavior is convenient, until it hides a real problem.

The Classification Metrics section is the largest, because label problems rarely reduce to one number. It covers the binary ConfusionMatrix. You build a ConfusionMatrix from hard 0/1 labels that you threshold yourself, and it panics on anything else. Its counts derive accuracy, precision, recall, specificity, F1, MCC, and balanced accuracy. The section also covers MulticlassConfusionMatrix, with macro, micro, and weighted aggregation through the Average enum. It covers the standalone accuracy, roc_auc, average_precision, log_loss, cohen_kappa, and top_k_accuracy functions, plus the roc_curve and precision_recall_curve threshold sweeps. Watch the input types. Some functions take bool labels with f64 scores. Others take usize class indices with a probability matrix.

The Clustering Metrics section splits along one important line. Extrinsic metrics (adjusted_rand_index, normalized_mutual_info, adjusted_mutual_info, homogeneity, completeness, V-measure, and fowlkes_mallows_score) compare a clustering against ground-truth labels. Intrinsic metrics (silhouette_score, davies_bouldin_score, calinski_harabasz_score) instead score a clustering from feature geometry alone, for when no ground truth exists. silhouette_score takes a DistanceCalculationMetric from Distance Metrics. This lets you evaluate under the same distance you used to cluster.

Read the sections in order. They all inherit the (y_true, y_pred) order and the panic convention from above, and 5.1 sets the tone for the rest. First train a model, with Classical Machine Learning or Neural Networks. Then hold out a test set with Train-Test Split. This chapter gives you the most after those 2 steps. A metric only means something on data the model never saw during fitting. The one hard prerequisite is a working grasp of Working with ndarray, since every function here consumes and returns ndarray types.