5.1. Regression Metrics

Regression metrics turn a vector of predictions and a vector of ground-truth targets into one scalar. That scalar states how good the fit is. RustyML groups these metrics under rustyml::metrics. The category module rustyml::metrics::regression re-exports them flat, so you reach any of them as rustyml::metrics::mean_squared_error. You can also reach them through the prelude with use rustyml::prelude::*;.

This page covers the 7 regression functions the crate ships. It explains the math behind each function, how each function reacts to outliers, and how argument order changes the result. It also shows how to read the functions together after you fit a Linear Regression model. The function names and the (y_true, y_pred) argument order match scikit-learn. RustyML diverges from scikit-learn in 2 ways. It panics instead of returning a Result (see Section 5.1.6), and it does not provide adjusted_r2 (see Section 5.1.2).

5.1.1. The 7 functions and their signatures

Every regression metric has the same shape. It takes 2 one-dimensional arrays and returns one f64 value. The functions are generic over ndarray storage. y_true and y_pred can each be an owned array (Array1<f64>) or a view (ArrayView1<f64>), for example a .column(j) slice of a 2-D array. Both arrays must hold f64 values. There is no f32 overload.

// where S1: Data<Elem = f64>, S2: Data<Elem = f64>
pub fn mean_squared_error<S1, S2>(y_true: &ArrayBase<S1, Ix1>, y_pred: &ArrayBase<S2, Ix1>) -> f64;
pub fn root_mean_squared_error<S1, S2>(y_true: &ArrayBase<S1, Ix1>, y_pred: &ArrayBase<S2, Ix1>) -> f64;
pub fn mean_absolute_error<S1, S2>(y_true: &ArrayBase<S1, Ix1>, y_pred: &ArrayBase<S2, Ix1>) -> f64;
pub fn median_absolute_error<S1, S2>(y_true: &ArrayBase<S1, Ix1>, y_pred: &ArrayBase<S2, Ix1>) -> f64;
pub fn mean_absolute_percentage_error<S1, S2>(y_true: &ArrayBase<S1, Ix1>, y_pred: &ArrayBase<S2, Ix1>) -> f64;
pub fn r2_score<S1, S2>(y_true: &ArrayBase<S1, Ix1>, y_pred: &ArrayBase<S2, Ix1>) -> f64;
pub fn explained_variance_score<S1, S2>(y_true: &ArrayBase<S1, Ix1>, y_pred: &ArrayBase<S2, Ix1>) -> f64;

The argument order is (y_true, y_pred): ground truth first, predictions second. This matches scikit-learn and the clustering metrics’ (labels_true, labels_pred) order. For 4 of the 7 functions (MSE, RMSE, MAE, MedAE), the order does not matter. Each of these builds on abs(y_true - y_pred) or (y_true - y_pred)^2, and both forms are symmetric. MAPE’s denominator uses only the first argument, so MAPE is order-sensitive too (see Section 5.1.5).

Order matters a great deal for the 2 variance-explained scores. r2_score and explained_variance_score normalize by the spread of the first argument. Swapping the arguments does not raise an error. It silently returns a different, wrong number. This is the most common way to corrupt a regression evaluation. Section 5.1.5 shows the cost of this mistake.

5.1.2. What each metric measures and when to trust it

MSE averages the squared error. Squaring makes every residual positive, so the argument order does not matter. Squaring also weights a large residual more than a small one. An error of 4 contributes 16. 4 errors of 1 contribute 4 in total.

This quadratic weighting is why most optimizers minimize MSE as a loss function. It also makes MSE hypersensitive to outliers, because one bad sample can dominate the whole number. The unit of MSE is the square of the target unit, so MSE is hard to interpret directly. A statement like “the error is 0.02 squared dollars” carries no clear meaning.

RMSE is sqrt(MSE). Taking the square root returns the number to the target’s own units, so you can say “the typical error is about 0.15 dollars”. MSE is never negative, so the square root always exists. RMSE keeps the quadratic emphasis on large errors, so it stays outlier-sensitive.

Report RMSE when large mistakes cost disproportionately more and you want the answer in real units. RMSE is never smaller than MAE, and the 2 are equal only when every error has the same size. A large gap between RMSE and MAE signals that a few big residuals inflate RMSE.

MAE averages the absolute error. Each sample contributes in proportion to its own error, not to the square of the error. This makes MAE less sensitive to outliers than RMSE, and MAE stays in the target’s units. Use MAE when every unit of error costs the same and you do not want a few extreme points to steer the score.

MedAE takes the median of the absolute errors instead of the mean. The median ignores the size of the tail completely. Up to half the samples can be arbitrarily wrong without moving the median. This makes MedAE the metric in the set that outliers affect the least.

Use MedAE on data with heavy-tailed noise or known bad records. MedAE says nothing about the tail it ignores. Do not report MedAE alone when the large errors are the ones that matter.

Ranked by outlier sensitivity, from most to least reactive: MSE and RMSE (squared), then MAE (linear), then MedAE (rank-based, least affected). MAPE does not fit this ranking. MAPE reweights errors by the size of the true value, not by the size of the error.

MAPE is the mean of the per-sample relative errors: abs(y_true - y_pred) / max(abs(y_true), eps), with eps set to f64::EPSILON. The result is a fraction. Multiply it by 100 to state it as a percent. MAPE is scale-free. That property matters when targets span several orders of magnitude, because a fixed absolute error means different things at different scales.

Two behaviors are worth watching. First, the denominator uses abs(y_true), so a negative target works normally. A y_true value at or near zero gets floored at f64::EPSILON instead of causing a division by zero. This floor makes that sample’s term explode to about 10^13, which drags up the whole mean. Treat a MAPE in the trillions as a sign that some y_true value is zero, not as a sign the model failed.

Second, MAPE is not symmetric. Under-prediction is bounded at 100 percent, but over-prediction has no upper bound. As a result, MAPE quietly rewards a model that predicts values on the low side.

R^2 is the coefficient of determination. The formula is 1 - SSE / SST. SSE = sum((y_pred - y_true)^2) is the residual sum of squares. SST = sum((y_true - mean(y_true))^2) is the total variance of the targets around their own mean. R^2 states what fraction of the target’s variance the model explains, compared to the baseline of always predicting the mean. A value of 1.0 marks a perfect fit.

A value of 0.0 means the model is no better than predicting mean(y_true). R^2 has no lower bound. It goes negative whenever SSE is greater than SST, that is, whenever the model is worse than the constant-mean baseline. A negative R^2 is a real and meaningful signal. It usually has one of 3 causes: a mis-specified model, evaluation on data from a different distribution than the model trained on, or swapped arguments. Because SST comes from y_true alone, R^2 is not symmetric in its arguments (see Section 5.1.5).

When y_true is constant, the ratio is undefined. RustyML follows scikit-learn here: it returns 1.0 for an exact fit and 0.0 otherwise, so a constant target never produces a NaN. RustyML decides constancy by comparing the values to each other, not by testing SST against a threshold. An earlier version used an absolute 1e-10 threshold on the unnormalized sum of squares. That threshold reported a false 1.0 for a genuinely varying target with a small spread, for example [1e-6, 2e-6, 3e-6], whose SST is 2e-12. A test for exact SST == 0.0 fails in the other direction, because computing the mean does not round-trip every constant value exactly.

EVS, the explained variance score, replaces the residual sum of squares in R^2 with the variance of the residuals. The formula is 1 - Var(y_true - y_pred) / Var(y_true). Subtracting the residual mean before squaring means a constant prediction bias does not lower the score.

Say a model is always off by exactly +1. Its residuals have zero variance, so EVS equals 1.0, even though the predictions are systematically wrong. R^2 correctly penalizes that same bias. The gap EVS - R^2 measures how biased the predictions are. Consider an unbiased fit, for example any model with an intercept fitted by least squares on its own training data. Its residual mean is close to 0, so EVS is close to R^2.

RustyML does not provide an adjusted-R^2 function. Plain R^2 never decreases when you add a feature, so it cannot compare models with a different number of features. Compute the adjustment yourself if you need it. The formula is adj = 1 - (1-r2)*(n-1)/(n-p-1), where n is the sample count and p is the predictor count.

5.1.3. Choosing a metric for model selection

Report at least one scale-aware error metric together with one variance-explained score. A single number hides too much. Use RMSE when large errors cost disproportionately more and you want the answer in the target’s units. Use MAE when errors scale linearly with cost and you distrust a few extreme points. Use MedAE when the data has known bad records or heavy tails and you want a number the tail cannot move. Use MAPE only when targets are strictly positive and comparable across scales, and never when a target can be zero.

Use R^2 to state how good a model is in a dimensionless way. Use EVS instead when you want to factor out a constant bias. For hyperparameter search and cross-model comparison, pick one metric up front and hold it fixed. Comparing model A’s RMSE against model B’s MAE has no meaning. Always compute these metrics on a held-out split (see Train-Test Split). Overfitting the training set trivially minimizes every metric on this page.

5.1.4. A worked example

This example fits a Linear Regression model on 5 noisy points, predicts on the same inputs, and computes every metric on this page. LinearRegression uses its default closed-form solver here. That solver is exact and instant, with no learning rate or iteration count to tune, so the example stays deterministic and fast. The metric calls stay the same no matter how the predictions were produced.

use ndarray::array;
use rustyml::machine_learning::LinearRegression;
use rustyml::metrics::{
    explained_variance_score, mean_absolute_error, mean_absolute_percentage_error,
    mean_squared_error, median_absolute_error, r2_score, root_mean_squared_error,
};

fn main() {
    // 5 samples, 1 feature. Roughly y = 2x, with a little measurement noise.
    let x = array![[1.0], [2.0], [3.0], [4.0], [5.0]];
    let y_true = array![2.1, 3.9, 6.2, 7.8, 10.1];

    // Closed-form ordinary least squares: exact, no hyperparameters, runs instantly.
    let mut model = LinearRegression::new(true);
    model.fit(&x, &y_true).unwrap();

    let y_pred = model.predict(&x).unwrap();

    // Ground truth first, predictions second. Order matters for R^2 and EVS.
    println!("MSE   = {:.5}", mean_squared_error(&y_true, &y_pred));
    println!("RMSE  = {:.5}", root_mean_squared_error(&y_true, &y_pred));
    println!("MAE   = {:.5}", mean_absolute_error(&y_true, &y_pred));
    println!("MedAE = {:.5}", median_absolute_error(&y_true, &y_pred));
    println!("MAPE  = {:.5}", mean_absolute_percentage_error(&y_true, &y_pred));
    println!("R2    = {:.5}", r2_score(&y_true, &y_pred));
    println!("EVS   = {:.5}", explained_variance_score(&y_true, &y_pred));

    // A fitted LinearRegression can also give you R^2 directly, without predict():
    println!("score = {:.5}", model.score(&x, &y_true).unwrap());
}

These are the values the program prints, rounded. The closed-form solver makes the output deterministic, so the same input always prints the same numbers.

MSE   ~ 0.021       (squared y-units)
RMSE  ~ 0.146       (y-units)
MAE   ~ 0.136       (y-units)
MedAE ~ 0.130       (y-units)
MAPE  ~ 0.026       (fraction -> ~2.6%)
R2    ~ 0.997
EVS   ~ 0.997
score ~ 0.997

Read together, these numbers tell a consistent story. R^2 is about 0.997, so the fitted line explains almost all the variance in y. RMSE is about 0.15 and MAE is about 0.14, both in the units of y, which ranges from 2 to 10. These 2 values confirm that the typical miss is about a seventh of a unit, small relative to the spread of y. RMSE sits just above MAE, so no single residual dominates the error. MedAE is close to MAE, another sign that the errors are evenly sized rather than tail-heavy.

MAPE is about 2.6 percent, which states the same accuracy in a scale-free way. EVS matches R^2 at this precision. A least-squares fit with an intercept produces residuals with a mean close to 0, so there is no bias left for EVS to forgive. model.score(&x, &y_true) reproduces r2_score(&y_true, &y_pred), because LinearRegression computes R^2 internally using the same definition. Use score for convenience on a fitted model. Use the free function r2_score when you score predictions from anything else, for example a neural network, a KNN regressor, or an external model.

5.1.5. The argument-order trap

r2_score and explained_variance_score normalize by the variance of their first argument. Calling r2_score(&y_pred, &y_true) by mistake does not raise an error. It computes a well-formed but wrong number. The example below scores the same 2 arrays both ways. It also shows R^2 going negative for a genuinely bad model.

use ndarray::array;
use rustyml::metrics::r2_score;

fn main() {
    let a = array![1.0, 2.0, 4.0];
    let b = array![2.0, 3.0, 4.0];

    // Swapping the arguments changes R^2, because SST comes only from the first array.
    println!("r2_score(a, b) = {:.4}", r2_score(&a, &b)); // ~ 0.5714  (= 4/7)
    println!("r2_score(b, a) = {:.4}", r2_score(&b, &a)); // 0.0000

    // A model whose predictions run opposite to the truth is worse than predicting
    // the mean, so R^2 goes negative. This is a real signal, not an error condition.
    let y_true = array![1.0, 2.0, 3.0];
    let y_pred = array![3.0, 2.0, 1.0];
    println!("negative R²    = {:.4}", r2_score(&y_true, &y_pred)); // -3.0000
}

The symmetric metrics (MSE, RMSE, MAE, MedAE, and the numerator of MAPE) are immune to this problem, because they only ever see y_true - y_pred. MAPE is the exception among the error metrics. Its denominator uses the first argument, so mean_absolute_percentage_error is order-sensitive too. One habit prevents all of this. Always pass ground truth first, and name your variables y_true and y_pred, so a swap is visible at the call site.

5.1.6. Input validation, panics, and NaN behavior

The model APIs in Chapter 2 return Result<_, Error> (see Error Handling). The metrics module works differently. It panics on a precondition violation instead of returning an error. This matches how ndarray itself panics on a dimension mismatch.

All 7 functions run the same check first. The lengths must be equal, and the inputs must not be empty. The length check runs before the emptiness check, so a mismatch is reported even when one side is empty. The panic messages mirror the crate’s Error wording:

dimension mismatch: expected 3, found 2
input is empty: y_true and y_pred

You cannot use ? to propagate a panic. Validate the lengths in your own code before you call these functions. Wrap the call in std::panic::catch_unwind if you need to recover from a panic. In practice, you control the lengths of the arrays you pass in, for example a predict output and its matching target. The panics never fire when the lengths already match.

The functions handle non-finite input inconsistently, by design. This matters when your data may contain NaN or inf. r2_score uses plain sums. A single non-finite value in either array propagates through the sum, so the result is NaN. This surfaces corrupt data loudly instead of hiding it, which is usually what you want.

explained_variance_score takes the opposite approach. Its variance helper silently skips non-finite samples and averages over the finite subset. A few bad entries leave a normal-looking score computed from the rest. This behavior is convenient, but it can mask a data problem. Clean your inputs first if a dropped sample would matter.

The error metrics MSE, RMSE, and MAE also propagate NaN through their sums. MedAE sorts with total_cmp, which orders NaN deterministically instead of panicking.

Two edge cases round this out. When y_true is constant, R^2 and EVS would divide by zero. R^2 returns 1.0 only for an exact fit and 0.0 otherwise. EVS returns 1.0 whenever the residuals have zero variance, even with a constant offset, and 0.0 otherwise. Neither path produces a NaN.

RustyML reads constancy off the values themselves, not from a tolerance on the variance. This choice keeps a target with a genuinely tiny spread from being mistaken for a constant and scored a false 1.0.

For MAPE, a y_true entry of zero does not cause a division by zero. It floors the denominator at f64::EPSILON instead, so that sample’s term explodes and drags up the mean with it. Treat an extremely large MAPE as a sign that y_true contains a zero, not as a verdict on the model.

For the companion metrics on the other 2 task families, see Classification Metrics and Clustering Metrics.