4.2. Standardization and Normalization

RustyML provides 2 operations that make numbers comparable in different ways. Standardization recenters and rescales each feature so it has zero mean and unit variance (a z-score). Normalization rescales each sample so its vector norm equals 1. The crate exposes both as stateless free functions, rustyml::utils::standardize and rustyml::utils::normalize.

Each call to these functions computes its statistics from the array you hand it, and returns a new array. The function stores no training mean between calls. A user who knows scikit-learn should note this difference before using them on a test split.

RustyML also provides stateful counterparts in rustyml::utils: StandardScaler, MinMaxScaler, MaxAbsScaler, RobustScaler, and Normalizer. These are fit/transform objects. They store what they learn from the training matrix, and reapply it to every later batch.

The choice between the free function and a scaler depends on whether a second batch needs the same scaling. If it does, for example a test split, a validation fold, or a single sample at inference, use a scaler. Section 4.2.5 covers this in full.

Read 4.1. Train-Test Split first. The leakage trap on this page applies only relative to a split.

4.2.1. Standardize and normalize are different operations

The 2 functions answer different questions. They are almost never interchangeable. A min-max scaler that squashes features into [0, 1] is a third, separate thing. RustyML provides it as MinMaxScaler among the stateful scalers, not here. normalize scales by a vector norm (L1, L2, Max, or Lp), not by feature range. standardize produces z-scores.

The “couples across samples” row decides whether a leakage problem can occur at all. Section 4.2.5 returns to this point.

4.2.2. Standardization (z-score)

standardize is a single generic free function. It works on an f64 array of any dimension:

pub fn standardize<S, D>(
    data: &ArrayBase<S, D>,
    axis: StandardizationAxis,
) -> Result<Array<f64, D>, Error>
where
    S: Data<Elem = f64>,
    D: Dimension;

The function borrows the input. It returns a new owned array of the same shape. The original array is never mutated.

StandardizationAxis has 3 variants. The mapping to physical axes is the reverse of what “row” and “column” suggest at first glance:

• Column standardizes along Axis(0). Each lane is one column, so this gives per-feature z-scores. Use this for the usual layout, where samples are rows and features are columns.
• Row standardizes along the last axis. Each lane is one row. Use this when a sample is itself a signal, and its internal scale needs normalizing. This is rarely what tabular features need.
• Global flattens the whole array. It standardizes the flattened array as one dataset.

For an N-D array with N greater than 2, Row operates on the last axis and Column operates on the second-to-last axis. Row and Column fail on a 1-D array, because it has only one axis. This returns Error::InvalidInput. Global works on any rank, including 1-D.

The divisor is the population standard deviation: sqrt(variance), with variance divided by n, not n - 1. This matches scikit-learn’s StandardScaler, which also uses ddof=0. Z-scores from RustyML match a ported scikit-learn pipeline.

The mean and variance come from a single numerically stable Welford pass. Above an internal size gate, the pass merges in parallel. Results stay the same across runs on the same machine. RustyML does not guarantee a bit-for-bit match across machines or thread counts (see 7.3. Performance Tuning and Parallelism).

use ndarray::{array, Axis};
use rustyml::utils::standardize::{standardize, StandardizationAxis};

fn main() {
    // Rows are samples. Columns are features on different scales.
    let x = array![
        [1.0, 2000.0],
        [2.0, 3000.0],
        [3.0, 4000.0],
        [4.0, 5000.0],
    ];

    // Standardize each feature (column) to zero mean, unit variance.
    let z = standardize(&x, StandardizationAxis::Column).unwrap();
    println!("shape: {:?}", z.shape()); // [4, 2]

    // Every column now has a population mean near 0 and a population variance near 1.
    for (j, col) in z.axis_iter(Axis(1)).enumerate() {
        let n = col.len() as f64;
        let mean = col.sum() / n;
        let var = col.iter().map(|v| (v - mean).powi(2)).sum::<f64>() / n;
        println!("feature {j}: mean={mean:.3e} var={var:.3}");
    }
}

4.2.3. Normalization (unit norm)

normalize takes an axis and a norm order. It divides each lane by its norm, so the lane’s norm becomes 1:

pub fn normalize<S, D>(
    data: &ArrayBase<S, D>,
    axis: NormalizationAxis,
    order: NormalizationOrder,
) -> Result<Array<f64, D>, Error>
where
    S: Data<Elem = f64>,
    D: Dimension;

NormalizationAxis mirrors the standardization variants. Row is the last axis. Column is the second-to-last axis. Global flattens the whole array. The same 1-D restriction applies to Row and Column. NormalizationOrder picks which norm to use:

Lp(1.0) and Lp(2.0) equal L1 and L2 exactly. The dedicated variants exist because they avoid the powf call. Dividing by a positive norm never flips a value’s sign, so [-3, 4] under L2 becomes [-0.6, 0.8].

The most common use is per-sample L2 normalization (Row, L2). It projects each sample onto the unit sphere, so only the sample’s direction matters, not its magnitude. This is what a cosine-similarity or dot-product model needs.

The row-wise case also has an object form, Normalizer, covered in section 4.2.5. It performs the same arithmetic, wrapped in the estimator contract.

use ndarray::array;
use rustyml::utils::normalize::{normalize, NormalizationAxis, NormalizationOrder};

fn main() {
    let x = array![[3.0, 4.0], [1.0, 2.0]];

    // Scale each row (sample) to unit L2 length. Row 0 becomes [0.6, 0.8].
    let l2 = normalize(&x, NormalizationAxis::Row, NormalizationOrder::L2).unwrap();
    for row in l2.rows() {
        let norm = row.iter().map(|v| v * v).sum::<f64>().sqrt();
        println!("row L2 norm = {norm:.6}");
    }

    // Same data under the Max (infinity) norm. Each row's largest absolute value becomes 1.
    let mx = normalize(&x, NormalizationAxis::Row, NormalizationOrder::Max).unwrap();
    println!("Max-normalized: {mx:?}");

    // A custom Lp order. p must be positive and finite, or the call returns Error::InvalidParameter.
    let lp = normalize(&x, NormalizationAxis::Row, NormalizationOrder::Lp(3.0)).unwrap();
    println!("Lp(3)-normalized: {lp:?}");
}

4.2.4. Which models need scaling

Scaling does not help every model. Whether it matters depends on how the model measures the data. Model families fall into 3 groups.

The distance-based row is the clearest case. See 6.1. Distance Metrics for background. These models sum squared per-feature differences, so a feature’s contribution scales with its variance. Standardize the columns, and every feature enters the distance on equal footing.

For the gradient-based family, standardization is the cheapest way to let a neural network train at a stable learning rate. Trees need no scaling at all. Standardizing before fitting a DecisionTree changes nothing about the result.

standardize (per-feature) and normalize (per-sample) are not substitutes for each other. Distance-based and gradient-based models almost always need column standardization. Row normalization fits a different case: sample magnitude is noise, and only direction carries signal. Text term-frequency vectors are a common example.

4.2.5. Reusing training statistics: the scaler family

The free function invites one particular mistake. Every call to standardize recomputes the statistics from the array you hand it. This has 2 distinct consequences. Conflating them is where mistakes happen.

First, the leakage trap. Calling standardize(&x_full, Column) on the whole dataset before splitting computes the per-feature mean and variance over rows that later land in the test set. The training transform absorbs information about the test distribution. The validation score becomes optimistic. The fix is a matter of order: split first, then scale.

Second, inconsistent transforms. Even after splitting first, calling standardize(&x_test, Column) scales the test set by the test set’s own column statistics. This is a different linear map than the one applied to the training data. A model trained on train-scaled features then sees test features scaled by different numbers. This degrades every distance-based and gradient-based model, without any error to flag it. It is also impossible at real inference time, because a single incoming sample has no meaningful per-column standard deviation of its own.

The same discipline solves both problems. Compute the statistics once, on the training matrix only, and apply those frozen numbers to every later batch. StandardScaler holds these numbers for you, with the same fit/transform contract as scikit-learn’s:

use rustyml::utils::StandardScaler;

let mut scaler = StandardScaler::new();
scaler.fit(&x_train)?;                       // Learn mean and std from training rows only.
let x_train_z = scaler.transform(&x_train)?; // Or call fit_transform(&x_train) in one step.
let x_test_z = scaler.transform(&x_test)?;   // Same numbers, applied to the test split.

fit and transform split the responsibilities. fit is the only method that looks at the data statistically. transform only applies what fit stored.

Once fitted, the scaler is a fixed linear map. Hand it one row, a thousand rows, or the training matrix again. Every value passes through the same (x - mean) / scale. You are responsible for the order: split before you fit, and never fit again on anything but training data.

use ndarray::{array, Array1};
use rustyml::utils::StandardScaler;
use rustyml::utils::train_test_split::train_test_split;

fn main() {
    let x = array![
        [1.0, 100.0],
        [2.0, 150.0],
        [3.0, 200.0],
        [4.0, 250.0],
        [5.0, 300.0],
        [6.0, 350.0],
    ];
    let y = Array1::from(vec![0, 1, 0, 1, 0, 1]);

    // 1. Split first, before touching any feature value.
    let (x_train, x_test, _y_train, _y_test) =
        train_test_split(x, y, Some(0.34), Some(42)).unwrap();

    // 2. Fit on the training matrix only. fit_transform returns the scaled training data.
    let mut scaler = StandardScaler::new();
    let x_train_z = scaler.fit_transform(&x_train).unwrap();

    // 3. The test split goes through the stored training statistics.
    let x_test_z = scaler.transform(&x_test).unwrap();

    // 4. A single live sample works the same way (the free function cannot do this).
    let one_sample = scaler.transform(&array![[2.5, 175.0]]).unwrap();

    println!("train mean: {:?}", scaler.get_mean().unwrap());
    println!("train scale: {:?}", scaler.get_scale().unwrap());
    println!("x_train_z shape: {:?}", x_train_z.shape());
    println!("x_test_z shape:  {:?}", x_test_z.shape());
    println!("one sample:      {one_sample:?}");
}

What StandardScaler holds and what it offers

partial_fit helps when the training set does not fit in memory. Feed it one chunk at a time. The per-feature moments merge (Chan et al.) into the same result a single fit over the concatenation would give.

inverse_transform matters whenever an answer needs the original units: recovering a reconstruction, or reversing the scale on a target before fitting a regressor.

The scaler is part of the trained pipeline, not throwaway state. A model saved without its scaler becomes unusable, because nothing records what divided the model’s inputs. Save both:

scaler.save_to_path("scaler.bin")?;
model.save_to_path("model.bin")?;

// At serving time
let scaler = StandardScaler::load_from_path("scaler.bin")?;
let model = LinearRegression::load_from_path("model.bin")?;
let prediction = model.predict(&scaler.transform(&incoming)?)?;

2 error paths exist here. Calling transform on an unfitted scaler returns Error::NotFitted("StandardScaler"). Handing it a matrix whose column count differs from the fitted one returns Error::DimensionMismatch. In a dynamically typed pipeline, a feature-order bug like this can silently corrupt predictions. Here, it is a typed error instead.

The scaler rejects non-finite input exactly as the free function does, with Error::NonFinite. This is a deliberate difference from scikit-learn. scikit-learn’s scaler treats NaN as missing data and skips it. RustyML requires you to impute or drop missing values first.

The scaler also implements the crate’s shared Fit, Transform, and FitTransform traits. It composes with generic code written over any transformer:

use rustyml::traits::{Fit, Transform};

Fit::fit(&mut scaler, &x_train)?;
let z = Transform::transform(&scaler, &x_test)?;

The rest of the family

StandardScaler is the default choice, not the only one. 4 siblings share its entire contract: the same fit, transform, and fit_transform methods, the same NotFitted and DimensionMismatch guards, and the same save_to_path. They differ only in what they learn:

MinMaxScaler squashes each feature onto [0, 1] by default. with_feature_range(-1.0, 1.0) retargets it, even on an already-fitted scaler, because the extrema stay stored and only the destination moves.

2 behaviors matter here. First, a later batch that exceeds the training extrema lands outside the range. This is the correct result. It shows that production data has drifted past what the scaler trained on. Pass with_clip(true) when a downstream component requires bounded input. Clipping is lossy, and inverse_transform cannot undo it.

Second, MinMaxScaler is the most outlier-sensitive of the 3 per-feature scalers here. One extreme training value stretches the denominator and squeezes every other sample into a small part of the range. A constant feature has no range at all, so its divisor becomes 1.0, and the column lands flat on feature_range.0.

MaxAbsScaler divides by max(|x|) and never shifts the data. A zero stays exactly zero, and every sign survives. This matters for count vectors, one-hot blocks, or TF-IDF rows, where zero means “absent”. Min-max scaling shifts the data, which turns those structural zeros into an arbitrary nonzero value. Training values land in [-1, 1]. An all-zero feature keeps a divisor of 1.0 and stays zero.

RobustScaler replaces the mean and standard deviation with the median and the interquartile range. Both are order statistics, so an extreme value changes them barely at all. By contrast, one outlier drags StandardScaler’s mean and inflates its standard deviation. An outlier also stretches MinMaxScaler’s denominator, squeezing the rest of the data into a small part of the range. Use RobustScaler for data with outliers you do not plan to remove.

with_quantile_range(10.0, 90.0) widens the spread it measures, covering more of the distribution and reacting more to the tails. This call discards any fitted statistics. The scaler read the stored quantiles at the old positions, and they cannot move without the training data.

2 limits apply here. There is no partial_fit, because quantiles do not merge across batches the way moments and extrema do. Robustness also needs enough samples. With only 4 rows, the 75th percentile interpolates straight into the outlier. This is the quantile rule working as designed, not a failure. The output has no fixed range and no unit variance, only a comparable middle.

Normalizer is the row-wise scaler in the family. A sample’s norm depends on that sample alone, so fit learns nothing but the feature count. It exists so row normalization shares the same estimator contract as the rest of the family. A mis-shaped batch then produces a typed error instead of a silent wrong answer.

Normalizer takes the same NormalizationOrder as the free function. It has no inverse_transform. Dividing by the norm discards the magnitude, and there is no way to recover it. For the Column or Global axes, or for N-D arrays, call normalize directly instead.

use ndarray::array;
use rustyml::utils::normalize::NormalizationOrder;
use rustyml::utils::{MaxAbsScaler, MinMaxScaler, Normalizer};

fn main() {
    let x_train = array![[1.0, 0.0, -4.0], [2.0, 3.0, 0.0], [3.0, 0.0, 2.0]];
    let x_test = array![[4.0, 1.0, -1.0]];

    // Bounded to [0, 1] by the training extrema. The test row exceeds them on purpose.
    let mut min_max = MinMaxScaler::new();
    let train_scaled = min_max.fit_transform(&x_train).unwrap();
    println!("train (min-max): {train_scaled:?}");
    println!("test  (min-max): {:?}", min_max.transform(&x_test).unwrap()); // > 1.0 in col 0

    // Clipped variant: same map, clamped into the range.
    let mut clipped = MinMaxScaler::new().with_clip(true);
    clipped.fit(&x_train).unwrap();
    println!("test  (clipped): {:?}", clipped.transform(&x_test).unwrap());

    // Magnitude only: the zeros in columns 1 and 2 stay exactly zero.
    let mut max_abs = MaxAbsScaler::new();
    println!("train (max-abs): {:?}", max_abs.fit_transform(&x_train).unwrap());

    // Per-sample direction. Each row ends up with L2 norm 1.
    let mut normalizer = Normalizer::new(NormalizationOrder::L2).unwrap();
    println!("train (rows):    {:?}", normalizer.fit_transform(&x_train).unwrap());
}

The next example makes robust scaling visible against a column with an outlier:

use ndarray::array;
use rustyml::utils::{RobustScaler, StandardScaler};

fn main() {
    // 2 identical features, except column 1's last value is an outlier.
    let x = array![
        [1.0, 1.0], [2.0, 2.0], [3.0, 3.0], [4.0, 4.0], [5.0, 5.0],
        [6.0, 6.0], [7.0, 7.0], [8.0, 8.0], [9.0, 1000.0],
    ];

    let mut robust = RobustScaler::new();
    robust.fit(&x).unwrap();
    let mut standard = StandardScaler::new();
    standard.fit(&x).unwrap();

    // Robust: both columns get the same center and scale. The outlier enters neither.
    println!("robust center: {:?}", robust.get_center().unwrap()); // [5, 5]
    println!("robust scale:  {:?}", robust.get_scale().unwrap());  // [4, 4]

    // Standard: the single extreme value dominates column 1's mean and std.
    println!("mean:  {:?}", standard.get_mean().unwrap());  // [5, ~115]
    println!("scale: {:?}", standard.get_scale().unwrap()); // [~2.6, ~313]
}

MinMaxScaler and MaxAbsScaler also support partial_fit. Their extrema merge exactly, so batches give the same answer as one big fit. Every per-feature scaler here has inverse_transform.

The crate does not yet provide the quantile and power transforms (QuantileTransformer, PowerTransformer), or RobustScaler’s unit_variance option. That option needs an inverse normal CDF the crate does not implement. For those cases, compute the statistics yourself, and apply them with the same freeze-on-train discipline.

When the free function is still the right call

standardize still has a place. It is the right tool whenever there is no second batch that needs consistent scaling:

• One-shot exploration. You are looking at one matrix, not training anything.
• Row or Global axes, or N-D arrays. StandardScaler is a 2-D, per-feature transformer, exactly like scikit-learn’s. Row-wise and whole-array standardization live only in the free function.
• Transforms that do not couple samples. Row normalization scales each sample by its own norm alone. normalize(&x_train, Row, L2) and normalize(&x_test, Row, L2) stay consistent by construction, because there is no cross-sample statistic to leak or freeze. Column and Global normalization do couple samples, and have no scaler object. To apply them consistently across a split, compute the norms yourself.

For a transform the family does not cover, for example a quantile or power transform, the same discipline applies. Compute the statistics on the training matrix, keep them, and apply those same numbers to every later batch by hand.

4.2.6. Edge cases: constant features, zeros, and non-finite input

Constant feature (zero variance). A column whose values are all equal has variance 0. Dividing by sqrt(0) directly yields NaN. standardize detects this case with a magnitude-relative, variance-based bound (the Chan-Golub-LeVeque error bound at machine precision), and sets the divisor to 1.0. The centered values, already near 0, then map to exactly 0.0, with no NaN or Inf. Other columns stay untouched.

The detection is variance-based, not an exact equal-to-zero test. A genuinely tiny but real spread, for example 2 values differing by 1e-8, still counts as a real feature, and standardize scales it normally. Only spread that sits within floating-point noise gets flattened to a constant.

StandardScaler follows this same rule. A constant column’s get_scale() entry reads 1.0. This is also why inverse_transform cannot recover such a feature: every value comes back as the training mean.

MinMaxScaler, MaxAbsScaler, and RobustScaler guard the same failure with the coarser test scikit-learn uses for them. A divisor below 10 * f64::EPSILON becomes 1.0. This lands a constant column on feature_range.0 under min-max, and on zero under the other 2.

Near-zero lane in normalization. RustyML treats a lane whose norm falls below 10 * f64::EPSILON as having norm 1.0, and leaves it exactly as is. A zero or near-zero vector has no direction to rescale, so dividing it would only produce NaN. An all-zero row stays all-zero. Global normalization of an all-zero array also stays all-zero.

Non-finite and degenerate input. Both functions validate the input before doing any work. They surface typed errors (see 1.6. Error Handling):

use ndarray::array;
use rustyml::error::Error;
use rustyml::utils::normalize::{normalize, NormalizationAxis, NormalizationOrder};
use rustyml::utils::standardize::{standardize, StandardizationAxis};

fn main() {
    // Constant column: zero variance. The divisor is forced to 1.0, so the centered
    // values collapse to exactly 0.0 with no NaN or Inf.
    let constant = array![[3.0, 1.0], [3.0, 3.0], [3.0, 5.0]];
    let z = standardize(&constant, StandardizationAxis::Column).unwrap();
    println!("constant-column result: {z:?}");
    assert!(z.iter().all(|v| v.is_finite()));

    // Zero row under normalization: norm below 10*f64::EPSILON is treated as 1.0, so
    // the near-zero lane is left exactly as is instead of dividing by close to 0.
    let with_zero_row = array![[3.0, 4.0], [0.0, 0.0]];
    let n = normalize(&with_zero_row, NormalizationAxis::Row, NormalizationOrder::L2).unwrap();
    println!("zero-row preserved: {n:?}");

    // Non-finite input is rejected up front, before any scaling happens.
    let bad = array![[1.0, f64::NAN], [3.0, 4.0]];
    match standardize(&bad, StandardizationAxis::Column) {
        Err(Error::NonFinite(_)) => println!("rejected non-finite input, as expected"),
        other => panic!("expected NonFinite, got {other:?}"),
    }
}

Because non-finite input is rejected up front, neither function can produce a NaN from clean data. If missing values are encoded as NaN, impute or drop them before scaling. Neither function passes them through silently.

4.2.7. Performance and determinism notes

Both functions clone the input once and never mutate it. They rescale in place on the clone.

The heavy statistics run per lane in sequence. This includes the Welford mean and variance pass for standardization, and the per-lane norm for normalization. Once the element count clears an internally calibrated gate, rayon parallelizes the work across lanes. The 2 levels of parallelism never nest.

Global standardization uses a deterministic blocked parallel reduction. It reproduces the same result across runs on the same machine, though not necessarily bit-for-bit across different machines or thread counts. For datasets small enough to fit in cache, the sequential path runs instead, with no threading overhead. None of this needs manual tuning.

A fitted StandardScaler runs on the same machinery, minus the statistics step. transform makes a single fused pass per row over the stored vectors. Scaling inside a tight loop costs 1 pass, instead of the Welford sweep the free function repeats on every call. That difference, not the parallelism, explains the time saved.

StandardScaler::fit_transform on a 2-D array is bit-for-bit identical to standardize(&x, Column). Both share the same Welford pass and the same constant-feature rule, so switching between them never changes a number.

See 7.3. Performance Tuning and Parallelism for the gate model. See 4.3. Label Encoding for the next preprocessing step.