Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

1.4. Your First End-to-End Model

1.4.1. The complete program

use ndarray::{Array1, Array2};
use rustyml::machine_learning::LogisticRegression;
use rustyml::metrics::{ConfusionMatrix, accuracy};
use rustyml::set_global_seed;
use rustyml::utils::StandardScaler;
use rustyml::utils::train_test_split::train_test_split;

fn main() {
    // Set the global seed for reproducibility
    set_global_seed(42);

    // Build a synthetic 2-class dataset
    let n_per_class = 30usize;
    let mut rows: Vec<f64> = Vec::with_capacity(2 * n_per_class * 2);
    let mut labels: Vec<f64> = Vec::with_capacity(2 * n_per_class);
    for i in 0..n_per_class {
        let a = (i % 6) as f64;
        let b = (i / 6) as f64;
        // Class 0 is centered near -0.6
        // Class 1 near +0.6
        rows.push(-0.6 + 0.3 * a);
        rows.push((-0.6 + 0.3 * b) * 100.0);
        labels.push(0.0);
        rows.push(0.6 + 0.3 * a);
        rows.push((0.6 + 0.3 * b) * 100.0);
        labels.push(1.0);
    }
    let n = labels.len();
    let x: Array2<f64> = Array2::from_shape_vec((n, 2), rows).unwrap();
    let y: Array1<f64> = Array1::from(labels);

    // Split into a training and a test set
    let (x_train, x_test, y_train, y_test) =
        train_test_split(x, y, Some(0.25), Some(42)).unwrap();

    // Standardize: fit the statistics on the training set only, then apply them to both splits
    let mut scaler = StandardScaler::new();
    let x_train_std = scaler.fit_transform(&x_train).unwrap();
    let x_test_std = scaler.transform(&x_test).unwrap();

    // Train
    let mut model = LogisticRegression::new(true, 0.5, 1000, 1e-6).unwrap();
    model.fit(&x_train_std, &y_train).unwrap();

    // Predict on the test set
    let preds: Array1<f64> = model.predict(&x_test_std).unwrap();

    // Evaluate
    let acc = accuracy(&y_test, &preds);
    let cm = ConfusionMatrix::new(&y_test, &preds);
    let (tp, fp, tn, fn_) = cm.get_counts();
    println!("test accuracy = {:.3}", acc);
    println!("TP={tp} FP={fp} TN={tn} FN={fn_}");
    println!("{}", cm.summary());
    println!("iterations run: {:?}", model.get_actual_iterations());

    // Save and load the model
    let path = "logreg_model.bin";
    model.save_to_path(path).unwrap();
    let loaded = LogisticRegression::load_from_path(path).unwrap();
    let preds_loaded = loaded.predict(&x_test_std).unwrap();
    println!("round-trip predictions identical: {}", preds == preds_loaded);
}

These println! calls produce a small report, laid out like this:

test accuracy = 0.XXX
TP=.. FP=.. TN=.. FN=..
Confusion Matrix:
+-----------------+--------------------+--------------------+
|                 | Predicted Positive | Predicted Negative |
+-----------------+--------------------+--------------------+
| Actual Positive | TP: ..             | FN: ..             |
| Actual Negative | FP: ..             | TN: ..             |
+-----------------+--------------------+--------------------+
... derived metrics ...
iterations run: Some(..)
round-trip predictions identical: true

1.4.2. Seeding for reproducibility

rustyml::set_global_seed takes a u64. It sets a thread-local seed, and every unseeded (random_state == None) component constructed afterwards on the same thread derives its RNG from it — the Keras-style global-seed model. See 7.1. Reproducibility and Random Seeds.

In this particular pipeline it changes none of the numbers. LogisticRegression::fit is plain full-batch gradient descent with no randomness at all (fitting twice on the same data gives bit-identical weights), and standardize is deterministic too. The only stochastic step is the shuffle inside train_test_split, and I pass it an explicit Some(42). Since a local seed, when set, outranks the global one, that step follows the local seed. Still, make a habit of calling set_global_seed at the top of your code.

1.4.3. The dataset and the label contract

The features are an Array2<f64> of shape (n_samples, n_features) and the labels are an Array1<f64>. The label element type is not incidental: LogisticRegression::fit requires y to be an f64 array whose values are exactly 0.0 or 1.0 (you can see the rule stated in its documentation), and anything else returns Error::InvalidInput. There is no LabelEncoder step folded into fit; if your labels are strings or arbitrary integers, you convert them yourself — see 4.3. Label Encoding.

1.4.4. Splitting the data

let (x_train, x_test, y_train, y_test) = train_test_split(x, y, Some(0.25), Some(42)).unwrap();

train_test_split consumes x and y, so clone them beforehand if you still need the originals afterwards. Passing None for test_size means 0.3 (a 70/30 split), and None for random_state means seed-from-entropy (or from the global seed, if one is set).

An empty dataset returns Error::EmptyInput, mismatched x/y lengths return Error::DimensionMismatch, a test_size outside (0, 1) is Error::InvalidParameter, and a dataset too small to split returns Error::InvalidInput. The function keeps at least one sample on each side: with ten rows and test_size = 0.99, the training side still has one row rather than zero.

On imbalanced data, a plain random split can push an entire class into the test set or into the training set. That is when train_test_split_stratified is the better choice, since it splits each class independently.

The split mechanics and the stratified variant are covered in 4.1. Train-Test Split.

1.4.5. Standardizing without leaking test information

Logistic regression is fit by gradient descent, and gradient descent cares about feature scale. The code above builds feature 2 a hundred times larger than feature 1, which lets feature 2 dominate the gradient, so each feature needs standardizing to zero mean and unit variance.

There are two ways to standardize:

  • Call the function standardize(data, axis): a stateless, one-shot transform that computes the mean and standard deviation from the array you pass it, returns a new standardized array, and keeps nothing.
  • Use the StandardScaler struct and its methods: it holds state, so fit computes the statistics and stores them, and transform applies the stored statistics to any array you give it later.

With the function, the common mistake is to call standardize once on x_train and again on x_test, which leaves the training and test sets on two different scales. Standardizing the whole matrix before splitting folds the test set’s mean and variance into the numbers the model trains on, which inflates your accuracy.

The correct approach is the StandardScaler struct and its methods — fit once on the training set, then transform everything:

use rustyml::utils::StandardScaler;

let mut scaler = StandardScaler::new();
let x_train_std = scaler.fit_transform(&x_train).unwrap(); // learns mean/std HERE, from train
let x_test_std = scaler.transform(&x_test).unwrap();       // applies the same statistics to test

Use fit/fit_transform on the training set only, and transform for everything else. The statistics are readable (scaler.get_mean(), get_scale()). The divisor is the population standard deviation (ddof = 0, matching scikit-learn’s StandardScaler).

The function is the right call when you do not need statistics to stay consistent, or when you need the Row/Global axes the struct does not cover:

use ndarray::array;
use rustyml::utils::standardize::{standardize, StandardizationAxis};

fn main() {
    let x = array![[1.0, 200.0], [3.0, 400.0], [5.0, 600.0]];
    let z = standardize(&x, StandardizationAxis::Column).unwrap();
    println!("standardized shape: {:?}", z.dim());
    println!("{:?}", z);
}

Output:

standardized shape: (3, 2)
[[-1.224744871391589, -1.224744871391589],
 [0.0, 0.0],
 [1.224744871391589, 1.224744871391589]], shape=[3, 2], strides=[2, 1], layout=Cc (0x5), const ndim=2

For the fuller tutorial, see 4.2. Standardization and Normalization.

1.4.6. Training the model

let mut model = LogisticRegression::new(true, 0.5, 1000, 1e-6).unwrap();
model.fit(&x_train_std, &y_train).unwrap();

LogisticRegression::new returns Result<Self, Error> because it validates its hyperparameters first and returns Error::InvalidParameter if they do not meet the requirements. You can also use LogisticRegression::default() for a model with default values (fit_intercept = true, learning_rate = 0.01, max_iter = 100, tol = 1e-4). Compared with those defaults, the learning rate and iteration budget passed to new here are both larger, because standardized features tolerate a bigger step. Regularization is off by default; enable it with .with_regularization(RegularizationType::L2(alpha)), whose return value is likewise wrapped in a Result (a negative or non-finite alpha returns an error). RustyML measures penalty strength the same way scikit-learn’s SGDClassifier/SGDRegressor does, while a LogisticRegression(C=c) corresponds to alpha = 1 / (c * n) for n training samples. The complete conversion table is in RegularizationType’s own documentation.

fit takes references to the feature matrix and the target vector. Its return value is wrapped in a Result to guard against:

  • the 0.0/1.0 label check
  • empty input (Error::EmptyInput)
  • mismatched x/y lengths (Error::DimensionMismatch)
  • non-finite feature values (Error::NonFinite). It also trips Error::NonFinite mid-loop if the weights or the loss overflow.

Iteration stops once the loss change between consecutive iterations falls below tol, or once the iteration count reaches max_iter. model.get_actual_iterations() tells you which of the two stopped it: if it equals max_iter, the model hit the iteration ceiling rather than converging, and you should raise max_iter or revisit the learning rate. For more on logistic regression, see 2.2. Logistic Regression.

1.4.7. Predicting

let preds: Array1<f64> = model.predict(&x_test_std).unwrap();

predict returns Result<Array1<f64>, Error>, each element a hard class label 0.0 or 1.0 (the sigmoid probability thresholded at 0.5). If you want the underlying probabilities rather than the labels, call predict_proba.

predict’s return value is likewise wrapped in a Result to guard against:

  • calling it before fit, which is Error::NotFitted
  • passing a feature count different from the one the model was trained on, which is Error::DimensionMismatch

1.4.8. Accuracy and the confusion matrix

let acc = accuracy(&y_test, &preds);
let cm = ConfusionMatrix::new(&y_test, &preds);
let (tp, fp, tn, fn_) = cm.get_counts();
println!("{}", cm.summary());

accuracy(y_true, y_pred) -> f64 computes accuracy, the fraction of matching labels. ConfusionMatrix::new(y_true, y_pred) builds the confusion matrix, and get_counts() returns (tp, fp, tn, fn). The confusion matrix takes hard labels only: every element of both arrays must be exactly 0.0 or 1.0, and anything else panics. If your labels use a different pair — the -1.0/+1.0 a margin classifier emits, say — use ConfusionMatrix::new_with_labels(y_true, y_pred, negative_label, positive_label), which corresponds to scikit-learn’s confusion_matrix(..., labels=[neg, pos]). Its methods are accuracy(), precision(), recall(), specificity(), f1_score(), balanced_accuracy(), and mcc(), and you can print summary() for the formatted table shown earlier. On imbalanced data, do not use raw accuracy — reach for balanced_accuracy() or mcc() instead.

For more classification metrics, see 5.2. Classification Metrics.

1.4.9. Saving and loading the model

model.save_to_path("logreg_model.bin").unwrap();
let loaded = LogisticRegression::load_from_path("logreg_model.bin").unwrap();

save_to_path(&self, path: &str) -> Result<(), Error> uses postcard to serialize the entire model — weights, intercept flag, learning rate, iteration count, regularization setting — into binary. load_from_path(path: &str) -> Result<Self, Error> reads it back. For the details, see 3.9. Saving and Loading Weights and 7.2. Model Persistence in Depth.