Supervised learning

Use regression to predict numeric values. Use classification to predict labels.

Enable the machine_learning feature. These examples also require ndarray = "0.17.2".

Prepare inputs

Store features in an Array2<f64> with shape (samples, features). Each row contains one sample.

Store targets in an Array1 with one target per sample. Keep feature order consistent during training and prediction.

Use separate training and test samples to measure performance. Fit preprocessing only on training samples.

Choose a model

Use linear regression for numeric targets. Start with the default LeastSquaresSolver::Normal solver.

Use logistic regression for binary targets and positive-class probabilities. Use KNN for labels with multiple classes and distance-based decisions.

Compare models on held-out samples. Scale features before KNN or gradient-based training when their ranges differ.

Train and predict

LinearRegression::new(true) enables an intercept. Its targets and predictions contain f64 values.

KNN::<i32>::new(k) creates a classifier with integer labels. Choose k between one and the training sample count.

The example uses one neighbor and avoids distance ties.

use ndarray::array;
use rustyml::prelude::machine_learning::{KNN, LinearRegression};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let x = array![[0.0], [1.0], [2.0], [3.0]];
    let y = array![1.0, 3.0, 5.0, 7.0];
    let mut regression = LinearRegression::new(true);
    regression.fit(&x, &y)?;
    let predictions = regression.predict(&array![[4.0], [5.0]])?;
    println!("regression: {:.1}, {:.1}", predictions[0], predictions[1]);

    let labels = array![0_i32, 0, 1, 1];
    let mut classifier = KNN::<i32>::new(1)?;
    classifier.fit(&x, &labels)?;
    let predictions = classifier.predict(&array![[0.1], [2.9]])?;
    println!("classes: {:?}", predictions.to_vec());
    Ok(())
}

Output:

regression: 9.0, 11.0
classes: [0, 1]

Configure logistic regression and penalties

Logistic regression requires Array1<f64> targets containing only 0.0 and 1.0.

LogisticRegression::new(fit_intercept, learning_rate, max_iterations, tolerance) configures training. Use positive, finite learning rates and tolerances.

Set max_iterations above zero. Increase it when training needs more iterations, and reduce the learning rate if training becomes unstable.

predict_proba(&x) returns one positive-class probability per row. predict(&x) returns 1.0 when that probability is at least 0.5.

Both methods return Result<Array1<f64>, Error>. Logistic regression uses the same fit(&x, &y) input shapes as linear regression.

Add with_regularization(RegularizationType::L2(alpha)) to either regression model. Use finite, nonnegative penalty strengths.

Linear regression’s default solver supports L2. Select LeastSquaresSolver::GradientDescent for L1, with learning_rate, max_iter, and tol settings.

The example configures logistic L2 regularization and linear L1 regularization.

use ndarray::array;
use rustyml::prelude::machine_learning::{
    LeastSquaresSolver, LinearRegression, LogisticRegression, RegularizationType,
};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let x = array![[-2.0], [-1.0], [1.0], [2.0]];
    let y = array![0.0, 0.0, 1.0, 1.0];
    let mut classifier = LogisticRegression::new(true, 0.1, 10_000, 1e-8)?
        .with_regularization(RegularizationType::L2(0.1))?;
    classifier.fit(&x, &y)?;
    let queries = array![[-1.5], [1.5]];
    let probabilities = classifier.predict_proba(&queries)?;
    println!(
        "positive probabilities: {:.3}, {:.3}",
        probabilities[0], probabilities[1]
    );
    println!(
        "binary labels: {:?}",
        classifier.predict(&queries)?.to_vec()
    );

    let targets = array![-3.0, -1.0, 3.0, 5.0];
    let mut regression = LinearRegression::new(true)
        .with_solver(LeastSquaresSolver::GradientDescent {
            learning_rate: 0.05,
            max_iter: 10_000,
            tol: 1e-8,
        })?
        .with_regularization(RegularizationType::L1(0.01))?;
    regression.fit(&x, &targets)?;
    println!("L1 predictions: {}", regression.predict(&queries)?.len());
    Ok(())
}

Output:

positive probabilities: 0.101, 0.899
binary labels: [0.0, 1.0]
L1 predictions: 2

Method contracts

The signatures below use owned arrays for clarity. These methods also accept compatible ndarray views.

Each call returns Result<_, Error>. Predictions contain one value per input row.

KNN labels must implement Clone, Hash, and Eq. Integer labels satisfy these requirements.

Handle errors

Call fit before predict. Unfitted predictions return Error::NotFitted.

Use finite feature values and matching target counts. Prediction inputs must have the training feature count.

Handle EmptyInput, DimensionMismatch, NonFinite, and invalid parameter errors. Numerical training failures can also return errors.

Propagate failures with ?, as the example does. See Model files to save fitted models.