1.3. Working with ndarray

Every matrix type in RustyML comes from ndarray. If you want the more advanced parts of ndarray, such as broadcasting rules, the various iterators, and linear-algebra helpers, read the official ndarray documentation.

RustyML 0.15 depends on ndarray 0.17. Your own Cargo.toml needs to use the same ndarray version:

[dependencies]
ndarray = "0.17"
rustyml = { version = "0.15", features = ["full"] }

1.3.1. f64 versus f32

In RustyML the elements of a classical machine-learning matrix are f64, while the neural-network stack’s tensor elements are f32.

The classical estimators (see machine_learning) take a 2-D Array2<f64> feature matrix laid out as samples by features, with 1 row per sample. They also take a 1-D Array1<_> target vector. The target’s element type varies by model:

The neural-network stack uses Tensor, which is a type alias:

pub type Tensor = ArrayD<f32>;

Tensor differs from the classical machine-learning matrices in 2 ways:

• The element type is f32: half the precision, half the memory, and the width every deep-learning framework standardizes on. Neural networks are compute-bound, so the extra throughput is worth a few bits of precision.
• The dimensionality is dynamic: ArrayD (equivalently Array<f32, IxDyn>) carries its rank at run time rather than encoding it in the type. A wrong rank surfaces at run time as an Err. See 1.3.7 for the details.

1.3.2. Building arrays

Here are the common ways to build the ndarray matrix types:

use ndarray::{array, Array, Array1, Array2};

fn main() {
    // `array!`: the shape is inferred from the nesting, so this is a 3x2 `Array2<f64>`
    let m = array![[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]];
    assert_eq!(m.shape(), &[3, 2]);

    // `from_shape_vec`: (shape, flat row-major Vec)
    // The length must equal the product of the shape, or you get an Err
    let x = Array2::from_shape_vec((2, 3), vec![1.0, 2.0, 3.0, 4.0, 5.0, 6.0]).unwrap();
    assert_eq!(x.shape(), &[2, 3]);

    // `from_shape_fn`: give it a shape, plus a closure that produces each element
    let g = Array2::from_shape_fn((3, 2), |(i, j)| (i * 2 + j) as f64);
    assert_eq!(g[[2, 1]], 5.0);

    // `from_vec`: turn a `Vec<T>` into an `Array1<T>`
    let y = Array1::from_vec(vec![2.0, 4.0, 6.0]);
    // `from_iter`: fill an `Array1<T>` from an iterator
    let labels = Array1::from_iter(0..3i32);

    // Constant / all-zero / all-one fills, with the element type spelled out
    let zeros = Array2::<f64>::zeros((2, 2));
    let ones = Array2::<f64>::ones((2, 2));
    let filled = Array::from_elem((2, 2), 7.0f64);

    println!("{} {} {} {}", y.len(), labels.len(), zeros.sum(), ones.sum());
    println!("{}", filled.sum());
}

Most data arrives as nested Vecs, and that is when you reach for from_shape_vec. It reads the Vec in row-major order, and it wraps its return value in a Result. If you get an Err, the shape is almost certainly wrong.

Often you want to initialize a matrix randomly. The ndarray-rand crate provides Array::random for that, and you add it to your Cargo.toml:

[dependencies]
ndarray-rand = "0.16"
ndarray = "0.17"

Example:

use ndarray::Array;
use ndarray_rand::RandomExt; // the extension trait that gives ndarray's array types
                             // random construction. Bring it into scope for `Array::random`
use ndarray_rand::rand_distr::Uniform; // samples uniformly between the given bounds

fn main() {
    let w = Array::random((4, 3), Uniform::new(-1.0, 1.0).unwrap());
    println!("{:?}", w.shape());
}

When you use RustyML, do not seed ndarray-rand’s generator by hand. RustyML has its own randomness management (set_global_seed, a per-model with_random_state, and SequentialBuilder::new_with_seed). Control randomness through RustyML’s API instead. See 7.1 Reproducibility and Random Seeds for the details.

1.3.3. Inputs to the classical estimators

Here is the shape a typical supervised classical fit and predict signature takes:

pub fn fit<S1, S2>(&mut self, x: &ArrayBase<S1, Ix2>, y: &ArrayBase<S2, Ix1>) -> Result<&mut Self, Error>
where
    S1: Data<Elem = f64>,
    S2: Data<Elem = f64>,

2 things depart from it, both visible in the table in 1.3.1. LDA’s y is Data<Elem = i32> rather than f64. The unsupervised estimators (KMeans, DBSCAN, MeanShift, PCA, KernelPCA) take the feature matrix alone, with no y argument at all.

ndarray has 2 kinds of borrow:

• a reference to an owned array (&Array)
• a view (ArrayView), which is what .view() and .slice(..) produce
• Both are zero-copy, and RustyML accepts either. The parameter is a reference, so write &array or &array.view(). A bare array.view() does not type-check.

You also have to keep the feature matrix 2-D (Ix2) and the target vector 1-D (Ix1).

use rustyml::machine_learning::LinearRegression;
use ndarray::{s, Array1, Array2};

fn main() {
    let x = Array2::from_shape_vec((4, 1), vec![1.0, 2.0, 3.0, 4.0]).unwrap();
    let y = Array1::from_vec(vec![2.0, 4.0, 6.0, 8.0]);

    let mut model = LinearRegression::new(true);

    // A reference to the owned array
    model.fit(&x, &y).unwrap();

    // A reference to a view
    model.fit(&x.view(), &y.view()).unwrap();

    // A reference to a slice: train on the first three rows only
    // This subview is a borrow into `x`
    let x_head = x.slice(s![0..3, ..]);
    let y_head = y.slice(s![0..3]);
    model.fit(&x_head, &y_head).unwrap();
}

What .view() and .slice(..) really buy you is the ability to feed a subset of an array straight into fit or predict. The subset can be a range of rows, a strided window, or a single reshaped column. It does not need to become a new owned array first. To train on rows 0..800 of a 1000-row matrix, x.slice(s![0..800, ..]) is all it takes.

1.3.4. Inputs to the neural-network stack

The neural layers want ArrayD<f32>, but you almost never construct one directly. The usual path is to build a fixed-dimension array and then convert it to the dynamic dimension IxDyn. That conversion is a thin wrapper. It does not touch the data:

use rustyml::neural_network::Shape;
use rustyml::neural_network::sequential::SequentialBuilder;
use rustyml::neural_network::layers::{Activation, Dense};
use rustyml::neural_network::optimizers::Adam;
use rustyml::neural_network::losses::MeanSquaredError;
use ndarray::Array2;

fn main() {
    // 4 samples, 2 features: build an `Array2<f32>` first, then convert to `ArrayD`
    let x = Array2::from_shape_vec(
        (4, 2),
        vec![0.0f32, 0.0, 0.0, 1.0, 1.0, 0.0, 1.0, 1.0],
    )
    .unwrap()
    .into_dyn(); // convert to a dynamic-dimension tensor
    let y = Array2::from_shape_vec((4, 1), vec![0.0f32, 1.0, 1.0, 0.0])
        .unwrap()
        .into_dyn(); // convert to a dynamic-dimension tensor

    // `build` takes the shape of the tensor the model receives
    let mut model = SequentialBuilder::new()
        .add(Dense::new(4, Activation::ReLU).unwrap())
        .add(Dense::new(1, Activation::Linear).unwrap())
        .build(&Shape::known(x.shape()))
        .unwrap();
    model.compile(Adam::new(0.001, 0.9, 0.999, 1e-8, 0.0).unwrap(), MeanSquaredError::new());

    model.fit(&x, &y, 5).unwrap();
    let preds = model.predict(&x).unwrap();
    println!("prediction tensor shape: {:?}", preds.shape());
}

Output:

prediction tensor shape: [4, 1]

1.3.5. Input shapes

RustyML’s input format is 1 row per sample, 1 column per feature.

The tensor shapes for neural-network layers match Keras’s default (batch, height, width, channels). The channel axis is always the trailing one, and the batch axis is always the leading one, with the spatial axes in between. A Keras user should feel at home with this layout. Here are the exact layouts:

For example:

• A batch of 8 single-channel 28x28 images is [8, 28, 28, 1]
• A bank of 3x3 filters producing 16 feature maps has weight shape [3, 3, 1, 16]

Here is a code example:

use ndarray::Array;
use rustyml::neural_network::layers::{Activation, Conv1D};
use rustyml::neural_network::traits::UnaryLayer;
use rustyml::neural_network::{Ctx, Shape};

fn main() {
    // filters=2, kernel=3, stride=1. No shape here: the layer takes it from `build`
    let mut conv = Conv1D::new(2, 3, 1, Activation::Linear).unwrap();

    // The output shape is available before any tensor exists
    let input_shape = Shape::with_free_batch(&[1, 6, 1]);
    println!("output shape: {}", conv.compute_output_shape(&input_shape).unwrap());

    // Channels last: [batch=1, length=6, channels=1]
    let input = Array::from_shape_vec((1, 6, 1), vec![1.0f32, 2.0, 3.0, 4.0, 5.0, 6.0])
        .unwrap()
        .into_dyn();

    // `forward_mut` builds the layer from the tensor, then runs the pass.
    let mut ctx = Ctx::inference();
    let output = conv.forward_mut(&input, &mut ctx).unwrap();
    println!("conv output shape: {:?}", output.shape()); // [1, 4, 2]
}

The constructor takes the layer configuration alone. build hands the layer the [batch, length, channels] shape of its input. compute_output_shape answers with the output shape before any tensor exists. A direct forward_mut call builds the layer from the tensor it receives. A built layer then refuses a tensor whose rank or channel count disagrees, and it accepts any batch size. For the per-layer detail, see 3.5 Convolutional Layers and 3.6 Pooling Layers.

1.3.6. Practical advice

Real data rarely arrives as an Array2. The most common shape is a Vec<Vec<f64>>, 1 inner Vec per parsed CSV row. The conversion is to flatten it into a single row-major Vec<T> and hand it to from_shape_vec along with the (rows, cols) you counted:

use ndarray::Array2;

fn main() {
    // As if parsed from CSV: 3 rows, 2 columns
    let rows: Vec<Vec<f64>> = vec![
        vec![1.0, 2.0],
        vec![3.0, 4.0],
        vec![5.0, 6.0],
    ];
    let n_rows = rows.len();
    let n_cols = rows[0].len();

    // `flatten()` flattens the nested vectors in row-major order
    let flat: Vec<f64> = rows.into_iter().flatten().collect();
    let x = Array2::from_shape_vec((n_rows, n_cols), flat).unwrap();

    assert_eq!(x.shape(), &[3, 2]);
    println!("{:?}", x.row(0)); // [1.0, 2.0]
}

If your rows are ragged, flatten still flattens the nested vectors, but the length will not equal n_rows * n_cols and from_shape_vec returns an Err.

Other tools you will use:

• The s! macro expresses ranges and strides
• concatenate joins arrays along an existing axis
• outer_iter (equivalently axis_iter(Axis(0))) walks each row as a 1-D view, which is how you iterate sample by sample

Code example:

use ndarray::{array, concatenate, s, Axis};

fn main() {
    let x = array![[1.0, 2.0, 3.0], [4.0, 5.0, 6.0], [7.0, 8.0, 9.0]];

    // Slice: all rows, first two columns (drop a feature)
    let first_two_cols = x.slice(s![.., 0..2]);
    assert_eq!(first_two_cols.shape(), &[3, 2]);

    // Slice: a single row as a submatrix
    let middle = x.slice(s![1..2, ..]);
    assert_eq!(middle.shape(), &[1, 3]);

    // Concatenate two matrices along axis 0 (append rows)
    let more = array![[10.0, 11.0, 12.0]];
    let stacked = concatenate(Axis(0), &[x.view(), more.view()]).unwrap();
    assert_eq!(stacked.shape(), &[4, 3]);

    // Iterate each row as an `ArrayView1`
    // The sample-by-sample idiom
    for (i, row) in x.outer_iter().enumerate() {
        println!("row {i} sums to {}", row.sum());
    }
}

Turning raw labels into the Array1 a model wants is usually a map over your source data collected into an array. When a model or a loss expects one-hot targets rather than integer class ids, use RustyML’s to_categorical function. It turns an Array1<i32> of class ids into an Array2<f64> one-hot matrix:

use rustyml::utils::to_categorical;
use ndarray::Array1;

fn main() {
    // Map string labels to integer class ids, then into an `Array1<i32>`
    let raw = ["cat", "dog", "cat", "bird"];
    let ids: Array1<i32> = raw
        .iter()
        .map(|s| match *s {
            "cat" => 0,
            "dog" => 1,
            _ => 2,
        })
        .collect::<Vec<_>>()
        .into();

    // One-hot encode: 4 samples, 3 classes -> a (4, 3) `Array2<f64>`
    let onehot = to_categorical(&ids, None).unwrap();
    assert_eq!(onehot.shape(), &[4, 3]);
    println!("{:?}", onehot.row(1)); // [0.0, 1.0, 0.0]
}

See 4.3 Label Encoding for the full story on label encoding, its inverse, and the mapping-preserving variants. See 4.2 Standardization and Normalization for preprocessing your feature matrix with standardize and normalize.

1.3.7. Shape-mismatch errors

2 different error systems catch shape problems. Errors that happen while you are building an array come from ndarray. from_shape_vec returns ndarray’s ShapeError when the buffer length disagrees with the shape. Errors that happen while RustyML runs come back as RustyML’s own Error enum, which subdivides them further:

• Error::DimensionMismatch { expected, found }: 2 scalar counts disagree, such as the feature count at predict time versus at fit time, or x.nrows() versus y.len()
• Error::ShapeMismatch { expected, found }: 2 whole tensor shapes disagree, such as a gradient in the neural-network stack not matching the activation it flows into
• Error::InvalidInput(_): the input’s rank is wrong, such as handing a 2-D tensor to a Conv1D that expects 3-D

The one you will hit most often is a feature-count mismatch. You train on p features, then call predict with a matrix that has a different number of columns. RustyML validates this and returns DimensionMismatch:

use rustyml::machine_learning::LinearRegression;
use rustyml::error::Error;
use ndarray::{Array1, Array2};

fn main() {
    // Train on 2 features
    let x = Array2::from_shape_vec((3, 2), vec![1.0, 2.0, 2.0, 3.0, 3.0, 4.0]).unwrap();
    let y = Array1::from_vec(vec![6.0, 9.0, 12.0]);
    let mut model = LinearRegression::new(true);
    model.fit(&x, &y).unwrap();

    // Predict with 3 features
    // The model returns an error
    let bad = Array2::from_shape_vec((1, 3), vec![4.0, 5.0, 6.0]).unwrap();
    match model.predict(&bad) {
        Err(Error::DimensionMismatch { expected, found }) => {
            println!("expected {expected} features, got {found}");
        }
        other => println!("unexpected: {other:?}"),
    }
}

RustyML marks Error as #[non_exhaustive], so a match over it always needs a trailing _ => arm. That keeps your code compiling when new error variants are added. For the full error-handling story, see 1.6 Error Handling.