4.3. Label Encoding

Classifiers in RustyML never see your string classes. A softmax head emits a probability per column. A cross-entropy loss reads either a one-hot row or an integer index. Every metric in Chapter 5 counts integer class ids. Label encoding is the translation layer between the labels you have ("cat", "spam", 42) and the 2 numeric shapes the training code accepts. RustyML gives you 3 free functions for this. They live in rustyml::utils::label_encoding. RustyML re-exports them at rustyml::utils and through the prelude.

If you know scikit-learn, forget the stateful LabelEncoder or OneHotEncoder pattern, the one that calls fit once and transform many times. RustyML has no encoder object and nothing to persist. Each function is a pure transformation of the array you pass it. This design keeps the API small and thread-safe. It also moves 1 task onto you: you must hold the label-to-index mapping yourself. You need that mapping to decode predictions and to apply the same scheme to new data. The rest of this page shows how to do that.

4.3.1. The API surface: 3 stateless functions

These 3 functions are the entire API. There is no LabelEncoder struct, no fit, transform, or inverse_transform method, and no separate ordinal encoder. The surface stays flat on purpose.

2 type facts matter now, because they cause most of the friction later. First, to_categorical accepts only i32 labels. to_categorical_with_mapping accepts any type that satisfies Clone + Eq + Hash, so it also accepts &str, String, u8, or an enum. Second, the one-hot matrix comes back as Array2<f64> and the decoded labels as Array1<i32>. The neural-network stack uses Tensor = ArrayD<f32>, so it needs f32 instead. This f64 to f32 cast is a required step. It is easy to forget. Section 4.3.5 covers it.

4.3.2. Integer labels to one-hot: to_categorical

Use to_categorical when your labels are already consecutive integers, 0..n_classes. It builds an (n_samples, n_classes) matrix. Each row has a single 1.0, in the column named by the label.

use ndarray::array;
use rustyml::utils::to_categorical;

fn main() {
    let labels = array![0i32, 1, 2, 1, 0];

    // num_classes = None infers the width from max_label + 1 = 3.
    let onehot = to_categorical(&labels, None).unwrap();
    assert_eq!(onehot.shape(), &[5, 3]);

    // Pin a wider width so train / validation / test share a column layout even
    // when a split happens to miss the last class. Extra columns are all-zero.
    let padded = to_categorical(&labels, Some(4)).unwrap();
    assert_eq!(padded.shape(), &[5, 4]);
    println!("{onehot:?}");
}

The num_classes argument is the reason this function needs its own section. Passing None infers the width from max_label + 1. This is convenient, but it is dangerous across a train/test split. If the test fold contains no example of the highest class, to_categorical(&test_labels, None) produces a matrix with 1 column fewer than the training matrix. Every downstream shape check then rejects it. Set num_classes to the true class count on every split, so the layouts stay aligned. Widening is cheap, because the surplus columns are simply zero. This fix solves a common shape mismatch between the validation targets and the output layer.

RustyML rejects 2 kinds of input instead of silently mangling them. Both surface as typed errors:

use ndarray::array;
use rustyml::error::Error;
use rustyml::utils::to_categorical;

fn main() {
    // A negative label cannot index a one-hot column.
    let bad = array![0i32, -1, 2];
    assert!(matches!(
        to_categorical(&bad, None),
        Err(Error::InvalidInput(_))
    ));

    // num_classes narrower than max_label + 1 would drop a class.
    let labels = array![0i32, 1, 2];
    assert!(matches!(
        to_categorical(&labels, Some(2)),
        Err(Error::InvalidParameter { .. })
    ));
    println!("error paths behave as documented");
}

A negative label produces Error::InvalidInput, because it has no valid column. A num_classes value smaller than max_label + 1 produces Error::InvalidParameter, because it would truncate a real class. An empty input array is not an error. It returns shape (0, 1), with 1 class by default, so the matrix stays 2D.

4.3.3. Arbitrary labels to one-hot: to_categorical_with_mapping

Real datasets rarely arrive as 0..n. They arrive as "cat", "dog", "bird", or as non-consecutive integer ids like 10, 20, 30. to_categorical_with_mapping handles these in 1 pass. It assigns each distinct label a column index, in first-seen order. It one-hot encodes against that assignment. It then returns both the matrix and the AHashMap<T, usize> it built.

use rustyml::utils::to_categorical_with_mapping;

fn main() {
    let labels = vec!["cat", "dog", "bird", "dog", "cat"];
    let (onehot, mapping) = to_categorical_with_mapping(&labels, None).unwrap();

    assert_eq!(onehot.shape(), &[5, 3]);
    // First-seen order fixes the column assignment.
    assert_eq!(mapping["cat"], 0);
    assert_eq!(mapping["dog"], 1);
    assert_eq!(mapping["bird"], 2);
    println!("{mapping:?}");
}

The contract is first-seen order, not sorted order. "cat" is column 0 because it appears first, not because it sorts first. This matters for reproducibility. The same slice always yields the same mapping, but 2 datasets that introduce classes in a different order produce different column assignments. That is why the function returns the mapping instead of discarding it. The mapping is the only record of what each column means. Keep it. You need it to decode predictions and, if you encode more data later, to reproduce the same layout. See 4.3.7. The num_classes argument works as it does in to_categorical. None uses the unique-label count. Some(n) pads to a wider matrix, and returns an error if n is smaller than the number of distinct labels.

One edge case differs from to_categorical. An empty slice returns shape (0, 0) with an empty mapping, because zero unique labels infer a class count of zero. to_categorical on an empty array instead returns (0, 1). Neither result is wrong. If you branch on the column count of an empty batch, check which function produced it.

4.3.4. Decoding predictions and round-trips: to_sparse_categorical

to_sparse_categorical runs the inverse direction. It takes a 2D matrix and reduces each row to the index of its largest value. Feed it a strict one-hot matrix, and it recovers the original integer labels. Feed it a softmax probability matrix, and it returns the predicted class per sample. That second use is the common one. It turns a model’s predict output into class ids.

use ndarray::array;
use rustyml::utils::{to_categorical, to_sparse_categorical};

fn main() {
    let original = array![0i32, 1, 2, 1, 0];
    let one_hot = to_categorical(&original, None).unwrap();
    let recovered = to_sparse_categorical(&one_hot).unwrap();
    assert_eq!(recovered, original);
    println!("round-trip ok: {recovered:?}");
}

2 behaviors matter here. Ties resolve to the first (lowest) index. When 2 columns share the row maximum, the function picks the earlier one. This matches NumPy’s argmax, not max_by, which would keep the last. A non-finite value anywhere in the matrix triggers Error::NonFinite up front. The per-row comparison is then total, and it never treats NaN as a winner or a loser. A NaN in your probabilities is a bug in the model output. This function reports it instead of hiding it.

For the string case, the round trip needs 1 more step. to_sparse_categorical recovers only indices, because it has never seen your labels. Invert the mapping yourself, to get from index back to label:

use rustyml::utils::{to_categorical_with_mapping, to_sparse_categorical};
use std::collections::HashMap;

fn main() {
    let labels = vec!["cat", "dog", "bird", "dog", "cat"];
    let (one_hot, mapping) = to_categorical_with_mapping(&labels, None).unwrap();

    // Decode to class indices, then invert the mapping to recover the strings.
    let idx = to_sparse_categorical(&one_hot).unwrap();
    let inverse: HashMap<usize, &str> = mapping.iter().map(|(&k, &v)| (v, k)).collect();
    let recovered: Vec<&str> = idx.iter().map(|&i| inverse[&(i as usize)]).collect();

    assert_eq!(recovered, labels);
    println!("{recovered:?}");
}

Build the index to label inverse once, and reuse it. This is the standard way to report predictions in their original labels. Section 4.3.5 closes with this pattern.

4.3.5. Picking the target format: one-hot vs sparse integer

Multi-class classification in the neural-network stack offers 2 losses. Your choice of loss sets the encoding you feed to fit. The 2 losses are numerically equivalent: same forward value, same gradient. They differ only in how the target is stored. See 3.3. Loss Functions for the loss side of this.

CategoricalCrossEntropy wants the full one-hot matrix. to_categorical returns f64, but Tensor is f32. So the .mapv(|v| v as f32) cast is mandatory. Skip it, and the types will not match your f32 feature matrix:

use ndarray::array;
use rustyml::neural_network::{
    layers::{Activation, Dense},
    losses::CategoricalCrossEntropy,
    optimizers::Adam,
    sequential::Sequential,
};
use rustyml::utils::to_categorical;

fn main() {
    let x = array![
        [5.1f32, 3.5, 1.4, 0.2],
        [4.9, 3.0, 1.4, 0.2],
        [6.2, 3.4, 5.4, 2.3],
        [5.9, 3.0, 5.1, 1.8],
        [6.0, 2.2, 4.0, 1.0],
        [5.5, 2.4, 3.8, 1.1],
    ]
    .into_dyn();
    let labels = array![0i32, 0, 2, 2, 1, 1];

    // One-hot, then bridge f64 -> f32 and into the dynamic-dim Tensor shape.
    let y = to_categorical(&labels, None)
        .unwrap()
        .mapv(|v| v as f32)
        .into_dyn();

    let mut model = Sequential::new();
    model
        .add(Dense::new(4, 8, Activation::ReLU).unwrap())
        .add(Dense::new(8, 3, Activation::Softmax).unwrap())
        .compile(
            Adam::new(0.01, 0.9, 0.999, 1e-8, 0.0).unwrap(),
            CategoricalCrossEntropy::new(false),
        );

    model.fit(&x, &y, 5).unwrap();
    let preds = model.predict(&x).unwrap();
    println!("prediction shape: {:?}", preds.shape()); // [6, 3]
}

SparseCategoricalCrossEntropy skips the one-hot step entirely. It reads the class id directly from a [batch, 1] tensor. There is no matrix to build. You just reshape your integer codes into a column, and cast to f32. With 2 classes this saves little. With thousands of classes, such as word vocabularies or product catalogs, the one-hot matrix is almost all zeros. The sparse form then saves both memory and the allocation:

use ndarray::{array, Axis};
use rustyml::neural_network::{
    layers::{Activation, Dense},
    losses::SparseCategoricalCrossEntropy,
    optimizers::Adam,
    sequential::Sequential,
};

fn main() {
    let x = array![
        [5.1f32, 3.5, 1.4, 0.2],
        [4.9, 3.0, 1.4, 0.2],
        [6.2, 3.4, 5.4, 2.3],
        [5.9, 3.0, 5.1, 1.8],
        [6.0, 2.2, 4.0, 1.0],
        [5.5, 2.4, 3.8, 1.1],
    ]
    .into_dyn();

    // Sparse targets: integer class ids as a [batch, 1] f32 column. No one-hot.
    let labels = array![0i32, 0, 2, 2, 1, 1];
    let y = labels.mapv(|v| v as f32).insert_axis(Axis(1)).into_dyn();

    let mut model = Sequential::new();
    model
        .add(Dense::new(4, 8, Activation::ReLU).unwrap())
        .add(Dense::new(8, 3, Activation::Softmax).unwrap())
        .compile(
            Adam::new(0.01, 0.9, 0.999, 1e-8, 0.0).unwrap(),
            SparseCategoricalCrossEntropy::new(false),
        );

    model.fit(&x, &y, 5).unwrap();
    println!("target shape fed to fit: {:?}", y.shape()); // [6, 1]
}

The sparse target needs shape [batch, 1]. insert_axis(Axis(1)) turns the length-batch label vector into a column. SparseCategoricalCrossEntropy validates this shape. It rejects a bare [batch] vector, a negative or non-finite label, and any label >= num_classes. Each case gets a descriptive error instead of an out-of-bounds panic. A non-integer label is not rejected. 1.6 rounds silently to class 2 through .round(). Encode class ids as exact integers to avoid this.

The full loop for a string-labeled dataset ties this together. Encode with a mapping, train, predict, and decode back to the original labels for reporting:

use ndarray::{array, Ix2};
use rustyml::neural_network::{
    layers::{Activation, Dense},
    losses::CategoricalCrossEntropy,
    optimizers::Adam,
    sequential::Sequential,
};
use rustyml::utils::{to_categorical_with_mapping, to_sparse_categorical};
use std::collections::HashMap;

fn main() {
    let x = array![[0.1f32, 0.2], [0.9, 0.8], [0.15, 0.25], [0.85, 0.95]].into_dyn();
    let raw = vec!["cat", "dog", "cat", "dog"];

    // Encode targets, keep the mapping, and size the output layer from it.
    let (y_f64, mapping) = to_categorical_with_mapping(&raw, None).unwrap();
    let y = y_f64.mapv(|v| v as f32).into_dyn();
    let n_classes = mapping.len();

    let mut model = Sequential::new();
    model
        .add(Dense::new(2, 8, Activation::ReLU).unwrap())
        .add(Dense::new(8, n_classes, Activation::Softmax).unwrap())
        .compile(
            Adam::new(0.01, 0.9, 0.999, 1e-8, 0.0).unwrap(),
            CategoricalCrossEntropy::new(false),
        );
    model.fit(&x, &y, 5).unwrap();

    // Predict -> f32 probabilities -> f64 2D -> argmax indices -> original labels.
    let probs = model.predict(&x).unwrap();
    let probs_2d = probs.mapv(|v| v as f64).into_dimensionality::<Ix2>().unwrap();
    let class_ids = to_sparse_categorical(&probs_2d).unwrap();

    let inverse: HashMap<usize, String> =
        mapping.iter().map(|(k, &v)| (v, k.to_string())).collect();
    let predicted: Vec<String> = class_ids
        .iter()
        .map(|&i| inverse[&(i as usize)].clone())
        .collect();
    println!("predicted labels: {predicted:?}");
}

The 2 type conversions on the return path match the cast on the way in. predict yields f32 in the dynamic ArrayD shape. to_sparse_categorical wants a 2D f64 array. So you cast to f64, and fix the dimensionality to Ix2, before decoding. Sizing the output layer from mapping.len(), instead of a fixed constant, keeps the network in step with the encoding.

4.3.6. Ordinal encoding: when integer codes lie

Integer codes are convenient, and that convenience hides a real trap. For a classification target fed to a cross-entropy loss, the code is just an identity. The loss looks up which column is the true one. It never compares 2 against 1 as magnitudes. So SparseCategoricalCrossEntropy treats class 2 as a bare integer safely. The trap is using the same integer codes to encode a categorical input feature, for a model that does arithmetic on its inputs. Linear and logistic regression, SVMs, and every distance-based method, such as KNN or KMeans, read those codes as numbers on a line. With red=0, green=1, blue=2, the model reads green as exactly between red and blue, and blue as twice green. Those relationships are fabricated. A linear model still fits a coefficient to them, and generalizes the fiction.

The fix is to one-hot the categorical feature, so no false order or spacing survives. Every category becomes its own axis, mutually equidistant:

use ndarray::{array, concatenate, Axis};
use rustyml::utils::to_categorical;

fn main() {
    let price = array![[10.0f64], [12.0], [11.0]];
    let color_code = array![0i32, 2, 1]; // red, blue, green

    // WRONG for a linear / distance model: the codes invent an order and spacing.
    let color_ordinal = color_code.mapv(|c| c as f64).insert_axis(Axis(1));
    let ordinal = concatenate(Axis(1), &[price.view(), color_ordinal.view()]).unwrap();
    assert_eq!(ordinal.shape(), &[3, 2]);

    // RIGHT: one-hot so red, green, blue are mutually equidistant.
    let color_onehot = to_categorical(&color_code, None).unwrap();
    let encoded = concatenate(Axis(1), &[price.view(), color_onehot.view()]).unwrap();
    assert_eq!(encoded.shape(), &[3, 4]); // price + 3 color columns
    println!("{encoded:?}");
}

A bare integer code is fine as a feature in 2 cases. The first case is a genuinely ordinal category, where the integers respect its order, such as small=0, medium=1, large=2. The order the model reads is then real, though the spacing between the values is still an assumption. The second case is a decision tree or tree ensemble. These models split on thresholds instead of multiplying features by weights. So they tolerate arbitrary integer codes far better than a linear model does, at the cost of needing deeper splits to isolate individual categories. For every linear or distance-based model with a nominal feature, use one-hot encoding. The extra columns are the price of not lying to the model about geometry. One-hot columns pair naturally with the scalers in 4.2. Standardization and Normalization. The indicator columns are already 0 or 1, and you usually leave them as-is while you scale the continuous features.

4.3.7. Unseen categories and the stateless model

These functions are stateless, so the scikit-learn question about an unseen category at transform time has a different answer here. There is no fit step and no stored encoder. A fresh call to to_categorical_with_mapping on new data builds a new mapping from whatever that data contains. With first-seen ordering, that new mapping can assign different columns than the mapping your model trained against. Re-encoding your inference data from scratch is the real risk. It is silent, because it produces a valid-looking matrix that means something different.

The correct pattern is to encode once, keep the returned mapping, and look labels up against that saved mapping at inference time, instead of re-encoding. Remember 1 sharp edge: indexing the map with mapping[key] panics if the key is absent. This is exactly the unseen-category case. Use .get instead, so a novel label becomes a value you handle, not a crash:

use rustyml::utils::to_categorical_with_mapping;

fn main() {
    let train = vec!["red", "green", "blue"];
    let (_matrix, mapping) = to_categorical_with_mapping(&train, None).unwrap();

    // At inference you hold the mapping yourself and look labels up.
    let incoming = ["green", "purple"]; // "purple" was never seen at fit time
    for label in incoming {
        match mapping.get(label) {
            Some(&idx) => println!("{label} -> class {idx}"),
            None => println!("{label} -> UNSEEN, route to a fallback"),
        }
    }
    // Indexing panics on a missing key, so prefer `.get`:
    // let _ = mapping["purple"]; // would panic: key not found
}

What you do with an unseen label is a modeling decision. RustyML leaves it to you. You can drop the row. You can route it to a reserved “unknown” column, by training with an explicit num_classes set 1 wider than the observed classes. Or you can reject the request. RustyML does not invent an “unknown” bucket for you. This is the honest choice: a category the model never trained on has no learned representation. Folding it silently into an existing class would only hide that fact.

The mapping is an ordinary AHashMap. Persisting it is your job too, because it is not part of the model weights that save and load handle. A model reloaded without its label mapping can still emit column indices. Nothing can then turn those indices back into "cat" and "dog". Serialize the mapping alongside the weights, since serde handles HashMap directly. You then have a complete, reproducible pipeline. See 7.2. Model Persistence in Depth for the broader story.