3.1. The Sequential Model

Sequential is the container that turns a stack of layers into a trainable model. Construction is a separate type. A caller adds layers in order to a SequentialBuilder, then builds it against the shape of the input. That call gives back a Sequential, which the caller compiles with an optimizer and a loss before fit and predict.

3 properties set this model apart. Training is full-batch by default. There is no callback machinery, so early stopping and learning-rate schedules are plain loops. The shape of the input reaches the model 1 time, at build, in place of an input layer or a per-layer shape argument.

SequentialBuilder and Sequential both live in rustyml::neural_network::sequential. The layers, optimizers, and losses that plug into them each have their own page (3.2, 3.4, 3.3).

3.1.1. The lifecycle at a glance

The 6 calls below are the entire public surface that training a network needs. add takes the builder by value and gives it back, so a whole stack reads as 1 expression. build returns Result<Sequential, Error>, and every training call belongs to that built model. fit returns Result<History, Error>, with 1 loss value per epoch. predict returns Result<Tensor, Error>.

let mut model = SequentialBuilder::new()
    .add(/* a layer */)
    .add(/* another layer */)
    .build(&Shape::known(x.shape()))?;    // Result<Sequential, Error>
model.compile(/* optimizer */, /* loss */);
model.summary();
let history = model.fit(&x, &y, epochs)?; // epochs: u32 (history.loss() returns one f32 per epoch)
let y_hat = model.predict(&x_new)?;       // Tensor = ArrayD<f32>

The prelude exports everything on this page: SequentialBuilder, Sequential, History, Shape, Dense, Activation, every optimizer and loss, and the Tensor alias. 1 glob is therefore the whole import list of every example below:

use rustyml::prelude::*; // SequentialBuilder, Sequential, History, Shape, Dense, Adam, losses, Tensor

Every tensor in the framework has type Tensor, an alias for ndarray::ArrayD<f32>. It has a dynamic rank and always holds f32 values. Call .into_dyn() to convert a statically-ranked array, such as Array2 or Array3, into IxDyn before it enters a layer. RustyML has no f64 path. The whole stack uses single precision for cache and SIMD performance.

3.1.2. Building the model: SequentialBuilder and build

Model construction is a separate type. SequentialBuilder collects layers, and Sequential is a built model. SequentialBuilder::build is the only way to reach a Sequential. fit, fit_with_batches, train_batch, evaluate, predict, and summary are methods of the built model alone. The weight save and load calls are too. Training a model that was never built is therefore a compile error, and no run-time flag says whether a model is ready.

SequentialBuilder::new() creates an empty builder with no layer and no shuffle seed. add takes any L: 'static + Layer by value, boxes it as Box<dyn Layer>, and appends it to the stack. build takes the shape of the tensor that enters the model:

pub fn add<L: 'static + Layer>(self, layer: L) -> Self;
pub fn build(self, input_shape: &Shape) -> Result<Sequential, Error>;

add consumes the layer, so 1 expression constructs it and moves it: .add(Dense::new(8, Activation::ReLU).unwrap()). It also consumes and returns the builder, so the whole stack chains as 1 expression. Layer constructors can fail on their own. An activation with an unusable parameter, for example, returns Error::InvalidParameter. That is why .unwrap() appears on the layer constructor and not on add.

add does no validation, and build does all of it. add cannot fail, because nothing about a layer can disagree with the stack until a shape runs through it. build walks the stack once from the input. It gives every layer the shape that reaches it, and it threads each output shape into the next layer. A layer that refuses its shape stops the walk, and nothing past that position is allocated. See 3.1.8.

The walk is also where every weight comes from. A layer constructor takes the configuration of the layer and nothing else. A kernel extent comes from the input, and the input is not there yet. build is where a layer learns that shape and allocates. A builder therefore holds no weight at all, and a model that reached Sequential holds every weight it needs.

The layer traits, briefly

Using Sequential needs no trait implementation at all. Reading the contract explains what fit and predict call. The framework splits it over 3 traits:

// What every layer holds, whatever number of inputs it takes.
pub trait LayerBase: std::any::Any + Send + Sync {
    fn layer_type(&self) -> &str;
    fn param_count(&self) -> ParamCounts;
    fn weights(&self) -> Vec<WeightRef<'_>>;               // every array, named, for a checkpoint
    fn weights_mut(&mut self) -> Vec<WeightMut<'_>>;
    fn parameters_mut(&mut self) -> Vec<ParamRef<'_>>;     // the trainable arrays, for the optimizer
    fn is_built(&self) -> bool;
    fn apply_state(&mut self, state: &mut StateSlot<'_>);  // takes back the state a pass proposed
}

// A layer with 1 input, which is almost every layer of the crate.
pub trait UnaryLayer: LayerBase {
    fn forward(&self, input: &Tensor, ctx: &mut Ctx) -> Result<Tensor, Error>;
    fn backward(&self, grad_output: &Tensor, ctx: &mut Ctx) -> Result<Tensor, Error>;
    fn build(&mut self, input: &Shape) -> Result<(), Error>;               // allocates, from the shape
    fn compute_output_shape(&self, input: &Shape) -> Result<Shape, Error>; // pure, allocates nothing
    fn forward_mut(&mut self, input: &Tensor, ctx: &mut Ctx) -> Result<Tensor, Error>;
}

// The general interface, and the one a model holds a layer through.
pub trait Layer: LayerBase {
    fn arity(&self) -> Arity;                                              // Exactly(1), AtLeast(2), ...
    fn forward_many(&self, inputs: &[&Tensor], ctx: &mut Ctx) -> Result<Tensor, Error>;
    fn backward_many(&self, grad: &Tensor, ctx: &mut Ctx) -> Result<Vec<Tensor>, Error>;
    fn build_many(&mut self, inputs: &[Shape]) -> Result<(), Error>;
    fn compute_output_shape_many(&self, inputs: &[Shape]) -> Result<Shape, Error>;
}

// A layer with 1 input gets the general interface for free.
impl<T: UnaryLayer> Layer for T { /* ... */ }

build and compute_output_shape are the pair that makes a build-time walk possible. build allocates, so it takes &mut self and runs 1 time per layer. compute_output_shape is pure. It reads the layer configuration and the input shape, and it touches no cache that a forward pass wrote.

It answers on a layer that was never built and never run. A Shape holds 1 entry per axis, and each entry is either a fixed extent or a free axis. Layer::output_shape, which summary prints, is compute_output_shape_many run against the shapes the layer holds.

There is 1 forward path, and the mode lives in the context rather than in the layer. forward takes &self, so a layer writes nothing into itself during a pass. Ctx::training() builds the context of a training pass, and Ctx::inference() builds the context of an inference pass. A training pass parks in the context whatever the backward pass needs. An inference pass parks nothing, and every mode-dependent layer, such as dropout and batch normalization, takes its inference behavior. Because forward borrows &self and LayerBase requires Send + Sync, several threads can run inference against 1 model with no lock.

The context has 4 channels. The training flag is the first. The cache stack is the second, and a forward pass writes it for the matching backward pass. The gradient store is the third, and ctx.grads() gives every parameter gradient of the pass, keyed by ParamId::new(scope, name).

The state channel is the fourth, and it carries the non-trainable values that a training pass changes. The running statistics of a normalization layer and the random stream of a dropout layer travel there. LayerBase::apply_state moves them into the layer after the pass.

UnaryLayer::forward_mut is the entry point for a caller that drives 1 layer by hand. It builds the layer from the tensor when the layer holds no build, runs the forward pass, and then applies the proposed state. A model never calls it, because SequentialBuilder::build has already built every layer it holds.

Backward propagation is pure math. It does not sanitize NaN or Inf values on purpose. A non-finite gradient propagates and surfaces at the next forward pass or as a NaN loss, instead of being masked silently.

3.1.3. compile: wiring the optimizer and loss

pub fn compile<O, LFunc>(&mut self, optimizer: O, loss: LFunc) -> &mut Self
where O: 'static + Optimizer, LFunc: 'static + Loss;

compile stores the optimizer and loss as trait objects. It returns &mut Self so it chains. That is all it does. It runs no shape inference and it allocates nothing, because build already gave every layer its arrays.

Training is the only thing that needs compile. fit returns Error::NeuralNetwork(NnError::NotCompiled(_)) if the optimizer or the loss is missing. predict never reads the optimizer or the loss, so it works on an uncompiled model.

evaluate sits between the 2. It needs a loss to score with, and no optimizer to step with. This split matters for serving. After a weight load into a fresh architecture, predict runs right away without a compiled optimizer.

Pick the optimizer and loss from 3.4 and 3.3.

Each loss family normalizes its value in a different way. MeanSquaredError, MeanAbsoluteError, and BinaryCrossEntropy average over every element. CategoricalCrossEntropy sums over the trailing class axis, then averages over the prediction sites. A prediction site is a sample for a [batch, classes] target. For a channels-last convolutional softmax head that predicts 1 class per pixel, a prediction site is a pixel. The divisor is then batch * height * width.

A switch between loss families rescales the gradient magnitude, which changes the effective learning rate. Retune the step size after a change of loss.

3.1.4. summary: reading the architecture

summary() prints a table to stdout. Use it to check the wiring and the parameter count before spending epochs on training. Here is the table for a Dense(2 -> 8, ReLU) layer followed by a Dense(8 -> 2, Softmax) layer:

Model: "sequential"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Layer (type)                    ┃ Output Shape           ┃       Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ dense (Dense)                   │ (None, 8)              │            24 │
│ dense_1 (Dense)                 │ (None, 2)              │            18 │
└─────────────────────────────────┴────────────────────────┴───────────────┘
 Total params: 42 (168 B)
 Trainable params: 42 (168 B)
 Non-trainable params: 0 (0 B)

RustyML generates layer names per type. It names the first Dense layer dense, the next dense_1, and so on. None in the output shape is a free axis: the batch axis of a model built with Shape::with_free_batch. Build the same stack with Shape::known(&[4, 2]) instead, and the same 2 rows print (4, 8) and (4, 2).

The printed shape is a shape the model produces. build gave each layer the shape that reaches it, so every row comes from a shape that the whole stack already agreed on. A model that has never seen a tensor still prints the true output shape of every position. Sequential::input_shape() and Sequential::output_shape() report the 2 ends of that walk, and output_shape() needs no forward pass either.

Parameter counts come from the param_count() method of each layer. A Dense(in -> out) layer reports in * out + out parameters, the weight matrix plus the bias vector. So 2 * 8 + 8 = 24, and 8 * 2 + 2 = 18. The byte figures assume 4 bytes per f32.

param_count() returns a ParamCounts, which holds a trainable field and a non_trainable field. A layer reports both counts at once, so a layer is not trainable or non-trainable as a whole. summary adds each field over the model. It prints “Trainable” from the first field, “Non-trainable” from the second, and “Total” from the sum of the 2.

A parameter-free layer, such as an activation or a pooling layer, reports 0 in both fields. BatchNormalization reports a non-zero value in both. For C channels it holds 2 * C trainable elements, which are gamma and beta, and 2 * C non-trainable elements, which are moving_mean and moving_variance. Its total is therefore 4 * C. That count assumes the default settings. with_center(false) drops beta, and with_scale(false) drops gamma, so each flag takes C off the trainable count and leaves the non-trainable count alone.

summary borrows &self and never changes the model, so call it at any time, before or after training.

3.1.5. fit and the training loop

pub fn fit(&mut self, x: &Tensor, y: &Tensor, epochs: u32) -> Result<History, Error>;

fit is full-batch. Each epoch runs exactly 1 gradient step over the whole of x and y. There is only 1 batch, so nothing is shuffled, and the model never reads its shuffle seed. epochs counts these full-dataset steps. If 1 gradient step per epoch converges too slowly for the data set, use fit_with_batches instead. If a single forward pass does not fit the memory budget, also use it (see below).

Each epoch, fit runs 1 train_batch call. The subsection below uses this same public, single-step method to build custom loops. train_batch performs these steps, in order:

1. Build a training context with Ctx::training().
2. Forward through every layer, and move the proposed state of each layer into it.
3. Compute the scalar loss.
4. Compute the loss gradient with respect to the output.
5. Advance the global step of the optimizer once, so Adam moves its bias-correction timestep 1 time per step and not 1 time per layer.
6. Backpropagate through the layers in reverse, so each layer adds its parameter gradients to ctx.grads().
7. Apply clip-by-global-norm, if the optimizer asks for it (see Optimizer::global_clipnorm).
8. Update the parameters of every layer, from the gradient store.

step() runs before the per-layer update() calls. This order keeps step-dependent optimizers correct across several layers. The context lives for 1 step alone, so no gradient and no cache can survive into the next step.

fit returns a History. Its entire API is loss(): 1 f32 per epoch, in epoch order. epochs = 0 gives an empty slice. Each entry is the mean per-sample loss measured during the epoch, before the update of that epoch. Every batch contributes the loss from the forward pass that ran before its own weight update. Each entry therefore describes the weights the model held while the epoch ran, and never the weights the epoch ends with.

Reading the last entry as the final loss of the trained model is wrong in both directions. While training converges, the entry reads above the truth, because the updates of that epoch already improved on the weights it measured. Once the step size overshoots, the entry reads below the truth, because those same updates made things worse. evaluate (see below) reports the loss of the model in hand.

The show_progress feature only adds a display. It is not the only way to read the loss. When it is on, fit renders a live progress bar. The bar shows the loss of the current epoch with 6 decimal places. fit_with_batches renders a similar bar that tracks the running average loss as its batches complete:

[dependencies]
rustyml = { version = "0.15", features = ["neural_network", "show_progress"] }

[00:00:00] ████████████████████████████████████████ 400/400 | Loss: <current loss>

Without the feature, training runs silently. The returned History holds the same numbers either way, so no caller depends on whether the feature is on.

Mini-batch training with fit_with_batches

pub fn fit_with_batches(&mut self, x: &Tensor, y: &Tensor, epochs: u32, batch_size: usize)
    -> Result<History, Error>;

fit_with_batches is the mini-batch loop. It reshuffles the sample order at the start of every epoch, then trains on fixed-size chunks. The shuffle makes this the 1 place where the seed of the model matters. Set it with SequentialBuilder::new_with_seed(seed) or Sequential::set_seed(seed) for a reproducible shuffle order (see 7.1). This seed governs the shuffle alone. It does not affect weight initialization, which each layer seeds through its own with_random_state call.

A batch_size of 0, or one larger than the data set, returns Error::InvalidParameter. A batch_size equal to n_samples degenerates to a single full-batch step per epoch, which matches fit.

Its History entries mean what the entries of fit mean, except for 1 refinement. That refinement shows only when batch_size does not divide the data set evenly. Each batch contributes to the epoch loss in proportion to its sample count, and not as 1 vote per batch. A short trailing batch therefore pulls the epoch figure less than a full batch does. This makes each entry exactly the mean per-sample loss over the whole data set. A plain mean over batches would not give that number.

use ndarray::Array;
use rustyml::prelude::*;

fn main() {
    let x = Array::ones((8, 3)).into_dyn();
    let y = Array::zeros((8, 1)).into_dyn();

    // Seed the per-epoch shuffle so the run is reproducible.
    let mut model = SequentialBuilder::new_with_seed(0)
        .add(Dense::new(6, Activation::ReLU).unwrap())
        .add(Dense::new(1, Activation::Sigmoid).unwrap())
        .build(&Shape::known(x.shape()))
        .unwrap();
    model.compile(
        SGD::new(0.05, 0.9, false, 0.0).unwrap(),
        BinaryCrossEntropy::new(),
    );

    // 4 mini-batches of 2 samples per epoch, reshuffled every epoch.
    let history = model.fit_with_batches(&x, &y, 5, 2).unwrap();
    assert_eq!(history.loss().len(), 5);

    // External LR schedule: read the step size, halve it, write it back. The optimizer keeps
    // its momentum buffers across the change.
    let lr = model.learning_rate().unwrap();
    model.set_learning_rate(lr * 0.5);
    let resumed = model.fit_with_batches(&x, &y, 5, 2).unwrap();

    // Training resumes where it left off rather than restarting.
    assert!(resumed.loss()[0] < *history.loss().last().unwrap());
    assert_eq!(model.predict(&x).unwrap().shape(), &[8, 1]);
}

set_learning_rate, used above, is the hook for external schedules such as step decay or warmup. It retunes the step size in place. The optimizer keeps all of its accumulated state, such as momentum buffers and Adam moments, across the change. It does nothing on a model that was never compiled.

learning_rate() is its read half. It returns None on an uncompiled model. This read-scale-write pattern avoids a second copy of the rate beside the model, which could drift out of sync with the optimizer. Unlike the optimizer constructors, set_learning_rate validates nothing. learning_rate() returns exactly the value that was set, even a zero or a negative rate.

Writing the loop by hand: train_batch and evaluate

pub fn train_batch(&mut self, x: &Tensor, y: &Tensor) -> Result<f32, Error>;
pub fn evaluate(&self, x: &Tensor, y: &Tensor) -> Result<f32, Error>;

train_batch is the single step that both fit variants build on. It is public. Any other epoch structure is a loop the caller writes, instead of a fork of the library. Examples include curriculum ordering, a per-step schedule, or a probe between steps.

The whole of x is the batch. Nothing is split, nothing is shuffled, and mode-dependent layers run in training mode. The returned f32 is the loss from the forward pass, measured before the update of that call. This is the number fit records for each epoch. It is also the value that fit_with_batches averages, weighted by sample count, into each History entry.

train_batch validates its own inputs instead of trusting the caller. A call on an uncompiled model therefore returns a NotCompiled error, and not a panic.

evaluate is the other half. It runs 1 inference-mode forward pass over the whole of x and scores it with the compiled loss. It updates nothing: no gradients, no parameters, and no batch-norm running statistics. It borrows &self, so scoring a model between training steps cannot disturb it. It also draws from no random number generator, so a call inside a fit_with_batches loop cannot perturb the shuffle stream.

Layers behave exactly as they do in predict. Dropout and noise layers act as the identity, and batch normalization reads its running statistics. On a model with such layers, evaluate and the number fit recorded for the same data therefore disagree. Training-mode dropout inflates the number fit records. evaluate gives the more accurate estimate of the 2.

Together, train_batch and evaluate turn early stopping, learning-rate schedules, and checkpoint selection into ordinary code. The loop below takes 1 full-batch step at a time. It scores the model it holds after each step. It halves the step size after 10 steps without an improvement, and it stops once the schedule runs out:

use ndarray::Array;
use rustyml::prelude::*;

fn main() {
    // 8 points of y = 2x + 1.
    let x = Array::from_shape_vec((8, 1), (0..8).map(|i| i as f32 / 8.0).collect::<Vec<_>>())
        .unwrap()
        .into_dyn();
    let y = x.mapv(|v| 2.0 * v + 1.0);

    let mut model = SequentialBuilder::new()
        .add(Dense::new(8, Activation::Tanh).unwrap().with_random_state(0))
        .add(Dense::new(1, Activation::Linear).unwrap().with_random_state(0))
        .build(&Shape::known(x.shape()))
        .unwrap();
    model.compile(SGD::new(0.1, 0.9, false, 0.0).unwrap(), MeanSquaredError::new());

    let start = model.evaluate(&x, &y).unwrap();
    let (mut best, mut stale, mut previous) = (start, 0, start);

    for _ in 0..500 {
        // The step reports the loss it started from. This stack has no dropout, so that
        // number is the previous `evaluate` measured again, one step stale.
        let during = model.train_batch(&x, &y).unwrap();
        assert!((during - previous).abs() < 1e-5);

        // This one scores the weights the step produced.
        let after = model.evaluate(&x, &y).unwrap();
        previous = after;

        if after < best - 1e-6 {
            best = after;
            stale = 0;
            continue;
        }

        // 10 steps without progress: halve the step size, read from the optimizer instead of
        // a copy kept here. Once it is that small, there is nothing left to try.
        stale += 1;
        if stale == 10 {
            let lr = model.learning_rate().unwrap();
            if lr < 1e-4 {
                break;
            }
            model.set_learning_rate(lr * 0.5);
            stale = 0;
        }
    }

    assert!(best < start / 100.0);
}

3.1.6. predict: forward-only inference

pub fn predict(&self, x: &Tensor) -> Result<Tensor, Error>;

predict builds an inference context with Ctx::inference(), runs every layer against it, and returns the output Tensor. It borrows &self, it writes no backward cache, and it puts mode-dependent layers into inference behavior. Dropout is disabled, and batch normalization uses its running statistics, which is the correct behavior for serving. predict is deterministic. 2 calls on the same input return identical tensors. Unlike training and evaluation, predict does not need compile.

The result shape is whatever the last layer emits, (batch, units) for a Dense tail on a rank-2 input. Sequential::output_shape() reports it without a forward pass, because it runs the same pure shape walk build ran.

The input must agree with the shape the model was built for, on every axis except the batch axis. A built layer refuses an input that disagrees, and it accepts any batch size, so a partial final mini-batch always passes. A Dense layer built for a last axis of 8 refuses a tensor of rank 1. It refuses a last axis of any other width as well, with Error::InvalidInput.

3.1.7. A complete example: learning XOR

XOR is the smallest problem that a linear model cannot solve. It is the classic proof that a hidden layer does real work. This example uses 2 Dense layers: a Tanh hidden layer and a Softmax head over 2 classes. Training with Adam and categorical cross-entropy separates the classes cleanly.

The targets are one-hot: class 0 is [1, 0], and class 1 is [0, 1]. A seed of with_random_state(0) on both layers makes the run reproducible. The builder only records that seed, and build spends it, so the layer draws its arrays 1 time.

use ndarray::Array;
use rustyml::prelude::*;

fn main() {
    // XOR inputs, shape (4, 2).
    let x = Array::from_shape_vec((4, 2), vec![0.0_f32, 0.0, 0.0, 1.0, 1.0, 0.0, 1.0, 1.0])
        .unwrap()
        .into_dyn();

    // One-hot targets: XOR is class 1 for (0,1) and (1,0), class 0 otherwise.
    let y = Array::from_shape_vec((4, 2), vec![1.0_f32, 0.0, 0.0, 1.0, 0.0, 1.0, 1.0, 0.0])
        .unwrap()
        .into_dyn();

    let mut model = SequentialBuilder::new()
        .add(Dense::new(8, Activation::Tanh).unwrap().with_random_state(0))
        .add(
            Dense::new(2, Activation::Softmax { axis: -1 })
                .unwrap()
                .with_random_state(0),
        )
        .build(&Shape::with_free_batch(x.shape()))
        .unwrap();
    model.compile(
        Adam::new(0.1, 0.9, 0.999, 1e-8, 0.0).unwrap(),
        CategoricalCrossEntropy::new(false),
    );

    model.summary();
    let history = model.fit(&x, &y, 400).unwrap();
    assert!(*history.loss().last().unwrap() < history.loss()[0] / 100.0);

    let preds = model.predict(&x).unwrap();
    for i in 0..4 {
        // argmax over the 2 class probabilities
        let class = if preds[[i, 0]] >= preds[[i, 1]] { 0 } else { 1 };
        println!("row {i} -> class {class}  probs [{:.3}, {:.3}]", preds[[i, 0]], preds[[i, 1]]);
    }
}

After 400 full-batch epochs, the network assigns the 2 true XOR rows to class 1 and the 2 false rows to class 0. Each probability sits at or near 1.000. The History records that descent. Its last epoch sits more than 2 orders of magnitude below its first, which is what the assertion checks. The CategoricalCrossEntropy::new(false) argument tells the loss that the head already produces probabilities, from the Softmax layer, so the loss does not apply its own log-softmax. When the last layer emits raw logits, pass true instead.

3.1.8. Errors, shape checking, and the build-time refusal

fit, fit_with_batches, and train_batch validate the model and the inputs before they touch any layer. The checks run in this order:

1. An optimizer is present.
2. A loss is present.
3. The model has at least 1 layer.
4. The inputs have a batch axis.
5. The inputs are not empty.
6. x and y agree on batch size.

evaluate runs the same checks, except for the optimizer check, because only a parameter update needs an optimizer. The table below maps each failure to its error variant. All of them are rustyml::error::Error values:

The rank-0 row needs an explanation. A 0-dimensional tensor is not an empty one. It holds exactly 1 element, so is_empty does not reject it, and the batch-axis index that follows has nothing to read. That case returns InvalidInput.

These are recoverable Result values. The next example shows the compile requirement. It shows how training and evaluation each demand something different from compile, and how predict needs neither:

use ndarray::Array;
use rustyml::error::Error;
use rustyml::neural_network::NnError;
use rustyml::prelude::*;

fn main() {
    let x = Array::ones((4, 2)).into_dyn();
    let y = Array::ones((4, 1)).into_dyn();

    let mut model = SequentialBuilder::new()
        .add(Dense::new(1, Activation::Linear).unwrap())
        .build(&Shape::known(x.shape()))
        .unwrap();

    // fit needs an optimizer and a loss. Without compile, it fails fast and names the first
    // thing it found missing.
    match model.fit(&x, &y, 1) {
        Err(Error::NeuralNetwork(NnError::NotCompiled(missing))) => assert_eq!(missing, "optimizer"),
        other => panic!("expected NotCompiled, got {other:?}"),
    }

    // evaluate updates nothing, so it asks only for the loss it scores with.
    match model.evaluate(&x, &y) {
        Err(Error::NeuralNetwork(NnError::NotCompiled(missing))) => {
            assert_eq!(missing, "loss function")
        }
        other => panic!("expected NotCompiled, got {other:?}"),
    }

    // predict, by contrast, never needs compile: it is forward-only.
    assert_eq!(model.predict(&x).unwrap().shape(), &[4, 1]);

    // A builder that holds no layer has nothing to build.
    match SequentialBuilder::new().build(&Shape::known(&[4, 2])) {
        Err(Error::NeuralNetwork(NnError::EmptyModel)) => {}
        _ => panic!("expected EmptyModel"),
    }
}

A stack whose shapes do not agree is the last case. build refuses a shape mismatch before any data moves, and the message names the layer. No adjacent pair can disagree in silence, because a layer no longer declares the width it expects. build gives each layer the shape that reaches it, and a layer that cannot accept that shape stops the walk. The example below asks a 4-by-4 pooling window to run over a 3-by-3 feature map:

use rustyml::prelude::*;

fn main() {
    // A 3x3 valid convolution over a 5x5 image emits 3x3, and a 4x4 pooling window does not
    // fit in it. `build` walks the stack, so the refusal arrives here.
    let refused = SequentialBuilder::new()
        .add(Conv2D::new(4, (3, 3), (1, 1), Activation::ReLU).unwrap())
        .add(MaxPooling2D::new((4, 4)))
        .add(Flatten::new())
        .add(Dense::new(2, Activation::Linear).unwrap())
        .build(&Shape::with_free_batch(&[8, 5, 5, 1]));

    let message = match refused {
        Ok(_) => panic!("the 4x4 window does not fit"),
        Err(error) => error.to_string(),
    };
    assert!(message.contains("layer 1"), "{message}");
    assert!(message.contains("MaxPooling2D"), "{message}");
    println!("{message}");
}

invalid input: layer 1 (`MaxPooling2D`) refused the input shape (None, 3, 3, 4): invalid
parameter `pool_size`: cannot exceed the corresponding input dimension

Read the 3 parts of that message. layer 1 is the position in the stack, counted from the input. MaxPooling2D is the type at that position. (None, 3, 3, 4) is the shape that actually arrived, which the convolution in front of it produced. The 3 parts together point at 1 line of the model.

This is the reason the shape reaches the model at build. The same mistake used to surface in the middle of a forward pass, deep inside a model, and with no layer named. It arrived after the first epoch had already started. It now surfaces on the line that assembles the model, and nothing past the layer at fault is ever allocated. Sequential::output_shape() answers the same question in the other direction, and it needs no data either.

Each building block has its own page from here. See Dense layers and activations, loss functions, optimizers, the convolutional and recurrent layers, and saving and loading weights. For a model that a chain cannot express, see graph models and merge layers.