Keras Interview Questions (2026): Most Asked, With Answers

Keras is a high-level API for building and training neural networks. TensorFlow, JAX, and PyTorch are the lower-level frameworks that actually run the numerical computation, manage tensors, and handle automatic differentiation. Keras sits on top and gives you layers, models, an optimizer/loss/metric interface, and a training loop, so you write far less boilerplate than working directly against a backend's tensor operations.

For years Keras shipped bundled inside TensorFlow as tf.keras. That's no longer the whole picture. Keras 3 is a standalone package that can run its model code on TensorFlow, JAX, or PyTorch depending on which backend you configure, which is the detail that trips up candidates who learned Keras strictly as "the easy part of TensorFlow."

François Chollet created Keras in 2015, and the design goal from the start was reducing the distance between an idea and a working model. That priority shows up everywhere in the API: sensible defaults, a small number of core objects (layers, models, optimizers, losses), and an explicit refusal to expose every backend detail unless you go looking for it.

It matters in interviews because it explains the tradeoffs candidates get asked to reason about. Keras optimizes for readability and fast iteration first, raw control second. When a question asks "why would you drop to a lower-level API instead of Keras," the honest answer usually involves needing control Keras deliberately keeps out of the way for everyone else.

A Layer is the basic building block. It holds weights and defines a computation in its call method, and it can be composed inside other layers or models. A Model is a Layer with extra bookkeeping added on top: compile(), fit(), evaluate(), and predict() all live on Model, not on plain Layer.

from keras import layers, Model

dense = layers.Dense(64, activation="relu") # a Layer
model = Model(inputs=inputs, outputs=outputs) # a Model, has.fit()

In practice this means you can nest a trained Model inside a larger model as if it were just another layer, which is exactly what transfer learning does with a pretrained backbone.

Sequential is the right tool when your model is a straight stack of layers with exactly one input and one output flowing through them in order. It's less code and reads clearly for exactly that shape.

model = keras.Sequential([
  layers.Dense(128, activation="relu"),
  layers.Dropout(0.3),
  layers.Dense(10, activation="softmax"),
])

The moment you need two inputs, two outputs, a skip connection, or a layer that's shared across two different paths through the network, Sequential can't express it and you move to the Functional API.

The activation function is applied element-wise to the layer's output after the linear transformation (the weighted sum plus bias). Without a non-linear activation, stacking multiple Dense layers collapses mathematically into one single linear transformation, no matter how many layers you add, because the composition of linear functions is still linear.

layers.Dense(64, activation="relu")  # linear transform, then ReLU
layers.Dense(64)            # linear transform only, "linear" activation by default

Leaving activation unset defaults to the identity function, which is fine for a final regression output layer but a mistake anywhere in the middle of a network meant to learn non-linear patterns.

ReLU is computationally cheap, just a max against zero, and it doesn't saturate for positive inputs the way sigmoid and tanh do. Sigmoid and tanh squash their input into a bounded range, and once the input gets large in magnitude, their gradient approaches zero, which slows or stalls learning in deep networks, a problem generally described as vanishing gradients.

The tradeoff: ReLU has its own failure mode, the "dying ReLU" problem, where a unit that consistently receives negative input before the activation stops updating entirely, since its gradient is exactly zero there. Variants like LeakyReLU and ELU exist specifically to keep a small non-zero gradient for negative inputs.

Binary crossentropy is for two-class problems, or independent multi-label problems where each output is its own yes/no decision. Categorical crossentropy is for single-label multi-class problems where the labels are one-hot encoded vectors. Sparse categorical crossentropy is mathematically the same loss as categorical crossentropy, but it expects integer class labels directly instead of one-hot vectors, saving you the encoding step.

# labels as integers: [2, 0, 1]
model.compile(loss="sparse_categorical_crossentropy",...)

# labels as one-hot: [[0,0,1], [1,0,0], [0,1,0]]
model.compile(loss="categorical_crossentropy",...)

Passing the wrong shape of label to the wrong loss is one of the most common bugs candidates hit live, it usually still runs, just trains toward garbage.

batch_size sets how many training examples are grouped together for one forward pass, loss computation, and weight update. epochs sets how many complete passes through the entire training dataset happen before fit() stops. One epoch is made up of however many batches of that size it takes to cover the full dataset once.

model.fit(x_train, y_train, batch_size=32, epochs=20, validation_split=0.2)

A smaller batch size gives noisier but more frequent gradient updates per epoch and generally uses less memory. A larger batch size gives smoother gradient estimates and better hardware utilization, but takes fewer, larger steps per epoch and sometimes needs a higher learning rate to compensate.

validation_split carves off a fraction of the training data you already passed in, taking that fraction from the end of the array without shuffling first. validation_data lets you supply a completely separate dataset for validation, one you control the composition of directly.

model.fit(x, y, validation_split=0.2)        # last 20% of x, y
model.fit(x_train, y_train, validation_data=(x_val, y_val)) # explicit set

The gotcha with validation_split: if your data isn't already shuffled and happens to be sorted by class or by time, the last 20% can be wildly unrepresentative of the training distribution, which is a subtle way to get a misleading validation score without any obvious error.

Dropout randomly zeroes out a fraction of a layer's activations during each training step, forcing the network to not rely too heavily on any single unit, since that unit might be dropped on the next batch. It's a regularization technique meant to reduce overfitting by preventing units from co-adapting to compensate for each other's specific quirks.

layers.Dropout(0.3) # zeroes 30% of activations, only during training

At inference time you want the model's full capacity and a deterministic output, so Dropout is automatically disabled, it passes activations through unchanged, when the layer is called with training=False, which is what predict() and evaluate() do by default.

L2 regularization adds a penalty term to the loss proportional to the sum of squares of the model's weights, scaled by a chosen coefficient. Since the optimizer is now minimizing loss plus that penalty, it's pushed toward smaller weight magnitudes overall, not toward exact zeros, but toward a spread where no single weight dominates the model's output.

layers.Dense(64, kernel_regularizer=keras.regularizers.l2(0.01))

Smaller, more evenly distributed weights tend to make a model less sensitive to any one input feature or noisy training example, which is the mechanism by which L2 reduces overfitting. It's a different tool from L1, which pushes some weights toward exact zero and can be used for feature selection.

model.fit()'s default training step covers the standard case: compute predictions, compute loss against a single set of labels, backpropagate, update weights. You need to override train_step when your training logic doesn't fit that shape: a GAN's alternating generator and discriminator updates, a custom loss that needs access to intermediate activations rather than just final predictions, or gradient penalty terms that require a second forward pass.

Overriding train_step lets you keep using fit(), and everything built on top of it, callbacks, progress bars, distributed training, while replacing exactly the part of the loop that doesn't match the default assumption.

Load a pretrained model without its top classification layer, freeze it so its weights don't update, add new trainable layers on top sized for your specific task, and train just those new layers while the frozen base extracts features. Once that converges, optionally unfreeze the base model and continue training the whole thing at a much lower learning rate, a step called fine-tuning.

base_model = keras.applications.Xception(
  weights="imagenet", input_shape=(150, 150, 3), include_top=False
)
base_model.trainable = False
inputs = keras.Input(shape=(150, 150, 3))
x = base_model(inputs, training=False)
x = layers.GlobalAveragePooling2D()(x)
outputs = layers.Dense(1)(x)
model = keras.Model(inputs, outputs)

This two-stage process is documented directly in Keras's own transfer learning guide, and reflects roughly how most production computer vision pipelines actually adopt a pretrained backbone (Keras documentation, Transfer learning and fine-tuning).

model.save() saves the complete model, architecture, weights, and the optimizer's state and configuration, in one file, so keras.models.load_model() can reconstruct a working model ready to continue training or run inference. model.save_weights() saves only the numeric weight values, nothing about the architecture, so you need to already have an identical model built before you can load those weights into it.

model.save("my_model.keras")      # full model
model.save_weights("my_weights.weights.h5") # weights only, needs matching architecture

Weights-only saving is smaller and useful when you're already reconstructing the architecture in code each time, transfer learning checkpoints during training being the common case. Full model saving is what you want for handing a model to someone else who has no reason to already have your model-building code.

compile() attaches the pieces a model needs to train: the loss function, the optimizer, and any metrics you want tracked during training and evaluation. It doesn't run any computation itself, it just wires those objects into the model so fit() knows what to optimize and what to report.

model.compile(
  optimizer="adam",
  loss="sparse_categorical_crossentropy",
  metrics=["accuracy"],
)

A common mistake is assuming compile validates your model architecture end to end. It doesn't check shape compatibility beyond what's needed to build the graph. Shape mismatches inside custom layers frequently surface only once you call fit() with real data.

Constructing a layer, layers.Dense(64), sets configuration like the number of units and the activation function, but it doesn't create any weights yet, because it doesn't know the input shape. The first time the layer is actually called with real data, it runs a build() step that creates the weight tensors sized to match that input, then runs the forward computation.

This is why a subclassed model's weights don't exist until you call it once, even after __init__ has run. Interviewers use this to check whether you understand lazy shape inference or whether you assume everything is materialized at construction time.

The Functional API builds a model by explicitly wiring an input tensor through a graph of layer calls, then wrapping the input and output nodes in a Model object. Because you're describing a graph rather than a linear list, it naturally supports multiple inputs, multiple outputs, layer sharing, and non-linear topology like residual connections.

inputs = keras.Input(shape=(784,))
x = layers.Dense(64, activation="relu")(inputs)
x = layers.Dense(64, activation="relu")(x)
outputs = layers.Dense(10)(x)
model = keras.Model(inputs=inputs, outputs=outputs)

Functional models are also data structures, not arbitrary Python code, which means Keras can validate shapes at construction time, plot the graph, and serialize the whole thing automatically (Keras documentation, The Functional API).

Subclassing means writing a class that extends keras.Model, defining layers in __init__, and writing the forward pass yourself inside a call method. It's implemented as regular Python code, not a graph, which means you get full control, including loops, conditionals, and recursive structure that isn't expressible as a directed acyclic graph.

class MLP(keras.Model):
  def __init__(self):
    super().__init__()
    self.dense_1 = layers.Dense(64, activation="relu")
    self.dense_2 = layers.Dense(10)

  def call(self, inputs):
    x = self.dense_1(inputs)
    return self.dense_2(x)

You need subclassing for genuinely dynamic architectures, tree-structured RNNs, models where the number of layers applied depends on the input itself. For anything expressible as a DAG, Functional gets you the same flexibility with automatic serialization and inspection, which subclassing doesn't give you for free (Keras documentation, The Functional API).

Sequential and Functional models are stored as data, a literal graph of layer configs and connections, so Keras can call get_config() on them automatically and reconstruct the exact same architecture from that config later. A subclassed model's forward pass lives in Python bytecode inside call(), which isn't something Keras can introspect or serialize on its own.

Practically, this means if you subclass, you're responsible for writing get_config() and from_config() yourself if you want full save-and-restore of the architecture, not just the weights. Skipping this is one of the most common reasons a "works on my machine" subclassed model fails to load somewhere else.

Yes, and it's common in practice. A Functional model can call a subclassed layer inside it, and a subclassed layer's call method can invoke a Functional or Sequential model it holds as an attribute, treating it like any other layer.

class CustomBlock(layers.Layer):
  def __init__(self, base_model):
    super().__init__()
    self.base_model = base_model # a Functional model reused here

  def call(self, inputs):
    return self.base_model(inputs)

Interviewers ask this to check whether you think the three APIs are mutually exclusive choices, or building blocks you compose based on which part of the model needs which properties.

Initialization sets the starting values of a layer's weights before training begins. Poor initialization, all zeros, or values that are too large or too small, can make gradients vanish or explode in the first few steps of training before the optimizer has a chance to correct course.

layers.Dense(64, kernel_initializer="glorot_uniform") # Keras default for Dense

Keras defaults most layers to Glorot (Xavier) uniform initialization, which scales the initial weight range based on the number of input and output units so the variance of activations stays roughly stable as signal passes through the layer. For ReLU-heavy networks, He initialization is often preferred since it accounts for ReLU zeroing out roughly half its inputs.

kernel_size sets the spatial dimensions of the sliding filter window, a 3x3 kernel looks at a 3x3 patch of the input at each position. strides controls how many pixels the window moves between applications, a stride of 2 skips every other position and halves the output resolution. padding controls whether the output shrinks at the edges: "valid" applies no padding and the output shrinks based on kernel size, "same" pads the input so the output keeps the same spatial dimensions as the input when stride is 1.

layers.Conv2D(32, kernel_size=3, strides=1, padding="same")

Getting these wrong is the single most common reason a candidate's manually-computed output shape doesn't match what the model actually produces during a live coding round.

Mathematically an Embedding layer does the same thing a Dense layer would do on a one-hot encoded input, look up the row of a weight matrix corresponding to the input index. The difference is efficiency: Embedding takes an integer index directly and does a lookup, never materializing the one-hot vector or doing a full matrix multiply against mostly zeros.

layers.Embedding(input_dim=10000, output_dim=64) # vocab of 10k, 64-dim vectors

For a vocabulary of any real size, tens of thousands of tokens and up, the one-hot-plus-Dense approach wastes enormous memory and compute on multiplying by zeros. Embedding is the practical, and only sane, choice once vocabulary size grows past a few hundred.

By default, crossentropy losses expect the model's output to already be a valid probability distribution, values between 0 and 1 that sum to 1 across the class dimension, typically produced by a softmax activation on the final layer. Setting from_logits=True tells the loss function that the raw, unnormalized scores are being passed in instead, and it applies the softmax internally as part of computing the loss.

# final layer has no activation, outputs raw logits
outputs = layers.Dense(10)(x)
model.compile(loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True))

The practical reason to prefer from_logits=True over a separate softmax layer plus the default loss is numerical stability: computing softmax and crossentropy in one fused operation avoids intermediate values that can overflow or lose precision when probabilities get very close to 0 or 1.

Adam maintains a per-parameter adaptive learning rate by tracking running estimates of both the first moment (mean) and second moment (variance) of past gradients, which lets it take larger, more confident steps on parameters with small, consistent gradients and smaller steps on noisy ones. Plain SGD applies the same learning rate uniformly to every parameter, which is simpler but generally needs more careful tuning and more epochs to reach the same result.

keras.optimizers.Adam(learning_rate=1e-3)
keras.optimizers.SGD(learning_rate=1e-2, momentum=0.9)

SGD, usually with momentum, still shows up in production image classification pipelines because models trained with SGD sometimes generalize slightly better at convergence, and because it's cheaper per step since it doesn't track the extra moment estimates Adam does. It's a real tradeoff, not just an older default nobody updated.

Yes. A custom metric function just needs to take y_true and y_pred as arguments and return a tensor representing the computed value, the same signature as a loss function.

def mean_pred(y_true, y_pred):
  return keras.ops.mean(y_pred)

model.compile(optimizer="adam", loss="mse", metrics=[mean_pred])

For metrics that need to accumulate state across batches within an epoch, rather than being computed independently per batch and averaged, you subclass keras.metrics.Metric instead and implement update_state, result, and reset_state. A stateless function is fine for something like mean prediction; something like precision, which needs running counts of true and false positives across an entire epoch, needs the stateful class.

During training, the loss shown for each epoch is an average of the per-batch loss computed as the model's weights were changing batch to batch, so early batches in that epoch were scored against slightly worse weights than the final batches. model.evaluate() runs a full separate pass using the final, fixed weights after training on that epoch finished, so every batch is scored with the same, more up-to-date weights.

That's why the last reported training loss in an epoch and the evaluate() loss on the exact same training data afterward are close but rarely identical. It's not a bug, and candidates who assume it should be an exact match usually haven't reasoned through what the epoch-average metric is actually averaging.

EarlyStopping monitors a chosen metric, typically validation loss, and stops training once it fails to improve for a set number of epochs, controlled by the patience argument. Without restore_best_weights=True, the model finishes with whatever weights it had at the final epoch, which by definition is one of the worse-performing epochs since training only stopped after the metric had already gotten worse for several epochs in a row. Setting restore_best_weights=True rolls the model back to the weights from the actual best-performing epoch (Keras documentation, EarlyStopping).

keras.callbacks.EarlyStopping(
  monitor="val_loss",
  patience=3,
  restore_best_weights=True,
)

Skipping restore_best_weights is a common way people accidentally ship a slightly worse model than the one they actually trained, and it's an easy detail to overlook since the training run still appears to finish successfully.

ModelCheckpoint saves the model, or just its weights, automatically during training, typically after every epoch or only when a monitored metric improves. Calling model.save() manually after training only captures whatever state the model happens to be in at that final moment, whether or not that was actually the best epoch.

keras.callbacks.ModelCheckpoint(
  filepath="best_model.keras",
  monitor="val_accuracy",
  save_best_only=True,
)

Using save_best_only=True with ModelCheckpoint means you never lose your best epoch even if training crashes, gets interrupted, or overfits badly in the epochs after that peak, which is a real risk on any run long enough to leave unattended.

ReduceLROnPlateau watches a metric and multiplies the learning rate by a given factor once that metric stops improving for a set number of epochs, the same patience-based idea as EarlyStopping but adjusting the learning rate instead of halting training.

keras.callbacks.ReduceLROnPlateau(
  monitor="val_loss", factor=0.5, patience=5, min_lr=1e-6
)

A fixed schedule, decaying the learning rate every N epochs regardless of what's actually happening, works fine when you already know roughly how training behaves for that dataset. ReduceLROnPlateau is the better default when you don't have that prior knowledge, since it reacts to the actual training curve instead of a guessed schedule.

Yes, by subclassing keras.callbacks.Callback and overriding whichever lifecycle hooks you need, on_epoch_end, on_train_batch_end, and so on. A real use case: logging metrics to an external experiment tracker after every epoch, or triggering a Slack alert if the loss goes non-numeric mid-run so you don't come back hours later to a training job that silently diverged.

class NanGuard(keras.callbacks.Callback):
  def on_epoch_end(self, epoch, logs=None):
    if logs and (logs.get("loss") != logs.get("loss")): # NaN check
      self.model.stop_training = True

Setting self.model.stop_training = True inside a callback is the mechanism most people don't know exists until they need it, it's the clean way to abort a run from inside a callback without raising an exception that would also corrupt whatever state you were trying to save.

Keras preprocessing layers get baked into the model itself, so the exact same preprocessing logic runs automatically at inference time without you having to remember to replicate it in a separate script. A Normalization layer's mean and variance get computed once by calling adapt() on training data, then stored as part of the model, not as separate constants you have to track and reapply by hand.

normalizer = layers.Normalization()
normalizer.adapt(x_train) # computes mean/variance from training data

model = keras.Sequential([normalizer, layers.Dense(64, activation="relu"),...])

Doing the same thing in NumPy works, but it creates a real risk: whoever deploys the model has to know exactly which mean and standard deviation to apply, and any drift between the training script's preprocessing and the serving code's preprocessing produces silent, hard-to-diagnose accuracy loss in production.

NumPy arrays have to be fully loaded into memory before training starts, which doesn't work once a dataset is larger than available RAM. tf.data.Dataset streams data, applies transformations lazily batch by batch, and can prefetch the next batch on CPU while the GPU is still working on the current one, which keeps the accelerator from sitting idle waiting on data loading.

ds = tf.data.Dataset.from_tensor_slices((x, y))
ds = ds.shuffle(10000).batch(32).prefetch(tf.data.AUTOTUNE)
model.fit(ds, epochs=10)

Keras 3's multi-backend design still supports tf.data.Dataset as an input regardless of which backend, TensorFlow, JAX, or PyTorch, is actually training the model, since it's treated purely as a data-feeding mechanism rather than something tied to the TensorFlow backend specifically.

Data leakage is any situation where information from outside the training set, that wouldn't be available at real prediction time, ends up influencing training or artificially inflating a validation score. The obvious case is literal duplicate rows across train and validation. A less obvious one: computing normalization statistics, mean and standard deviation, across the full dataset before splitting, which means the validation set's own values subtly influenced the numbers used to scale the training set.

Another common one with time-series or user-level data: splitting randomly at the row level when multiple rows belong to the same user or the same time window, so the model sees some of that user's or that time period's data during training and some during validation, making the validation score look better than what the model will actually achieve on a genuinely new user or a genuinely new time period.

A custom train_step receives a batch of data, computes the forward pass and loss inside a gradient-tracking context, computes gradients with respect to the trainable weights, applies them through the optimizer, updates any tracked metrics, and returns a dictionary mapping metric names to their current values, which is what fit() uses to populate the progress bar and history object.

class CustomModel(keras.Model):
  def train_step(self, data):
    x, y = data
    with keras.backend.GradientTape() as tape:
      y_pred = self(x, training=True)
      loss = self.compute_loss(y=y, y_pred=y_pred)
    gradients = tape.gradient(loss, self.trainable_variables)
    self.optimizer.apply_gradients(zip(gradients, self.trainable_variables))
    for metric in self.metrics:
      metric.update_state(y, y_pred)
    return {m.name: m.result() for m in self.metrics}

Missing the return dictionary is the most common mistake, the training loop still runs and updates weights correctly, but the progress bar and returned history silently show nothing useful.

The training argument propagates down to every layer in the model that behaves differently between training and inference, Dropout and BatchNormalization being the two most common. training=True tells Dropout to actually zero out activations and tells BatchNorm to use the current batch's statistics and update its running averages. Omitting it, or explicitly setting training=False, silently disables Dropout and freezes BatchNorm's running statistics during what's supposed to be an actual training step.

This is one of the easiest mistakes to make when hand-writing train_step, because the model still trains and the loss still goes down. It just trains with weaker regularization than intended, and the bug rarely surfaces until someone notices the model overfits worse than an equivalent one trained through default fit().

Compiling a model locks in which weights the optimizer is tracking as trainable at that moment. Flipping trainable on a layer after compiling doesn't retroactively change what the already-compiled optimizer state considers trainable, so the change silently has no effect on subsequent training unless you call compile() again after making the change.

base_model.trainable = True
model.compile(optimizer=keras.optimizers.Adam(1e-5), loss=loss_fn) # required
model.fit(dataset, epochs=10)

This is a genuinely easy mistake to make and a genuinely quiet one, training runs without error, the loss might even move slightly, but the base model's weights stay frozen despite trainable reading True, because the optimizer was never told to start tracking them.

The base model's weights already encode useful, generally-applicable features learned from a large pretraining dataset. A learning rate anywhere close to what you'd use training from scratch would apply large updates to those weights based on a comparatively small, task-specific dataset, which risks overwriting the useful general features faster than the model can learn anything better to replace them with, a failure mode often called catastrophic forgetting.

model.compile(optimizer=keras.optimizers.Adam(1e-5), loss=loss_fn) # 1e-5, not 1e-3

A learning rate around 1e-5, often two orders of magnitude below what you'd use training a fresh model, lets the pretrained weights adjust gently to the new task instead of getting overwritten by noisy, oversized updates from a small dataset.

The .keras format is Keras 3's native saving format, a single archive file containing the model's architecture, its weights, and its full compile configuration, optimizer, loss, and metrics, so training can be resumed exactly where it left off. HDF5, the older .h5 format, covers roughly the same information but was designed around Keras 2's internals and is now considered the legacy option, kept mainly for compatibility with older codebases.

model.save("model.keras")  # current default, recommended
model.save("model.h5")   # legacy format, still supported

Unless you're maintaining an existing pipeline built around .h5 files, .keras is the format to default to for new projects, since it's what current Keras development and tooling actually targets.

get_config() returns just the architecture as a plain, JSON-serializable Python dictionary, no weights, no optimizer state, just the structural description needed to rebuild an identical, freshly-initialized model via from_config(). model.save() already includes this same architecture information plus everything else, so for a full save-and-restore workflow, get_config() isn't something you'd normally call directly.

config = model.get_config()
new_model = keras.Model.from_config(config) # same architecture, fresh random weights

Where it's genuinely useful on its own: writing custom layers or models that need to support serialization correctly, since implementing get_config() yourself is exactly what makes a custom subclassed layer safely saveable as part of a larger model, rather than causing load_model() to fail with an unrecognized layer error.

The common path is exporting the model to TensorFlow's SavedModel format for serving through TensorFlow Serving, or converting it to ONNX or TensorFlow Lite depending on the target hardware, then wrapping inference in whatever API framework the rest of the service uses. Before assuming it's production-ready, I'd check inference latency under a realistic batch size and hardware target, not the batch size used during training, since training-time batching assumptions rarely match real-time request patterns.

I'd also explicitly verify the model runs in inference mode correctly, that Dropout is disabled and BatchNorm is using its running statistics rather than batch statistics, since a serving setup that accidentally calls the model in training mode produces nondeterministic outputs for the same input, which is a genuinely confusing bug to chase down once it's already live.

Keras 3 rewrote its internals around a universal ops namespace, keras.ops, that reimplements the NumPy API plus neural-network-specific functions like softmax and convolution in a way that dispatches to whichever backend is active (Keras documentation, Keras 3 release announcement). Write a custom layer using keras.ops instead of calling tf.nn or torch.nn.functional directly, and the same layer code runs unmodified whether the active backend is TensorFlow, JAX, or PyTorch.

You switch backends by setting the KERAS_BACKEND environment variable before importing Keras. The catch candidates usually miss: mixing raw backend calls, calling tf.reduce_mean inside a custom layer instead of keras.ops.mean, breaks portability even though the code runs fine on whichever backend you tested against.

Anything where the computation graph itself depends on the input, not just the values flowing through a fixed graph. A recursive tree-LSTM that walks a parse tree of variable depth is the classic example: the number of layer applications and which children feed which parent changes per example, so there's no single static graph you could draw ahead of time the way Functional requires.

A subtler case: a model that needs a Python-level loop with a data-dependent stopping condition, an early-exit network that stops computing once confidence crosses a threshold, checked with a plain Python if against a tensor value rather than a fixed number of layers. Functional models describe a static graph at construction time; conditional control flow that depends on runtime tensor values inside the forward pass is exactly what subclassing exists for.

During training, BatchNormalization normalizes each batch using that batch's own mean and variance, and separately keeps a running estimate of the population mean and variance as non-trainable weights, updated with a moving average each step. During inference, it stops using the current batch's statistics and instead normalizes using the accumulated running mean and variance, since a batch of size one, or a batch with different characteristics than training, would otherwise give an unstable normalization.

The bug shows up constantly in transfer learning: setting layer.trainable = False freezes the weights from being updated by the optimizer, but it does not by itself stop the running statistics from updating if the layer is still called in training mode. You have to call the model with training=False explicitly to force BatchNorm into inference mode and stop it from corrupting statistics that took an entire pretraining run to learn (Keras documentation, Transfer learning and fine-tuning).

Training loss dropping while validation loss climbs is the standard signature of overfitting: the model is increasingly fitting the specific noise and idiosyncrasies of the training set rather than the general pattern that also holds on unseen data. That's expected behavior past some point in training, not automatically a bug.

It becomes a data pipeline bug specifically when the divergence happens almost immediately, within the first epoch or two, or when validation loss is worse than what a trivial baseline would produce. The usual culprits: a preprocessing step, normalization statistics, tokenizer vocabulary, applied differently to train and validation data, or a validation split that leaked duplicate or near-duplicate examples from the training set. The fix for genuine overfitting is regularization or more data. The fix for a pipeline bug is auditing exactly what transformation each split goes through, since no amount of dropout or weight decay repairs a validation set that isn't actually independent.

The usual cause is BatchNormalization, or any layer relying on batch statistics. Layers that compute statistics per batch during training behave differently depending on batch size, since a batch of 8 gives a noisier estimate of the batch mean and variance than a batch of 256 does. If a model was trained with small batches and evaluated with very large ones, or the reverse, the running statistics BatchNorm accumulated during training may not match well with the statistics the evaluation batches actually have, especially early in training before those running statistics have stabilized.

This is one of the harder bugs to diagnose because everything about the model architecture and weights is identical between the two runs, only the batch size differs, and the failure mode looks exactly like a training bug rather than an evaluation-time configuration mismatch.

The first thing to check is whether the validation set was actually drawn from the same distribution production traffic comes from, and whether that distribution has shifted since the data was collected. A validation split, however carefully constructed, is still a snapshot from the same source and same time as the training data. It can't catch distribution shift, new categories of input, different user behavior, seasonal changes, that only shows up after deployment.

Concretely, I'd check the feature distributions of live traffic against the training distribution, look for categories or ranges the model never saw, and check whether the preprocessing pipeline in production matches training exactly, since a mismatch there produces exactly this pattern: great offline metrics, bad live behavior. A validation split answers "does this model generalize to more data like what I already have," not "will this model hold up against data I haven't seen yet," and conflating those two questions is a common and expensive mistake.

Overriding train_step still assumes you want the rest of fit()'s machinery, epoch iteration, callback hooks, the progress bar, distribution strategy handling. A raw GradientTape loop, written entirely by hand with an explicit for epoch in range(...) and for batch in dataset, is for situations where even that structure gets in the way: reinforcement learning loops where the number of gradient steps per episode varies unpredictably, meta-learning setups that need nested gradient computations, or research code where you're actively changing the training procedure itself and don't want to fight a framework's assumptions about what an epoch is.

The tradeoff is real: you lose callbacks, automatic metric aggregation, and a lot of the convenience code you'd otherwise get for free, and you take on responsibility for writing your own logging, checkpointing, and validation loop from scratch. It's the right call when the training procedure is genuinely non-standard, and overkill when the actual need is just a custom loss or a slightly unusual metric.

Most Keras optimizers accept clipnorm or clipvalue directly, which clips gradients uniformly before the optimizer applies them, and that covers the common case. Inside a custom train_step, you'd do it explicitly after computing gradients and before applying them, which is necessary when you need clipping logic the built-in argument doesn't support, different clip thresholds for different parts of the model, for instance.

gradients = tape.gradient(loss, self.trainable_variables)
gradients = [keras.ops.clip(g, -1.0, 1.0) for g in gradients]
self.optimizer.apply_gradients(zip(gradients, self.trainable_variables))

The built-in optimizer argument is the right choice for uniform clipping across the whole model. A custom train_step earns its keep when you genuinely need per-variable or per-layer clipping rules, which is rare enough that most candidates who reach for it unprompted in an interview are demonstrating they've actually needed it before, not just read about it.

The first suspect is the learning rate, whether it was actually lowered before fine-tuning started, since fine-tuning with the same learning rate used for training the new top layers is the single most common cause of this exact symptom. The second suspect is whether model.compile() was called again after setting base_model.trainable = True, since skipping that step means the optimizer isn't actually tracking the base model's weights at all, in which case the drop in accuracy is coming from somewhere else entirely and unfreezing didn't do anything.

The third suspect, and the one that's easy to miss, is whether the base model is still being called with training=False. If that flag got dropped when the code was refactored for fine-tuning, BatchNorm layers throughout the base model start updating their running statistics using the new, much smaller, and differently-distributed fine-tuning dataset, which can genuinely corrupt features that took a much larger pretraining dataset to learn correctly (Keras documentation, Transfer learning and fine-tuning).

Keras's built-in layers know how to serialize and deserialize themselves automatically because their configuration is fully described by simple arguments Keras already understands. A custom subclassed layer with its own __init__ arguments doesn't have that automatic support unless you explicitly implement get_config() to describe those arguments, and register the class so load_model() knows what Python class an entry in the saved file's config actually refers to.

@keras.saving.register_keras_serializable()
class MyCustomLayer(layers.Layer):
  def __init__(self, units=32, **kwargs):
    super().__init__(**kwargs)
    self.units = units

  def get_config(self):
    config = super().get_config()
    config.update({"units": self.units})
    return config

Skipping the registration decorator, or forgetting to include a custom argument in get_config(), produces exactly this failure: the model saves without complaint, since saving just serializes whatever config exists, and only fails later when load_model() tries to look up a class it was never told how to find, or tries to reconstruct a layer missing an argument it actually needs.