Scikit-learn Interview Questions (2026): Most Asked, With Answers

An estimator is any object that implements a fit method, which takes data (and optionally labels) and learns something from it, storing the result as attributes ending in an underscore, like coef_ or cluster_centers_. Beyond that base contract, most estimators add either predict (for supervised models that output labels or values), transform (for preprocessing steps that output a modified version of the input), or both.

from sklearn.linear_model import LinearRegression

model = LinearRegression()
model.fit(X_train, y_train)  # learns coef_ and intercept_
predictions = model.predict(X_test)

The consistency of that contract is the whole point. Swap LinearRegression for RandomForestRegressor or SVR and the surrounding code, the fit call, the predict call, the pipeline it sits in, doesn't need to change at all.

For most transformers they produce the same result, fit_transform(X) is just a convenience method that calls fit(X) then transform(X) and returns the output in one step. The distinction that actually matters is where you're allowed to call which one. fit_transform belongs on training data. On validation or test data you call transform only, using statistics already learned from training, never fit_transform again.

from sklearn.preprocessing import StandardScaler

scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train) # learns mean/std from train
X_test_scaled = scaler.transform(X_test)     # reuses train's mean/std

Calling fit_transform on the test set, or worse, on the full dataset before splitting, is the single most common source of data leakage I see in take-home submissions.

It's a naming convention scikit-learn uses to distinguish parameters you set before fitting from attributes the model learned during fitting. n_estimators, no trailing underscore, is something you chose. feature_importances_, trailing underscore, only exists after fit has run. Trying to access a fitted attribute before calling fit raises a NotFittedError, which is scikit-learn's way of catching a whole category of "did I actually train this thing" bugs at the point of the mistake instead of downstream.

predict returns the single class label the model thinks is most likely for each row. predict_proba returns the full probability distribution across all classes instead, one row per sample, one column per class, all summing to 1. You reach for predict_proba anytime the raw probability matters more than the label itself, ranking leads by likelihood to convert, setting a custom decision threshold instead of the default 0.5, or computing a metric like ROC-AUC or log loss that needs a probability, not a hard label, to even be defined.

model.predict(X_test)     # array(['spam', 'not_spam', 'spam',...])
model.predict_proba(X_test)  # array([[0.12, 0.88], [0.95, 0.05],...])

Pipeline bundles a sequence of transformers and a final estimator into a single object that behaves like any other estimator, one fit call, one predict call. The real value isn't convenience, it's correctness under cross-validation. When you call fit on the whole pipeline inside a cross-validation fold, every preprocessing step, scaling, imputation, encoding, gets refit on that fold's training data only, and applied to the fold's validation data using transform, not fit_transform. Manually scaling the whole dataset once before splitting it into folds skips that isolation entirely.

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression

pipe = Pipeline([
  ("scaler", StandardScaler()),
  ("clf", LogisticRegression()),
])
pipe.fit(X_train, y_train)
pipe.predict(X_test)

Passing stratify=y tells train_test_split to preserve the class proportions of y in both the resulting train and test sets, the same idea as StratifiedKFold but for a single split rather than several folds. Without it, a random split on a small or imbalanced dataset can easily produce a test set with a noticeably different class balance than the training set, purely from the luck of the draw, which makes the reported test accuracy less trustworthy as a stand-in for how the model performs on the real class distribution.

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(
  X, y, test_size=0.2, stratify=y, random_state=42
)

A single split gives you one estimate of performance based on whichever rows happened to land in the test set, and that estimate carries variance from the split itself, not just from the model. A model can look great on one random split and mediocre on another, especially with a small dataset, purely by chance in which rows ended up where. Cross-validation, typically 5-fold or 10-fold, fits and evaluates the model multiple times across different splits and reports the mean and spread, giving you a far more honest sense of how the model performs and how much that performance actually varies.

from sklearn.model_selection import cross_val_score

scores = cross_val_score(model, X, y, cv=5, scoring="accuracy")
print(scores.mean(), scores.std())

Plain KFold splits rows into folds without regard to the label distribution, so a fold could end up with wildly different class proportions than the overall dataset, purely by chance. StratifiedKFold preserves the class proportions in every fold, so if your dataset is 95 percent negative and 5 percent positive, every fold stays roughly 95/5 instead of one unlucky fold ending up with almost no positive examples at all.

from sklearn.model_selection import StratifiedKFold, cross_val_score

skf = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
scores = cross_val_score(model, X, y, cv=skf, scoring="roc_auc")

It matters most with imbalanced classification, which in practice is most real classification problems. It's also why scikit-learn's cross_val_score silently switches to stratified splitting by default for classifiers when you just pass an integer for cv, rather than plain KFold.

LinearRegression minimizes plain squared error with no penalty on coefficient size, which means with enough correlated features or too little data relative to the number of features, coefficients can blow up to extreme, unstable values that fit the training data closely but generalize poorly. Ridge adds an L2 penalty, controlled by alpha, that shrinks coefficients toward zero, trading a little bias for meaningfully less variance whenever features are correlated or the dataset is small relative to its dimensionality.

from sklearn.linear_model import Ridge

ridge = Ridge(alpha=1.0) # higher alpha = more shrinkage, more bias, less variance
ridge.fit(X_train, y_train)

An unconstrained decision tree keeps splitting until every leaf contains a single class or a single value, which means it's memorizing individual training examples rather than learning a general pattern. That gives you close to 100 percent training accuracy and noticeably worse performance on anything the tree hasn't seen, the textbook definition of overfitting. max_depth, min_samples_split, and min_samples_leaf are the three parameters that most directly control this, by capping how deep the tree grows and how few samples are allowed to justify a further split.

from sklearn.tree import DecisionTreeClassifier

tree = DecisionTreeClassifier(max_depth=5, min_samples_leaf=10)

K-means requires you to specify the number of clusters, k, up front, and it assumes clusters are roughly spherical and similarly sized, since it assigns points to the nearest centroid by Euclidean distance. Real data with elongated, unevenly sized, or non-convex clusters, think two crescent moons overlapping, breaks that assumption badly, and k-means will produce clusters that look wrong even though the algorithm converged successfully.

from sklearn.cluster import KMeans

kmeans = KMeans(n_clusters=4, n_init=10, random_state=42)
kmeans.fit(X)

Dropping rows (dropna()) is safe only when the missing data is a small fraction of the dataset and genuinely random, otherwise you're throwing away information and possibly biasing what remains, if the rows missing a value aren't a random sample of all rows. Imputing, filling with a mean, median, mode, or model-based estimate, keeps every row but introduces some amount of made-up data in its place. In practice I check what fraction of rows are affected and whether missingness correlates with anything else in the data before deciding. Losing 0.5 percent of rows at random is usually fine to drop. Losing 30 percent of rows that all happen to share some other characteristic is a sign dropping them would bias the dataset.

model.score(X, y) is a convenience method every estimator implements, but it's hardcoded to one specific metric depending on the estimator type, accuracy for classifiers, R-squared for regressors, and you can't change which metric it computes. Calling a metric function directly, accuracy_score, f1_score, mean_absolute_error, from sklearn.metrics, lets you compute exactly the metric you actually care about, against predictions you already have, rather than being limited to whatever the estimator's default happens to be.

from sklearn.metrics import accuracy_score, f1_score

y_pred = model.predict(X_test)
print(model.score(X_test, y_test))    # always accuracy for a classifier
print(accuracy_score(y_test, y_pred))   # same number, computed explicitly
print(f1_score(y_test, y_pred))      # a metric.score() can't give you

joblib is the standard tool, it's more efficient than raw pickle for objects containing large NumPy arrays, which is exactly what a fitted estimator is. The real risk, whether you use joblib or pickle directly, isn't performance, it's version compatibility: a model pickled with one version of scikit-learn (or even one version of NumPy) can fail to load, or worse, load silently with subtly wrong behavior, under a different version.

import joblib

joblib.dump(model, "model.joblib")
loaded_model = joblib.load("model.joblib")

The practical fix is pinning the exact scikit-learn version in whatever environment loads the model, and treating a scikit-learn upgrade in production the same way you'd treat any other breaking dependency change, with a re-validation step before deploying, not an assumption that a pickled model just keeps working forever.

random_state seeds whatever randomness an estimator or splitting function relies on, weight initialization, bootstrap sampling in a random forest, the shuffle order in train_test_split. Set it to a fixed integer and you get the exact same result every run. Leave it as the default None and you get a different result each time, drawn from NumPy's global random state.

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(
  X, y, test_size=0.2, random_state=42
)

In an interview it signals you understand your results need to be reproducible for someone else to check. In a real project it's what lets you actually isolate whether a change in accuracy came from your code change or just from a different random split.

Every estimator implements get_params(), which returns a dictionary of every constructor argument and its current value, and set_params(**params), which updates them in place without re-instantiating the object. GridSearchCV and RandomizedSearchCV both rely on this pair internally: for each candidate hyperparameter combination, they clone the base estimator, call set_params to configure it, fit it on a training fold, and score it on the held-out fold.

from sklearn.ensemble import RandomForestClassifier

clf = RandomForestClassifier()
print(clf.get_params())
clf.set_params(n_estimators=200, max_depth=5)

This is also why a custom transformer that stores its constructor arguments under different attribute names than the arguments themselves silently breaks with grid search. It's a scikit-learn convention worth knowing before you write your first custom estimator, not just trivia.

ColumnTransformer applies different transformers to different subsets of columns and concatenates the results back into one array. A plain Pipeline applies each step to the entire input, which doesn't work when you need to scale numeric columns and one-hot encode categorical columns at the same time. Trying to force that through a single Pipeline means writing a custom transformer that internally splits and rejoins columns yourself, which is exactly what ColumnTransformer already does.

from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import StandardScaler, OneHotEncoder

numeric_features = ["age", "income"]
categorical_features = ["region", "plan_type"]

preprocessor = ColumnTransformer([
  ("num", StandardScaler(), numeric_features),
  ("cat", OneHotEncoder(handle_unknown="ignore"), categorical_features),
])

Wrapping that ColumnTransformer as the first step of a Pipeline, followed by a model, is the pattern I'd expect any candidate who's touched tabular data professionally to reach for without prompting.

StandardScaler centers data to zero mean and unit variance, and it's the reasonable default for anything assuming roughly normal data, linear models, SVMs, PCA. MinMaxScaler squashes values into a fixed range, usually 0 to 1, which matters for algorithms that expect bounded input, some neural network activation functions, or when you specifically want to preserve zero values as meaningful (sparse data). RobustScaler uses the median and interquartile range instead of the mean and standard deviation, which makes it the one to reach for when your data has real outliers that would otherwise distort a standard scaler's mean and variance estimates.

from sklearn.preprocessing import RobustScaler

scaler = RobustScaler()
X_scaled = scaler.fit_transform(X_train) # median and IQR, resistant to outliers

Decision trees split on a single feature at a time by choosing a threshold, and the split it picks doesn't change whether that feature ranges from 0 to 1 or 0 to a million, only the relative ordering of values matters. Linear models compute a weighted sum across features, and gradient descent (or the closed-form solution) converges very differently when one feature spans 0 to 1 and another spans 0 to 100,000, the loss surface becomes badly conditioned. Distance-based methods, SVMs with an RBF kernel, KNN, k-means, are even more directly sensitive, since an unscaled large-range feature dominates every distance calculation regardless of whether it's actually more informative.

By default, OneHotEncoder raises an error the moment it sees a category at transform time that it never saw during fit. Setting handle_unknown="ignore" tells it to instead encode an unseen category as all zeros across that feature's one-hot columns, letting the pipeline keep running instead of crashing.

from sklearn.preprocessing import OneHotEncoder

encoder = OneHotEncoder(handle_unknown="ignore")
encoder.fit(X_train[["region"]])
encoder.transform(X_test[["region"]]) # new region value encoded as all-zero row

In a notebook, your train and test split usually come from the same static dataset, so every category shows up in both. In production, a new region, a new product SKU, a new browser user agent, will eventually show up in live traffic that your training data never saw. Without handle_unknown="ignore", that's an unhandled exception in front of a real user, not a metric in a report.

SimpleImputer fills missing values with a chosen statistic, mean, median, most frequent, or a constant. add_indicator=True additionally appends a binary column per imputed feature marking which rows originally had a missing value, so the model can learn that "this value was missing" is itself potentially informative, separate from whatever value got substituted in.

from sklearn.impute import SimpleImputer

imputer = SimpleImputer(strategy="median", add_indicator=True)
X_imputed = imputer.fit_transform(X_train)
# original columns plus one missingness-indicator column per feature with NaNs

It's worth turning on whenever missingness isn't random, a survey field left blank because the respondent didn't want to answer, a sensor reading missing because the sensor failed under specific conditions. In those cases the fact of missingness carries signal that a plain mean-fill throws away.

GroupKFold ensures that all rows belonging to the same group end up entirely in one fold, never split across train and validation. Regular KFold doesn't know or care about groups, so if your dataset has multiple rows per patient, per user, per store, plain KFold will happily put some of a given patient's rows in training and others in validation. The model then effectively gets to "see" that patient during training and gets evaluated on more rows from that same patient, which inflates the validation score in a way that won't hold up on a genuinely new patient the model has never encountered.

from sklearn.model_selection import GroupKFold

gkf = GroupKFold(n_splits=5)
for train_idx, val_idx in gkf.split(X, y, groups=patient_ids):
  pass # all rows for a given patient_id stay in one fold

Any dataset with a natural entity that generates multiple rows, patients with repeated visits, users with repeated sessions, stores with repeated daily records, needs GroupKFold or the score is measuring something closer to memorization than generalization.

TimeSeriesSplit generates train/test folds where the test set always comes chronologically after the training set, and each successive fold expands the training window forward in time rather than randomly resampling. Shuffling time-ordered data before splitting lets the model train on future rows and get validated against past ones, which is a form of leakage: in production the model will never have access to future data when predicting the past, so a cross-validation setup that allows it produces an optimistic score that has nothing to do with real deployment conditions.

from sklearn.model_selection import TimeSeriesSplit

tscv = TimeSeriesSplit(n_splits=5)
for train_idx, test_idx in tscv.split(X):
  pass # test_idx always occurs after train_idx in time

If fraud makes up roughly 2 percent of transactions, a model that predicts "not fraud" for literally every transaction gets 98 percent accuracy while catching zero fraud, which makes 95 percent look bad by comparison rather than good. Accuracy treats every class equally, and on an imbalanced dataset the majority class dominates the number regardless of what the model actually learned about the minority class that matters.

from sklearn.metrics import precision_recall_curve, average_precision_score

# for imbalanced classification, look at precision, recall, and PR-AUC
# instead of, or alongside, accuracy and ROC-AUC
ap = average_precision_score(y_test, y_scores)

I'd want to see precision and recall on the fraud class specifically, and the precision-recall curve rather than the ROC curve, since ROC-AUC can also look deceptively good on heavily imbalanced data because it's dominated by the huge number of true negatives.

scoring tells GridSearchCV which metric to use when comparing candidate hyperparameter combinations across folds, and by default it uses the estimator's own .score() method, accuracy for classifiers, R-squared for regressors, which isn't always the metric you actually care about. Optimizing for accuracy versus optimizing for recall on the positive class can genuinely select a different best hyperparameter combination, especially near a decision threshold, because the two metrics penalize different kinds of mistakes differently.

from sklearn.model_selection import GridSearchCV

grid = GridSearchCV(model, param_grid, scoring="recall", cv=5)
grid.fit(X_train, y_train)

Picking the wrong default here, letting a fraud or churn model get tuned for accuracy when recall on the rare class is what the business actually needs, is a mistake that's invisible unless you specifically ask what scoring was used.

C is the inverse of the regularization strength, so a smaller C means stronger regularization and a larger C means weaker regularization, which trips up plenty of candidates who assume a bigger number always means "more" of whatever the parameter controls. The reasoning: scikit-learn's LogisticRegression minimizes the loss function plus a penalty term scaled by 1/C. Shrink C toward zero and that penalty term grows huge, forcing the coefficients toward zero and simplifying the decision boundary.

from sklearn.linear_model import LogisticRegression

strong_reg = LogisticRegression(C=0.01)  # heavily regularized, simpler boundary
weak_reg = LogisticRegression(C=100)   # lightly regularized, closer to unpenalized fit

A random forest averages predictions across many trees that are each individually overfit and noisy but make different mistakes from each other, and averaging cancels out uncorrelated noise while preserving the signal every tree agrees on. It needs randomness in two places to make sure the trees actually are different from each other: bootstrap sampling gives each tree a different (with-replacement) sample of the training rows, and random feature selection at each split (controlled by max_features) considers only a random subset of columns for that split rather than all of them. Skip either source of randomness and the trees end up highly correlated with each other, and averaging correlated predictions doesn't reduce variance nearly as much.

from sklearn.ensemble import RandomForestClassifier

rf = RandomForestClassifier(n_estimators=300, max_features="sqrt", random_state=42)

Bagging (what random forests do) trains many models independently and in parallel on different bootstrap samples, then averages or votes across them, each model has no idea what the others are doing. Boosting (GradientBoostingClassifier, or HistGradientBoostingClassifier, which is scikit-learn's much faster histogram-based implementation) trains models sequentially, where each new model specifically targets the errors the previous models made, typically by fitting to the residuals or by reweighting misclassified examples more heavily. Bagging reduces variance, since it's averaging independent noisy estimators. Boosting reduces bias, since each new model is explicitly correcting what came before, though it can overfit if you add too many rounds without proper regularization.

The kernel trick lets a support vector machine draw a curved or complex decision boundary in the original feature space without ever explicitly computing the coordinates of a higher-dimensional space where that boundary would actually be a straight line. It does this by replacing every dot-product computation the SVM needs with a kernel function that returns the same result you'd get by transforming both points into the higher-dimensional space first and then taking their dot product there, skipping the (potentially infinite-dimensional) transformation itself entirely.

from sklearn.svm import SVC

svm = SVC(kernel="rbf", C=1.0, gamma="scale") # rbf kernel handles nonlinear boundaries
svm.fit(X_train, y_train)

The practical upside: an RBF-kernel SVM can separate classes no straight line could, without you ever having to manually engineer the nonlinear features that would make it separable in a linear model.

class_weight='balanced' automatically sets each class's weight inversely proportional to its frequency in the training data, so the loss function penalizes a mistake on a rare class more heavily than a mistake on the common class, without changing the actual rows the model sees. It's the lower-effort option compared to oversampling or undersampling, since it requires no extra data manipulation and no risk of duplicating rows in a way that could cause overfitting on the minority class.

from sklearn.linear_model import LogisticRegression

clf = LogisticRegression(class_weight="balanced")

I'd reach for it first, as the cheap baseline, before bringing in SMOTE or manual resampling, and only escalate to resampling if class_weight alone doesn't close the gap, since resampling adds real complexity and its own leakage risks if done incorrectly inside cross-validation.

The elbow method plots within-cluster sum of squared distances (inertia) against different values of k and looks for the point where adding another cluster stops meaningfully reducing that number, though in practice the "elbow" is often ambiguous rather than a clean bend. The silhouette score is usually the more reliable second check: it measures how similar each point is to its own cluster versus the nearest other cluster, on a scale from -1 to 1, and you pick the k that maximizes the average silhouette score across points.

from sklearn.metrics import silhouette_score

for k in range(2, 10):
  km = KMeans(n_clusters=k, n_init=10, random_state=42).fit(X)
  print(k, silhouette_score(X, km.labels_))

Neither method replaces genuine domain knowledge about how many segments actually make business sense, both are just a numerical sanity check on top of that judgment.

PCA finds the directions (principal components) along which the data varies the most, and re-expresses the data in terms of those directions instead of the original features, letting you drop the components that explain the least variance while keeping most of the information. The assumption that trips people up: PCA is entirely about linear combinations of the original features and about variance specifically, it has no idea which features are predictive of your target label. A direction with huge variance might be completely irrelevant to the thing you're trying to predict, and a direction with tiny variance might be exactly the signal that separates your two classes.

from sklearn.decomposition import PCA

pca = PCA(n_components=0.95) # keep enough components to explain 95% of variance
X_reduced = pca.fit_transform(X_train_scaled)
print(pca.explained_variance_ratio_)

Always scale features before PCA, unless every feature is already on a genuinely comparable scale, since PCA will otherwise treat a feature with a naturally larger numeric range as more "important" purely because of its units.

DBSCAN groups together points that are densely packed, defined by two parameters: eps, the radius to search around each point, and min_samples, the minimum number of neighbors within that radius for a point to count as a "core point." Clusters grow outward by connecting core points to their neighboring core points, and any point that doesn't have enough nearby neighbors within eps, and isn't reachable from a core point, gets labeled as noise (given a cluster label of -1) rather than forced into some cluster it doesn't really belong to.

from sklearn.cluster import DBSCAN

db = DBSCAN(eps=0.5, min_samples=5)
labels = db.fit_predict(X)
# labels == -1 marks points classified as noise, not assigned to any cluster

That built-in ability to label outliers as noise instead of shoehorning them into the nearest cluster is the main reason people reach for DBSCAN over k-means for anomaly-adjacent problems, fraud rings, unusual sensor readings, geographic clustering with genuinely sparse regions.

PCA, for a downstream model. t-SNE and UMAP are built for visualization, they preserve local neighborhood structure well enough to make a nice 2D scatter plot, but the distances and axes they produce don't have a stable, reusable meaning you can apply to new data the way PCA's components do. scikit-learn's TSNE doesn't even implement a proper transform method for new points by default, only fit_transform, because the embedding is specific to the exact dataset it was computed on.

from sklearn.decomposition import PCA
# use PCA to feed a downstream model
pca = PCA(n_components=20)
X_reduced = pca.fit_transform(X_scaled)
model.fit(X_reduced, y)

# t-SNE / UMAP: great for a 2D plot to eyeball cluster structure, not for feeding a model

Plain random oversampling duplicates existing minority-class rows exactly, which fixes the class ratio but adds zero new information, the model just sees the same handful of minority examples repeated many times. SMOTE (Synthetic Minority Oversampling Technique, from imbalanced-learn, not core scikit-learn) instead generates new synthetic minority examples by interpolating between a real minority point and one of its nearest minority neighbors, creating plausible new points rather than exact duplicates.

from imblearn.over_sampling import SMOTE

X_resampled, y_resampled = SMOTE(random_state=42).fit_resample(X_train, y_train)
# note: SMOTE must be fit only on X_train, never on the full dataset before splitting

The risk with SMOTE: it interpolates purely in feature space, with no awareness of whether the synthetic point it creates is actually realistic or falls in a region that genuinely belongs to the majority class, especially near the decision boundary. It can also leak badly if applied before splitting, since a synthetic minority point can end up in the training set while being interpolated from points now sitting in the test set.

scikit-learn's classifiers default to predicting the class with probability above 0.5 as the positive prediction, but that 0.5 cutoff is just a default, not a rule. Lower the threshold and you flag more cases as positive, catching more true positives (higher recall) but also flagging more false positives (lower precision). Raise the threshold and it's the reverse. The right threshold depends entirely on the relative cost of a false positive versus a false negative for your specific problem, a threshold tuned for cancer screening should look nothing like one tuned for flagging spam email.

from sklearn.metrics import precision_recall_curve

probs = model.predict_proba(X_test)[:, 1]
precisions, recalls, thresholds = precision_recall_curve(y_test, probs)
# pick the threshold that hits your target recall or precision, not necessarily 0.5

Train-test leakage is about improperly sharing information between your training and evaluation sets, the kind of thing a Pipeline and correct cross-validation fix. Target leakage is a different, often worse problem: a feature that's available at training time but wouldn't actually be available at prediction time in the real world, because it's downstream of, or a proxy for, the target itself. A classic example: predicting whether a customer will churn using a feature like "number of support tickets in the cancellation month." That feature is only known once churn has already effectively happened, it doesn't exist yet at the moment you'd actually need to make the prediction.

Target leakage doesn't show up as a difference between your train and test accuracy, both look great, since the leaked feature is present and correlated with the target in both splits equally. It only shows up once the model runs against genuinely new data in production, where that future-dependent feature simply isn't available yet, and performance collapses in a way cross-validation never warned you about.

class_weight applies one weight per class uniformly to every row in that class. sample_weight lets you assign an individual weight to every single row, which covers cases class_weight can't touch at all, weighting recent transactions more heavily than old ones in a fraud model, downweighting rows you know came from a noisier data source, or giving more importance to high-value customers in a churn model regardless of which class (churned or not) they fall into.

weights = np.where(recent_mask, 2.0, 1.0)
model.fit(X_train, y_train, sample_weight=weights)

Subclass BaseEstimator and TransformerMixin, implement fit (which must return self) and transform, and store constructor arguments as attributes with exactly the same names as the arguments, without any extra logic in __init__. That last detail isn't a style preference, it's what makes get_params and clone work correctly, which GridSearchCV and cross-validation both depend on.

from sklearn.base import BaseEstimator, TransformerMixin

class LogTransformer(BaseEstimator, TransformerMixin):
  def __init__(self, offset=1.0):
    self.offset = offset # store exactly as passed in, no logic here

  def fit(self, X, y=None):
    return self # nothing to learn, but must return self

  def transform(self, X):
    return np.log(X + self.offset)

Inheriting TransformerMixin is what gets you fit_transform for free, built from your fit and transform methods, instead of writing it yourself.

A sparse matrix stores only the nonzero values and their positions, instead of every cell including all the zeros the way a dense NumPy array does. Text data run through CountVectorizer or TfidfVectorizer produces a matrix where each row is a document and each column is a vocabulary word, and for any single document, nearly every column is zero since a document only actually uses a tiny fraction of the full vocabulary. Storing that as a dense array would waste enormous amounts of memory on zeros, so scikit-learn's text vectorizers return a SciPy sparse matrix (typically CSR format) by default instead.

from sklearn.feature_extraction.text import CountVectorizer

vectorizer = CountVectorizer()
X_sparse = vectorizer.fit_transform(documents)
print(type(X_sparse)) # <class 'scipy.sparse._csr.csr_matrix'>
print(X_sparse.shape, X_sparse.nnz) # nnz = number of stored nonzero values

Not every estimator accepts sparse input, and calling .toarray() or .todense() on a genuinely large sparse matrix, tens of thousands of documents against a vocabulary of tens of thousands of words, can blow past available memory almost instantly. Knowing which estimators handle sparse input natively, most linear models and SVMs do, some others don't, is worth checking before that conversion, not after a crash.

sklearn.base.clone reads an estimator's constructor parameters via get_params and builds a brand new instance from scratch with those same parameters, deliberately discarding any fitted attributes (the underscore-suffixed ones) learned during a previous fit call. That's intentional, not an oversight. Cross-validation and grid search both need a fresh, untrained copy of the estimator for every fold and every hyperparameter combination, otherwise fold two would start from whatever fold one already learned, which would leak information across folds and quietly inflate every score.

from sklearn.base import clone
from sklearn.linear_model import LogisticRegression

clf = LogisticRegression(C=0.5)
clf.fit(X_train, y_train)

fresh_clf = clone(clf) # same C=0.5, but unfitted, no coef_ attribute

Anyone writing their own cross-validation loop by hand instead of using cross_val_score and forgetting to clone the estimator inside the loop will get numbers that look better than they should, and it's a subtle enough bug that I've seen it in production code, not just interview answers.

The classic one, beyond fitting a scaler on the full dataset before splitting, is feature selection or imputation done the same way. Say you impute missing values with the column mean computed across the entire dataset, then split into train and test. The test set's imputed values were influenced by the test set's own data, since the mean baked in every row including the ones you're supposed to be evaluating against blind. Same problem with SelectKBest run on the full dataset before splitting: the features you kept were chosen partly based on their correlation with labels in the test set.

# leaky
from sklearn.impute import SimpleImputer
imputer = SimpleImputer(strategy="mean")
X_imputed = imputer.fit_transform(X) # sees everything, including test rows
X_train, X_test = train_test_split(X_imputed)

# correct
X_train, X_test = train_test_split(X)
imputer = SimpleImputer(strategy="mean")
X_train_imputed = imputer.fit_transform(X_train)
X_test_imputed = imputer.transform(X_test)

The fix is always the same shape: any step that learns a statistic from data belongs inside the cross-validation loop, ideally inside a Pipeline, never applied once to the whole dataset up front.

Pipeline chains steps sequentially, each step's output feeds the next step's input. FeatureUnion (largely superseded now by ColumnTransformer for most use cases) runs multiple transformers in parallel on the same input and concatenates their outputs side by side into one wider feature matrix, rather than passing data through in sequence. remainder="passthrough" on a ColumnTransformer is a related idea: columns not explicitly listed in any transformer tuple get passed through unchanged instead of being dropped, which matters if you only want to transform a subset of columns and keep the rest as-is.

from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import StandardScaler

ct = ColumnTransformer(
  [("scale", StandardScaler(), ["income"])],
  remainder="passthrough", # other columns pass through untouched
)

Forgetting that the default for remainder is "drop", not "passthrough", is a real bug I've seen ship: a model silently trained on fewer features than the author thought, because every column not named in the ColumnTransformer just disappeared.

Bias is error from a model too simple to capture the real pattern, it makes the same kind of mistake consistently regardless of which training data it saw. Variance is error from a model too sensitive to the specific training data it happened to get, one that would produce a meaningfully different model if trained on a slightly different sample. A decision tree with max_depth=1 (a stump) is high bias: it can only make one split, so it underfits almost any real dataset and performs similarly badly whether trained on this year's data or last year's. A decision tree with no depth limit is high variance: it grows until every leaf is pure, memorizing the training data's noise along with its signal, and a slightly different training sample produces a visibly different tree with different predictions.

from sklearn.tree import DecisionTreeClassifier
from sklearn.model_selection import cross_val_score

for depth in [1, 3, 5, None]:
  scores = cross_val_score(DecisionTreeClassifier(max_depth=depth), X, y, cv=5)
  print(depth, scores.mean())
# training accuracy climbs toward 1.0 as depth increases (or is unbounded)
# cross-validated accuracy typically peaks somewhere in the middle, then drops

The interview-worthy part isn't defining the two terms, it's being able to point at a specific hyperparameter, max_depth here, or C in an SVM, or alpha in Ridge, and explain which direction moving it trades one kind of error for the other.

Each tree in a random forest is trained on a bootstrap sample, which on average leaves out about a third of the training rows for that particular tree, since bootstrap sampling draws with replacement. The out-of-bag score evaluates each row using only the trees that never saw it during training, which gives you a validation-like estimate of performance without setting aside a separate validation split at all, effectively getting cross-validation almost for free as a side effect of how bagging already works.

from sklearn.ensemble import RandomForestClassifier

rf = RandomForestClassifier(n_estimators=300, oob_score=True, random_state=42)
rf.fit(X_train, y_train)
print(rf.oob_score_) # estimate of generalization accuracy, no separate val set needed

It's not a full substitute for held-out validation in every case though. OOB score only exists for bagging-style ensembles, and once you start doing real hyperparameter search or comparing entirely different model families, you still want a consistent, model-agnostic validation or cross-validation setup that isn't tied to a specific ensemble's internal bootstrap mechanics.

The default feature_importances_ in scikit-learn's tree ensembles is computed from mean decrease in impurity, how much each feature reduces Gini impurity or entropy when it's chosen as a split, summed across all trees. That measure is biased toward high-cardinality features (a continuous feature or one with many unique categories has more possible split points to choose from, which mechanically gives it more chances to look useful) and it splits credit unpredictably between correlated features, since a tree might use either of two nearly-identical features to make the same split, understating both of their true importances.

from sklearn.inspection import permutation_importance

result = permutation_importance(model, X_test, y_test, n_repeats=10, random_state=42)
# shuffles each feature independently and measures the drop in model performance
# not biased toward high-cardinality features the way impurity-based importance is

Permutation importance, which measures the actual drop in held-out performance when a feature's values are randomly shuffled, is the fix scikit-learn's own documentation recommends whenever this bias matters, and it's the answer I'd want to hear from anyone who's actually shipped a feature-importance chart to a stakeholder.

explained_variance_ratio_ tells you what fraction of the total variance in the data each principal component captures, a single number per component. The loadings (accessed via pca.components_) tell you something completely different: how much each original feature contributes to a given component, a full vector per component with one weight per original feature. People confuse the two because both come from the same PCA object and both get described loosely as "importance," but explained variance ratio answers "how much information is in this component," while the loadings answer "what does this component actually represent in terms of the original features."

pca = PCA(n_components=2).fit(X_scaled)
print(pca.explained_variance_ratio_) # e.g. [0.62, 0.18], how much variance each PC captures
print(pca.components_) # shape (2, n_features), how each original feature loads onto each PC

To actually interpret what principal component 1 "means" in business terms, you read the loadings, seeing which original features have the largest absolute weight on that component, not the explained variance ratio.

A well-calibrated classifier's predicted probabilities match observed frequencies, among all the times it predicts 0.7, roughly 70 percent of those should actually turn out positive. Random forests in particular tend to be poorly calibrated out of the box, their predict_proba output is really just the fraction of trees that voted for each class, which tends to be pulled toward the middle (rarely near 0 or 1) even when the model is quite confident, since averaging across many trees smooths out extreme votes.

from sklearn.calibration import CalibratedClassifierCV

calibrated = CalibratedClassifierCV(base_estimator=rf, method="isotonic", cv=5)
calibrated.fit(X_train, y_train)

This matters anywhere the actual probability value gets used downstream, not just the ranking, expected-value calculations for pricing or risk, or any dashboard that reports "70 percent likely to churn" as a number a human will act on directly rather than just a relative ranking.

partial_fit updates a model incrementally on a batch of data at a time, without requiring the entire dataset to fit in memory at once and without discarding what the model already learned from previous batches the way calling fit again would. It's how scikit-learn supports out-of-core learning on datasets too large to load into RAM, and how you'd update a model with a stream of new data without retraining from scratch every time.

from sklearn.linear_model import SGDClassifier

model = SGDClassifier()
classes = np.unique(y_full) # must be provided on the first partial_fit call
for X_batch, y_batch in batches:
  model.partial_fit(X_batch, y_batch, classes=classes)

Not every estimator supports it, only a specific subset does, SGDClassifier and SGDRegressor, MiniBatchKMeans, MultinomialNB and a few other estimators built around gradient-based or incremental update rules. Tree-based models and standard linear regression have no partial_fit at all, because their fitting procedure genuinely requires seeing the whole dataset at once to compute a split or a closed-form solution.

n_jobs=-1 tells scikit-learn to use every available CPU core in parallel, spreading the work of fitting individual trees across processes. Spinning up that many parallel processes has real fixed overhead, serializing data to each worker, starting and coordinating the processes, that overhead is roughly constant regardless of dataset size. On a small dataset where a single tree fits in a fraction of a second, that parallelization overhead can end up larger than the actual time saved by splitting the work across cores, so the parallel version finishes slower than just running it single-threaded.

from sklearn.ensemble import RandomForestClassifier
import time

for n_jobs in [1, -1]:
  start = time.time()
  RandomForestClassifier(n_estimators=100, n_jobs=n_jobs).fit(X_small, y_small)
  print(n_jobs, time.time() - start)
# on a genuinely small dataset, n_jobs=1 can beat n_jobs=-1

The lesson isn't "never use n_jobs=-1," it's that parallelization has a real cost you should actually measure rather than assume, especially in a hyperparameter search loop that's already fitting hundreds of small models.