OpenCV Interview Questions (2026): Most Asked, With Answers

Mat is OpenCV's core n-dimensional dense array class. It stores an image (or any matrix of numbers) as a header describing rows, columns, type, and step, plus a pointer to the actual pixel data. Nearly every function in the library takes a Mat in and returns a Mat out, so once you understand its memory model you understand most of the API surface without needing to memorize individual functions.

The header and the data are separate on purpose. Copying a Mat by value copies the lightweight header and bumps a reference count, it does not copy the underlying pixel buffer unless you explicitly ask for that with clone(). That single design decision is why so many OpenCV bugs come down to "I thought I had my own copy and I didn't."

cv2.imread(path) loads the file and returns a Mat in BGR channel order by default, three channels for a color image, one for grayscale if you pass the IMREAD_GRAYSCALE flag. cv2.imshow with cv2.waitKey renders it in a window.

import cv2

img = cv2.imread("photo.jpg")     # BGR, uint8
gray = cv2.imread("photo.jpg", cv2.IMREAD_GRAYSCALE)

cv2.imshow("preview", img)
cv2.waitKey(0)
cv2.destroyAllWindows()

imread returns None instead of raising an exception if the path is wrong or the file can't be decoded, which trips people up constantly. Any real pipeline should check for None before doing anything else with the result.

BGR was the native pixel order in early Windows bitmap and camera APIs that OpenCV was built to interoperate with, so the library kept it for backward compatibility and never switched. It's a historical accident, not a technical advantage.

It bites people the moment an image crosses a library boundary. Load an image with OpenCV, hand it to matplotlib's imshow, or to a PyTorch model trained on RGB tensors, and the colors come out swapped, red and blue reversed, unless you explicitly convert with cv2.cvtColor(img, cv2.COLOR_BGR2RGB) first. This is probably the single most common first bug anyone hits learning the library, and interviewers ask about it specifically because it separates people who've actually shipped an OpenCV pipeline from people who copied a tutorial once.

cv2.imwrite(path, img) infers the codec from the file extension and encodes accordingly. Quality and compression are controlled through an optional params list rather than a separate function call.

import cv2

cv2.imwrite("out.jpg", img, [cv2.IMWRITE_JPEG_QUALITY, 90])
cv2.imwrite("out.png", img, [cv2.IMWRITE_PNG_COMPRESSION, 5])

JPEG quality runs 0 to 100, higher means less compression and bigger files. PNG compression runs 0 to 9 and is lossless either way, the number only trades encode time for file size, unlike JPEG where it trades visual fidelity.

A Gaussian blur replaces each pixel with a weighted average of its neighbors, weighted by a Gaussian distribution centered on that pixel, so nearby pixels matter more than distant ones. It smooths noise and edges evenly and is the standard pre-processing step before edge detection. A median blur replaces each pixel with the median value in its neighborhood instead of an average.

The practical difference shows up with salt-and-pepper noise, isolated bright or dark pixels scattered through an image. An average pulls toward those outliers a little, while a median simply ignores them because they're not the middle value in a sorted neighborhood. That makes median blur the better choice for cleaning up that specific kind of noise while keeping edges sharper than a Gaussian blur would.

A kernel is a small matrix of weights that slides across an image, and at each position you multiply the overlapping pixel values by the kernel's weights and sum the result to produce one output pixel. Sharpening, blurring, and edge detection are all just different kernels applied through the same underlying convolution operation.

import cv2, numpy as np

sharpen_kernel = np.array([
  [ 0, -1, 0],
  [-1, 5, -1],
  [ 0, -1, 0],
])
sharpened = cv2.filter2D(img, -1, sharpen_kernel)

In BGR, a single color like "red" is spread across a combination of all three channels that shifts noticeably as lighting changes, brightness and color are entangled together. HSV separates hue (the actual color), saturation (how vivid or washed out it is), and value (brightness) into independent channels, so you can isolate a color by hue alone and stay tolerant to changes in lighting or shadow that would otherwise throw off a BGR-based threshold.

import cv2, numpy as np

hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
lower_red = np.array([0, 120, 70])
upper_red = np.array([10, 255, 255])
mask = cv2.inRange(hsv, lower_red, upper_red)

findContours expects a binary image, typically the output of a threshold or Canny call, where foreground pixels are white and background is black. It returns a list of contours, each one a NumPy array of (x, y) points tracing the boundary of a connected white region, plus a hierarchy array describing nesting relationships between contours (which ones are holes inside other contours, and so on).

import cv2

contours, hierarchy = cv2.findContours(
  binary_img, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE
)
cv2.drawContours(img, contours, -1, (0, 255, 0), 2)

contourArea computes the actual enclosed area of the contour's shape using Green's theorem on the boundary points, which gives you the true pixel area of an irregular blob. The bounding box from boundingRect is the smallest axis-aligned rectangle that fully contains the contour, and its area (width times height) is almost always larger than the contour's actual area unless the shape genuinely is a perfect axis-aligned rectangle.

import cv2

area = cv2.contourArea(contour)
x, y, w, h = cv2.boundingRect(contour)
box_area = w * h

fill_ratio = area / box_area # how "rectangular" the shape actually is

The ratio between the two is itself a useful feature, a circle's contour area to bounding box area ratio is roughly 0.785 (pi over 4), which is a cheap way to distinguish round blobs from rectangular ones without building a full shape classifier.

A keypoint is a location in an image, typically a corner or blob-like region, that's distinctive enough to be found again reliably even if the image is rotated, scaled, or lit differently. A descriptor is a numerical vector computed from the pixels around that keypoint, designed so that the same real-world point in two different images produces two very similar descriptor vectors, even though the raw pixels around it look somewhat different.

Detection finds where the keypoints are. Description turns each one into something you can compare mathematically. You need both stages to actually match features between two images, detection alone just gives you a set of interesting locations with nothing to compare them against.

cv2.VideoCapture handles both cases through the same interface, a file path opens a video file, an integer index (usually 0) opens a connected camera. You call read() in a loop, which returns a success flag and the frame, and the loop exits once read() reports failure, either the file ended or the camera disconnected.

import cv2

cap = cv2.VideoCapture(0)
while True:
  ret, frame = cap.read()
  if not ret:
    break
  cv2.imshow("frame", frame)
  if cv2.waitKey(1) & 0xFF == ord("q"):
    break
cap.release()
cv2.destroyAllWindows()

cap.get(cv2.CAP_PROP_FPS) only reports what the camera or file metadata claims its frame rate is, which tells you nothing about how fast your own loop is actually consuming and processing those frames. To measure real throughput you time the loop yourself, across enough frames to smooth out noise.

import cv2, time

start = time.time()
frame_count = 0
while frame_count < 100:
  ret, frame = cap.read()
  if not ret:
    break
  #... processing...
  frame_count += 1

elapsed = time.time() - start
print(f"actual FPS: {frame_count / elapsed:.1f}")

The dnn module lets you load pre-trained neural network models exported from other frameworks, TensorFlow, PyTorch (via ONNX), Caffe, and Darknet among others, and run inference on them directly through OpenCV without needing that original framework installed. It's designed specifically for forward-pass inference, running a network testing (OpenCV dnn module documentation).

What it explicitly does not support is training a network from scratch, that's still the job of the framework you trained the model in. People occasionally expect to fine-tune a model loaded through cv2.dnn and end up confused when there's no backward pass available, that's a hard boundary of the module's design, not a missing feature waiting to be added.

Assigning one Mat to another, or passing one into a function by value, copies the header only. Both Mats then point at the same underlying pixel buffer and share a reference count, so modifying pixels through one changes what you see through the other. clone() allocates a brand new buffer and copies the actual pixel data into it, giving you a fully independent Mat that the original can't affect.

import cv2

img = cv2.imread("photo.jpg")
shallow = img     # same underlying buffer
deep = img.copy()    # independent buffer, cv2's equivalent of clone()

shallow[0, 0] = [0, 0, 0] # also changes img
deep[0, 0] = [255, 255, 255] # img is unaffected

In the Python bindings this maps to NumPy semantics: cv2 images are NumPy arrays, so a plain assignment shares the buffer, and.copy() is what actually clones it.

Every Mat that shares an underlying data buffer increments a shared reference counter when it's created and decrements it when it goes out of scope. The buffer only gets deallocated once that counter hits zero, meaning every Mat referencing it has been destroyed (OpenCV Mat class documentation). This is essentially the same idea as a shared_ptr in C++, applied to pixel buffers instead of arbitrary objects.

The practical upshot: you can pass a Mat around freely, store it in a container, return it from a function, without worrying about a use-after-free or a manual delete call, and without paying the cost of copying pixel data on every hand-off. The cost only shows up when you actually need an independent copy, which is exactly when you call clone().

Simple thresholding (cv2.threshold with a fixed value) applies one cutoff to every pixel in the image, which only works well when lighting is uniform across the whole frame. Adaptive thresholding (cv2.adaptiveThreshold) computes a different threshold for each pixel based on the intensities in its local neighborhood, which handles uneven lighting, a shadow across half a document, far better. Otsu's method picks a single global threshold automatically by analyzing the image's histogram to find the value that best separates it into two classes, so you don't have to guess a fixed number by hand.

import cv2

_, simple = cv2.threshold(gray, 127, 255, cv2.THRESH_BINARY)
adaptive = cv2.adaptiveThreshold(
  gray, 255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY, 11, 2
)
_, otsu = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)

Canny runs a Gaussian blur first to suppress noise, then computes intensity gradients with a Sobel filter to find both the gradient magnitude and direction at every pixel. It applies non-maximum suppression, keeping only pixels that are a local maximum along the gradient direction, to thin wide gradient responses down to single-pixel-wide lines. Finally it uses two threshold values in a hysteresis step: any pixel above the high threshold is a definite edge, any pixel below the low threshold is discarded, and pixels in between only survive if they're connected to a definite edge.

import cv2

blurred = cv2.GaussianBlur(gray, (5, 5), 0)
edges = cv2.Canny(blurred, 50, 150)

The two-threshold hysteresis step is the part candidates most often can't explain, and it's the actual reason Canny produces cleaner, more connected edge maps than a single-threshold gradient method.

Erosion shrinks bright regions by sliding a structuring element (a small kernel) across the image and keeping a pixel only if every pixel under the kernel is also foreground, which strips away thin protrusions and small noise specks. Dilation does the opposite, expanding bright regions by setting a pixel to foreground if any pixel under the kernel is foreground, which fills small holes and connects nearby regions.

import cv2, numpy as np

kernel = np.ones((5, 5), np.uint8)
opening = cv2.morphologyEx(mask, cv2.MORPH_OPEN, kernel)  # erode then dilate
closing = cv2.morphologyEx(mask, cv2.MORPH_CLOSE, kernel) # dilate then erode

Opening (erode then dilate) removes small noise blobs while roughly preserving the size of larger objects, since the dilation step grows things back after erosion shrank them. Closing (dilate then erode) fills small holes and gaps inside objects for the same reason in reverse. Picking the wrong order for a binary mask cleanup is a classic mistake candidates make under pressure.

Raising the thresholds only filters edges after gradients have already been computed on noisy pixel data. Sensor noise creates small, sharp, random intensity jumps between neighboring pixels, and a Sobel gradient will happily report those as high-magnitude edges regardless of how high you set the threshold, because the noise itself can locally look as strong as a real edge.

Blurring first removes that noise at the source, before gradients are computed, so the gradient map genuinely reflects real structure in the scene rather than sensor artifacts. Pushing thresholds up instead just trades false positives for false negatives, real weak edges get lost right alongside the noise, since both look similar in magnitude once you're only looking at the gradient output. That's why Canny bakes the blur into the algorithm rather than leaving it as an optional pre-step.

Histogram equalization spreads out an image's intensity values so they cover the full available range more evenly, which boosts overall contrast, especially useful on a washed-out or underexposed image. cv2.equalizeHist applies one global transformation across the entire image, which works fine when lighting is uniform.

import cv2

equalized = cv2.equalizeHist(gray)

clahe = cv2.createCLAHE(clipLimit=2.0, tileGridSize=(8, 8))
clahe_result = clahe.apply(gray)

CLAHE (Contrast Limited Adaptive Histogram Equalization) splits the image into small tiles and equalizes each tile separately, which handles images with both dark and bright regions far better than a single global equalization would. The clip limit caps how much contrast any one tile can gain, which is the piece that stops CLAHE from wildly amplifying noise in already-noisy dark regions, a problem plain adaptive histogram equalization has without that cap.

INTER_LINEAR interpolates a new pixel value from a weighted average of the four nearest pixels in the source image, and it's the standard general-purpose choice, especially for upscaling. INTER_AREA instead considers the area each output pixel maps back to in the source image, effectively averaging a whole region rather than just four points.

import cv2

upscaled = cv2.resize(img, None, fx=2.0, fy=2.0, interpolation=cv2.INTER_LINEAR)
downscaled = cv2.resize(img, None, fx=0.25, fy=0.25, interpolation=cv2.INTER_AREA)

That averaging is what makes INTER_AREA noticeably better for shrinking an image significantly, since a nearest-neighbor-style method for downscaling can alias or lose thin details entirely, while averaging an area preserves more of the original signal. Using INTER_AREA for upscaling doesn't cause problems, it's just wasted work since there's no larger area to average from.

An affine transform preserves parallel lines, so rotation, scaling, translation, and shearing are all affine, and it's fully defined by three point correspondences using a 2x3 matrix. A perspective transform can also handle vanishing points, mapping a trapezoid back to a rectangle, the kind of correction you need when a photo of a document was taken at an angle, and requires four point correspondences with a 3x3 matrix since it has more degrees of freedom.

import cv2, numpy as np

src_pts = np.float32([[0, 0], [300, 0], [0, 300], [300, 300]])
dst_pts = np.float32([[10, 30], [280, 0], [40, 300], [300, 280]])

M = cv2.getPerspectiveTransform(src_pts, dst_pts)
warped = cv2.warpPerspective(img, M, (300, 300))

A document scanner app is the canonical example: four corners of a skewed page get mapped to a proper rectangle with a perspective transform, something an affine transform can't correctly do because it can't model the way a photographed rectangle converges toward a vanishing point.

getRotationMatrix2D builds a 2x3 affine matrix that combines a rotation around a specified center point with an optional uniform scale factor. You pass a center because rotating "in place" around an arbitrary pivot, usually the image's own center, is what people actually want, rotating around the origin (0, 0) would swing the whole image far outside its own frame for anything but a tiny angle.

import cv2

(h, w) = img.shape[:2]
center = (w // 2, h // 2)
M = cv2.getRotationMatrix2D(center, angle=45, scale=1.0)
rotated = cv2.warpAffine(img, M, (w, h))

Note that rotating by 45 degrees with the output size unchanged crops the corners, since the rotated content no longer fits in the original bounding box. Computing a new output size to avoid that crop is a common follow-up question interviewers ask once you've gotten this far.

You can't slice a rotated rectangle directly with NumPy indexing since that only works on axis-aligned boxes. The standard approach is to compute the rotation matrix for the ROI's angle, rotate the whole image so the ROI becomes axis-aligned, then crop with ordinary slicing.

import cv2

# rect from cv2.minAreaRect: ((cx, cy), (w, h), angle)
(cx, cy), (rw, rh), angle = rect
M = cv2.getRotationMatrix2D((cx, cy), angle, 1.0)
rotated_img = cv2.warpAffine(img, M, (img.shape[1], img.shape[0]))
cropped = cv2.getRectSubPix(rotated_img, (int(rw), int(rh)), (cx, cy))

cv2.getRectSubPix handles the sub-pixel-accurate crop once the region is axis-aligned, including interpolation if the center coordinates aren't whole numbers. This whole pattern shows up constantly in document scanning and license plate recognition pipelines where the region of interest is rarely perfectly aligned with the frame.

CHAIN_APPROX_NONE stores every single boundary point along a contour, including all the redundant points along a straight edge. CHAIN_APPROX_SIMPLE compresses horizontal, vertical, and diagonal straight segments down to just their endpoints, discarding the intermediate points since they carry no extra information about the shape.

For a simple rectangle, CHAIN_APPROX_SIMPLE returns exactly four points instead of hundreds, which matters a lot when you're processing many contours per frame in a real-time pipeline. The tradeoff is negligible in practice, you lose nothing about the actual shape, only redundant points on straight lines, so CHAIN_APPROX_SIMPLE is the default choice almost everywhere.

cv2.approxPolyDP implements the Douglas-Peucker algorithm, which simplifies a curve by keeping only the points that deviate from a straight line between two anchor points by more than a given epsilon tolerance, recursively subdividing until every remaining segment is within tolerance.

import cv2

epsilon = 0.02 * cv2.arcLength(contour, True)
approx = cv2.approxPolyDP(contour, epsilon, True)

if len(approx) == 4:
  print("looks like a quadrilateral")

Epsilon is usually set as a fraction of the contour's own perimeter, from arcLength, rather than a fixed pixel value, since that scales naturally with how big the shape is in the frame. This is the standard technique behind detecting whether a contour is roughly a triangle, rectangle, or something else, by just counting the vertices approxPolyDP returns.

SIFT (Scale-Invariant Feature Transform) produces highly distinctive, reliable descriptors and was patented for years, which historically pushed a lot of commercial and open source projects away from it, though the patent has since expired and SIFT is back in OpenCV's main contrib modules without licensing concerns. ORB (Oriented FAST and Rotated BRIEF) was designed from the start as a free, much faster alternative, combining the FAST corner detector with a rotation-aware version of the BRIEF binary descriptor.

import cv2

orb = cv2.ORB_create(nfeatures=1000)
kp, des = orb.detectAndCompute(gray, None)

The real tradeoff today is speed versus reliability under difficult conditions, not licensing. ORB's binary descriptors can be compared with a cheap Hamming distance instead of a Euclidean distance, which matters a lot for real-time matching, but SIFT still tends to hold up better under significant scale change and blur. Pick ORB for anything real-time on modest hardware, SIFT when accuracy on hard images matters more than speed.

A brute-force matcher compares every descriptor in one image against every descriptor in the other, exhaustively, and returns the closest match for each. It's simple and always finds the true nearest neighbor, but its cost grows with the product of the two descriptor counts, which becomes slow once you're matching thousands of keypoints per frame.

import cv2

bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck=True)
matches = bf.match(des1, des2)

flann = cv2.FlannBasedMatcher(indexParams, searchParams)
matches = flann.knnMatch(des1, des2, k=2)

FLANN (Fast Library for Approximate Nearest Neighbors) builds an index structure that trades a small amount of accuracy for a large speedup on nearest-neighbor search, which matters when you're matching against a large database of descriptors rather than just two single images. Use brute-force for small, one-off comparisons and FLANN once the descriptor counts get large enough that exhaustive search becomes the bottleneck.

Even after a ratio test, some matches are still wrong, an outlier that happens to pass the distance check by coincidence. RANSAC (Random Sample Consensus) repeatedly picks a small random subset of the matched point pairs, fits a geometric model (typically a homography) to that subset, then checks how many of the remaining matches agree with that model within some tolerance.

import cv2

H, mask = cv2.findHomography(src_pts, dst_pts, cv2.RANSAC, ransacReprojThreshold=5.0)
inliers = mask.ravel().astype(bool)

Whichever random subset produces the model with the most agreeing matches, the most inliers, wins, and everything that doesn't fit that winning model gets discarded as an outlier. This is what lets homography estimation stay accurate even when a meaningful fraction of your feature matches are outright wrong, which happens routinely with real-world images.

The Hough transform represents every possible line through the parameter space of (rho, theta), where rho is the perpendicular distance from the origin to the line and theta is the angle of that perpendicular. For every edge pixel in the image, the algorithm votes for every (rho, theta) pair a line through that pixel could have, incrementing a cell in an accumulator array for each one. A cell that ends up with a lot of votes represents a line that many edge pixels agree passes through it, which is strong evidence that a real straight line exists there.

import cv2

edges = cv2.Canny(gray, 50, 150)
lines = cv2.HoughLinesP(
  edges, rho=1, theta=cv2.pi / 180, threshold=100,
  minLineLength=50, maxLineGap=10
)

HoughLinesP, the probabilistic variant, only samples a subset of edge points rather than voting from every single one, which is much faster and directly returns line segments with actual endpoints instead of infinite lines, making it the version almost everyone reaches for in practice over the classic HoughLines.

Background subtraction produces a mask separating moving foreground objects from a static background, which is the standard first step in motion detection and simple object tracking without running a full detector on every frame. cv2.createBackgroundSubtractorMOG2 models each pixel's intensity over time as a mixture of Gaussian distributions, learned continuously as frames stream in, rather than comparing against one fixed reference frame.

import cv2

backSub = cv2.createBackgroundSubtractorMOG2(history=500, varThreshold=16)

while True:
  ret, frame = cap.read()
  if not ret:
    break
  fgMask = backSub.apply(frame)

A pixel that consistently matches one of the learned Gaussian components gets classified as background, that's how MOG2 tolerates gradual lighting changes and repetitive background motion, like leaves swaying, without flagging them as foreground on every frame the way a naive frame-difference approach would.

UMat is a Mat-like class that lets OpenCV's Transparent API automatically dispatch supported operations to an OpenCL-capable GPU when one's available, and fall back to the CPU otherwise, all through the same function calls you'd use for a regular Mat. You don't have to write separate GPU and CPU code paths, you just wrap your data in a UMat and call the same cv2 functions.

import cv2

frame = cv2.imread("photo.jpg")
u_frame = cv2.UMat(frame)
blurred = cv2.GaussianBlur(u_frame, (15, 15), 0) # runs on GPU if available
result = blurred.get() # bring it back to a regular Mat / NumPy array

It helps most for operations that are genuinely expensive per pixel and run on large frames, heavy filtering, resizing, color conversion at high resolution, where the GPU's parallelism outweighs the overhead of moving data to and from the device. For small images or cheap operations, that transfer overhead can make UMat slower than just staying on the CPU, so it's not a blanket speedup, it's worth benchmarking on your actual workload.

A real lens doesn't project the world perfectly according to the ideal pinhole camera model, it introduces radial and tangential distortion that bends straight lines in the world into slightly curved lines in the image, especially noticeable near the edges of a wide-angle lens. Calibration solves for two things: the intrinsic parameters (focal length, optical center, and distortion coefficients specific to that lens and sensor) and, for stereo setups, the extrinsic parameters describing how multiple cameras are positioned relative to each other.

Without correcting for those intrinsics, any measurement you try to derive from pixel coordinates, distance to an object, the real-world size of something in frame, will be systematically wrong, and the error gets worse the further a point is from the image center. Calibration is what lets you convert between pixel coordinates and real-world coordinates with known, bounded error.

Radial distortion comes from the physical curvature of the lens and causes straight lines to bow, either barrel distortion, where lines bulge outward away from the center, most visible in cheap wide-angle lenses, or pincushion distortion, where they curve inward toward the center. Tangential distortion happens when the lens and the image sensor aren't perfectly parallel to each other, a small manufacturing misalignment, which makes the image look tilted or skewed in a way radial correction alone can't fix.

import cv2

undistorted = cv2.undistort(img, camera_matrix, dist_coeffs)
# dist_coeffs = [k1, k2, p1, p2, k3]
# k1, k2, k3 model radial distortion
# p1, p2 model tangential distortion

OpenCV's standard distortion model bundles both into one coefficient vector and corrects for them together with undistort, but understanding which coefficients correspond to which physical cause matters when you're debugging a calibration that produces a still-visibly-warped result even after correction, since it tells you whether the problem is more likely a bad lens or a poorly mounted sensor.

calibrateCamera solves for the camera's intrinsic parameters, focal length, optical center, and distortion, using multiple images of a known calibration object, and you typically do this once per camera as a setup step. solvePnP does something narrower and is meant to run every frame, or every time you need it: given a set of known 3D points, an object's real-world corners, say, and their detected 2D positions in a single image, plus the camera's already-known intrinsics, it solves for the camera's pose, its rotation and translation relative to that object.

import cv2

success, rvec, tvec = cv2.solvePnP(
  object_points_3d, image_points_2d, camera_matrix, dist_coeffs
)

In practice this is the core of most augmented reality tracking and marker-based pose estimation, calibration happens once offline, and solvePnP runs continuously at frame rate to figure out where the camera is relative to a known object each time a new frame comes in.

readNetFromONNX (or the more general readNet, which infers the format from the file extension) loads the model. Input images need to go through blobFromImage first, which handles resizing, mean subtraction, scaling, and channel reordering all in one call, since a raw BGR uint8 frame isn't the tensor format most models expect.

import cv2

net = cv2.dnn.readNetFromONNX("model.onnx")
blob = cv2.dnn.blobFromImage(
  frame, scalefactor=1/255.0, size=(640, 640), swapRB=True, crop=False
)
net.setInput(blob)
output = net.forward()

swapRB=True is the detail that trips people up most, since it's exactly the BGR versus RGB mismatch from earlier in this list showing up again: most ONNX models exported from PyTorch expect RGB input, and forgetting swapRB silently feeds them BGR data instead, which doesn't error, it just produces quietly wrong predictions.

A backend is the underlying inference engine that actually executes the network graph, options include OpenCV's own implementation, OpenVINO, CUDA, and others. A target is the specific hardware the chosen backend runs on, CPU, an integrated GPU through OpenCL, a discrete NVIDIA GPU through CUDA, or a Myriad VPU for embedded inference.

import cv2

net.setPreferableBackend(cv2.dnn.DNN_BACKEND_CUDA)
net.setPreferableTarget(cv2.dnn.DNN_TARGET_CUDA)

The default backend picks whatever's available and reasonably safe, which usually means running on CPU even if a capable GPU sits unused in the same machine. Setting these explicitly is how you actually get GPU acceleration in a deployed pipeline, and it's a common gap between a demo that runs fine on a dev laptop and a production deployment that's mysteriously slower than expected on a GPU-equipped server nobody configured it to use.

Many OpenCV functions parallelize internally across multiple CPU threads by default, using whichever parallel framework the build was configured with, TBB, OpenMP, or a built-in pthreads implementation. This is usually a net win, a single call to a heavy operation like resize or GaussianBlur on a large image finishes faster by spreading the work across cores.

import cv2

cv2.setNumThreads(1) # disable OpenCV's internal parallelism

It becomes a problem when you're already parallelizing at a higher level, running several independent video streams concurrently, each in its own process or thread, and each one also tries to spin up its own set of internal OpenCV threads. That oversubscribes the CPU with far more threads than cores, and throughput actually drops from contention. Calling cv2.setNumThreads(1) inside each worker in that scenario, and doing your parallelism at the outer level instead, is a standard fix once you notice the mismatch.

A Mat for a color image is laid out HWC, height, width, channels, with pixel values usually as uint8 from 0 to 255. Most deep learning frameworks expect a tensor in CHW order, channels first, as normalized float32 values, often scaled to 0 to 1 or standardized with a specific mean and standard deviation the model was trained with.

import cv2

blob = cv2.dnn.blobFromImage(
  frame, scalefactor=1/255.0, size=(224, 224),
  mean=(0.485, 0.456, 0.406), swapRB=True
)
# blobFromImage handles HWC->CHW, uint8->float32, and normalization in one call

Feeding a raw HWC uint8 Mat directly into a model expecting a normalized CHW float32 tensor doesn't throw an error in most bindings, it just silently produces garbage predictions, since the shapes might coincidentally still be compatible even though the values mean something completely different than the model expects. blobFromImage exists specifically to handle this conversion correctly in one call instead of leaving room for that mistake.

You take several photos of a checkerboard pattern with known, fixed square dimensions, from different angles and distances, since the calibration needs varied viewpoints to solve for the camera's parameters reliably. For each image, findChessboardCorners locates the checkerboard's inner corners, and calibrateCamera then uses all those correspondences between known 3D checkerboard points and their detected 2D image positions to solve for the intrinsic matrix and distortion coefficients.

import cv2

ret, corners = cv2.findChessboardCorners(gray, (9, 6), None)
if ret:
  objpoints.append(objp)
  imgpoints.append(corners)

ret, camera_matrix, dist_coeffs, rvecs, tvecs = cv2.calibrateCamera(
  objpoints, imgpoints, gray.shape[::-1], None, None
)

A common mistake is calibrating from too few images or images that are all shot from roughly the same angle. The math needs enough variation in checkerboard pose to disambiguate focal length from distortion, and a set of nearly identical shots gives a calibration that looks fine numerically but performs badly on new images.

Step is the number of bytes between the start of one row and the start of the next. For a tightly packed image it equals width times the number of channels times the byte size of each element, but Mats aren't always tightly packed. A submatrix carved out of a larger Mat with a Rect keeps the parent's row stride, so its step is larger than its own width would suggest, because each row of the submatrix has to skip over pixels belonging to the parent image that aren't part of the crop.

cv::Mat parent(480, 640, CV_8UC3);
cv::Mat roi = parent(cv::Rect(100, 100, 50, 50));
// roi.step == parent.step, not 50 * 3
// roi.isContinuous() is false, its rows aren't back to back in memory

This is why isContinuous() matters before you reshape a Mat or hand its raw pointer to a function expecting a flat buffer. Code that assumes rows are always contiguous silently corrupts data the first time it runs on a cropped image.

Human perception isn't equally sensitive to red, green, and blue light, the eye is far more sensitive to green than to blue. A straight average would make objects that are pure blue or pure red appear too bright or too dark relative to how a person actually perceives their luminance. cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) instead applies a weighted sum, roughly 0.299R + 0.587G + 0.114B in the standard formula, that matches perceived brightness far more closely than an unweighted average would.

This matters in practice whenever grayscale output feeds into something perception-sensitive, contrast-based thresholding, or a model trained on grayscale images generated the standard way. Reimplementing grayscale conversion as a naive channel average produces images that look subtly wrong and can shift the input distribution a downstream model was trained on.

A shape like a washer or the letter O has an outer boundary and an inner boundary around the hole, both of which are legitimate closed contours in a binary mask. Whether findContours reports both, and how they relate, depends on the retrieval mode. RETR_TREE builds a full hierarchy describing parent-child nesting between every contour, so you can tell that the hole's contour is a child of the outer contour.

import cv2

contours, hierarchy = cv2.findContours(mask, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
# hierarchy[0][i] = [next, previous, first_child, parent]
for i, h in enumerate(hierarchy[0]):
  if h[3] != -1:
    print(f"contour {i} is a hole inside contour {h[3]}")

RETR_EXTERNAL, by contrast, only returns the outermost contours and silently drops any nested holes, which is fine if you only care about outer boundaries but will quietly lose information if the shape's internal structure actually matters to whatever you're building.

For each descriptor in the first image, you find its two nearest neighbors in the second image rather than just the single best match. The ratio test keeps a match only if the best match is meaningfully closer than the second-best, typically requiring the best distance to be less than 0.7 or 0.75 times the second-best distance.

import cv2

matches = flann.knnMatch(des1, des2, k=2)
good = [m for m, n in matches if m.distance < 0.75 * n.distance]

Without this filter, ambiguous keypoints, a corner on a repetitive brick pattern that looks nearly identical to a dozen other corners, all get reported as confident matches even though there's no real way to tell which one is correct. Requiring a clear margin between the best and second-best candidate throws out exactly those ambiguous cases, and it's the single biggest lever for cutting false matches before you ever get to a geometric verification step like RANSAC.

Optical flow estimates the apparent motion of pixel intensities between two consecutive frames, effectively a per-pixel velocity vector describing where each piece of the image moved. Lucas-Kanade, the sparse variant, only tracks a specific, chosen set of points, usually corners found beforehand with something like goodFeaturesToTrack, and assumes motion is locally constant within a small window around each point.

import cv2

p0 = cv2.goodFeaturesToTrack(prev_gray, maxCorners=100, qualityLevel=0.3, minDistance=7)
p1, status, err = cv2.calcOpticalFlowPyrLK(prev_gray, curr_gray, p0, None)

flow = cv2.calcOpticalFlowFarneback(
  prev_gray, curr_gray, None, 0.5, 3, 15, 3, 5, 1.2, 0
)

Farneback's method computes a dense flow field, a motion vector for every single pixel in the frame, by fitting a polynomial approximation to local neighborhoods rather than tracking discrete points. Dense flow is far more expensive but gives you full-frame motion information, useful for things like motion-based segmentation, while sparse flow is cheap enough to run in real time when you only care about tracking a handful of specific points, corners on a face, say, across frames.

The camera keeps producing frames at its own fixed rate regardless of whether your processing loop keeps up. If reading and processing a single frame takes longer than the interval between incoming frames, whether because the resolution jumped from 720p to 4K or because a per-frame operation allocates a new buffer instead of reusing one, the queue backs up and either frames get dropped by the driver or memory usage climbs as unprocessed frames pile up.

The usual fixes are downscaling the frame immediately after capture if full resolution isn't actually needed for the algorithm, moving expensive per-frame operations like color conversion or resizing onto a GPU with UMat and OpenCV's transparent OpenCL path, and profiling to find which single call in the loop is actually the bottleneck rather than guessing. A pipeline that silently allocates a new Mat every iteration instead of writing into a preallocated one is a common, often-overlooked source of the slowdown, since allocation itself has real cost at video frame rates.

Two cameras separated by a known baseline distance each capture the same scene from a slightly different viewpoint, so a given real-world point projects to slightly different pixel positions in the left and right images, the disparity. Once both cameras are calibrated and rectified, so corresponding scan lines in both images line up horizontally, that disparity is inversely proportional to the point's actual distance from the cameras, closer objects have larger disparity, farther objects have smaller disparity.

import cv2

stereo = cv2.StereoSGBM_create(minDisparity=0, numDisparities=64, blockSize=5)
disparity = stereo.compute(left_gray, right_gray)
depth = (focal_length * baseline) / (disparity + 1e-6)

StereoSGBM (Semi-Global Block Matching) computes the disparity map by matching small patches between the rectified left and right images, and once you have disparity, converting to actual depth is a straightforward formula involving the calibrated focal length and the known baseline between the two cameras.

Start by checking assumptions that don't raise errors when violated: whether imread actually loaded the file correctly rather than returning an unexpectedly small or all-black Mat from a corrupted file, whether the image's channel count matches what downstream code assumes, a grayscale image sneaking into a pipeline written for three-channel color, and whether a BGR versus RGB mismatch is silently swapping color channels somewhere the code crosses a library boundary.

The next layer is checking dtype and value range assumptions specifically, a Mat that's float32 in the 0 to 1 range being fed into code written for uint8 in the 0 to 255 range won't error, it'll just silently threshold or clip incorrectly. None of these failure modes throw an exception, which is exactly why they're the ones that make it past a small, well-behaved test set and only show up on production images that happen to violate an assumption nobody wrote a check for. Adding explicit assertions on shape, dtype, and value range at pipeline boundaries, even temporarily while debugging, is usually the fastest way to isolate which assumption actually broke.

Every time you call a function that returns a new Mat instead of writing into an existing one you pass in, OpenCV allocates a fresh memory buffer for the result. Allocation isn't free, it involves a system call or a call into an internal allocator, and doing it every single frame at 30 or 60 frames per second adds up to real, measurable overhead, plus it fragments memory over a long-running process and can eventually trigger garbage collection pressure in the Python bindings.

import cv2, numpy as np

output = np.empty_like(frame)
cv2.GaussianBlur(frame, (5, 5), 0, dst=output) # writes into the preallocated buffer

Most OpenCV functions accept an optional dst parameter for exactly this reason, letting you reuse the same buffer across every frame instead of allocating a new one each time. For a pipeline that's already close to its frame budget, this is one of the cheapest optimizations available, and it's the kind of thing that separates code written to pass a tutorial from code that's actually been profiled under load.

On a server with a GPU, you'd typically run a larger, more accurate model and let CUDA absorb the cost. On a CPU-only embedded board, the priority flips to a lightweight architecture designed for that constraint, MobileNet-based detectors or a small YOLO variant, exported to a quantized int8 model rather than full float32, since quantization can cut inference time substantially with a modest accuracy tradeoff.

Beyond the model itself, you'd downscale input frames as aggressively as accuracy allows before feeding them to the network, since inference cost scales with input resolution, and you'd consider running detection on every second or third frame with a lightweight tracker filling in the gaps in between rather than running full detection on every single frame. The realistic goal on constrained hardware isn't matching server-grade accuracy, it's hitting a frame rate the application actually needs, which sometimes means accepting a real accuracy tradeoff on purpose rather than as an accident of hardware limits.