Generative AI Interview Questions (2026): Q&A

Discriminative models learn a boundary between categories, given an input, they predict a label: spam or not spam, cat or dog, fraud or legitimate. Generative models learn the underlying distribution of the data itself, well enough to produce new samples that look like they came from that distribution. An LLM isn't classifying text, it's modeling the probability of the next token given everything before it, and sampling from that distribution to produce new text nobody wrote before.

The practical difference shows up in what each type of model can do. A discriminative spam filter can tell you an email is spam. It can't write you a new one. A generative model can do both, in principle, because it has learned the shape of the data, not just a decision boundary through it.

Autoregressive means the model generates one token at a time, and each new token is conditioned on every token generated so far, including its own previous output. The model doesn't plan the whole sentence up front. It predicts the single most likely next token given the context, appends that token to the context, and repeats.

context = "The capital of France is"
for _ in range(3):
  next_token = model.predict_next(context) # one token at a time
  context += next_token
# "The capital of France is" -> "Paris" -> ", a city" ->...

This is also why a model can't "take back" an early token once it's generated. If token five was a mistake, every token after it is conditioned on that mistake being part of the sequence.

A token is the actual unit an LLM operates on, and it's usually a chunk of a word, a whole common word, or a single character, decided by a tokenizer trained separately from the model itself, usually with byte-pair encoding or a similar subword algorithm. "Interview" might be one token. "Interviewing" might split into "interview" and "ing" because the tokenizer learned that "ing" shows up often enough as its own unit to be worth a separate slot in the vocabulary.

This matters practically because context windows, pricing, and rate limits are all measured in tokens, not words or characters, and the ratio isn't fixed. Dense technical text with lots of rare words or code tokenizes less efficiently than plain English prose, so the same character count can cost meaningfully more tokens depending on what's in it.

Pretraining first: the model learns to predict the next token over a massive, broad corpus, this is where almost all of the compute cost and almost all of the general language ability comes from. Then supervised fine-tuning (SFT): the model trains on curated examples of the kind of response you actually want, question in, ideal answer out, to shift it from "complete this text" toward "answer this instruction." Finally, alignment: a preference-based stage, RLHF or a related method, that further shapes the model's behavior using human (or model) judgments about which of two responses is better.

Each stage narrows the model's behavior further, from raw text prediction toward something that reliably behaves like a helpful assistant rather than an autocomplete engine.

Prompt engineering changes what you ask the model at inference time, no weight changes, no training run, and it's fast to iterate on and reversible instantly. Fine-tuning changes the model's weights themselves, which is slower and costlier to set up but bakes the behavior in permanently, so you don't have to re-explain it in every request.

My rule of thumb: reach for prompting first, always, it's cheaper to test and cheaper to change your mind about. Reach for fine-tuning when you need a behavior that's hard to express reliably in a prompt, a very specific output format the model keeps drifting from, a tone or vocabulary that needs to be consistent across thousands of calls, or when you need to shrink the prompt itself because you're paying per token at real scale.

Temperature scales the probability distribution over the next possible token before sampling. Low temperature, near 0, sharpens the distribution so the model almost always picks the single highest-probability token, producing deterministic, focused output. High temperature flattens the distribution, giving lower-probability tokens a real chance of getting picked, which produces more varied, sometimes more creative, sometimes less coherent output.

logits = model.get_logits(context)
scaled = logits / temperature  # temperature < 1 sharpens, > 1 flattens
probs = softmax(scaled)
next_token = sample(probs)

It's a dial on randomness, not a dial on correctness. Cranking temperature down doesn't make a model more accurate, it makes it more predictable, which are related but not the same thing.

The context window is the maximum number of tokens, input plus output combined, a model can process in a single call. It isn't a soft limit you brush up against gracefully, it's a hard architectural ceiling tied to how the model was trained and how its attention mechanism scales.

Exceed it and the request either gets rejected outright with an error, or, depending on the API and client library, gets silently truncated, usually from the front, dropping the earliest part of the conversation or document. That second failure mode is the dangerous one in production: the call still succeeds, the model just quietly answers a question it never actually saw the full context for, and nothing in the response tells you that happened.

Zero-shot means the prompt asks the model to do a task with no examples at all, just an instruction. One-shot gives exactly one worked example before the actual task. Few-shot gives several, usually two to five, showing the model the pattern of input-to-output you want before it sees the real input.

Zero-shot:
Classify sentiment: "The flight was delayed four hours."

Few-shot:
"Great service, would fly again." -> Positive
"Never again, worst airline." -> Negative
"The flight was delayed four hours." -> ?

Few-shot generally improves reliability for tasks with a specific expected format, since the examples pin down exactly what "correct" looks like instead of leaving the model to infer it from the instruction alone. It costs more tokens per call, which is the actual trade-off most teams weigh.

An embedding is a fixed-length vector of numbers, typically hundreds to a couple thousand dimensions, that represents a piece of text, an image, or another kind of input in a way that captures its meaning rather than its literal characters. Text with similar meaning ends up with vectors that sit close together in that high-dimensional space, even if the actual words used are completely different.

embed("The cat sat on the mat")
embed("A feline rested on the rug")
# different words entirely, embeddings still land close together

That's the entire trick behind semantic search: you're not matching keywords, you're measuring distance between meaning-vectors, which is why a search for "how do I cancel my subscription" can surface a document titled "ending your plan" even though they share almost no words.

Retrieval-augmented generation retrieves relevant documents from an external knowledge source at query time and stuffs them into the model's context window alongside the user's question, so the model answers using information it was never trained on and doesn't have to memorize. Fine-tuning bakes knowledge into the model's weights during training, which is fixed the moment training ends.

RAG solves the freshness and traceability problem fine-tuning can't touch cheaply: your company's docs change weekly, and every fine-tuning run to keep the model current would cost real money and time. RAG just updates the document index instead, and because the model's answer is grounded in specific retrieved text, you can point to exactly which document it came from, something a fine-tuned model's internalized knowledge can't offer at all.

An LLM is fundamentally a next-token predictor trained to produce plausible-sounding text, not a database with a built-in notion of "I don't actually know this." When a prompt asks for something the model wasn't trained on, or wasn't trained on well enough, it still generates the statistically most likely continuation of the text, which can be a confident, coherent, completely fabricated answer, because fluency and factual accuracy are two separate properties the training process doesn't guarantee together.

It's not a bug that occasionally slips through, it's a direct consequence of what the model is optimized to do. Nothing in standard pretraining or instruction tuning explicitly rewards a model for saying "I don't know" over confidently guessing, unless that behavior is specifically trained in during alignment.

Precision, in a retrieval context, measures how many of the chunks you retrieved were actually relevant to the question. Faithfulness measures something downstream of that: whether the final generated answer is actually supported by the retrieved context, or whether the model wandered off and answered from its own training data instead.

You can have perfect retrieval precision and still get an unfaithful answer, if the model ignores the good context it was given. And you can have mediocre retrieval and still get a faithful answer, if the model correctly says "the provided context doesn't address this" instead of guessing. They're measuring two different stages of the same pipeline, and a RAG system that only tracks one of them is flying half-blind.

Latent space is a compressed representation of an image, produced by an encoder, that's much smaller than the actual pixel grid but still captures the meaningful structure of the image. Stable Diffusion runs its entire denoising process in this compressed latent space rather than on full-resolution pixels directly, then decodes the final latent back into a real image only at the very end.

That's the specific innovation "latent diffusion" added over earlier pixel-space diffusion models: doing the expensive iterative denoising on a much smaller representation cuts the compute cost dramatically, which is a big part of why Stable Diffusion could run on a consumer GPU while earlier pixel-space diffusion approaches needed far more hardware for comparable image quality.

A multimodal model processes and reasons over more than one type of input, text and images, or text and audio, within a single unified architecture, rather than running two separate models and passing text descriptions between them. Modern multimodal LLMs typically encode images into the same token-like representation space the language model already operates on, so the model attends across text tokens and image tokens together in the same forward pass.

Stitching a separate vision model's text description into an LLM's prompt (an image captioning model feeding a caption into a chat model, say) loses information at that handoff, whatever the captioning model chose not to mention is simply gone. A truly multimodal model can reason about fine visual detail directly, because the image representation never gets collapsed down to a lossy text summary before the language model sees it.

Guardrails are checks applied before a prompt reaches the model, after the model produces output, or both, to catch behavior the model's own training doesn't reliably prevent on its own. Input guardrails might filter or flag attempted prompt injections, off-topic requests, or PII before it ever reaches the model. Output guardrails might check a generated response against a policy before it reaches the user, blocking disallowed content, redacting leaked sensitive data, or catching an answer that doesn't match an expected format.

They're a separate system layered around the model, not a property of the model itself, precisely because you can't fully trust the model's own judgment to catch every case reliably, especially against adversarial input specifically designed to slip past it.

Batch inference waits for the entire response to finish generating before returning anything to the client. Streaming returns each token to the client as soon as it's generated, so the user sees the response appear incrementally, word by word, instead of waiting for the whole thing at once.

Chat products stream almost universally because generating a long response can take several seconds to tens of seconds, and perceived latency matters more to users than actual total completion time. Watching text appear immediately, even if the full answer takes just as long to finish, feels dramatically faster than staring at a blank loading spinner for the same duration.

A transformer is a neural network architecture built around self-attention instead of recurrence. The original paper, "Attention Is All You Need," proposed dropping RNNs and LSTMs entirely and instead letting every token attend directly to every other token in a sequence in parallel (Vaswani et al., 2017).

RNNs process a sequence one token at a time, carrying a hidden state forward, which means training can't parallelize across the sequence dimension and long-range dependencies degrade the further apart two related tokens sit. Self-attention computes relationships between all tokens in one matrix operation, which parallelizes cleanly on GPUs and doesn't lose signal over distance the same way. That's the actual reason transformers scaled to the sizes they did, the architecture happened to match the hardware.

A foundation model is trained from scratch on a huge, broad corpus, general web text, books, code, to learn general-purpose language and reasoning ability. GPT-4, Claude, and Llama base models are foundation models. A fine-tuned model starts from an existing foundation model's weights and continues training on a smaller, targeted dataset to specialize its behavior for a narrower task, a support chatbot trained on a company's own tickets, a code model trained further on one language's idioms.

The distinction matters for cost and scope. Training a foundation model from scratch takes enormous compute, tens of millions of dollars for a frontier-scale model. Fine-tuning an existing one is orders of magnitude cheaper, because you're adjusting weights that already encode general language ability, not building that ability from nothing.

Every token gets projected into three vectors: a query, a key, and a value. The attention score between two tokens is the dot product of one token's query with another token's key, scaled and passed through softmax to turn it into a weight between 0 and 1. Each token's new representation becomes a weighted sum of every value vector in the sequence, weighted by those attention scores.

# simplified, ignoring batching and multiple heads
scores = Q @ K.T / sqrt(d_k)   # how much each token should attend to every other
weights = softmax(scores, dim=-1)
output = weights @ V       # weighted blend of value vectors

Practically, this lets a token like "it" in a sentence attend heavily to whatever noun it actually refers to, regardless of how many words sit in between, something an RNN's fixed hidden state struggles to preserve over long distances.

RLHF, reinforcement learning from human feedback, trains a model using a reward signal derived from human preferences rather than a fixed correct answer. Humans rank multiple model outputs for the same prompt, a separate reward model learns to predict those rankings, and the base model is then fine-tuned with reinforcement learning to produce outputs the reward model scores highly.

It solves a problem that supervised fine-tuning alone can't: there's often no single "correct" response to an open-ended prompt, only better and worse ones, and RLHF gives the model a way to learn relative quality instead of memorizing one fixed target answer per prompt. It's the stage that pushed models from technically-correct-but-unhelpful toward the tone and format people actually associate with ChatGPT-style assistants.

LoRA, low-rank adaptation, freezes the original model's weights entirely and injects small trainable low-rank matrices into specific layers, usually the attention projections. Instead of updating a billion-plus original parameters, you train a much smaller set of new ones, often under one percent of the original parameter count, and add their output back into the frozen weights at inference time.

# conceptually: original weight update becomes a low-rank decomposition
# W_new = W_frozen + (A @ B) * scaling
# A: d x r, B: r x d, with r much smaller than d

This cuts both the GPU memory needed for training (no need to store gradients and optimizer state for the full model) and the storage cost of each fine-tuned variant, since you can ship just the small LoRA weights and swap them onto the same frozen base model. That's the difference between fine-tuning needing a lab's worth of A100s and fine-tuning fitting on a single consumer GPU for smaller models.

Instruction tuning is supervised fine-tuning on a dataset of (instruction, ideal response) pairs, teaching the model to follow explicit instructions rather than just continue a text pattern. It uses ordinary supervised learning, no reward model, no reinforcement learning loop, just gradient descent toward matching the target response as closely as possible.

RLHF comes after instruction tuning in most training pipelines and optimizes for relative preference rather than matching a fixed target. Instruction tuning teaches the model the shape of a good response. RLHF refines which of several already-reasonable responses humans actually prefer, catching things like unnecessary hedging, unhelpful refusals, or subtly unhelpful tone that a fixed-target loss doesn't directly penalize.

Top-k restricts sampling to a fixed number of the highest-probability tokens, say the top 40, regardless of how confident the model actually is. Top-p instead picks the smallest set of tokens whose cumulative probability adds up to at least p, say 0.9, so the size of the candidate set shrinks when the model is very confident (one or two tokens might already cover 90 percent of the mass) and grows when the model is uncertain across many plausible options.

Top-p adapts to the shape of the distribution at each step, which is usually the more sensible default. Top-k with a fixed cutoff can either exclude a reasonable fifth-place token when the model is genuinely torn between several options, or include implausible tail tokens when the model is actually very confident and the top few tokens already dominate.

Chain-of-thought prompting asks the model to produce intermediate reasoning steps before its final answer, instead of jumping straight to a conclusion. "Let's think step by step" is the classic trigger phrase, but the more reliable version explicitly shows a worked example with reasoning steps included, not just the instruction.

It helps because an autoregressive model's only mechanism for "thinking longer" is generating more tokens, there's no separate internal deliberation step hidden from the output. Forcing it to write out intermediate steps effectively gives it more computation, and more relevant context, to condition the final answer on, rather than requiring the correct answer to emerge in a single forward pass with no scratch space.

A system prompt is a special message role most chat-tuned models are trained to treat as higher-priority instructions, setting persona, constraints, and behavior for the whole conversation, distinct from the back-and-forth user and assistant turns that follow. Architecturally there's no hardware-level wall between them, it's still all just tokens fed into the same context window, but the model has been specifically trained during fine-tuning to weight system-role tokens as instructions to follow rather than content to respond to.

That distinction is trained-in behavior, not a hard technical guarantee, which is exactly why prompt injection attacks work at all: a cleverly worded user message can sometimes get the model to treat it as though it carries system-level authority, because the boundary is learned, not enforced by the runtime.

The user's question gets embedded into a vector. That vector gets compared against a pre-built index of embeddings for every chunk of your knowledge base, usually via approximate nearest-neighbor search, to pull back the handful of chunks most semantically similar to the question. Those chunks get inserted into the prompt as context, alongside the original question, and the whole thing goes to the LLM, which generates an answer grounded in that retrieved text rather than purely its own training data.

query_vector = embed(user_question)
top_chunks = vector_db.search(query_vector, k=5)
prompt = f"Context:n{top_chunks}nnQuestion: {user_question}nAnswer using only the context above."
answer = llm.generate(prompt)

Every failure mode in a RAG system traces back to one of these three steps: bad chunking means the wrong pieces of text get indexed, bad retrieval means the right chunks exist but don't get pulled back, and a weak prompt means even good retrieved context gets ignored or contradicted by the model anyway.

Chunking is splitting a large document into smaller pieces before embedding each one separately, since you can't usefully embed an entire 50-page PDF as a single vector without losing most of the specificity that makes semantic search work. Chunk size is a direct trade-off between context and precision.

Chunks too large dilute the embedding, a 2,000-word chunk covering five different subtopics produces a vector that's an average of all five, matching none of them precisely when a user asks about just one. Chunks too small lose surrounding context, a fifty-word chunk might contain a pronoun or a table row with no header, meaningless on its own even if it's the technically correct match. Most production systems land somewhere around 200 to 500 tokens per chunk with some overlap between consecutive chunks, but the right number is genuinely dataset-dependent, and I'd be suspicious of anyone who gives you one fixed number as a universal answer.

Cosine similarity measures the angle between two vectors, ignoring their magnitude. Dot product measures both angle and magnitude together. For embeddings that are already normalized to unit length, a common step many embedding models perform, the two are mathematically equivalent up to a constant scaling factor, so it genuinely doesn't matter which you use.

It starts mattering the moment your embeddings aren't normalized. Dot product on unnormalized vectors will favor longer vectors regardless of whether they're actually more semantically relevant, which can quietly bias retrieval toward whatever documents happen to produce larger-magnitude embeddings, often longer text chunks, for reasons that have nothing to do with relevance. Checking whether your embedding model normalizes its output, and picking your similarity metric to match, is a small detail that catches out more RAG builds than it should.

A vector database is optimized for approximate nearest-neighbor search over high-dimensional vectors at scale, using index structures like HNSW that trade a small amount of accuracy for search speed that stays fast even across millions of vectors. A brute-force comparison against every stored vector is exact but scales linearly with dataset size, fine for a few thousand documents, unworkable at real production volume.

You genuinely can just use Postgres, with the pgvector extension, for small-to-medium datasets, and plenty of production systems do exactly that rather than standing up a dedicated vector database. The trade-off shows up as your corpus grows into the millions of vectors or your latency requirements tighten, at which point a purpose-built vector store's indexing and sharding usually outperforms a general-purpose relational database bolted onto vector search after the fact.

Re-ranking takes the top candidates returned by the initial vector search, usually the top 20 to 50, and scores them again with a more expensive, more accurate model, often a cross-encoder that looks at the query and each candidate chunk together rather than comparing pre-computed independent embeddings, then keeps only the best handful to actually send to the LLM.

The initial embedding-based retrieval is fast but approximate, it compares vectors that were computed independently of each other, so it sometimes surfaces documents that are topically similar but not actually the best answer to the specific question asked. A cross-encoder re-ranker is too slow to run over an entire corpus, but running it over just the initial retrieval's short list is cheap, and it meaningfully improves precision on the final set of chunks that actually reach the model.

No, and this is worth saying directly because it's a common misconception. Larger models generally hallucinate less on well-represented topics, since they've seen more examples and encode more accurate world knowledge, but scale doesn't remove the underlying mechanism. A bigger model still generates the most probable next token, and for anything genuinely outside its training data, that probable-sounding continuation can still be false.

What actually reduces hallucination in practice is a combination of things: grounding answers in retrieved context (RAG), training the model specifically to express calibrated uncertainty, and constraining the task itself, asking a model to summarize a document it can see is far less hallucination-prone than asking it an open factual question with no supporting context at all.

BLEU and ROUGE measure n-gram overlap between generated text and a reference text, essentially counting how many words and short phrases match. They were built for machine translation and summarization tasks where there's a fairly narrow band of acceptable phrasing close to a reference answer.

They fall apart for open-ended generative tasks because a genuinely correct, well-written response can use completely different words than any reference answer and score poorly, while a response that copies reference phrasing verbatim but gets the actual meaning wrong can score well. Neither metric has any notion of semantic correctness, only surface-level overlap, which is exactly the gap that pushed the field toward embedding-based similarity metrics and LLM-as-judge evaluation instead.

LLM-as-judge uses a separate model call, often a stronger or differently-prompted model, to score or compare generated outputs against a rubric, instead of relying on exact-match or n-gram metrics. It's popular because it can evaluate genuinely open-ended criteria, helpfulness, tone, factual grounding, that fixed metrics like BLEU simply can't capture.

The catch is that you're using a model with its own biases and blind spots to evaluate another model, and those biases are well documented: judge models tend to favor longer responses, favor responses stylistically similar to their own default output, and can be gamed by a candidate response that's confidently worded even when it's wrong. Treating an LLM judge's score as ground truth without periodically checking it against real human judgment is a common way teams end up optimizing for "sounds good to another LLM" instead of "is actually correct."

During training, a diffusion model learns to reverse a process of gradually adding random noise to real images, step by step, until the image is pure noise. It's trained to predict, at each noise level, what noise was added, so it can be run backward: subtract a small amount of predicted noise, repeat many times, and a coherent image gradually emerges out of what started as pure static.

Generation starts from random noise instead of a real image and runs that same reverse process, guided by a text prompt at each step, so the "denoising" gradually converges toward an image consistent with the prompt rather than toward any specific training image. It's a fundamentally different generation mechanism from an autoregressive LLM: instead of producing output token by token in sequence, the entire image gets refined all at once, over many iterative passes.

A GAN trains two networks against each other, a generator producing fake images and a discriminator trying to tell fakes from real ones, with the generator improving as it learns to fool an increasingly capable discriminator. GANs can generate an image in a single forward pass, which made them fast, but that adversarial training setup is notoriously unstable, prone to mode collapse where the generator learns to produce only a narrow range of outputs that reliably fool the discriminator rather than genuinely diverse images.

Diffusion models train with a much more stable, straightforward objective, predict the noise, no adversarial dynamic to destabilize, and they cover a wider diversity of outputs as a result. The trade-off is speed: diffusion needs many iterative denoising steps to generate one image, which is why so much diffusion research since 2022 has focused on cutting the number of required steps down from the dozens originally needed. Diffusion's stability and output quality won out over GANs' speed advantage for most production image generation tools.

CLIP, contrastive language-image pretraining, trains a text encoder and an image encoder jointly so that matching text-image pairs end up close together in a shared embedding space, and mismatched pairs end up far apart, using a large dataset of images paired with captions scraped from the web.

Text-to-image diffusion models use this shared space as their steering mechanism: at each denoising step, the model checks how well the partially-denoised image's embedding aligns with the prompt's text embedding, and nudges the image toward better alignment. CLIP is effectively the bridge that lets a diffusion model, which fundamentally just knows how to remove noise, understand what "a golden retriever wearing sunglasses on a beach" should actually look like as an image.

Quantization reduces the numerical precision used to store a model's weights, from 32-bit or 16-bit floating point down to 8-bit integers or even lower, which shrinks the model's memory footprint and speeds up inference, since lower-precision arithmetic runs faster on most hardware and the whole model fits in less memory and VRAM.

The trade-off is a small amount of accuracy loss, since you're representing each weight with fewer possible values, which introduces rounding error throughout the network. In practice, well-implemented 8-bit or even 4-bit quantization often loses very little measurable quality for a lot of models, which is exactly why it's become a default step for running large models on limited hardware rather than a niche optimization technique reserved for edge cases.

Training cost is a one-time, front-loaded expense, you pay it once (or once per fine-tuning run) regardless of how many users end up using the model. Inference cost is recurring and scales directly with usage, every single user request costs compute, and that cost never goes away no matter how many requests came before it.

This changes how you should actually plan a generative AI feature. A team that optimizes hard for training cost but ships an inefficient inference setup, an oversized model for the task, no caching, no batching, will bleed money proportional to their growth curve. A feature that gets more popular should, ideally, get cheaper per request through caching and batching efficiencies, not linearly more expensive in a way that erodes margin as usage scales.

Prompt injection is an attack where untrusted input, a webpage the model is asked to summarize, a document a user uploads, a field in a support ticket, contains text specifically crafted to be interpreted as an instruction rather than data, hijacking what the model does next. A jailbreak specifically targets the model's own safety training, trying to get it to produce content it was explicitly trained to refuse.

They can overlap, but they're distinct threats. Prompt injection doesn't need to touch the model's safety behavior at all, it can just get a summarization tool to quietly exfiltrate data or perform an unintended action, because the model can't reliably tell the difference between "the user's actual instruction" and "text that happened to be inside the content I was asked to process." That's the core problem: instructions and data share the exact same channel, plain text in the context window, with no hard technical separation between them.

A chatbot takes a message, generates a response, and stops, one request in, one response out. An agent runs a loop: it decides on an action (often calling an external tool or API), observes the result of that action, and feeds that result back into its own context to decide the next action, repeating until it judges the task complete, rather than producing a single response and handing control back to the user.

User: "What's the weather in Austin and should I bring an umbrella?"
Agent step 1: decides to call get_weather(city="Austin")
Agent step 2: receives {"condition": "rain", "temp": 68}
Agent step 3: generates final answer using that tool result

The core architectural difference is that an agent's own prior actions and their results become part of its context for the next decision, which is also exactly why agent failures compound differently than chatbot failures do, one bad tool call or one hallucinated intermediate result can steer every subsequent step in the loop off course.

Context caching lets you send a large, unchanging block of context, a long system prompt, a big reference document, once, and have the provider reuse the already-processed representation of it on subsequent calls instead of reprocessing every token from scratch each time, cutting both latency and cost for that repeated portion. The underlying model weights don't change at all, it's purely an inference-time optimization on repeated input.

Fine-tuning changes the model's weights so the reference material's influence is baked in permanently, without needing to resend it in every request at all. Caching is the right call when the reference material might change (a document that gets updated weekly) or when you need it verbatim and traceable in the context. Fine-tuning is the right call when the material is stable long-term and you want to shrink your per-request token cost by not sending it at all anymore.

Self-attention is permutation-invariant by construction, it computes a weighted sum over all tokens with no built-in notion of order, so shuffling the input sequence produces the exact same attention output unless something injects position information. That's the trade-off for dropping recurrence: you gain parallelism, you lose the free ordering signal an RNN gets automatically from processing tokens one after another.

Positional encoding solves this by adding a vector that encodes each token's position directly into its embedding before attention runs, either a fixed sinusoidal pattern (as in the original paper) or a learned embedding per position. Newer architectures like rotary position embeddings (RoPE) bake position information into the attention computation itself rather than the input embedding, which handles longer sequences and extrapolation to unseen lengths more gracefully than the original fixed scheme did.

Catastrophic forgetting is what happens when further training on a narrow dataset overwrites weights that encoded broader capabilities the model had before, so the model gets better at the new task and measurably worse at things it used to handle fine. It happens because gradient descent has no built-in mechanism to protect knowledge it isn't currently being trained on, every update nudges every affected weight toward whatever minimizes loss on the current batch, regardless of what that weight was previously doing for an unrelated task.

I've seen this bite a team that fine-tuned a support model heavily on refund-policy examples, it got noticeably worse at general conversational tone within a few thousand steps, because the fine-tuning data skewed so narrow that the optimizer had no signal telling it to preserve the broader behavior. LoRA and lower learning rates reduce the risk by limiting how much of the model actually changes, and keeping a slice of general-purpose data mixed into the fine-tuning set is the more direct fix.

DPO reformulates the RLHF objective as a direct supervised loss over preference pairs, chosen response versus rejected response, without training a separate reward model or running a reinforcement learning loop at all. It uses a closed-form relationship between the optimal policy and the reward function to derive a loss you can just run standard supervised training against.

The appeal is mostly operational. RLHF's RL loop, PPO in particular, is notoriously unstable and sensitive to hyperparameters, and training a separate reward model is an extra stage with its own failure modes. DPO gets a comparable alignment effect with a training setup closer to ordinary fine-tuning, which is a big part of why it's shown up in more open fine-tuning pipelines since 2023, teams get most of the preference-alignment benefit without needing the RL infrastructure and expertise RLHF demands.

Setting temperature to 0 removes randomness from the sampling step, but it doesn't guarantee bit-for-bit identical floating-point computation across calls. Most production LLM serving stacks batch multiple requests together for GPU efficiency, and the exact batch composition, which other requests happen to be processed alongside yours, can change the order of floating-point operations in ways that produce tiny numerical differences in the logits.

Those differences are usually far too small to matter, until they land near a genuine tie between two tokens' probabilities, at which point a rounding difference in the fourth decimal place is enough to flip which token wins. Once one token flips, every token generated after it is different too, since the model is autoregressive and conditions on its own prior output. This is a known, documented behavior of hosted inference at scale, not a bug specific to any one provider, and it's one of the more counterintuitive things about production LLMs for engineers coming from deterministic software.

The model confidently answering using its own training data instead of the retrieved context, especially when the retrieved chunks are irrelevant or contradict what the model already "knows." Retrieval failing silently is worse than retrieval failing loudly, because the LLM doesn't refuse to answer just because the context it was handed is bad, it fills the gap with its parametric knowledge and produces a fluent, confident answer that happens to ignore the very documents it was supposed to be grounded in.

The fix isn't purely technical, it's a prompt-and-evaluation discipline: explicitly instructing the model to say "I don't know" when the retrieved context doesn't answer the question, and separately measuring faithfulness, whether the answer is actually supported by the retrieved text, not just whether the answer sounds plausible. Teams that only measure "did the user get an answer" and not "was the answer grounded in what we retrieved" tend to discover this failure mode from an angry customer instead of a test suite.

Model collapse is a degradation that occurs when a model is trained repeatedly on data generated by earlier models rather than genuine human-produced data. Each generation amplifies the statistical patterns the previous generation favored and loses the tails of the original distribution, rare but valid patterns of language, unusual but correct facts, edge cases that don't show up often but matter. Over successive generations, output narrows toward a blander, more repetitive average, and genuinely rare information can disappear from what the model can produce entirely.

It matters increasingly because a growing share of text on the open web is now AI-generated, which means models trained on fresh web scrapes going forward are training on an ever-larger proportion of synthetic data whether the researchers intend to or not. Labs address this with careful data curation and provenance tracking rather than assuming "more scraped web text" keeps working the way it did when nearly all of that text was human-written.

A single image just needs to be internally consistent, plausible lighting, coherent anatomy, sensible composition. Video needs all of that in every frame plus temporal consistency across frames: an object's shape, color, and identity have to stay stable as it moves, physics has to look roughly right across time, and small per-frame errors that would be invisible in a single still image become obvious and jarring as flicker or morphing once you play frames in sequence.

The compute cost compounds too, a few seconds of video at a usable frame rate is dozens of frames that all need to be generated coherently together, not independently, since generating each frame with an image model and stitching them would produce exactly the flickering inconsistency video models are built to avoid. Most production video generation systems handle this by extending diffusion into the temporal dimension directly, denoising a whole short clip's worth of frames jointly rather than one frame at a time, which is both why the field trails image generation in quality and why it needs so much more compute per output.

The model itself never executes any code. It's trained to recognize, from the tool definitions provided in its context, when a user's request matches a tool it has access to, and to output a structured, machine-parseable response, usually JSON, naming which tool to call and with what arguments, instead of a normal free-text answer. Your application code is what actually receives that structured output, executes the real function call against your real API or database, and then feeds the result back into the model's context as a new turn so it can produce a final natural-language answer.

{
 "tool_call": {
  "name": "get_weather",
  "arguments": { "city": "Austin" }
 }
}

The model is entirely a decision-maker and a formatter here, not an executor. Every actual side effect, hitting an API, writing to a database, sending an email, happens in your application's code, which is also exactly where the actual security boundary needs to live: never let a model's output directly trigger a destructive action without your own validation step in between.

The NIST AI Risk Management Framework is a voluntary framework published by the U.S. National Institute of Standards and Technology for identifying, measuring, and managing risks specific to AI systems, organized around four functions: govern, map, measure, and manage (NIST, 2023). It's not a certification or a legal requirement, it's a structured vocabulary and process for thinking about things like bias, reliability, and explainability across an AI system's lifecycle.

Companies reference it mostly for two practical reasons: it gives procurement and legal teams at enterprise customers a recognizable, standards-based answer when they ask "how do you manage AI risk" during a vendor security review, and it gives internal teams a shared checklist so risk management doesn't depend entirely on one engineer's personal judgment about what could go wrong. For an interview, knowing it exists and roughly what it covers signals you've thought about generative AI risk beyond "the model might say something embarrassing."