🧠 Top 200 LLM Interview Questions
Detailed answers, explanations & real‑world scenarios — curated for 2026 interviews at Google, OpenAI, Meta & more.
🚀 Ace your technical interview | FreeLearning365 — Job Interview Preparation Portal
Go to Portal
Jump to a category
1. LLM Fundamentals
What is a Large Language Model (LLM) and how does it work?
Answer: An LLM is a neural network trained on vast text data to predict the next token. It uses transformer architecture to learn patterns and relationships. Scenario: When you type a prompt, the model generates a response by iteratively predicting the most probable next word based on context.
How do LLMs generate text?
Answer: Through autoregressive generation: given a sequence, the model predicts the next token, appends it, and repeats until a stop condition. Scenario: Chatbots generate responses one token at a time, using sampling strategies like temperature to control randomness.
What is the difference between generative AI and traditional AI?
Answer: Generative AI creates new content (text, images) while traditional AI focuses on classification or regression tasks. Scenario: Generative AI writes poetry; traditional AI classifies spam emails.
Explain the concept of "next‑token prediction".
Answer: The core training objective: given previous tokens, predict the next token using cross-entropy loss. Scenario: During training, the model sees "The cat sat on the" and learns to predict "mat".
What is the difference between autoregressive and masked language modeling?
Answer: Autoregressive predicts left-to-right (GPT); masked predicts masked tokens in a bidirectional context (BERT). Scenario: BERT is used for understanding (e.g., sentiment), GPT for generation.
How does the Transformer architecture power LLMs?
Answer: Transformers use self-attention to capture dependencies regardless of distance, enabling parallel training and scalability. Scenario: All modern LLMs (GPT, LLaMA) are transformer-based, allowing them to handle long-range context.
What is the "context window" and why is it important?
Answer: It's the maximum number of tokens the model can process at once. Larger windows allow handling longer documents or conversations. Scenario: Claude 3 has a 200K context window, enabling processing of entire novels.
What happens when you exceed the context window?
Answer: The model truncates or ignores the excess tokens, losing information. Solutions include sliding window or summarization. Scenario: In a long chat, older messages are dropped if the window is exceeded.
How do you handle long documents that exceed the context limit?
Answer: Use chunking, summarization, or RAG to retrieve relevant parts. Also, use models with larger windows or hierarchical attention. Scenario: When summarizing a 100-page report, chunk it into sections and summarize each.
What is the "lost in the middle" phenomenon?
Answer: LLMs tend to forget information in the middle of a long context; they recall the beginning and end better. Scenario: When you place critical facts in the middle of a long prompt, the model may ignore them.
Explain the difference between encoder‑only, decoder‑only, and encoder‑decoder models.
Answer: Encoder-only (BERT) for understanding; decoder-only (GPT) for generation; encoder-decoder (T5) for sequence-to-sequence tasks. Scenario: BERT for sentence classification; GPT for text generation; T5 for translation.
Why do decoder‑only models dominate today’s LLM landscape?
Answer: They are simpler, scalable, and excel at in-context learning and generation, outperforming encoder-decoder for most tasks. Scenario: GPT-4, LLaMA, Claude are all decoder-only.
What are the key differences between GPT, BERT, and T5?
Answer: GPT is autoregressive, BERT is bidirectional, T5 is encoder-decoder with a unified text-to-text format. Scenario: Use GPT for creative writing, BERT for NLU, T5 for translation/summarization.
How does the size (number of parameters) affect model capability?
Answer: Larger models generally have better performance due to more capacity, but require more data and compute; scaling laws predict performance gains. Scenario: GPT-3 (175B) outperforms GPT-2 (1.5B) on complex reasoning.
What is the scaling law for LLMs?
Answer: Model performance improves predictably with increases in parameters, data, and compute, following a power-law relationship. Scenario: Used to estimate the optimal model size for a given budget.
What is the role of layer normalization in transformers?
Answer: It stabilizes training by normalizing activations across features, reducing covariate shift. Scenario: Applied after each sub-layer in transformer blocks.
Explain the concept of positional encoding.
Answer: Adds information about token position since self-attention is permutation-invariant. Usually sine/cosine functions or learned embeddings. Scenario: Allows the model to distinguish "dog bites man" from "man bites dog".
What are residual connections and why are they used?
Answer: Skip connections that add input to output of a layer, enabling gradients to flow easily and preventing vanishing gradients. Scenario: Essential for training very deep transformers.
How do LLMs handle different languages?
Answer: Through multilingual pretraining on diverse corpora, sharing tokenizers that cover many scripts, and sometimes using language-specific adapters. Scenario: XLM-R, mT5 are multilingual models.
What is the difference between a language model and a foundation model?
Answer: A language model is trained on text; a foundation model is a broader term for models trained on broad data (text, image, etc.) that can be adapted. Scenario: GPT-4 is both a language model and a foundation model.
2. Transformers & Attention
How does the self‑attention mechanism work?
Answer: It computes attention scores between every pair of tokens, allowing each token to attend to all others. Weights are derived from queries, keys, and values. Scenario: In "the cat sat on the mat", "cat" attends strongly to "sat".
Walk me through the multi‑head attention.
Answer: Multiple attention heads run in parallel, each with different learned projections, then concatenated and projected. This captures different relationships. Scenario: One head may focus on syntax, another on semantics.
What are queries, keys, and values in attention?
Answer: Queries are from the current position, keys are from all positions, values carry the content. Attention weights = softmax(QK^T) * V. Scenario: In a sentence, query for "ate" looks for "cake" key to get value.
How is scaled dot‑product attention computed?
Answer: Compute QK^T / sqrt(d_k), apply softmax, then multiply by V. The scaling prevents gradients from becoming too small. Scenario: Standard in transformer implementations.
What is the purpose of the feed‑forward network in a transformer block?
Answer: It applies a nonlinear transformation (MLP) to each position independently, adding capacity and allowing complex feature interactions. Scenario: After attention, the FFN refines the representation.
Explain the difference between self‑attention, cross‑attention, and causal attention.
Answer: Self-attention: Q,K,V from same sequence. Cross-attention: Q from one, K,V from another (encoder-decoder). Causal: masks future tokens. Scenario: Decoder uses causal self-attention and cross-attention to encoder.
What is FlashAttention and why is it important?
Answer: An efficient implementation of attention that reduces memory reads/writes using tiling, speeding up training and inference. Scenario: Used in training large models like GPT-4 to reduce cost.
How does attention scale with sequence length?
Answer: Standard attention has O(n^2) complexity in time and memory, becoming a bottleneck for long sequences. Scenario: For a 10K token sequence, attention becomes very expensive.
What are some efficient attention variants (e.g., sparse attention, linear attention)?
Answer: Sparse attention limits which tokens attend (e.g., local windows); linear attention approximates with O(n) complexity. Scenario: Longformer uses sparse attention for long documents.
What is the role of the encoder in a transformer?
Answer: Encoder processes the input sequence and outputs a contextualized representation; used in BERT and T5. Scenario: In translation, encoder reads source sentence.
What is the role of the decoder?
Answer: Decoder generates output tokens autoregressively, using cross-attention to encoder's output. Scenario: In translation, decoder produces target sentence.
How does the transformer process input sequences in parallel?
Answer: Self-attention computes all token interactions simultaneously, unlike RNNs that process sequentially. Scenario: Training is much faster due to parallelization.
What is the difference between pre‑norm and post‑norm in transformers?
Answer: Pre‑norm applies layer norm before the sub-layer, post‑norm after. Pre‑norm is more stable and common in modern transformers (e.g., GPT). Scenario: GPT-3 uses pre-norm.
How does ALiBi (Attention with Linear Biases) improve length extrapolation?
Answer: ALiBi adds linear bias to attention scores based on distance, no positional embeddings; allows better generalization to longer sequences. Scenario: Used in models like MPT.
What is Rotary Position Embedding (RoPE)?
Answer: RoPE encodes position by rotating the query and key vectors, allowing relative position information; used in LLaMA and PaLM. Scenario: LLaMA uses RoPE for better performance.
3. Tokenization & Embeddings
What is tokenization and why is it critical for LLM performance?
Answer: Tokenization converts text into tokens (subwords). It affects vocabulary size, model efficiency, and performance on rare words. Scenario: English tokenization differs from Chinese, impacting generation quality.
How does tokenization impact generation quality and cost?
Answer: Poor tokenization can fragment words, increase token count (cost), and reduce comprehension. Efficient tokenizers reduce tokens per message. Scenario: BPE tokenizes "tokenization" into fewer tokens than character-level.
Explain Byte‑Pair Encoding (BPE).
Answer: BPE merges frequent character pairs iteratively to create subword units, balancing vocabulary size and coverage. Scenario: Used in GPT and many LLMs.
What is WordPiece and how does it differ from BPE?
Answer: WordPiece also merges, but uses likelihood gain to select merges, often used in BERT. BPE uses frequency. Scenario: BERT uses WordPiece.
What is SentencePiece?
Answer: A tokenizer library that treats input as raw bytes, supports multiple languages, and can learn subword units without pre-tokenization. Scenario: Used in LLaMA, T5.
How do you handle out‑of‑vocabulary (OOV) tokens?
Answer: Subword tokenizers like BPE handle OOV by splitting into known subwords. Also use unknown token () as fallback. Scenario: Rare names are split into smaller pieces.
What is a token in the context of LLMs?
Answer: A token is the smallest unit of text processed by the model, typically a subword or word piece. Scenario: "I love AI" -> ["I", "love", "AI"] or ["I", "lo", "ve", "AI"] depending on tokenizer.
How do you estimate the cost of token usage for SaaS vs. open‑source LLMs?
Answer: Count tokens via tokenizer, multiply by per-token cost (SaaS) or estimate compute/energy for open-source. Scenario: GPT-4 costs ~$0.03 per 1K tokens.
What are vector embeddings and how are they used in LLM applications?
Answer: Embeddings are dense vector representations of text, used for semantic search, clustering, and as input to LLMs. Scenario: In RAG, embeddings retrieve relevant documents.
What is the difference between embedding short and long content?
Answer: Short content embeddings capture direct meaning; long content may lose granularity; often use pooling or mean. Scenario: For a paragraph, use mean of token embeddings.
How do you benchmark embedding models on your own data?
Answer: Use tasks like semantic similarity, retrieval accuracy, or classification; compare different models (e.g., BERT, E5). Scenario: Evaluate on a custom question-answering dataset.
How would you improve the accuracy of an embedding model if performance is low?
Answer: Fine-tune on domain data, use larger models, improve pooling, or use contrastive learning. Scenario: Domain-specific embeddings for legal texts.
Walk me through improving a sentence‑transformer model for embeddings.
Answer: Use labeled (query, positive, negative) triplets, train with triplet loss, and evaluate on STS tasks. Scenario: For a product search engine, fine-tune on product descriptions.
4. Prompt Engineering
What is prompt engineering and why is it important?
Answer: Crafting inputs to guide LLM outputs. It's crucial because well-designed prompts can drastically improve accuracy and reduce hallucination. Scenario: Adding "think step by step" improves reasoning.
Explain the basic structure of prompt engineering.
Answer: Typically includes instruction, context, input, and output format. System messages set behavior. Scenario: "You are a helpful assistant. Answer the question based on the text."
What is in‑context learning?
Answer: The model learns from examples provided in the prompt without weight updates. It adapts to the task via the context. Scenario: Few-shot prompts demonstrate how to answer.
What are the different types of prompt engineering?
Answer: Zero-shot, few-shot, chain-of-thought, role-based, structured prompts, etc. Scenario: Use chain-of-thought for math problems.
What strategies help write good prompts?
Answer: Be clear and specific, provide examples, use system role, constrain output format, and ask for reasoning. Scenario: "Explain as if to a 5-year-old" for simplicity.
What are few‑shot prompts and what aspects should you keep in mind?
Answer: Provide a few examples of input-output pairs. Ensure examples are diverse, representative, and correctly formatted. Scenario: Show 3 examples of sentiment classification before asking for new one.
What is Chain‑of‑Thought (CoT) prompting?
Answer: Encourages the model to output intermediate reasoning steps, improving performance on complex tasks. Scenario: "Let's think step by step" for arithmetic word problems.
How do you improve LLM reasoning through prompt engineering?
Answer: Use CoT, self-consistency, or Tree-of-Thoughts. Also prompt to verify answers. Scenario: Ask the model to critique its own answer.
What if your CoT prompt fails — how do you improve reasoning?
Answer: Try different wording, provide more detailed examples, or use self-consistency (sample multiple reasoning paths). Scenario: If one CoT fails, generate 5 and take majority.
What is self‑consistency decoding?
Answer: Generate multiple reasoning paths and select the most consistent answer, improving accuracy. Scenario: For math, generate 10 chains and pick majority final answer.
What is Tree‑of‑Thoughts (ToT) prompting?
Answer: Explores multiple reasoning branches and backtracks, like a search tree, for complex problems. Scenario: Solving a crossword puzzle or planning a trip.
What is ReAct (Reasoning + Acting) prompting?
Answer: Combines reasoning with tool use; the model generates reasoning traces and executes actions (e.g., search) iteratively. Scenario: An agent that looks up information before answering.
How can you control hallucination using prompt engineering?
Answer: Ask the model to cite sources, or say "if unsure, say 'I don't know'", or provide constraints. Scenario: "Only answer based on the provided text."
What is the difference between system, user, and assistant messages?
Answer: System sets behavior, user asks, assistant responds. They structure the conversation for the model. Scenario: In ChatGPT API, roles define the dialogue.
How do you structure prompts for multi‑turn conversations?
Answer: Include conversation history in context, with clear role markers, and maintain state. Scenario: Chat memory where each turn is appended to the prompt.
5. Pretraining & Fine‑tuning
What are common pretraining objectives for LLMs?
Answer: Causal language modeling (CLM) for decoder-only; masked language modeling (MLM) for encoder-only; also denoising autoencoding. Scenario: GPT uses CLM, BERT uses MLM.
What is the difference between pretraining and fine‑tuning?
Answer: Pretraining learns general language understanding on large corpus; fine-tuning adapts to a specific task with smaller labeled data. Scenario: Pretrain on internet text, fine-tune on legal documents.
What is supervised fine‑tuning (SFT)?
Answer: Training the model on labeled data (input-output pairs) to perform a specific task like classification or summarization. Scenario: Fine-tune GPT-3 on customer support dialogues.
What is instruction tuning?
Answer: Fine-tuning on instruction-response pairs so the model follows instructions well, enhancing generalizability. Scenario: InstructGPT, LLaMA-2-chat.
What is LoRA and how does it work?
Answer: Low-Rank Adaptation adds trainable low-rank matrices to weight layers, enabling efficient fine-tuning with few parameters. Scenario: Fine-tune a 70B model on a single GPU using LoRA.
What is QLoRA and how does it differ from LoRA?
Answer: QLoRA quantizes the base model to 4-bit and adds LoRA adapters, drastically reducing memory usage. Scenario: Fine-tune LLaMA-65B on a single 24GB GPU.
What are Adapters and Prefix‑tuning?
Answer: Adapters are small trainable modules added between layers. Prefix-tuning prepends trainable continuous vectors to the input. Scenario: Used for parameter-efficient multi-task learning.
What is P‑Tuning and P‑Tuning v2?
Answer: P-tuning uses trainable continuous prompts; v2 adds deep prompt tuning to multiple layers, improving performance. Scenario: Tuning prompts for few-shot NLU tasks.
What is Parameter‑Efficient Fine‑Tuning (PEFT)?
Answer: A set of methods (LoRA, Adapters, etc.) that fine-tune only a small fraction of parameters, saving memory and compute. Scenario: Essential for fine-tuning large models on consumer hardware.
When would you use fine‑tuning instead of RAG?
Answer: When you need the model to internalize knowledge (e.g., style, domain expertise) or when latency is critical; RAG is for external knowledge retrieval. Scenario: Fine-tune a legal chatbot vs. using RAG for up-to-date news.
What is catastrophic forgetting and how do you mitigate it?
Answer: Forgetting previously learned knowledge during fine-tuning. Mitigate using regularization, replay, or parameter-efficient tuning. Scenario: When fine-tuning on new tasks, older skills degrade.
What is the difference between full fine‑tuning and LoRA in terms of compute and quality?
Answer: Full fine-tuning updates all weights, requires more compute and memory, but may yield slightly better quality; LoRA is cheaper with comparable performance. Scenario: LoRA is preferred in production for cost-effectiveness.
How do you choose the right fine‑tuning method for a given task?
Answer: Consider data size, compute budget, and task complexity. For small data, use PEFT; for large data and high quality, full fine-tuning. Scenario: If you have 100K examples, full fine-tuning may be worth it.
What is multi‑task fine‑tuning?
Answer: Fine-tuning on multiple tasks simultaneously, enabling the model to generalize across tasks. Scenario: T0, FLAN models.
What is the role of a validation set during fine‑tuning?
Answer: To monitor performance on unseen data, prevent overfitting, and select the best checkpoint. Scenario: Early stopping based on validation loss.
6. RAG & Vector Databases
What is Retrieval‑Augmented Generation (RAG) and how does it work?
Answer: RAG retrieves relevant documents from an external knowledge base and uses them as context to augment generation, reducing hallucination. Scenario: A QA system that searches a company's internal documents.
Explain the complete RAG pipeline.
Answer: 1. Indexing: chunk documents, embed, store in vector DB. 2. Query: embed user query, retrieve top-k chunks. 3. Generate: feed retrieved chunks + query to LLM. Scenario: Building a customer support bot.
What are the benefits of using a RAG system?
Answer: Up-to-date knowledge, reduced hallucination, ability to cite sources, and easier knowledge updates without retraining. Scenario: News Q&A system that pulls current articles.
What is chunking and why do we chunk data?
Answer: Splitting documents into smaller pieces to fit the LLM's context window and improve retrieval granularity. Scenario: A 100-page PDF is split into paragraphs or sections.
What factors influence chunk size?
Answer: Context window, nature of content (semantic boundaries), and retrieval performance. Larger chunks may contain more context but less precision. Scenario: For code, chunk by function; for articles, by paragraph.
What are different chunking strategies (length‑based, semantic, structure‑based)?
Answer: Length-based: fixed token count. Semantic: split by sentence or paragraph boundaries. Structure-based: by markdown headers or HTML tags. Scenario: Use semantic chunking for better coherence.
How do you find the ideal chunk size for your use case?
Answer: Experiment with different sizes, evaluate retrieval accuracy and generation quality, and tune based on performance metrics. Scenario: A/B test chunk sizes on a validation set.
What is a vector database and how does it differ from a traditional database?
Answer: Vector DBs store embeddings and enable similarity search using distance metrics (e.g., cosine). Traditional DBs are for structured data with exact queries. Scenario: Pinecone, Milvus are vector DBs.
How does a vector database work internally?
Answer: Uses indexing structures like HNSW, IVF to speed up approximate nearest neighbor (ANN) search over high-dimensional vectors. Scenario: Retrieving top-5 similar vectors out of millions in milliseconds.
What is the difference between a vector index, a vector DB, and a vector plugin?
Answer: Vector index is the data structure; vector DB is a full database (index + storage + query); plugin is an add-on to existing DBs (e.g., pgvector). Scenario: pgvector is a Postgres extension.
Explain vector search strategies: clustering, LSH, product quantization (PQ).
Answer: Clustering (IVF) partitions vectors; LSH hashes similar vectors; PQ compresses vectors by sub-space quantization. All for efficient ANN. Scenario: PQ reduces memory 70% while keeping accuracy.
When would you use text search vs. vector search?
Answer: Text search (BM25) for keyword matching, vector search for semantic similarity. Hybrid combines both. Scenario: Search for exact product IDs vs. finding similar concepts.
How do you evaluate a RAG system?
Answer: Evaluate retrieval (precision, recall, MRR) and generation (faithfulness, relevance, answer accuracy). Also user feedback. Scenario: Use metrics like ROUGE, BLEU, and human evaluation.
What metrics are used to evaluate RAG?
Answer: Retrieval: Hit Rate, NDCG, MAP. Generation: Faithfulness, Answer Relevance, Context Relevancy. Also end-to-end accuracy. Scenario: Use RAGAS metrics.
How do you increase accuracy and reliability in RAG?
Answer: Improve chunking, use hybrid search, re-rank retrieved documents, and prompt engineering. Also filter low-confidence retrievals. Scenario: Add a cross-encoder for re-ranking.
What is the "lost in the middle" problem in RAG context?
Answer: LLM may ignore relevant information placed in the middle of the retrieved context; place critical info at the beginning or end. Scenario: When feeding 10 chunks, put the best ones first and last.
How do you handle cases where no relevant information is found?
Answer: Respond with "I don't know" or ask a clarifying question; also use fallback mechanisms like web search. Scenario: A customer bot says "I couldn't find that, can you rephrase?".
What are hybrid search approaches?
Answer: Combine vector similarity and keyword search (BM25) with weighted scores to get the best of both. Scenario: Use BM25 for exact terms and vector for semantics.
What is re‑ranking in RAG and why is it used?
Answer: After initial retrieval, use a more expensive model to re-score and re-order top candidates, improving relevance. Scenario: Cross-encoder re-ranker after initial ANN.
7. Alignment (RLHF / DPO)
What is Reinforcement Learning from Human Feedback (RLHF)?
Answer: A method to align model outputs with human preferences by training a reward model on human comparisons and then fine-tuning using reinforcement learning. Scenario: ChatGPT was trained with RLHF.
Explain the RLHF pipeline step by step.
Answer: 1. Collect human comparisons of model outputs. 2. Train a reward model to score outputs. 3. Use PPO to fine-tune the base model against the reward model. Scenario: Used to make models more helpful and harmless.
What is a reward model and how is it trained?
Answer: It's a model that predicts a scalar reward for a given response. Trained on pairwise comparisons using ranking loss (e.g., Bradley-Terry). Scenario: Trained on thousands of human preferences.
What is Proximal Policy Optimization (PPO) and how is it used in RLHF?
Answer: PPO is an RL algorithm that updates the policy while constraining updates to avoid drastic changes. Used to fine-tune the LLM to maximize reward. Scenario: PPO is the standard RLHF algorithm.
What is Direct Preference Optimization (DPO) and how does it differ from RLHF?
Answer: DPO directly optimizes the policy using preference data without a separate reward model or RL, using a closed-form loss. Simpler and more stable. Scenario: DPO is often used as a cheaper alternative to RLHF.
What are the tradeoffs between RLHF and DPO?
Answer: RLHF can be more expressive but complex and unstable; DPO is simpler and cheaper but may have performance gaps. Scenario: RLHF often yields higher quality but requires more compute.
What is Constitutional AI?
Answer: A method where the model critiques and revises its own responses based on a set of principles, reducing need for human feedback. Scenario: Used in Anthropic's Claude.
How do you collect preference data for alignment?
Answer: Generate multiple responses for a prompt, have human annotators rank them based on quality, safety, helpfulness, etc. Scenario: Collect via platforms like Scale AI.
What is the KL penalty in RLHF and why is it used?
Answer: Adds a penalty for diverging too far from the original model, preventing catastrophic forgetting and maintaining fluency. Scenario: Keeps the model from exploiting the reward model.
How do you evaluate alignment without human evaluation?
Answer: Use automated metrics (e.g., BLEU, ROUGE) or LLM-as-judge to score responses for safety, helpfulness, and honesty. Scenario: Use GPT-4 to evaluate responses.
8. Evaluation & Hallucination
What is hallucination in LLMs and what causes it?
Answer: Generating content that is factually incorrect or ungrounded. Causes: data biases, insufficient context, overgeneralization, and model uncertainty. Scenario: Model invents a citation that doesn't exist.
How can you detect and mitigate hallucination?
Answer: Detect using consistency checks, fact-checking, or uncertainty estimation. Mitigate with RAG, better prompts, or fine-tuning with high-quality data. Scenario: Ask the model to provide sources or verify claims.
What is faithfulness, relevance, and coherence in LLM evaluation?
Answer: Faithfulness: consistency with given context. Relevance: relevance to prompt. Coherence: logical flow and readability. Scenario: Key dimensions for assessing generation quality.
How do you measure consistency of LLM outputs?
Answer: Ask the same question multiple times with slight variations; compute agreement (e.g., semantic similarity) across responses. Scenario: High consistency indicates robustness.
What is the difference between determinism and consistency?
Answer: Determinism: same input gives exactly same output (temperature=0). Consistency: outputs are similar semantically even with different phrasing. Scenario: Deterministic is a special case of consistency.
What are common evaluation metrics for LLMs (e.g., BLEU, ROUGE, METEOR)?
Answer: BLEU (n-gram precision), ROUGE (recall-based for summarization), METEOR (harmonized precision/recall). But they correlate poorly with human judgment. Scenario: Used for machine translation and summarization.
What are LLM‑based evaluation methods (e.g., G-Eval, Prometheus)?
Answer: Using a strong LLM to evaluate outputs based on given criteria, often more aligned with human judgment. Scenario: G-Eval uses GPT-4 to score reasoning.
What is the Chain of Verification technique?
Answer: A method where the model verifies its own outputs by checking consistency across multiple generated answers, reducing hallucination. Scenario: Generate answer, then generate a verification question and check.
How do you evaluate RAG‑based systems?
Answer: Use end-to-end metrics (answer accuracy) and component metrics (retrieval recall, generation faithfulness). Also use human evaluation. Scenario: Use RAGAS or ARES frameworks.
What are the different metrics for evaluating LLMs?
Answer: Perplexity (language modeling), accuracy (task-specific), BLEU/ROUGE (generation), and human evaluation scores. Scenario: Perplexity for base models, accuracy for fine-tuned.
How do you ensure consistent and accurate outputs in multi‑step workflows?
Answer: Use deterministic settings, validate intermediate steps, and implement feedback loops with verification. Scenario: In a data extraction pipeline, validate each extraction step.
What is the role of human evaluation in LLM assessment?
Answer: Gold standard for measuring quality, but expensive and slow. Used for final validation and to train reward models. Scenario: Human raters judge safety and helpfulness.
9. Agents & Tool Use
What is an LLM‑powered agent?
Answer: An agent uses an LLM as its brain to plan, reason, and execute actions (like searching, calling APIs) to achieve goals. Scenario: AutoGPT, task automation.
How do you design a tool‑using agent?
Answer: Define tools (functions) with descriptions, provide them to the LLM in a structured format (e.g., JSON schema), and let the model decide when to call them. Scenario: A travel agent that can call flight APIs.
What is the ReAct framework for agents?
Answer: ReAct interleaves reasoning (thought) and acting (tool use) in a loop, enabling the agent to handle complex tasks. Scenario: An agent that searches for info, thinks, and takes more actions.
What is function calling and how does it work in LLMs?
Answer: The LLM is fine-tuned to generate structured outputs that call specific functions, which are then executed externally. Scenario: GPT-4's function calling API.
How do you handle multi‑step reasoning in agents?
Answer: Use a loop where the agent thinks, acts, observes results, and repeats until goal is met. Manage state and memory. Scenario: A scientific reasoning agent that runs experiments sequentially.
What is the difference between a tool and a plugin?
Answer: Tool is a generic function; plugin is a specific integration (e.g., browser plugin) that may include UI or authentication. Scenario: A calculator is a tool; a Slack plugin sends messages.
How do you manage state and memory in an agent?
Answer: Store conversation history, retrieved information, and intermediate results in a structured format within the context window. Scenario: Use a buffer or vector memory for long-term.
What are the challenges of building reliable agents?
Answer: Error propagation, infinite loops, cost, and fragility if the LLM makes bad decisions. Scenario: Agent might get stuck if it repeatedly calls a failing tool.
What is the difference between a deterministic and a stochastic agent?
Answer: Deterministic always makes the same decision for a given state; stochastic uses randomness (e.g., sampling) for exploration. Scenario: Stochastic agents can avoid repetitive loops.
How do you evaluate agent performance?
Answer: Measure task success rate, efficiency (steps taken), and cost. Also evaluate the quality of intermediate reasoning. Scenario: Success rate on benchmark tasks (e.g., WebShop).
What is the "agent‑tool" loop?
Answer: The cycle where the agent generates an action, calls a tool, gets a result, and decides next action. Scenario: The core of ReAct.
What are some common failure modes of LLM agents?
Answer: Not recognizing when to stop, hallucinating tool outputs, getting stuck in loops, and misinterpreting tool results. Scenario: Agent calls search repeatedly with same query.
10. Deployment & Optimization
How do you deploy an LLM in production?
Answer: Use inference servers (e.g., Triton, vLLM), containerization, API gateways, and load balancing. Choose deployment type (cloud, on-prem, edge). Scenario: Deploy LLaMA with vLLM for high throughput.
What are the main challenges in deploying LLMs?
Answer: High latency, memory footprint, cost, scaling, and maintaining consistency. Also model updates and versioning. Scenario: Serving a 70B model requires multiple GPUs.
How do you reduce inference latency?
Answer: Use model quantization, pruning, batch inference, speculative decoding, and optimized kernels (FlashAttention). Scenario: Use TensorRT for latency-sensitive apps.
What is speculative decoding?
Answer: Generate multiple tokens with a smaller draft model, then verify with the large model in parallel, speeding up generation. Scenario: Can speed up decoding by 2-3x.
What is KV caching and why is it important?
Answer: Stores past keys and values to avoid recomputation for each new token, crucial for autoregressive generation speed. Scenario: Without KV cache, generation is O(n^2).
How do you handle streaming responses?
Answer: Use chunked transfer encoding or server-sent events (SSE) to send tokens as they are generated, improving user experience. Scenario: ChatGPT streams responses token by token.
What is model parallelism and pipeline parallelism?
Answer: Model parallelism splits layers across devices; pipeline parallelism splits layers in stages for pipelined execution. Used for large models. Scenario: Megatron-LM uses both.
How do you serve multiple models efficiently?
Answer: Use model multiplexing, caching, or serverless functions. Also consider model compression and shared compute. Scenario: Serve both a small and large model, route based on complexity.
What is the difference between online and batch inference?
Answer: Online inference is real-time (low latency), batch inference processes many requests offline (high throughput, lower cost). Scenario: Chatbots use online; document summarization uses batch.
How do you monitor LLM performance in production?
Answer: Track metrics like latency, throughput, error rates, and output quality (e.g., toxicity, hallucination). Use logging and alerting. Scenario: Set alerts if hallucination rate exceeds threshold.
What is LLMOps and how does it differ from MLOps?
Answer: LLMOps focuses on LLM-specific challenges: prompt engineering, model evaluation, cost management, and safety monitoring. Scenario: MLOps for classical ML; LLMOps for generative models.
How do you handle versioning and A/B testing of LLMs?
Answer: Use model registries, canary deployments, and compare metrics (quality, cost) across versions using A/B tests. Scenario: Roll out new model to 10% of traffic for evaluation.
What are the cost considerations for running LLMs at scale?
Answer: Compute costs (GPU hours), inference cost per token, storage, and data transfer. Optimize with batching, quantization, and efficient serving. Scenario: For a high-traffic app, costs can be significant.
11. Quantization & Efficiency
What is quantization and why is it used for LLMs?
Answer: Reducing precision of weights (e.g., from FP32 to INT8) to decrease memory and increase inference speed with modest quality loss. Scenario: Run a 70B model on a single GPU with 4-bit quantization.
What are the different quantization levels (INT8, INT4, FP16)?
Answer: FP16 is half-precision (16-bit); INT8 (8-bit integer); INT4 (4-bit). Lower bits = smaller size but more quality loss. Scenario: INT8 is common; INT4 for extreme compression.
What is GPTQ and how does it work?
Answer: A post-training quantization method that adjusts weights to minimize error, often using layer-wise optimization. Scenario: GPTQ is used to quantize LLaMA models.
What is AWQ (Activation‑aware Weight Quantization)?
Answer: Quantizes weights based on activation distribution, protecting important weights to maintain accuracy. Scenario: AWQ achieves better quality than GPTQ at similar size.
What is the difference between quantization‑aware training and post‑training quantization?
Answer: Quantization-aware training simulates quantization during training, leading to better accuracy; post-training quantizes pre-trained model without retraining. Scenario: QAT is more accurate but expensive.
How does quantization affect model quality?
Answer: May cause accuracy drop, especially for smaller models or when using low bit widths. Techniques like GPTQ minimize the loss. Scenario: Quantization from FP16 to INT8 often loses <1% accuracy.
What is pruning and how is it applied to LLMs?
Answer: Removing less important weights (or neurons) to reduce model size. Can be unstructured (zeroing) or structured (removing entire channels). Scenario: Sparsity can speed up inference.
What is distillation in the context of LLMs?
Answer: Training a smaller student model to mimic a larger teacher model's outputs, achieving similar performance with less cost. Scenario: DistilBERT is a distilled version of BERT.
How do you balance model size, speed, and accuracy?
Answer: Choose the right model size for the task, use quantization/pruning for speed, and evaluate tradeoffs on a validation set. Scenario: For a real-time chatbot, use a 7B quantized model.
What is the memory footprint of a typical LLM?
Answer: Approximately 4 bytes per parameter (FP32) or 2 bytes (FP16). A 7B model in FP16 ~14GB, 70B ~140GB. Scenario: Quantization to 4-bit reduces memory to ~1/8.
How do you estimate GPU memory requirements for inference?
Answer: Model parameters * bytes per param + KV cache + activations. Use tools like `transformers` model memory calculator. Scenario: For 7B model in FP16, ~14GB model + 1-2GB cache.
12. System Design
Design a chatbot system using an LLM.
Answer: Use an LLM as the core, with prompt management, conversation memory, and fallbacks. Scale with load balancing and caching. Scenario: A customer support chatbot.
How would you build a document Q&A system with RAG?
Answer: Ingest documents, chunk, embed, store in vector DB. For each query, retrieve chunks, generate answer using LLM. Scenario: Internal knowledge base Q&A.
Design a system to process huge PDF reports with an LLM.
Answer: Extract text, split into chunks, summarize each chunk, then aggregate summaries. Use parallel processing. Scenario: Summarizing financial reports.
How do you design a system that handles millions of daily LLM requests?
Answer: Use microservices, autoscaling, request queueing (Kafka), and cost-aware routing to different model sizes. Scenario: Large-scale chatbot service.
What database fits the problem best — SQL, NoSQL, or vector?
Answer: SQL for structured data, NoSQL for flexible schemas, vector DB for embeddings. Often combine (e.g., PostgreSQL with pgvector). Scenario: Use Postgres for user metadata and vector for semantic search.
How do you design a multi‑tenant LLM service?
Answer: Use isolation (separate models or namespaces), rate limiting per tenant, and tenant-specific fine-tuning if needed. Scenario: SaaS platform serving different companies.
What are the key components of an LLM‑powered search engine?
Answer: Query processor, retriever (vector+keyword), re-ranker, and LLM summarizer/generator. Scenario: Enterprise search with natural language answers.
How would you architect a system for real‑time LLM applications?
Answer: Use streaming, low-latency inference (e.g., vLLM), and edge caching. Minimize overhead and use efficient networking. Scenario: Real-time translation or live captioning.
What are the tradeoffs between using a hosted LLM API vs. self‑hosting?
Answer: API: easy, low upfront, but cost and data privacy concerns. Self‑host: control, privacy, but high infra and maintenance costs. Scenario: For sensitive data, self-host.
How do you design fallback mechanisms for LLM failures?
Answer: Use circuit breakers, retries, fallback to a smaller model, or rule-based responses. Also monitor health. Scenario: If LLM times out, return a canned response.
13. Miscellaneous
What is temperature and how does it affect output?
Answer: Temperature controls randomness: lower (e.g., 0.2) makes output more deterministic; higher (e.g., 0.8) more diverse/creative. Scenario: Use low for factual QA, high for story generation.
What is top‑p (nucleus) sampling and when do you use it?
Answer: Top-p selects from the smallest set of tokens whose cumulative probability exceeds p, balancing diversity and quality. Scenario: Used with temperature for controllable generation.
What is top‑k sampling?
Answer: Samples only from the top-k most likely tokens, truncating the distribution. Scenario: Often used together with top-p.
Explain greedy decoding vs. beam search.
Answer: Greedy picks the highest probability token each step; beam search keeps multiple hypotheses (beams) and chooses the best overall. Scenario: Beam search for translation to avoid greedy errors.
What is the difference between temperature, top‑p, and top‑k?
Answer: Temperature adjusts probabilities, top-k truncates, top-p filters by cumulative probability. They can be used together. Scenario: Temperature first, then top-p.
How do you define stopping criteria in LLMs?
Answer: Use a max token limit, stop sequences (e.g., "\n\n"), or when the model outputs an end-of-text token. Scenario: Stop at "###" or after 100 tokens.
How do you use stop sequences?
Answer: Provide a list of strings; if the model generates any, generation stops. Helps control output format. Scenario: Stop at "END" to keep output concise.
What is prompt hacking and how do you defend against it?
Answer: Prompt hacking is manipulating prompts to elicit unintended outputs. Defend with input sanitization, system prompts, and monitoring. Scenario: An attacker injects "ignore previous instructions".
What is jailbreaking in the context of LLMs?
Answer: Circumventing safety filters to get the model to generate harmful content. Mitigated with alignment and robust filtering. Scenario: Using role-playing to bypass content policies.
What are the ethical considerations of deploying LLMs?
Answer: Bias, misinformation, privacy, job displacement, and environmental impact. Need transparency and accountability. Scenario: Ensure model doesn't generate racist content.
How do you ensure fairness and reduce bias in LLM outputs?
Answer: Use debiasing techniques, diverse training data, fairness metrics, and post-processing. Scenario: Monitor demographic parity in outputs.
What is data privacy and how does it apply to LLMs?
Answer: Privacy concerns include memorization of personal data. Use differential privacy, data anonymization, and avoid training on sensitive info. Scenario: Don't train on user conversations.
What is the role of synthetic data in training LLMs?
Answer: Synthetic data augments limited real data, but may introduce biases. Used for instruction tuning and safety training. Scenario: Generate instruction pairs with GPT-4.
What are the limitations of current LLMs?
Answer: Hallucination, lack of true reasoning, context length, cost, and inability to update knowledge without retraining. Scenario: They fail at long-term planning.
What is the future of LLMs and GenAI?
Answer: More efficient models, multimodal, better reasoning, longer contexts, and integration with the physical world (robotics). Scenario: Multimodal LLMs like GPT-4V.
14. Case Studies
How would you use an LLM to summarize financial reports?
Answer: Extract key metrics, compare quarters, and generate a concise executive summary with citations. Use RAG for accurate numbers. Scenario: Automated quarterly earnings summary.
Design a system for automated code generation and review.
Answer: Use an LLM to generate code from specs, then run static analysis and tests. Use another LLM to review for bugs and style. Scenario: AI pair programmer.
How would you build a customer support agent using an LLM?
Answer: Combine FAQ retrieval (RAG) with a chatbot. Handle common issues automatically, escalate complex ones to human. Scenario: E-commerce support bot.
How would you use an LLM for personalized education?
Answer: Adapt explanations to student's level, generate practice questions, and provide feedback. Track progress and adjust difficulty. Scenario: AI tutor for math.
Design a system for real‑time translation using LLMs.
Answer: Use a multilingual LLM, preprocess input, translate, and post-process. Optimize for latency with smaller models or quantization. Scenario: Live subtitles for meetings.
How would you implement an LLM‑powered recommendation system?
Answer: Generate user and item embeddings, use similarity search, and then use LLM to generate natural language recommendations with reasoning. Scenario: Personalized movie recommendations with explanations.
What are the challenges of using LLMs in healthcare?
Answer: Privacy (HIPAA), accuracy (medical errors), explainability, and bias. Requires rigorous validation and human oversight. Scenario: Clinical decision support.
How would you build a legal document analysis tool?
Answer: Use RAG on legal databases, extract key clauses, and summarize. Ensure compliance with legal standards. Scenario: Contract review system.
Design a system for automated content moderation.
Answer: Use LLM to classify content (toxic, spam, etc.) and apply rules. Combine with human review for borderline cases. Scenario: Social media moderation.
How would you use an LLM for creative writing assistance?
Answer: Provide prompts, suggest plot ideas, generate drafts, and edit for style and tone. Use few-shot to mimic authors. Scenario: AI co-writer for novels.
🎯 Land your dream job | FreeLearning365 — curated interview prep for developers
Visit Interview Portal
0 Comments
thanks for your comments!