Prompt Engineer Interview Questions

"I came in sideways. I was a technical writer, and our team started using a model to draft help articles. The first drafts were vague, so I began rewriting the prompts, and I kept a spreadsheet of inputs and what came out. That spreadsheet turned into a proper test set, and after a few months I was the person people sent their broken prompts to. I'd say my strongest skill is turning a fuzzy request like 'make it sound better' into something I can test, with examples of good and bad output agreed up front. This role looks like more of exactly that, across several products, which is why I applied."

"I'd agree that the trick-phrasing part of the job is shrinking, and I think that's a good thing. But most of what I do was never about magic words. It's getting a team to agree on what a good answer looks like, writing that down so the model and the reviewers see the same thing, building a test set, and working out why the output is wrong on the few cases that matter. A better model raises the floor, but someone still has to decide the format, the tone, the edge cases and the trade-off between quality and cost. In my last project, the biggest gain came from rewriting the definition of the task, not from any wording trick."

"The one I'm proudest of turned long insurance claim notes into a short handover summary for the next agent. Version one just said 'summarise the notes', and the summaries read well but skipped the open action, which is the one thing the next agent needs. Version two added a fixed structure: what happened, what's been promised, what's still open. That fixed most cases, but it invented dates when the notes had none. Version three told it to write 'no date given' instead of guessing, and added two short examples. I checked every version against about eighty real notes the team lead had labelled, and I counted how many summaries had the open action right and how many contained anything invented. Both counts improved at each step."

"First I'd ask what the manager uses the summary for, because that changes everything. Say they need to decide whether to act. Then I'd rewrite it roughly like this: you're writing for a manager with two minutes; start with a one-line bottom line; then up to three actions, each with an owner and a due date if the document gives one; then any risks it names. Keep the whole thing under about 120 words. If the document doesn't say who owns something, write 'not stated' rather than guessing. I'd put the document inside clear tags so the model can't mix it up with my instructions. Then I'd run it on five or six real documents before calling it done."

"The system prompt is for what stays true in every conversation: who the assistant is, the rules, the tone, the output format, and what to do when it doesn't know. The user message carries this particular request and its data, like the customer's email or the document. The split matters for a few reasons. Models are generally trained to give system instructions more weight, so rules there hold up better over a long chat. It keeps the stable part identical between calls, which helps with caching and with tracking versions. And it keeps untrusted text, like whatever a user pastes in, separate from my rules. One thing I never do is put secrets in the system prompt, because users can often coax the model into repeating it."

"I put the contract first, wrapped in tags with a clear label, and the question and instructions after it. For long inputs, some provider guides recommend the question at the end, and another suggests repeating the key instructions before and after the document, so I test both layouts on my own examples instead of assuming. Then I ask it to first pull out the exact clauses that relate to the question, and to answer only from those quotes. That points the model at the right text and gives the reader something to check. If details still get missed, especially from the middle of a very long document, I stop sending the whole thing. I either find the relevant sections first with search, or split the contract into parts, ask the question of each part, and combine the answers."

"I split when one prompt is juggling different jobs, like pulling out facts, making a decision and then writing a customer-facing reply, and the failures are hard to pin on any one part. As separate steps, each prompt is shorter and clearer, I can test each one on its own, and I can often run the easy steps on a smaller, cheaper model. The cost is more calls, so more latency, and a mistake in step one gets carried into every later step. So I pass structured output between steps and validate it before moving on. I don't split by default, though. If one prompt passes the test set and is fast enough, a chain is just more to maintain."

"A bare 'never' tells the model what to avoid, but not what to do when a customer asks about a competitor directly. So it may dodge awkwardly, or mention them anyway while trying to be helpful. I'd rewrite it with the action and the reason. Something like: if the customer asks about another company's product, say you can only speak about our products, then help with their underlying need using what we offer, because we can't make accurate claims about other companies. Now the model has a clear path. I'd also check whether that one line is really the issue, since a long list of don'ts can crowd the prompt. And I'd add test cases where the customer pushes hard on competitors, because that's where the rule really gets tested."

"I'd layer it. If the API supports structured outputs with a schema, I turn that on first, because it constrains what the model can produce instead of just asking nicely. A plain JSON mode is weaker: it gives valid JSON, but not always the fields I need. In the prompt I describe the schema plainly, give one short example, and say to return only the JSON. I keep the schema small and flat, and every field gets a rule for unknown values, like null instead of a guess. Then in code I never trust it. I parse it, validate it against the schema, and if it fails I retry once with the error included. If it fails again, I log the input and output and send that case to a person or a safe default. Those logged failures become new test cases."

"The model picks each next token from a probability distribution, and temperature reshapes that distribution. Lower values make it stick to the most likely tokens, so output is more focused and repeatable. Higher values give less likely tokens more of a chance, so output is more varied. For extraction, classification or anything feeding code, I keep it low. For brainstorming taglines or writing variations, I raise it. Top-p is a related setting that limits choices to the smallest group of likely tokens that covers a set share of the probability, and I usually tune one of the two, not both. Two cautions: low temperature doesn't make an answer right, it just makes the same answer more likely, and even at zero I wouldn't assume identical output every time. Some models don't expose these settings, so I check what the API allows."

"Models tend to fill a gap with something plausible, so I make not knowing a valid, expected answer. The prompt says to answer only from the provided text, and if the text doesn't contain the answer, to reply with an exact phrase like 'The document does not say.' The exact wording helps, and it makes those cases easy to count in code. I also ask it to quote or cite the part of the text it used, which makes invented answers easier to spot. Then I test it directly: I add questions to the test set that sound answerable but aren't covered by the text, and I measure how often it invents something versus declining. I watch the other side too, because if it starts declining questions it could answer, I've pushed too far."

"I'd start from real tickets, not ones I invent, and strip personal details first. I'd sample across a few months so seasonal issues show up. Then I'd make sure every category has enough examples, even the rare ones, because a set that's mostly billing will hide a broken account security label. I'd add awkward cases on purpose: tickets with two issues, very short ones, angry ones, other languages if we get them, and a few that try to mess with the prompt. For labels, I'd write a short guideline with the support leads, have two people label a slice, and look at where they disagree, because if people can't agree, the model can't either. Finally, I'd keep part of the set aside that I never look at while tuning, so my score isn't just me fitting the test."

"I break 'good' into a few things you can check separately: does it include the customer's main issue, is every fact supported by the call, is the next step clear, is it within the length. For each one I prefer pass or fail, or a small scale with a written description and an example for each point, because a one-to-ten quality score drifts between people. Then I test the rubric itself. Two people score the same thirty or so outputs, and where they disagree I tighten the wording. If I want a model to grade at scale, I score a sample by hand first and check how often the model grader agrees with us, especially on the failures. I also watch for graders that favour longer answers, and I recheck agreement whenever the grader prompt or model changes."

"First I'd gather the actual failures with their full inputs, the exact prompt version and the settings. Then I'd rerun each failing input maybe twenty times. That tells me which kind of problem I have. If the same input fails only sometimes, the prompt is probably ambiguous or has two instructions pulling against each other, and the model is choosing between readings. Lowering the temperature can hide that, but I'd rather fix the ambiguity. If certain inputs always fail, I look for what they share: very long, a missing field, an odd format, a case the instructions never mention. Then I make the smallest change that addresses it, rerun the whole test set, not just the failures, and add those inputs to the set so the bug can't quietly come back."

"At my last company, we had a prompt pulling order numbers and issue types out of customer emails. A support lead noticed a batch of tickets linked to the wrong order. I pulled the logs for that week and found the failing emails had one thing in common: they were replies in a thread, so they quoted older emails with different order numbers, and the model often grabbed the first number it saw. Our test set only had fresh single emails, so we'd never seen it. I changed the prompt to prefer the newest message and to return 'multiple orders found' when it wasn't clear, and we stripped quoted text in code before the model saw it. The bigger change was adding threaded emails to the test set and sampling real traffic every month to catch gaps like this."

"I start by being honest that no wording fully stops this, so I plan for it to sometimes work. In the prompt, I keep the rules in the system prompt, wrap the customer text in clear tags, and say that anything inside them is content to respond to, never instructions to follow. That helps, but the bigger protection is limiting what a successful attack can do. If the assistant can call tools, it only gets the ones it needs, with the customer's own permissions, and anything risky like refunds or sending emails needs a check in code or a person's approval. I also screen outputs, for example blocking replies that include internal notes or links we didn't supply. And I keep attack examples in the test set, including instructions hidden in documents or web pages the model reads, not only in the chat box."

"I'd start by collecting the refused requests and sorting them with a clinical lead: which ones were fine, which were rightly refused. Usually the model refuses because it doesn't know who it's talking to. So in the system prompt I'd say this is an internal tool for licensed clinical staff, that dose questions about medicines in our approved reference are in scope, and that answers should come from that reference, which we pass in, not from the model's memory. I'd also spell out what's still out of scope and why, with a couple of examples on each side of the line. Then I'd test on two sets together: the questions it wrongly refused, and the ones it should still refuse or send to a pharmacist. A fix that cuts refusals but lets risky answers through isn't a fix, and the clinical lead signs off on the result."

"I'd give it three inputs: the ticket, a short customer history, and the help articles found for this ticket. The instructions would say to answer only from those articles, in the brand's tone, which I'd show with one or two approved replies. It should always acknowledge the problem in one line, give the fix or the next step, and say what happens next. It must never promise a refund, a credit or a delivery date unless the article allows it, never make up a policy, and never mention internal notes. If the articles don't cover the question, it should tell the agent that and draft a holding reply instead of guessing. Since an agent reviews every draft, I'd also have it name the article it used, so they can check quickly."

"With code, vague context turns into confident wrong code, so I give much more of it: the language and framework, the versions that matter, the relevant files, and a short note on the team's conventions. I'm explicit about scope, like 'change only this function' or 'don't add new dependencies', and about the output format, for example a unified diff or the whole file, because that decides how we apply it. For reviews, I tell it what to focus on, like correctness and security, and to point to the exact line with a reason, or it pads the review with style nitpicks. The big difference from prose is that code can be checked automatically, so my evaluation is running tests and linters on the output rather than scoring it by eye."

"I'd first read twenty of them side by side with the marketing lead and name what's samey: usually the same opening, the same few stock adjectives, and very little that's specific to the product. Then I'd fix the inputs, not just the wording. I'd feed in real product facts and what customers actually say in reviews, add a few descriptions the team loves as voice examples, and give a short list of phrases to avoid. I'd also ask for different structures, like opening with a use case or a customer problem, and raise the temperature a little for this task. Marketing copy carries a risk too: the model can make claims the product can't back up. So every claim has to come from the product data, and marketing reviews a blind sample of old versus new."

"I was asked to build prompts for a team that handles freight shipping claims, and I knew nothing about shipping terms. In the first week I read about fifty closed claims end to end and sat with two claims handlers for an afternoon each, just listening to how they made decisions. I kept a glossary of their terms and the mistakes they said new staff make, like mixing up who's responsible at different points of the journey. That glossary went into the prompt as definitions, and their common mistakes became test cases. The key thing I learned was that I couldn't judge the outputs myself, so a handler scored the first rounds. After a few weeks I could spot most errors, but a handler still signed off every change to the scoring."

"I'd measure before touching anything: average input tokens, output tokens, and where the time goes. Output length usually drives latency most, since the model writes one token at a time, so I'd first ask whether we need all that text. Maybe a shorter format, or structured fields instead of paragraphs. Then the input: long instructions, lots of examples, and whole documents where only a section is needed. I'd trim those and rerun the test set after each cut. If the provider supports prompt caching, I'd move the stable parts to the start so they can be reused. Then routing: often most requests are easy, so a smaller, faster model can handle those and only the hard ones go to the big model. For users, streaming the answer also makes the wait feel shorter."

"I'd say yes to testing it this week and let the data decide. I'd run our full test set on the current and the new model with the same prompts, and compare quality scores, format failures, refusal rates, cost per request and latency. New models often behave differently in small ways, like being wordier or following an old workaround in the prompt too literally, so I'd expect to adjust a few prompts and rerun. If it's better or equal on what matters, I'd pin the exact model version in config, send a small share of real traffic to it first, and watch the same numbers plus user feedback. The old version stays one config change away. Next week is realistic if the tests are clean. If they aren't, I'd show my manager exactly which cases got worse."

"First I'd get the prompt out of wherever people are editing it live and into one place with history, usually the code repo, or a prompt registry if the team uses one. Every change goes through a small review, like a pull request, with a one-line reason and the test results attached. I'd have the test set run automatically on each change, so if someone improves the tone but breaks the format, we see it before it ships. Each production output gets logged with the prompt version and model version, so when quality drifts we can see exactly which change caused it and roll back in minutes. I'd also give the prompt one owner. Anyone can propose changes, but one person keeps the overall intent coherent."

"I wrote the prompt for a feature that suggested tags for new help articles. The first thing I sent the engineers was just the prompt text, and they came back with lots of questions: which model, what settings, what happens if the article is empty, what the output looks like when it's unsure. So I put together a proper handover: the prompt with a version number, the model and settings, the exact input and output format with a schema, the test set with expected results, and a short list of known weak spots. They wired the test set into their automated tests, so any later change from either side got checked. After launch we agreed I'd propose prompt changes through the same review process as code, which kept things calm."

"I'd start by agreeing the five results are promising, because they are. Then I'd explain what five examples can't tell us: how it handles rare cases, angry users or odd inputs, and whether it broke something the old prompt did well. I wouldn't say no to tomorrow. I'd offer to run it tonight on our existing test set, compare it with the current prompt, and share a short summary in the morning against a clear bar, like no drop on the categories that matter and no new format failures. If it passes, we launch to a small share of users first and watch the logs for a day. If something fails, the product manager sees the actual failing examples, which usually makes the decision easy for everyone."

"I was building a prompt that wrote first drafts for a legal team's internal questions. I thought the drafts were good: short and readable. The senior lawyer thought they were risky, because they stated conclusions without the conditions behind them. Instead of arguing in general, I picked fifteen real outputs and we went through them together. Every time she flagged one, I asked her to say exactly what was missing. By the end we had four written criteria, including 'names the conditions that would change the answer' and 'says when to check with a lawyer'. Some of my readable drafts failed, and she was right. We updated the prompt and the rubric together, and she became the person who approved changes to it, which made later updates much faster."

"Our sales lead wanted the assistant to work out custom price quotes straight from a customer's email. I tried it on thirty real emails, and it got the arithmetic or the discount rules wrong in a handful of them, which was too many for something going to customers. Rather than say models are bad at maths, I showed her three of the wrong quotes next to the right ones. Then I offered a split: the model reads the email and pulls out the products and quantities, which it did very well, and our existing pricing code does the calculation. She got most of the time saving she wanted, and the numbers were always right. I think it landed because I showed real examples instead of explaining how models work."

"I read the documentation and release notes for the models we actually use, because that's where real behaviour changes show up. I skim wider discussion too, but I treat tips from social media as ideas to test, not facts. A lot of them worked on one older model or one hand-picked example. When something looks useful, I try it on our own test set for a relevant task, and I only change a production prompt if it moves our numbers. Every month or so I write a short note for the team on what I tried and what did or didn't help, so we don't all run the same experiments. That keeps me learning without rewriting prompts every week."

"A good review starts with the reason for the change and the test results, so we're discussing evidence, not taste. Then I read the prompt as if I were the model with no context: is anything ambiguous, do two instructions conflict, is there a case it doesn't cover? When I give feedback I try to point at a real output. Something like, this ticket got the wrong category, I think because these two lines pull in different directions, what if we merge them? That's much easier to act on than saying it feels unclear. Over time I turn the comments that keep coming up into a short checklist the team uses, so reviews get quicker and the prompts start to read like one team wrote them."