Give AI the Right Context: Source Material and Long Documents

Ask an assistant, "What does my college's attendance policy say about medical leave?" and it will happily answer. It has never seen your college's policy. It will produce what a typical attendance policy says, in the confident voice of someone who has read yours.

The fix is almost embarrassingly simple: give it the policy. But how you give it, and what you ask it to do with it, decides whether you get a reliable answer or a more convincing guess.

• What grounding is and why it is the most effective single technique for factual tasks.
• A reusable pattern for supplying sources: label, instruct, quote, cite.
• How to work with documents that are too long or messy.
• How to check that the answer really came from the source.
• What can go wrong even when you supply the material.

There are two ways to ask a question:

• From memory: "What does the policy say?" The model answers from patterns in its training. It cannot see your document, so it is guessing what documents of that type usually say.
• From the page: "Here is the policy. What does it say about medical leave?" The model reads what you gave it and answers from that text.

The second is called grounding: tying the answer to specific source material. It does not make the model infallible, but it changes the job from recall to reading comprehension, and reading is both easier for the model and much easier for you to verify. You can compare the answer with the page.

Use this four-part pattern whenever facts matter.

1. Label and separate the material. Make it obvious where your instructions end and the source begins. Delimiters work well: markdown headings, triple dashes or tags. OpenAI's prompting guide mentions using Markdown headers and XML tags to create logical boundaries, and Anthropic's examples wrap documents in tags. Pick one style and use it consistently.

2. Say what the model may use. For example: "Answer using only the text between the tags. Do not use outside knowledge."

3. Give it a safe way out. "If the text does not answer the question, write 'not stated in the text'." Anthropic's guidance on reducing hallucinations recommends permitting the model to say it does not know, because otherwise guessing is the only way to produce an answer.

4. Ask for evidence. "For each point, quote the exact sentence that supports it." The same documentation recommends having the model extract word-for-word quotes first for long documents, then base its analysis on them, and verify claims with citations, retracting any that it cannot support.

Together:

Task: answer my question using only the document below.Rules:- If the document does not say, write "not stated".- For every point, give the exact quote that supports it, in quotation marks.- Do not add background knowledge.Question: Can I get an attendance exemption for medical leave, and what proof is needed?<document>[paste the policy section here]</document>

Goal. Find out whether you can get an attendance exemption.

Initial request, with no grounding.

Does my college allow attendance exemptions for medical leave?

What goes wrong. The assistant cannot know. It may answer, "Yes, with a medical certificate, typically within seven days." It sounds plausible, and it is a guess based on typical policies.

Improved request. The pattern above with the real section pasted in.

Example response (illustrative, as the real answer depends on your document).

Yes, in some cases. The document says: "Students with a medical certificatefrom a registered practitioner may apply for condonation of shortage of up to10 percent." Proof needed: "a medical certificate" (the text does not say howmany days after the absence it must be submitted: not stated).

Verify. You search the document for the quoted phrase. Two failure modes are possible and both are easy to spot: the quote does not exist (invented), or it exists but says something different from the claim (misread). Here, you also notice that the answer admits one detail is "not stated". That is the safe way out working correctly. You then go to the college office for that missing detail instead of accepting a guess.

The workflow. Ground, require quotes, allow "not stated", then verify quotes against the page. Four steps, and each one removes a way for the answer to go wrong.

Bigger context windows let you paste whole reports, but longer is not automatically better. As the "Lost in the Middle" study found, models often use information at the start and end of a long input better than information in the middle. Newer models may differ, but the safe approach is the same.

Three strategies:

1. Focus. Paste only the sections that matter. If you do not know which, ask the model to list the section headings and tell you which are likely to be relevant, then paste those. Fewer, more relevant tokens beat more, noisier ones.
2. Stage the work. First request: "Extract every sentence that mentions deadlines, exactly as written, with section numbers." Second request, using that extract: "Now list the deadlines in date order." The extraction can be checked against the document, and the second step works on a short, reliable list.
3. Split and combine. Process the document in parts (chapters, sections), take notes per part, then ask for a combined view. Keep each part's quotes so you can trace claims back.

A good habit is to make long-document answers navigable: ask for section numbers or page references, so you can jump to the source in seconds.

• Relevance beats volume. Remove boilerplate, signatures, unrelated emails, old threads.
• Say what each source is. "Source A: official circular dated 12 March. Source B: a news article." The model can then reason about reliability, and so can you.
• Include dates. Add "today is 5 October 2026" for time-sensitive work, and say how recent each source is.
• Handle conflicts explicitly. "If the sources disagree, show both versions and do not pick one."
• Keep a clean thread. If a long chat is cluttered or has gone off track, start a new one with a short summary of the key facts.
• Do not supply what you should not share. Anything you paste is processed by the product. If the document is confidential or contains personal data, check your policy first.

Grounding reduces errors but does not remove them:

• Misreading. The answer cites a real quote but draws the wrong conclusion from it.
• Selective reading. It found one relevant sentence and missed a later exception ("except where...").
• Hidden gaps. The retrieval step or the attachment parser missed part of the material.
• Bad sources. The text you gave is wrong, out of date, or one-sided. The model has no way to know.
• Hostile text. A pasted web page or email can contain instructions aimed at the model ("ignore the above and..."). This is called prompt injection, and article 28 covers it. Be especially careful with text from sources you do not control.

The honest summary: grounding moves the problem from "did it invent this?" to "did it read this correctly?", and the second question is cheaper to answer.

• Asking about a document without supplying it, or assuming a link is enough. The model may not open links, and in some products it will pretend to.
• Pasting the whole thing when only one section is relevant.
• Skipping the safe way out, so the model feels obliged to answer.
• Trusting a paraphrase when a quote is available. Ask for quotes on anything important.
• Not checking the quote. Quotes can be invented too. Search for them.
• Forgetting the date. The same question can have a different answer after a rule change.

Answer using only the terms below. If the terms do not say, write "not stated".For each rule, quote the exact sentence. Include any conditions or exceptionsthat appear in the same section. Question: what are the refund rules?<terms>[paste the terms]</terms>

• Next: Examples, Output Formats and Structured Output teaches you to shape the answer, including output you can check by program.
• For the full checking routine, see How to Fact-Check AI-Generated Answers.
• For long documents in practice, Meetings, Notes and Long Documents: Summaries You Can Trust applies this pattern.