What Is RAG and How Does It Decide Which Content to Cite?

Do you know that when you ask ChatGPT or Google's AI Overview a question, the answer you get is very often built from pages which the system has gone and fetched for you at that very moment?
This happens because of RAG, which is the method that a lot of AI search tools use to pull in information which is current and reliable before they write their answer. So once you get your head around how RAG works, it becomes a lot easier to see why your page shows up in one answer and is nowhere to be found in the next one.

The full form of RAG is Retrieval-Augmented Generation, and honestly the name gives most of it away, since there are two jobs going on here and one happens after the other.

Picture yourself asking a friend something they're not sure about. They'd Google it, skim a few pages and then explain it to you in their own words, right? RAG does pretty much that.

Your friend doing the Googling? That's retrieval, only the AI is digging through web pages, databases and APIs. Your friend explaining it back to you in plain words is generation, and in RAG that job goes to the large language model (LLM). So you're not stuck with some answer the model memorised ages ago.

Open their profiles and see what they've actually got. Then look at the bio. If a business has a website, it'll almost always put the link there. No link? There's a decent chance they don't have one at all.

You might not be aware, but it does not, because some questions can be answered from the memory which the LLM already has. Take "When was Gandhi born?" for example, the model already has that one in its memory, and the date is not going to change on it, so there is no point in it going off to search.

But ask it "Which is the best digital marketing institute in Hyderabad?" and it is a whole other thing, because courses get added, reviews come in and rankings move around every few months, so the system has to go and find sources that are current and that it can actually trust and point you to. This is exactly the point where RAG comes in.

In the first step, the system reads the query which the user has typed and works out what the intent behind it is. Moreover, it does not stop at the single query, and it expands it into several smaller searches which are called sub-queries, so that the answer it gives is complete.

For example, when you search for "best dental clinics in Hyderabad", the system may also look for the dental clinics in Hyderabad which are the most affordable, the ones which have won awards, the ones which patients trust the most and the ones which are considered the safest. It then fetches results for each of these from web search, databases, APIs or internal data, and so the final answer covers your question from more than one side.

In the second step, the system goes through the pages which it has found and pulls out the parts which actually answer the query, and it throws away any material which is repeated or duplicated.

The thing to note here is that it is looking for a direct answer. For example, if you have published a piece about CRM software in Hyderabad and the answer is written clearly near the top, the system can lift it easily, whereas if the answer is buried under several paragraphs of build-up, there is a good chance that it will be passed over.

It is recommended that you place a short and direct answer in your introduction and above the fold, and also in your headings, your subheadings, the first paragraph under each heading and your schema markup, so that the AI has a clear answer which it can quote.

Say two clinics have both written about root canal costs in Hyderabad. One page actually answers the cost question, it's on a site people trust and it was updated last month. The other wanders around the topic and hasn't been touched since 2022.

You can guess which one wins, and that's relevance, authority and freshness doing their thing. Then there is the question of what other people are saying about you, and that is where your backlinks, citations and mentions on other sites come into the picture. Last of all it checks diversity, or whether the sources it has picked, put together, give the reader more than one side of the story.

In the last step, the LLM takes the information which it has gathered from several sources, cross-checks the facts and verifies the sources, and then it writes a summary which becomes the overview that you see. So what this means is that if your content has made it through the earlier steps, it may be cited as one of the sources in that answer.

There is no trick to this, and it mostly comes down to making your page easy for the system to trust and easy for it to quote.

Keep updating your pages, and don't hide the "last updated" date at the bottom where nobody, not even the AI, is going to spot it. Get the main answer in right at the top and then only add what your reader needs on top of that, since repeating one point three times on the same page really doesn't help you here.

Schema is worth adding as well, as long as it actually matches what is written on that page and you are not just stuffing it in. And then work on getting mentioned and cited on websites that are relevant to your field and that people take seriously. None of these on its own guarantees a citation, but put together they push your odds up quite a bit.