LLM Intent Classification: A RAG Pipeline for Keyword Research
Most LLM intent classification stops at a demo. Ground the classifier in the live SERP, emit a four-axis intent vector, and make intent auditable.
LLM intent classification promises to replace brittle, rule-based intent tags with models that read a query the way a searcher means it. Most implementations stop at a demo: embed some keywords, retrieve a few neighbours, ask a model to guess. This guide argues for a sharper design. Ground the classifier in the live search results page, emit intent as a calibrated vector rather than a single label, and treat the whole pipeline as a measurement system you can audit — not a clever prompt you run once and trust.
Why LLM Intent Classification Beats Heuristic Tags
Heuristic intent tagging solves a problem that no longer exists. The old model assumed a query maps to one of three tidy buckets — informational, commercial, or transactional — a taxonomy that traces back to Andrei Broder's 2002 work on web search and still underpins most keyword tools. That assumption held when a results page was ten blue links. It collapses when a single query returns an AI overview, a product carousel, a video pack, and a People Also Ask block stacked on top of each other.
LLM intent classification changes the unit of analysis. Instead of matching a query against a keyword list, you retrieve evidence about how the search engine itself is currently answering that query, then let the model infer what the searcher is trying to accomplish. Retrieval-augmented generation is the mechanism that makes this reliable: it pairs a model's parametric memory with an explicit, updatable retrieval index, which is exactly what Lewis and colleagues designed RAG to do in their 2020 paper — give a language model provenance for its decisions and a way to manipulate knowledge it was never trained on.
The common pushback is that this is heavy machinery for a job a regular expression used to handle. Here is where that reasoning breaks down. A regex cannot tell you that "best crm for nonprofits" has quietly shifted from a listicle results page to a comparison-table results page, which changes the winning format overnight. A grounded model can, because its input is the live page, not a frozen label from the last crawl.
Architecture: Building an LLM+RAG Intent Pipeline
The architecture is a six-stage flow, and each stage has exactly one job. Ingest pulls the raw material: the seed terms you would build with any keyword research workflow, plus, for each query, a snapshot of the live results page — the organic titles, the SERP feature mix, and the People Also Ask questions. Embed turns those artifacts into vectors. A vector store holds them. A retriever assembles the most relevant evidence for a given query. The classifier reads that evidence and emits an intent vector. Export writes the result into your keyword groups and briefs.
Embeddings and Vector Stores
Two decisions dominate this layer: which embedding model, and where the vectors live. Resist the urge to default to whatever model your LLM vendor happens to ship. The Massive Text Embedding Benchmark evaluated 33 models across 8 tasks and 58 datasets and found that no single embedding model dominates every task — retrieval, clustering, and classification reward different models. Benchmark two or three on a labelled sample of your own queries before you commit to one.
For storage, you rarely need a dedicated vector database on day one. pgvector adds exact and approximate nearest-neighbour search directly to Postgres, with cosine, inner-product, and L2 distance operators, so your embeddings sit beside the keyword metadata you already store and stay inside ordinary JOINs and transactions. Reach for a specialised index like FAISS only when latency or corpus size genuinely demands it — not because a demo told you to.
Retrieval Orchestration and Prompt Templates
Retrieval orchestration decides what evidence reaches the model and in what shape. For intent inference, retrieve three things per query: the top organic titles and meta descriptions, the list of SERP features present, and the two or three nearest queries whose intent you have already labelled. Keep the passage count small. More context is not more signal, and it quietly inflates cost on every call.
The prompt template is where most pipelines leak accuracy. A robust intent-classification prompt states the task plainly, hands the model the retrieved SERP evidence, and constrains the output to a fixed schema: a probability for each intent axis plus a one-line justification that must quote a retrieved feature. Constraining the shape is what keeps the pipeline stable as SERP layouts drift — the model reasons over whatever features it was handed instead of inventing a category. Version the template like code, because a prompt change is a schema change in disguise.
Operationalizing Intent Outputs for Keyword Grouping and Briefs
A single intent label throws away the most useful part of the output: the runner-up. Treat the model's response as a four-axis vector across informational, commercial, transactional, and navigational intent, each with a probability. The dominant axis sets the page format; the runner-up sets the section that wins the modern results page — the comparison table inside a how-to, the pricing block inside a definition. This is the same reasoning we develop in our breakdown of search intent as a vector, now produced automatically rather than by hand.
From there, the mapping to editorial artifacts is mechanical. Feed the vector plus the retrieved evidence into a second prompt — the brief-field extractor — whose only job is to populate structured fields: the working title's intent, the primary user need, the awareness stage, and a recommended format. A third prompt handles cluster labelling: given a group of queries and their vectors, it returns a human-readable cluster name and the single pillar the cluster should map to. Priority scoring falls out of the vector too — a high transactional probability paired with a weak incumbent page is a better target than a purely informational query with an entrenched answer. Our guide to keyword clustering for content planning covers the grouping mechanics this stage automates.
Measurement, Validation, and Feedback Loops
Here is the part the demos skip. A classifier is only as good as your ability to prove it agrees with real searcher behaviour, and to notice when it stops. Start with a labelled validation set: a few hundred queries a human has tagged. Track agreement between the model's dominant axis and the human label, but weight errors by cost — mislabelling a transactional query as informational sends a buyer to a blog post, which is an expensive miss.
The stronger signal comes from downstream behaviour. For pages already published against a predicted intent, correlate the prediction with click-through rate and engaged sessions from Search Console and analytics. When a page underperforms its predicted intent, that underperformance becomes a labelled example for your next evaluation round.
This is where grounding earns its keep. Because every classification is tied to a dated SERP snapshot, you can diff intent over time: when "project management software" drifts from informational to transactional, the pipeline shows you the exact retrieval that changed the call. Intent stops being an opinion the model regenerates differently next month and becomes a measurement you can version, audit, and hand to a writer with confidence. The same discipline applies whether you determine search intent by hand or at the scale of a whole content plan.
Actionable Implementation Checklist and Code-First Templates
You can stand up a working pipeline in four weeks without a research team. The point of a phased rollout is to get a grounded, auditable signal into one writer's hands early, then widen. Anticipate the objection before it lands: this is not a moon-shot, it is four contained deliverables.
- Week one — instrument ingestion. For a seed list of a few hundred queries, capture the live SERP for each: organic titles, the feature mix, and People Also Ask. Store every snapshot with a timestamp so it stays diffable later.
- Week two — embed and store. Pick an embedding model off a public benchmark, write vectors into a pgvector table beside your keyword metadata, and confirm nearest-neighbour retrieval returns sane matches on a handful of known queries.
- Week three — classify. Ship the intent-classification prompt with the fixed vector schema, run it across the seed list, and build the labelled validation set in the same pass so you can measure from day one.
- Week four — operationalize. Wire the brief-field extractor and the cluster-labeller, export into your content plan, and put the output in front of one writer for a real brief. Widen only after that writer trusts it.
The code-first core is smaller than it looks. Enabling vector search is a single CREATE EXTENSION statement, and a nearest-neighbour lookup is one ORDER BY on the cosine-distance operator against your embedding column. The OpenAI Cookbook's embedding and retrieval examples are a faithful starting point for the model calls around that query. Everything else is plumbing you already own.
Risks, Limitations, and When to Fall Back to Heuristics
Grounding reduces hallucination; it does not abolish it. Three failure modes deserve explicit guardrails. Model hallucination: a classifier can invent a justification the retrieved evidence does not support, so require that justification to quote a retrieved feature and reject responses that do not. SERP noise: personalised or volatile results can skew a single snapshot, so sample a query more than once before you trust a large intent shift. Cost and latency: running full retrieval and inference on every query is how these pipelines quietly die in production.
The fix is a confidence router, and it doubles as the honest answer to when heuristics still win. Most head terms are unambiguous — a regex tags them correctly and for free. Send only the ambiguous middle band, the queries where the model's top two axes sit close together, through the full LLM and RAG path. You spend inference where it changes a decision and nowhere else. Heuristics are not the enemy; they are the cheap first pass that keeps grounded classification affordable. And on anything touching health, finance, or safety, keep a human in the loop regardless — Google's guidance ties ranking to demonstrated experience and trust, which no classifier can manufacture.
How VarynForge Fits In
Building this end to end is a real project, and most teams want the output, not the pipeline. VarynForge already runs grounded, SERP-aware intent analysis and turns it into prioritized keyword clusters and writer-ready briefs, so you get the intent vector and the brief fields without standing up embeddings and a vector store yourself. If you would rather ship briefs than maintain infrastructure, see what the platform includes.
Key Takeaways
Model-driven intent is not about a smarter label; it is about a better system. Ground the classifier in the live results page so its input is current, emit a four-axis intent vector so you keep the runner-up that decides format, and tie every call to a dated snapshot so intent becomes auditable rather than disposable. Route cheap heuristics on the unambiguous majority and spend inference on the ambiguous middle. Do that, and keyword research stops guessing at intent and starts measuring it.
Further Reading
- FAISS: efficient similarity search for dense vectors
- OpenAI Cookbook: embeddings and retrieval examples
- Google Search Central: supported structured data markup
- Google Search's guidance on AI-generated content
Sources
Frequently asked questions
What is retrieval-augmented generation, and why does it help infer search intent?
Retrieval-augmented generation, or RAG, pairs a language model's built-in knowledge with an external, updatable index that it queries at run time. Instead of relying only on what the model memorised during training, the pipeline retrieves relevant, current evidence and hands it to the model as context before it answers. For search intent this matters because intent is not a fixed property of a keyword; it is revealed by how the results page is answering the query right now. RAG lets the classifier read that live evidence, so its judgement reflects the current SERP rather than a stale assumption baked in months ago. It also gives you provenance: every classification points back to the specific evidence that produced it.
How do embeddings tell apart similar keywords like buy and compare?
An embedding maps a piece of text to a point in a high-dimensional space where semantically similar items sit close together and different ones sit far apart. Two keywords that look similar on the surface can land in very different regions once the model accounts for the context they usually appear in. A term with commercial urgency clusters near other transactional phrases and near results pages full of product listings, while a research-style term clusters near guides and comparison content. The pipeline does not decide intent from the keyword string alone. It embeds the query alongside the retrieved SERP evidence, so the surrounding signals, not just the words, drive where the query lands and which intent it most resembles.
Which embedding model and vector database should I use for an intent pipeline?
There is no universally best embedding model. Public benchmarking of dozens of models across many tasks has shown that retrieval, clustering, and classification each reward different models, so the right move is to test two or three candidates on a labelled sample of your own queries and pick the one that separates intent classes cleanly. For storage, start simple. If your keyword data already lives in Postgres, a vector extension lets you store embeddings beside that data and run nearest-neighbour search with standard SQL, which keeps everything inside familiar transactions and joins. Only graduate to a specialised vector database when query latency or corpus size genuinely forces the move. Buying scale you do not need is a common early mistake.
How do I turn intent probabilities into keyword priority and brief fields?
Treat the classifier output as a vector of probabilities across informational, commercial, transactional, and navigational intent rather than a single winning label. The dominant axis decides the page format. The runner-up axis decides which secondary section you need, such as a comparison table inside a how-to guide. Priority scoring combines the vector with SERP difficulty: a strong transactional signal against a weak incumbent page is a higher-value target than an informational query with an entrenched answer. For briefs, feed the vector and the retrieved evidence into a second prompt that populates structured fields: the working title's intent, the primary user need, the awareness stage, and the recommended format. Each field traces back to a specific signal, so writers understand why the brief says what it says.
Which metrics prove that LLM-inferred intent actually improves results?
Use two layers of measurement. The first is agreement: hold out a validation set of queries a human has labelled and track how often the model's dominant intent matches the human call, weighting mistakes by business cost since sending a buyer to a blog post is worse than the reverse. The second layer is downstream behaviour, which is the stronger evidence. For pages you have already published against a predicted intent, correlate that prediction with click-through rate and engaged sessions from Search Console and your analytics. When a page underperforms its predicted intent, that gap becomes a fresh labelled example that sharpens the next evaluation round. Accuracy on a benchmark is a starting point, not proof; real behaviour closes the loop.
What are the common failure modes of LLM intent classification, and how do I mitigate them?
Three failure modes recur. First, hallucinated justifications: the model asserts an intent the evidence does not support. Require it to quote a specific retrieved feature and reject answers that cannot. Second, SERP noise: personalised or volatile results skew a single snapshot, so sample a query more than once before trusting a large intent shift. Third, runaway cost: running full retrieval and inference on every query is unsustainable at scale. Route by ambiguity instead. Cheap heuristics handle unambiguous head terms for free, and only the queries whose top two intent axes sit close together pay for the full retrieval and model path. On sensitive topics like health or finance, keep a human reviewer in the loop regardless.
