_____ GENERATIVE AI GUIDE

Generative AI - known as Gen AI for short - is already shaping how people search, from Google to social platforms and everyday tools. As users switch from keywords to conversational questions, brands risk disappearing from the answer box if they don’t understand how Gen AI works.

In fact, research has found that AI Overviews now appear in 12.8% of Google searches. When they do, clicks to traditional results drop by over a third (34.5%). In this day and age, visibility isn't just about rankings anymore, but about being referenced in the answers themselves. 

As an experienced GEO agency, we’ve created this guide to cut through the AI jargon. It exaplins how Gen AI and large language models (LLMs) work, and what they mean for your search, content, and SEO strategies., breaking down the jargon and demonstrating what it means for search, content, and SEO teams.

Cutting through the jargon

What is the difference between AI, ML, LLM, NLP, Gen AI, and Neural Networks?

Since the explosion of ChatGPT in late 2022 and the rapid rise of other Generative AI tools, you've probably seen a lot of hype and terminology like AI, ML, LLM, NLP, and Gen AI being thrown around everywhere.

But what do they all actually mean? Let’s break it down.

AI

At the broadest level, AI (Artificial Intelligence) refers to the field of building machines that can perform tasks we typically associate with human intelligence, such as reasoning, learning, planning, and understanding language.

ML

Machine Learning (ML) is a key technique that allows computers to learn from data and improve their performance over time without being explicitly programmed. Many modern ML systems rely on neural networks, which are computational models inspired by the way the human brain processes information. These networks are particularly good at recognising complex patterns, and they form the backbone of today's most powerful AI systems.

NLP

Natural Language Processing (NLP) is a specialised branch of AI focused on enabling machines to understand, interpret, and generate human language. It underpins technologies like virtual assistants, translation apps, and automated summarisation.

LLM

A step further, we have Large Language Models (LLMs), massive AI models trained on huge amounts of text to understand and generate human-like language. ChatGPT is an example of an LLM, capable of responding to questions, writing stories, summarising documents, and more.

Gen AI

Generative AI (Gen AI) is the broader category of AI models designed to create new content, whether it’s text, images, music, code, or even video. LLMs fall under this umbrella, alongside tools like DALL·E (image generation) and Sora (video generation).

Understanding these terms is key to navigating today’s rapidly evolving tech landscape.

How do Gen AI models work?

At a high level, Generative AI works by learning patterns from massive datasets and then using those patterns to generate new content (such as text, images, or code) that resembles what it has seen. 

A common type of Gen AI is the Large Language Model (LLM), which is specifically trained on huge volumes of text from books, websites, and other written sources to understand and produce human-like language.

The core technology powering LLMs is a type of neural network called a transformer: a deep learning architecture designed to understand the relationships between words in context. You can think of it as a giant pattern-matching engine that learns not just word definitions, but how words relate to each other across vast contexts.

How LLMs work: An in-depth look

1. Training phase 

The process begins with training, where the model is exposed to trillions of words sourced from books, websites, code repositories, and other digital text data. However, the model doesn’t process text in raw form. It first undergoes a crucial step called tokenisation.

Tokenisation breaks down text into smaller units called tokens, which may represent full words, subwords, or characters, depending on the tokenisation algorithm. For instance, the sentence “The cat sat on the mat” might be tokenised into subword units like: ["The", "Ġcat", "Ġsat", "Ġon", "Ġthe", "Ġmat"], where the "Ġ" represents a preceding space.

Once tokenised, the model is trained using a technique called causal language modelling, where it learns to predict the next token in a sequence based on all prior tokens. So, for the input “The cat sat on the...”, the target is “mat.” The model doesn’t memorise exact sentences. Instead, through exposure to billions of such sequences, it begins to internalise language patterns, grammar, world knowledge, and even forms of reasoning. This stage involves adjusting billions (or even trillions) of parameters (internal weights in the model) using backpropagation and gradient descent to minimise the difference between its predictions and the correct next token.

2. Neural Network Architecture: The Transformer

At the heart of these models lies the Transformer architecture, a powerful neural network structure built specifically to handle sequential data like text. Unlike older models that processed words one at a time in order, Transformers can look at entire sequences in parallel, making them faster and more effective at capturing long-range dependencies in language.

The embedding layer converts each token into a high-dimensional vector capturing semantic relationships between words. Since Transformers don’t inherently understand word order, positional encoding is added to provide information about the position of each token in the sequence.

The core of the Transformer is the multi-head self-attention mechanism. This allows the model to weigh the relevance of all other words in the sequence when processing each token. For example, in the sentence “The cat, which was very fluffy, sat on the mat,” the model learns that “cat” and “sat” are closely related, even though several words come in between. Attention scores help determine which parts of the sentence are most relevant when making predictions.

Each attention block is followed by a feedforward neural network, and both components are wrapped in layer normalisation and residual connections to stabilise training. These layers are stacked repeatedly (12 layers in GPT-2, and 96 in GPT-4), allowing the model to build deeper, more abstract representations of language as it progresses through the network.

3. Generation phase: Predicting tokens step by step

Once the model is trained, it enters the generation phase, where it can produce new text based on a user-provided prompt. This is done one token at a time, using the internal logic and patterns it learned during training.

When a user inputs a prompt like “How does a rocket work?”, the model tokenises it, passes it through the Transformer layers, and produces a probability distribution over its vocabulary for the next most likely token. It selects a token based on that distribution, appends it to the prompt and repeats the process for the next token. So, a single sentence is built token by token:

Each word is chosen based on everything that came before it, allowing the model to maintain coherence and relevance throughout its response. Importantly, the model doesn’t think or plan, it predicts based on learned statistical patterns.

Whilst this example refers to generating and predicting sentences to explain and answer questions. From a search perspective, the model works exactly the same if you are asking questions like “What is the best accounting software in 2025?”. The model tokenises, runs through the transformer layers and produces a probability distribution for the next likely token, which will return sources and information it has been trained on. 

In short, a Generative AI model’s ability to produce fluent, human-like language stems from this intricate process: converting raw text into tokens, learning billions of contextual relationships through the Transformer’s layered attention mechanisms, and generating output one token at a time with remarkable sensitivity to language structure and meaning. The combination of vast training data, complex architecture, and probabilistic generation is what allows these systems to answer questions, write stories, code software, and so much more.

What it means for Search

There has been a significant shift in where people are turning to for information and how they’re finding it. With the rise of Gen AI models and their integration into search engines (e.g., AI Overviews and AI Mode in Google Search), users are shifting away from short, keyword-driven queries to more conversational, context-heavy questions.

For example, instead of “best cheap car 2024”, users are tending towards context-based questions such as “What are the best cars in 2024 for less than £200 per month?”. This reflects a deeper reliance on AI to interpret intent, understand nuance and generate direct, relevant answers all in one go. 

As a result, the need to click through various websites to piece together multiple pieces of information is diminishing. This naturally means a dip in traditional click-through rates. However, rather than spelling the end of SEO, this marks a shift: it’s no longer just about ranking for keywords, it’s about being cited in AI-generated responses and getting your brand embedded into the model's understanding of the world and your industry.

Ranking still matters, especially for driving traffic, but now earning a place in AI responses is just as crucial (likely more so in the years ahead). As AI algorithms focus so heavily on context and intent, it is more important than ever to be creating content that demonstrates genuine experience, expertise, authoritativeness and trustworthiness (E-E-A-T) that is well aligned with the intent behind what people are asking AI tools for help with. Ultimately, establishing your brand with high authority signals will also help to increase the chances of being cited in AI-generated answers, both at the training stage and when those AI models use search tools and features to find real-time sources to inform their answers.

From a technical standpoint, traditional SEO fundamentals still apply, but with a fresh lens. To increase your chances of being included in AI responses, make sure your content is structured and easy to parse. Implement schema markup to help AI understand the context and purpose of your content, and make pulling your content into its response quicker and cheaper for the LLM tool providers. And don’t forget the basics: site speed, mobile responsiveness, and clean UX still matter, not just for users, but because AI models are trained on what they can scrape and understand. It is an exciting time to be working in the field, it is important to continuously experiment with new strategies and monitor how AI affects site traffic and rankings in this fast-evolving landscape.

The focus has shifted from purely ranking for keywords to becoming a trusted source that AI models reference and surface in their responses. To stay competitive, it's crucial to create high-quality, well-structured content that demonstrates genuine authority and expertise, while ensuring your site remains technically sound and easy for AI systems to find and interpret.

As search continues to evolve, so too must our strategies. Those who adapt early will be best positioned to maintain visibility and drive meaningful conversions and traffic in this new era.

Sources

1. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., … Polosukhin, I. (2017). Attention is all you need (arXiv:1706.03762). arXiv. https://arxiv.org/abs/1706.03762 arxiv.org

2. Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., … Amodei, D. (2020). Language models are few-shot learners (arXiv:2005.14165). arXiv. https://arxiv.org/abs/2005.14165 arxiv.org

3. OpenAI. (2023). GPT-4 technical report (arXiv:2303.08774). arXiv. https://arxiv.org/abs/2303.08774 arxiv.org

4. Devlin, J., Chang, M-W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding (arXiv:1810.04805). arXiv. https://arxiv.org/abs/1810.04805 arxiv.org

5. Kudo, T., & Richardson, J. (2018). SentencePiece: A simple and language-independent subword tokenizer and detokenizer for neural text processing (arXiv:1808.06226). arXiv. https://arxiv.org/abs/1808.06226 arxiv.org

6. Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., … Scialom, T. (2023). Llama 2: Open foundation and fine-tuned chat models (arXiv:2307.09288). arXiv. https://arxiv.org/abs/2307.09288 arxiv.org

7. Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., … Liu, P. J. (2019). Exploring the limits of transfer learning with a unified text-to-text transformer (arXiv:1910.10683). arXiv. https://arxiv.org/abs/1910.10683 arxiv.org

8. Bengio, Y., LeCun, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436-444. https://doi.org/10.1038/nature14539 nature.com

9. Google. (2023, November). Search Quality Rater Guidelines: An overview [PDF]. https://services.google.com/fh/files/misc/hsw-sqrg.pdf services.google.com

10. Venkatachary, S. (2024, October 28). AI Overviews in Search are coming to more places around the world. Google Blog. https://blog.google/products/search/ai-overviews-search-october-2024/ blog.google

11. Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., & Sutskever, I. (2019). Language models are unsupervised multitask learners [Technical report]. OpenAI. https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf