Lesson 2: Inside ChatGPT - How it Thinks

Why Understanding ChatGPT Matters

ChatGPT has become an integral part of our digital landscape, powering everything from writing assistants to customer service platforms. By understanding how this AI system "thinks," we can better leverage its strengths, recognize its limitations, and use it more effectively in our daily lives.

This knowledge transforms us from passive users into skilled collaborators with AI technology. For instance, by understanding how ChatGPT interprets prompts, you can craft questions that yield more precise answers. When we comprehend how ChatGPT generates answers, we can craft more effective prompts, critically evaluate its responses, and make informed decisions about when and how to incorporate AI assistance into our workflows.

‍

Learning Objectives

By the end of this lesson, you will be able to:

• Describe the Transformer architecture that serves as ChatGPT's foundation
• Explain the three-phase training process that shapes ChatGPT's capabilities
• Understand how the model processes language through tokenization and maintains conversational context
• Explain the probability-based method ChatGPT uses to generate responses
• Identify ChatGPT's key limitations and why it sometimes produces incorrect information

‍

Key Concepts in AI Language Models

Before exploring ChatGPT's inner workings, let's establish a foundation of essential concepts:

Machine Learning (ML): A branch of artificial intelligence where systems learn from data to make predictions or decisions, rather than following explicit programming instructions. ML systems improve their performance as they encounter more examples, gradually identifying patterns and relationships within data.

Neural Network: A computing architecture inspired by the human brain's network of neurons. These systems consist of interconnected "nodes" organized in layers that process information. During training, neural networks adjust the connections between nodes (called "weights") to recognize patterns and make predictions.

Transformer: A specific neural network architecture introduced in 2017 that revolutionized natural language processing. Transformers use a mechanism called "self-attention" to process all words in a sequence simultaneously, allowing the model to consider relationships between words regardless of their position in the text.

Natural Language Processing (NLP): The field focused on enabling computers to understand and generate human language. NLP tasks include translation, sentiment analysis, summarization, and conversation.

‍

‍

The Architecture: Understanding Transformers

At the core of ChatGPT lies the Transformer architecture—indicated by the "T" in GPT (Generative Pre-trained Transformer). This breakthrough design revolutionized how AI systems process language.

Inside the Transformer

The Transformer architecture consists of multiple layers of specialized components:

• Input Embedding: Converts words into numerical vectors that the neural network can process
• Positional Encoding: Adds information about word position, since the parallel processing would otherwise lose sequence information
• Multi-Head Self-Attention: The key innovation that allows the model to focus on different aspects of the input simultaneously
• Feed-Forward Networks: Process the attention outputs through standard neural network layers
• Layer Normalization: Stabilizes the learning process across deep layers

These components are stacked in multiple layers (GPT-3 has 175 billion parameters across many layers), allowing the model to build increasingly sophisticated representations of language. The early layers might capture basic syntactic patterns, while deeper layers develop more abstract semantic understanding.

The self-attention mechanism works by creating three representations of each word—query, key, and value vectors—and then computing attention scores that determine how much each word should focus on every other word. This happens across multiple "attention heads" that can each specialize in different types of relationships.

Self-Attention: How Transformers Process Language

Unlike earlier models that analyzed text word-by-word in sequence (like humans reading), Transformers evaluate entire text passages simultaneously through a mechanism called self-attention. This allows the model to:

• Process words in parallel rather than sequentially
• Identify relationships between words regardless of distance
• Recognize which words are most relevant to understanding meaning
• Maintain awareness of context throughout a passage

Imagine reading a paragraph and instantly knowing which sentences relate to each other, rather than reading word-by-word. For example, in the sentence "The cat, which was hungry, sat on the mat," a Transformer can directly connect "hungry" with "cat" despite the words between them. This capability to maintain long-range dependencies is crucial for understanding complex language. If you're at a party and trying to follow multiple conversations at once, your attention shifts to what’s most relevant at the moment. Transformers do something similar.

‍

‍

The Training Process: From Raw Text to Helpful Assistant

ChatGPT's capabilities emerge through a sophisticated three-phase training process, each building upon the previous to develop increasingly refined abilities.

‍

Phase 1: Pre-training (Learning Language Patterns)

The foundation of ChatGPT's knowledge comes from exposure to an enormous corpus of internet text—including books, articles, websites, and forums. During pre-training, the model develops a statistical understanding of language by repeatedly attempting to predict the next word in sequences.

This process works as follows:

1. The model is shown a segment of text with the final word masked
2. It must predict the missing word based on context
3. The system compares its prediction with the actual word
4. The model's internal parameters adjust to improve future predictions
5. This cycle repeats billions of times across diverse texts

Like a student checking their answers against a key and learning from mistakes, through this intensive training, the model gradually internalizes grammatical rules, factual information, common reasoning patterns, and cultural references. At this stage, the model functions as a general-purpose language model with broad knowledge but no specific alignment toward being a helpful assistant.

‍

Phase 2: Supervised Fine-Tuning (Learning to Be Helpful)

After pre-training, the model is refined through supervised fine-tuning (SFT) to shape its behavior toward being a helpful assistant:

• Human AI trainers create a dataset of example conversations
• Each example includes a prompt and an ideal, helpful response
• The model is trained to generate responses that match these expert demonstrations
• This teaches the model to follow instructions and provide useful information

For instance, if the prompt is "How do I boil an egg?", the demonstration might show a clear, step-by-step explanation. Through exposure to thousands of such examples, the model learns to produce helpful responses rather than just statistically likely text completions. This is what differentiates ChatGPT from the autocomplete feature on smartphones and word processors.

‍

Phase 3: Reinforcement Learning from Human Feedback (Learning Human Preferences)

The final training phase uses Reinforcement Learning from Human Feedback (RLHF) to further refine the model's behavior according to human values and preferences:

1. Comparison Data Collection:
   
   • The model generates multiple possible responses to a prompt
   • Human evaluators rank these responses from best to worst
   • These rankings create a dataset of human preferences
2. Reward Model Training:
   
   • A separate "reward model" is trained to predict human preferences
   • It learns to score responses based on helpfulness, accuracy, safety, and other desired qualities
3. Policy Optimization:
   
   • The main ChatGPT model is further trained using reinforcement learning
   • It generates responses, receives scores from the reward model, and adjusts to maximize these scores
   • This process continues until the model consistently produces highly-rated responses

RLHF is crucial for addressing problems like harmful outputs, bias, or hallucinations (more on this below). It's what enables ChatGPT to decline inappropriate requests, admit uncertainty, and generally behave in alignment with human values. This is also why it's important for you to give a "thumbs up" or "thumbs down" to responses; this is literally human feedback that ties into the model's reinforcement learning!

‍

Tokenization and Context: How ChatGPT Processes Text

Before ChatGPT can work with any text, it must first convert human language into a format the neural network can process. This conversion happens through a process called tokenization.

‍

Understanding Tokens: The Building Blocks of Language Processing

Tokens are the fundamental units that ChatGPT works with—they can be whole words, parts of words, or even individual characters. Think of tokens as puzzle pieces. The model's tokenizer applies specific rules to break text into these manageable pieces:

• Common words like "the" or "and" often become single tokens
• Longer words, like those with prefixes/suffixes or compound words, get split into multiple tokens (e.g., "unprecedented" might become "un" + "precedent" + "ed")
• Numbers and punctuation marks typically form their own tokens or follow unique token patterns

For example, the sentence "Hello, how are you doing today?" might be tokenized as: ["Hello", ",", "how", "are", "you", "doing", "today", "?"]

The model doesn't actually see the words themselves but rather numerical IDs that correspond to each token in its vocabulary. Just like breaking a sentence into syllables or words for easier pronunciation, tokenization breaks text into manageable pieces for the AI to understand.

‍

Managing Conversation Context

ChatGPT's ability to maintain coherent conversations relies on its context management system. When you chat with the model:

• Context Window: The model has a fixed-size "window" of tokens it can consider at once (typically a few thousand tokens).
• Conversation History: Each time you send a new message, the system combines your current message, relevant previous messages, and system instructions that guide the model's behavior.
• Token Management: If a conversation exceeds the context limit, earlier portions may be summarized or removed to make room for new content while preserving essential information.

This context mechanism enables ChatGPT to reference earlier parts of your conversation, answer follow-up questions, and maintain a coherent thread across multiple exchanges. However, this only works within its context window limits. Once information falls outside this window, the model effectively "forgets" it unless reminded, which can lead to all kinds of issues.

‍

Word Prediction: How ChatGPT Generates Responses

Remember earlier when I said that ChatGPT is different from the autocomplete function on your phone? Well, that's actually not too far off, despite the unique differences mentioned earlier; at its core, ChatGPT generates text through a sophisticated process of predicting the most likely next word given all previous words. This sequential prediction is what allows it to create coherent, contextually relevant responses.

‍

The Prediction Process

When ChatGPT generates a response, it follows these steps:

1. Analyzing Input Context: The model processes your prompt and any relevant conversation history, building an internal representation of the context.
2. Token-by-Token Generation: Starting with this context, the model:
   
   • Calculates probability scores for every possible next token in its vocabulary
   • Selects a token based on these probabilities
   • Adds this token to the growing response
   • Updates its understanding of the context to include the new token
   • Repeats this process until it generates an end token or reaches a length limit
3. Sampling with Temperature: Rather than always choosing the single highest-probability token, ChatGPT uses "sampling" with a controllable "temperature" parameter. Higher temperature settings increase randomness and creativity, while lower settings make responses more deterministic and focused. You can use the OpenAI Playground to manually adjust the temperature of your model to find the right mix of predicability and creativity for your specific needs.

This process happens extremely rapidly, with the model potentially evaluating tens of thousands of possible tokens at each step to build a coherent response.

‍

How Probability Shapes Responses

The probability distribution generated at each step reflects the model's understanding of language patterns. For example, after the phrase "The capital of France is," the token "Paris" would receive an extremely high probability, while unrelated words would score very low.

These probabilities are influenced by multiple factors:

• Grammar and syntax: Proper sentence structure receives higher probability
• Factual associations: Words that reflect real-world knowledge score higher in appropriate contexts
• Thematic coherence: Tokens that maintain the topic of discussion are favored
• Stylistic consistency: The model attempts to maintain a consistent tone and style

It's important to understand that ChatGPT doesn't retrieve pre-written answers from a database. Instead, it constructs responses word by word based on patterns it learned during training. This explains why:

• The same prompt can generate slightly different responses each time
• The model can create entirely novel text it has never seen before
• Responses reflect statistical patterns rather than explicit programming

‍

Key Learnings & Practical Applications

Now that we've explored how ChatGPT works, here are some key insights and their practical implications for effectively using this technology.

1. ‍Transformer Architecture
   ChatGPT processes language using a Transformer model with self-attention, allowing it to understand entire contexts at once.
   Tip: Give full context upfront, no matter how long-winded your sentences are. Complete, well-structured prompts yield better responses than fragmented ones.‍
   ‍
2. Training Process
   ChatGPT learns through three stages: pre-training on internet text, supervised fine-tuning, and reinforcement learning from human feedback.
   Tip: It's trained to be helpful, harmless, and honest. It may refuse harmful or ambiguous requests, which is yet another reason why context matters.‍
   ‍
3. Token-Based Processing
   Text is broken into tokens (word pieces), which limits how much it can process at once.
   Tip: For long tasks, break inputs into smaller chunks and prioritize key info.‍
   ‍
4. Probabilistic Output
   Responses are generated one token at a time, based on probability. There are no fixed answers.
   Tip: Answers can vary, so try rephrasing or repeating prompts or modifying the model's temperature to cross-check accuracy.‍
   ‍
5. Built-In Limitations
   Constraints include hallucinations, outdated knowledge, bias, and sensitivity to phrasing.
   Tip: Always verify important outputs with manual research. Treat ChatGPT as a collaborator, not a sole authority.

‍

By reading through this lesson, you should feel demystified with what might initially seem like technological magic.

Knowing how ChatGPT processes context enables you to craft better prompts: provide comprehensive context, break complex requests into clear components, and specify exactly what you need. It also sharpens your critical evaluation because you know what mistakes to look for. With time, you will become better at spotting potential hallucinations, accounting for knowledge limitations, and recognizing when additional context might help.

Most importantly, understanding ChatGPT's capabilities and constraints helps you identify where it truly excels: creative ideation, content drafting, explaining complex concepts, and problem-solving assistance. You can leverage these strengths while remaining mindful of its limitations.