Sushil’s Dev Notes

From Confused to Confident: LLM Fundamentals for Full Stack Developers

Introduction

After taking a short career break, I got the opportunity to work on an AI-powered product. I was super excited. Naturally, the first thing I did was open ChatGPT because let’s be honest, we’ve mostly replaced Google at this point.

I assumed it would be simple. Just another tool in the market, right?

Wrong.

I hit a wall almost immediately.

Suddenly, terms like tokens, context window, embeddings, and temperature were everywhere. The worst part? Nobody clearly explained how much of this a developer actually needs to understand just to ship features.

After a few hours of confusion (and way too many articles), things finally started to click.

This article is the guide I wish I had when I was stuck and overwhelmed, a practical path from confusion to confidence while working with LLMs.

If you’re a developer who wants to integrate AI into your apps but feels overwhelmed by the jargon, this is for you.

What Even Is an LLM?

LLM stands for Large Language Model. Let’s break that down:

At its core, an LLM is a very sophisticated autocomplete on steroids.

You’ve used autocomplete on your phone before. LLMs work in a similar way but at a massive scale, with far more context, and trained on an absurd amount of text.

The Important Realization

LLMs don’t think or understand the world the way humans do.

They are extremely good pattern-matching systems trained on billions of words. Given some input, they predict what comes next based on patterns they’ve seen during training.

Once this clicks, a lot of confusing LLM behavior suddenly makes sense.

Why This Matters for Developers

Understanding how LLMs actually work helps you use them effectively and avoid surprises:

Treat LLMs less like a source of truth and more like a powerful, probabilistic tool and you’ll build better AI-powered features.

Tokens: The Currency of LLMs

What Confused Me

My first chatbot was eating through my API budget like crazy. I assumed users were asking long questions.

Turns out, my prompt was the real problem.

What Tokens Actually Are

A token is the basic unit of text an LLM processes.

Think of it as roughly ¾ of a word or 3–4 characters in English (on average).

Examples:

Not exactly words, not exactly characters somewhere in between.

Why Tokens Matter (The Hard Way)

Everything costs tokens:

I learned this when: My chatbot had a 500-word system prompt explaining its personality, rules, etc. That's roughly 650 tokens. Every single message was costing me those 650 tokens PLUS the actual conversation. My expensive mistake was having a verbose system prompt that repeated instructions over and over. Better approach: Keep it concise. "You are a helpful assistant. Be concise and accurate." just ~15 tokens. Cost impact: Reduced my per-message cost by 80%.

The Real Impact

Tokens aren’t just a technical detail they directly affect:

You need to be conscious of token usage from day one.

Context Window: Your Memory Limit

What Confused Me

User: "Remember what I said 10 messages ago?"
My app: Returns completely different information
Me: "Why doesn't it remember?!"

What Context Window Actually Is

The context window is the total amount of text (measured in tokens) an LLM can see at once.

Think of it like RAM for the model:

Current Typical Limits

Why This Matters

Imagine your chatbot conversation:

Total: 8,000 tokens

If your model has an 8K context window, you’re at the limit. The next message won’t fit.

What Happens When You Exceed It

My Solution: Smart Context Management

The key is managing what you send to the model:

For long conversations, you need to be intentional about what stays in context and what gets summarized or dropped.

Neural Networks: What's Actually Happening

What Confused Me

"It's a neural network" told me nothing. I needed to understand how it actually works.

The Simplest Explanation

Think of a neural network as a giant collection of numbers that transforms your input into output.
That's really it. No magic.

Basic flow:

Weights: The Numbers That Do Everything

Here's what finally made it click for me:

When you use GPT-4 or Claude, you're basically using billions of numbers stored in a file. Those numbers are the model. They are the intelligence.

Think of weights like recipe measurements:

Each number controls how much something matters.

In the neural network:

The model just picks words based on which numbers are biggest.

Biases: The Starting Point

Bias is even simpler it's just a default setting before anything else happens.

Back to the recipe analogy:

In the neural network:

Every neuron has:

Training: Where These Numbers Come From

This blew my mind when I finally understood it:

Those billions of numbers started completely random. Totally useless.

Then training happened:

  1. Show model: "The cat says meow"
  2. Model guesses: "hello" (weights are random, so wrong answer)
  3. Computer: "Wrong! Adjust the numbers"
    • Weight for "meow" goes up a tiny bit
    • Weight for "hello" goes down a tiny bit

Repeat this process trillions of times on billions of sentences.

After enough repetition, the numbers become accurate.
The model "learned" that cats say meow not because someone programmed it, but because the numbers adjusted to match the pattern.

Why This Matters for Developers

Understanding this simple truth helps explain LLM behavior:

The Bottom Line

You don't need to understand complex math to use LLMs effectively.

Just remember:

No magic just math at enormous scale.

Attention Mechanism: The Game Changer

What Confused Me

Everyone said “attention is why modern LLMs are so good”, but nobody explained what it actually does.

The Problem It Solves

Sentence:

"The bank of the river was flooded"

Question:
What does “bank” mean here?

A human instantly knows: riverbank (not a financial institution).

Why?
Because we naturally look at “river” and “flooded” for context.

How Attention Works (Simply)

For each word, the model asks:

“Which other words in this sentence are most relevant to understanding THIS word?”

When processing “bank”:

Conclusion

“bank” is strongly connected to “river” and “flooded”
→ It means riverbank, not a financial institution.

The model computes these relevance scores automatically and uses them to build a context-aware understanding of each word.

Why This Matters for Developers

Attention is what gives modern LLMs their superpower. It enables:

Practical Impact

Without good attention (old models):

User:

"What's the return policy? I bought a laptop."

Old model:

"Our return policy is 30 days."

(Generic answer doesn’t connect “return policy” to “laptop”)

With attention (modern LLMs):

User:

"What's the return policy? I bought a laptop."

Modern LLM:

"For laptops, we offer a 30-day return policy. If the laptop has been opened, a 15% restocking fee applies."

(Understands that the policy should be specific to laptops and provides relevant details)

This breakthrough is what makes modern LLMs contextually aware, use-case specific, and genuinely powerful for real-world applications.

Temperature: Controlling Creativity

What Confused Me

Sometimes my chatbot was too creative (making things up).
Other times it was too boring (repetitive and robotic).

I had no idea how to control that behavior.

What Temperature Actually Is

Temperature is a parameter (usually ranging from 0.0 to 1.0+) that controls how random the model’s outputs are.

Think of it as a creativity dial:

How It Works Internally

When predicting the next word, the model assigns probability scores to all possible options.

Example:

How temperature affects the choice:

The Core Idea

My Practical Guide

Here’s how I now choose temperature based on the use case:

When I Got Burned

Mistake:
Using temperature 0.9 for a medical advice chatbot

Result:
Creative but potentially dangerous hallucinations about treatments

Fix:
Dropped to temperature 0.2 for factual accuracy and safety

The Lesson

Always match temperature to your use case.
When accuracy and safety matter, keep it low.

Embeddings: Numbers That Capture Meaning

What Confused Me

"Convert text to embeddings" showed up everywhere, but nobody explained what that actually means.

I also kept seeing "RAG" mentioned but didn't understand what problem it solves or how embeddings fit in.

The Concept

An embedding turns words into numbers that capture their meaning.

Think of it like addresses for concepts:

The Key Insight

Similar meanings = Similar numbers

This is how computers understand meaning, not just match exact words.

Why This Matters: Finding What You Actually Mean

Old-school search (keyword matching):

You search: "headache remedy"
Results: Only pages with the exact words "headache" AND "remedy"
Misses: Pages about "migraine relief" or "pain treatment"

Smart search (with embeddings):

You search: "headache remedy"
Results include:

Why? The embeddings recognize these all mean similar things, even though the words are different.

RAG: Teaching AI About YOUR Information

The Problem RAG Solves

Here's the issue:

LLMs like GPT-4 or Claude were trained on general internet data up to a certain date. They don't know:

Without RAG: The LLM can only answer based on its training data (generic knowledge, often wrong for your use case).

With RAG: The LLM can answer questions about YOUR specific documents and data.

What Is RAG?

RAG = Retrieval-Augmented Generation

Break it down:

Simple explanation: Before asking the LLM to answer, you first retrieve relevant information from your documents and include it in the prompt.

This is where embeddings become essential they power the retrieval step.

Real Example: Pizza Delivery Support Bot

Scenario: You run a pizza delivery service and want a support bot that answers customer questions.

Step 1: Prepare Your Knowledge Base (One-time setup)

Documents:

What you do:

  1. Take each document
  2. Convert it into an embedding (those 1,536 numbers)
  3. Store the embeddings in a vector database (like Pinecone or ChromaDB)

Now your knowledge is ready to be searched by meaning.

Step 2: Customer Asks a Question

Customer asks: "Can I get my money back if the pizza is cold?"

Step 3: Retrieval (Using Embeddings)

  1. Convert the customer's question into an embedding
  2. Compare that embedding against all stored document embeddings
  3. Find the closest match: "Refunds available within 24 hours if pizza arrives cold"

The magic: Customer said "money back" but the document says "refunds" embeddings understood these mean the same thing!

Step 4: Augmentation (Build the Prompt)

Use this information to answer the question: "Refunds available within 24 hours if pizza arrives cold"

Question: Can I get my money back if the pizza is cold?

Step 5: Generation (LLM Answers)

The LLM receives your prompt (with YOUR document included as context) and responds:

"Yes, you can get a refund if your pizza arrives cold. You have 24 hours to request it."

Why Embeddings Make RAG Work

Different customers ask the same question in different ways:

All these different phrasings produce similar embeddings, so they all retrieve the same refund policy document.

Traditional keyword search would fail:

Embeddings succeed: They understand the meaning behind the words.

Another Example: Delivery Time

Customer asks: "How long until my pizza gets here?"

Retrieval with embeddings:

  1. Question becomes an embedding
  2. System searches all documents
  3. Finds closest match: "Delivery takes 30-45 minutes in normal conditions"

Why this works:

LLM response: "Your pizza should arrive in 30-45 minutes under normal conditions."

Complete RAG Workflow

Setup Phase (Do Once)

  1. Collect all your documents (FAQs, policies, guides)
  2. Split into chunks if needed (remember context window limits!)
  3. Generate embeddings for each chunk
  4. Store embeddings in a vector database

Runtime Phase (Every Question)

  1. User asks a question
  2. Convert question to an embedding
  3. Search vector database for 3-5 most similar document embeddings
  4. Retrieve those relevant document chunks
  5. Build a prompt: question + retrieved documents
  6. Set temperature to 0.3 (low for factual accuracy)
  7. LLM generates answer based on YOUR documents
  8. Return answer to user

What This Complete System Uses

✅ Embeddings → Convert text to numbers that capture meaning
✅ Vector database → Store and search embeddings efficiently
✅ Context window → Retrieved chunks sized to fit within limits
✅ Tokens → Count everything: question + documents + response
✅ Temperature → Set low (0.3) for accurate, factual answers
✅ Attention → Model connects question to relevant document parts

All the fundamentals working together to create something powerful!

Why RAG Is Essential for Real Applications

RAG transforms LLMs from general knowledge systems into specialists for YOUR domain.

Without RAG:

With RAG:

The Bottom Line

Embeddings let you search by meaning, not just keywords.

RAG uses embeddings to give LLMs access to your specific information.

Together, they solve the core problem: How do I make an LLM that knows about MY business?

Think of it this way:

This is how you build AI features that actually work for real-world applications.

Common Mistakes I Made (So You Don’t Have To)

1. Not Counting Tokens

Mistake:
Sent the entire conversation history with every API call

Result:
Hit context limits frequently and racked up expensive API costs

Fix:
Implement context management from day one track token counts and trim history intelligently

2. Using High Temperature for Factual Tasks

Mistake:
Set temperature to 0.9 for customer support responses

Result:
Creative, but often inaccurate or misleading answers

Fix:
Match temperature to the task use low values (0.2–0.4) when accuracy matters

3. Ignoring Streaming

Mistake:
Waited for the full response before showing anything to users

Result:
Users thought the app was frozen during the 10–20 second wait

Fix:
Implement streaming responses so users see output as it’s generated
(Huge UX improvement)

4. Not Handling Errors

Mistake:
Assumed the API would always work perfectly

Result:
App crashed on rate limits, network timeouts, and server errors

Fix:
Add proper error handling:

5. Not Testing Edge Cases

Mistake:
Only tested with normal, well-formed inputs during development

Result:
Weird, broken responses in production

Fix:
Build a comprehensive test suite:

Learn from these early they’ll save you time, money, and headaches in production.

Resources That Actually Helped Me

Documentation

Tools

Conclusion

Learning LLMs as a web or full-stack developer can feel overwhelming at first. There’s a lot of new terminology, hidden constraints, and unexpected behavior that doesn’t show up in a simple “Hello World” demo.

But here’s the key takeaway:

You don’t need to become an AI researcher to build great AI-powered products.

What you do need is a solid mental model of how LLMs work:

Once these concepts click, LLMs stop feeling unpredictable and start feeling like any other powerful engineering tool one with trade-offs you can reason about and control.

Final Thoughts

What I Wish I Knew From Day One

The Shift in Mindset

From:

"I’ll just call the API and it’ll work perfectly."

To:

"I need to manage tokens carefully, handle context window limits, choose the right temperature, implement proper error handling, and thoroughly test edge cases."

Once you understand these fundamentals, you can build incredibly powerful AI features that seemed impossible just months ago.

Ignore them, and you’ll keep debugging “weird AI behavior” that isn’t weird at all.

Thanks for sticking around and taking the time to read this! 🙏 🚀