LLM Cost Optimization for Beginners: The Playbook for Cheaper, Faster AI Apps

Meera is a developer in Pune. Her team shipped a small AI helper that answers questions from customers. For two weeks everyone was happy. Then the first full bill arrived, and it was three times what she had guessed.

Her manager asked a fair question: “Can we make this cheaper without making it worse?” Meera had never looked at LLM cost optimization before. She did not know where the money was going, or why some answers felt slow.

This playbook follows her as she finds out. No maths degree needed. Every term is explained when it first shows up.

Last verified: 6 November 2026. AI tools change fast; check the linked sources for anything newer.

Who this is for: anyone who has called an AI model from code and wants a smaller bill and quicker replies. Time needed: about 30-45 minutes to read and try the first two plays.

What is LLM cost and latency, in simple words

Think of a taxi meter. An AI model (an LLM, or large language model) charges you for the text it reads and the text it writes. That text is counted in tokens, which are small pieces of words. If tokens are new to you, read our How LLMs Work playbook first.

The simple formula: cost is the input tokens times the input price, plus the output tokens times the output price. Output tokens usually cost more than input tokens, though the gap differs by provider, so check your own price page.

Latency is how long you wait for the answer. Both cost and latency grow with the number of tokens, so cutting waste often helps both at once.

What’s new: pricing keeps moving

Providers change prices and add discount options often. Our Gemini 4 Argon price increase playbook covers one recent example. The lesson is general: never hard-code a price in your head. Read the official price page each month.

The playbook at a glance

1. Measure first: find where the tokens go.
2. Trim the prompt and cap the answer length.
3. Use prompt caching for text you repeat.
4. Batch the work that can wait.
5. Match the model to the job, then re-test quality.

Play 1: Measure before you change anything

You cannot cut a bill you cannot read. Goal: know which feature, prompt and model spend the most.

1. Most APIs return the input and output token counts with every reply. Save them.
2. Tag each call with the feature name, such as “answer-question” or “summarise-ticket”.
3. Add up tokens and cost per feature for one week.

Meera logs the counts and finds that one feature, which re-sends a long set of company rules with every question, makes up most of the bill. Our observability playbook shows how to log AI calls properly.

You are done when: you can say which feature costs the most and why.

Play 2: Trim the prompt and cap the answer

Goal: stop paying for words nobody needs. Think of a courier who charges by weight. You would not pack the box with old newspapers.

1. Remove repeated or unused instructions from the prompt.
2. Send only the few documents or messages the model needs, not the whole history.
3. Set a maximum output length, and ask for short answers when short is enough.

Meera cuts her rules from a long essay to a tight list and keeps only the last few chat messages. The answers stay just as good, and each call is lighter and faster.

You are done when: average tokens per call drop and your test questions still get good answers.

Play 3: Use prompt caching for repeated text

Goal: stop paying full price to re-read the same opening text. Prompt caching means the provider remembers the processed form of the start of your prompt. When the next request begins with the same text, that part is cheaper and quicker. It is like a waiter who already knows your usual order.

1. Put the parts that never change (rules, examples) at the start of the prompt.
2. Put the parts that change (the customer question) at the end.
3. Read your provider’s docs: some cache automatically, some need you to turn it on, some charge a small fee to write the cache, and many have a minimum prompt length.

Here is a tiny illustration of the idea. Exact names differ between tools, so treat it as a sketch:

RULES = "...long, never-changing company rules..."

def ask(question):
    prompt = RULES + "\n\nQuestion: " + question
    reply = model.generate(prompt)
    log(reply.input_tokens, reply.output_tokens)
    return reply.text

In plain steps: the rules always come first and stay identical, so the provider can reuse them. Only the question changes. Meera also checks the “cached tokens” number in the reply, if her provider shows one.

Be realistic. Savings depend on how often requests share the same start and arrive before the cache expires. Blog posts that promise huge savings describe best cases.

You are done when: your logs show a share of input tokens being served from the cache.

Play 4: Batch the work that can wait

Goal: pay less for jobs nobody is waiting on. Many providers offer a batch option: you hand over a big pile of requests and get the results later, often within a day, at a lower price. It is like posting parcels by slow mail instead of express.

1. List your AI jobs and mark each one “someone is waiting” or “can wait”.
2. Move the “can wait” jobs, such as nightly summaries or tagging old tickets, to the batch option.
3. Check the provider’s batch discount and delay on its price page.

Meera’s team summarises every closed ticket each night. Nobody reads those summaries until morning, so she moves the job to batch.

You are done when: every job that can wait runs in batch, and live chat stays on the fast path.

Play 5: Match the model to the job, then re-test

Goal: stop using the biggest model for easy work. You do not hire a senior architect to sort the post. Smaller models cost less and usually answer faster, but they may be weaker on hard questions.

1. Pick one simple feature, such as labelling a ticket as “billing” or “bug”.
2. Try a smaller model on your saved test questions.
3. Compare quality, cost and speed side by side. Keep the small model only if quality holds.

The test step matters. Without it you are guessing. Our AI evals playbook shows how to build a small test set in an afternoon.

You are done when: each feature uses the cheapest model that still passes your tests.

Which lever should you pull first?

Mistakes to avoid

• Optimising without measuring. Fix: do Play 1 first, so you cut the real cost, not a guess.
• Choosing a model by price per token alone. Fix: compare the real cost of your own workload, including the mix of input and output.
• Putting changing text before fixed text. Fix: keep the fixed part first, or caching cannot help.
• Switching models without tests. Fix: re-run your test questions after every change.

How to know it is working

• Cost per feature, or per customer question, goes down week over week.
• Average reply time stays the same or gets faster.
• Your test score does not drop.

Your checklist

• ☐ I log input and output tokens for every call.
• ☐ I know my most expensive feature.
• ☐ Prompts are trimmed and output length is capped.
• ☐ Fixed text comes first so caching can work.
• ☐ Jobs that can wait use batch.
• ☐ Each feature has a test set before I change its model.

Frequently asked questions

Why are output tokens usually more expensive?

Writing text is more work for the model than reading it, so most providers charge more for output. The exact gap varies, so check your provider’s price page.

What is prompt caching?

It lets the provider reuse its work on the repeated start of your prompt. Repeated text costs less and is processed faster. Rules differ by provider.

When should I use batch processing?

Use it when nobody is waiting for the answer, such as nightly reports. You trade speed for a lower price.

Will a smaller model lower my quality?

Sometimes. For easy tasks it is often fine. Always test on your own questions before you switch.

Meera’s next bill

A month later, Meera showed her manager a simple chart: cost per question going down, answer quality unchanged. She had not done anything clever. She measured, trimmed, cached, batched and tested.

Your next step: do Play 1 today. Log your token counts for one day and find your most expensive feature.

Sources: Infracost on cached token pricing, Developers Digest on caching and batch tiers, TrueFoundry LLM cost optimization guide. Figures vary by provider, so confirm with each official price page.