Fine-Tuning Is a Data Audit in Disguise

Direct answer

Fine-tuning does not create knowledge. It compresses whatever you already have, including the mess. Before commissioning a fine-tune, check whether you can produce roughly 10,000 examples of one repeated decision, recorded in one place, with the outcome attached, the good separated from the bad, and a named owner. If you cannot, the data work is the project and the fine-tune is the easy part.

Key takeaways

• Custom AI became a premium product in July 2026. OpenAI opened early access to GPT-5 fine-tuning for enterprise customers while winding down self-serve fine-tuning: new organisations are already blocked from creating jobs, and new job creation ends for all customers by January 2027.
• The model is now the commodity and your data is the differentiator. Two competitors can buy the identical fine-tune. Only the one with clean, labelled, owned examples buys an advantage.
• Fine-tuning is the cheapest data audit available. It fails loudly and immediately. Inconsistent training examples produce inconsistent output on day one, which is why so many teams discover the true state of their data only after they have signed the invoice.
• Fine-tuning and retrieval are not competitors. Retrieval gives a model access to documents at query time. Fine-tuning bakes in tone, format, terminology and decision patterns. Most organisations should earn the right to fine-tune by getting retrieval working first.
• The gold standard is a management artefact, not an AI one. A written record of what "done right" looks like for a repeated decision is simultaneously your training data, your quality benchmark, your onboarding material and your agent's specification.

What changed in July 2026

For three years the industry sold a simple story: the model is the product. Buy access to the best one and capability follows.

In July 2026 the most valuable AI company started selling something different. OpenAI opened early access to fine-tuning for GPT-5, letting enterprises train private versions of the model on their own data, terminology and knowledge domains rather than assembling retrieval pipelines around a generic model. The pricing puts it out of reach of individual developers and small teams. In the same period, OpenAI has been restricting its self-serve fine-tuning platform: new organisations can no longer create fine-tuning jobs, and new job creation ends for existing customers by January 2027.

Read those two moves together and the message is unmistakable. Customising a frontier model on proprietary knowledge is now a large-enterprise capability, priced accordingly, and what you get out of it depends entirely on what you can put into it.

That is the part nobody is selling, because nobody can sell it.

The uncomfortable arithmetic

Picture two insurers. Both sign the same fine-tuning contract, on the same model, on the same day.

The first has fifteen years of claims decisions in a single system, each with the decision recorded, the outcome attached, the disputed cases flagged, and a data owner who can produce a clean export by Friday.

The second has a shared drive, three overlapping case systems from two acquisitions, and a folder called "Final_v3".

They bought the same model. Only one of them bought an advantage.

Fine-tuning does not reason about your business. It finds the patterns in the examples you give it and reproduces them. If your examples encode inconsistency, the model reproduces the inconsistency, faster and at scale. This is not a hypothetical failure mode. It is the single most common reason a fine-tuning project produces a model nobody trusts enough to deploy.

The four-question test

Before you price a fine-tuning engagement, take one high-value decision your organisation repeats often. A credit approval. A claims assessment. A quote. A support resolution. A supplier risk rating. Then answer four questions honestly.

1. Are the past decisions recorded in one place? Not "recoverable from three systems by someone who knows where to look". One place, exportable.

2. Is the outcome recorded next to each decision? A decision without its outcome is an opinion. The model needs to learn which answers were right, which means you need to know.

3. Can you separate the good examples from the bad ones? A gold standard requires that someone can say "this one was handled correctly and this one was not". If every historical case is treated as equally valid, you are training a model on your average performance, not your best.

4. Does anyone own that data set? A named person, accountable for its quality. If the answer is "IT" or "the data team, kind of", the answer is no.

Four yeses and you are genuinely fine-tunable. Anything less and you have just found this quarter's real project, which is worth more than the fine-tune would have been.

Fine-tuning or retrieval? A working rule

The two are routinely presented as alternatives. They solve different problems.

Retrieval (RAG) gives the model access to your documents at the moment of the query. It is easier to update, cheaper to start, and better when the knowledge changes often: policies, prices, product documentation, live records. When the source changes, the answer changes. Nothing to retrain.

Fine-tuning changes the model itself. It is the right tool when you need consistent tone, a house format, domain vocabulary, or a decision pattern reproduced reliably across thousands of cases. It does not keep your model current on facts, and updating it means training again.

The working rule: if the answer lives in a document, retrieve it. If the answer lives in judgement your best people exercise repeatedly, fine-tune it, but only once you can show that judgement to a machine.

Most organisations should earn the right to fine-tune. Get retrieval working, watch where it fails, and you will discover exactly which patterns need to be trained in rather than looked up. That failure log is a better specification than any consultant will write for you.

Build the gold standard first

Every fine-tune needs a gold standard: a set of examples that represent the work done right. Almost no organisation has one written down, which is why so many AI projects begin by inventing one under deadline pressure, usually by a junior analyst guessing.

Here is the cheapest high-value asset most companies could build this quarter. Take your highest-volume repeated decision. Ask the person who does it best to work through twenty real cases and write down, in plain language, why each answer is the right one. Not the policy. The reasoning.

That document is four things at once:

• Training data. It is the seed of any fine-tune you ever commission.
• A quality benchmark. You can now score any model, any vendor, any new hire against it.
• Onboarding material. It is the thing you wish you had when the expert retires.
• An agent specification. No agent can be trusted with a process nobody has defined.

It costs about two days of a senior person's time. Compare that to the cost of a fine-tuning engagement that produces a model nobody trusts.

What Nordic leaders should do this month

The Nordic pattern is consistent: high ambition, strong engineering, and data spread across systems that were never designed to be joined. The region does not have a model access problem. It has a decision-record problem. Three concrete moves:

1. Name one decision. Not a department, not a use case family. One repeated decision with commercial weight.
2. Run the four-question test on it this week. It takes an afternoon and the result tells you whether your next AI investment is a fine-tune or a data project. Either answer is useful; only the guess is expensive.
3. Commission the twenty-case gold standard. Whatever you decide about models, you will own something the vendors cannot supply.

They will sell you the model. They will sell you the engineers to deploy it. They cannot sell you the memory of how your business makes good decisions. That is the whole asset, and it is the one thing on the list a competitor cannot buy with a purchase order.

Architecture before acceleration.

Could you hand a supplier 10,000 clean, labelled examples of your best work tomorrow? For most organisations the answer is no, and that answer decides what any AI investment is worth. We will map it with you in 30 minutes: which decision to start from, what your data actually supports, and what to fix first. No pitch, no deck.

Book an AI Readiness Session

Frequently asked questions

What is GPT-5 fine-tuning and who can use it?

OpenAI opened early access to fine-tuning for GPT-5 in July 2026, letting enterprise customers train private versions of the model on proprietary data, terminology and knowledge. The pricing targets large enterprises. At the same time OpenAI is winding down self-serve fine-tuning: new organisations are blocked from creating jobs, and new job creation ends for all customers by January 2027.

Is fine-tuning better than RAG for enterprise AI?

They solve different problems. Retrieval gives a model access to documents at query time and is easier to update. Fine-tuning bakes tone, format, terminology and decision patterns into the model itself. Fine-tuning only pays off when you have a large, clean, consistently labelled set of examples of the work done correctly. Without that, retrieval is usually the better first step.

How much data do you need to fine-tune a model?

It depends on the task, but the practical bar for an enterprise decision-making use case is on the order of thousands of high-quality, consistently labelled examples, not dozens. Quality matters more than volume: 1,000 examples of the work done correctly beat 50,000 examples of mixed quality, because the model learns the inconsistency along with everything else.

What is a gold standard data set?

A set of examples that represent a repeated decision done correctly, with the reasoning recorded. It is built by having your best practitioner work through real cases and explain why each answer is right. It serves at once as training data, as a benchmark for evaluating models and vendors, as onboarding material, and as the specification any AI agent needs before it can be trusted with the process.

Why do enterprise fine-tuning projects fail?

Almost never because of the model. They fail because the training examples were inconsistent, the outcome of each past decision was not recorded, nobody could separate the good cases from the bad, or no one owned the data set. Fine-tuning compresses whatever you give it. Give it a mess and it will reproduce the mess reliably, at scale, and faster than before.

---

Published by Avalerion Consulting AB, Malmö and Stockholm. Read the weekly briefing at Avalerion Intelligence.