The current status of OpenAI fine-tuning
OpenAI's optimization guide says that the fine-tuning platform is winding down. New users cannot access it. Existing users can still create training jobs for the coming months, and fine-tuned models remain available for inference until their base models are deprecated. Check the current deprecation guidance before depending on an existing fine-tuned model.
That changes the recommendation for a new build. Do not start a new support project by collecting training examples for an API surface that you cannot newly access. First define the support outcome, prepare an evaluation set, and test whether clear instructions and current, approved knowledge resolve the problem.
What fine-tuning is designed to improve
OpenAI describes supervised fine-tuning as training a supported base model with example inputs and desired outputs. Its documented uses include classification, nuanced translation, a specific output format, and correcting instruction-following failures.
The important word is defined. Fine-tuning is a fit only when you can state what good output looks like and measure it. It can help a model apply a stable response pattern. It does not decide whether a particular customer qualifies for an exception, whether a current policy changed yesterday, or whether an order system completed a refund.
| If the problem is | Start by testing | What you still need to verify |
|---|---|---|
| The reply needs a repeatable label or format | Instructions, examples, and an eval against that format | The format is useful and the data is representative |
| A support policy changes often | Approved current knowledge plus instructions | Source freshness, access, exceptions, and the answer's evidence |
| A reply could trigger a refund, account change, or other action | A held or handoff path | The policy allows it and the connected system completed it |
| An existing team can still run fine-tuning | Baseline versus trained-model evals | The measured improvement justifies data, training, and maintenance work |
Run evals before you change a model
OpenAI's SFT guide says to set up evals before investing in fine-tuning. An eval gives a team a baseline for deciding whether a fine-tuned model outperforms a base model. Its training data uses JSONL; OpenAI allows a minimum of 10 examples, reports improvements from 50 to 100 examples, and recommends starting with 50 well-crafted demonstrations before evaluating.
For support, a useful evaluation set contains more than easy questions. Include the normal policy case, a request outside the policy, a request with missing evidence, an outdated policy wording, and an action that must be held for a person. Keep evaluation cases separate from the examples used to train a model. Otherwise, a good-looking score can only show that the model has seen the answer already.
Compare each candidate against the same cases:
- A baseline model with clear instructions and approved context.
- The same model after a prompt or knowledge change.
- An existing fine-tuned model, if your organization is still eligible to use it.
Review not only the text but also the safety boundary. The response should ask for missing evidence, avoid promising a refund, and hand the case to a person when policy requires it. If any workflow takes an external action, confirm the result in the system that performed the action; a model response is not proof that the action happened.
Use eesel CLI to test the support setup first
Fine-tuning and eesel operate at different layers. Fine-tuning changes an OpenAI model's behavior from training examples. eesel is a ready-to-work helpdesk teammate. Its CLI is another way to operate that teammate: a person can use a terminal, a script can consume JSON, and Claude Code, Codex, or Cursor can help perform a bounded review. The CLI and dashboard work on the same teammate and workspace.
This is useful when a support lead wonders whether a policy problem needs a trained model or a better source and instruction. First, a coding agent can make a read-only report: which knowledge sources are connected, whether their content has downloaded, which manual files exist, and what instruction governs a refund exception. It should not connect accounts, change permissions, create an OpenAI job, or enable automation while preparing that report.
With Node.js 18.17 or newer, identify the workspace and the intended teammate:
npx @eesel/cli login
npx @eesel/cli whoami
npx @eesel/cli agents
TARGET_EESEL_AGENT="paste-agent-id-or-name-here"
Then use an explicit agent scope for each inspection. That prevents a script or coding agent from relying on a saved default teammate in the wrong workspace:
npx @eesel/cli status --agent "$TARGET_EESEL_AGENT"
npx @eesel/cli integrations --agent "$TARGET_EESEL_AGENT"
npx @eesel/cli files ls --agent "$TARGET_EESEL_AGENT"
npx @eesel/cli instructions --agent "$TARGET_EESEL_AGENT"
If a support lead approves a revised policy file, preview the upload first. --dry-run prints the server call without sending it:
npx @eesel/cli files upload ./approved-refund-policy.pdf --agent "$TARGET_EESEL_AGENT" --dry-run
Only after approval, upload the file and confirm it is available. Start a new named chat for a redacted test case:
npx @eesel/cli files upload ./approved-refund-policy.pdf --agent "$TARGET_EESEL_AGENT"
npx @eesel/cli files ls --agent "$TARGET_EESEL_AGENT"
npx @eesel/cli new --name "refund-policy-decision" --agent "$TARGET_EESEL_AGENT"
npx @eesel/cli chat "A customer requests a refund after the normal window because the item arrived damaged. State the approved evidence needed, do not promise a refund, and say when a human must decide." --agent "$TARGET_EESEL_AGENT"
Review whether the reply uses the approved policy, asks for missing evidence, and avoids an invented exception. This is a test of the eesel teammate's support setup, not a test of an OpenAI fine-tuned model. It does not create a training job or prove that a refund happened. If the teammate creates a held action, inspect it with npx @eesel/cli approvals --agent "$TARGET_EESEL_AGENT"; then confirm the outcome in the connected system before telling the customer it succeeded.
The Activity view gives a support lead a dashboard-side record to compare with the CLI's JSON output after a controlled test.
Choose the smallest change you can measure
For a new OpenAI user, do not plan on starting a fine-tuning project. For an existing user, treat it as one option after an eval shows that instructions and current knowledge do not meet a clearly defined standard.
For support operations, keep policy sources current, narrow the teammate's authority, and test representative edge cases before expanding automation. Try eesel if you want to inspect and test a support teammate in the same workspace as the dashboard, rather than build an OpenAI training workflow for a policy problem that may be configuration instead.
Frequently asked questions
Can new users access the OpenAI Fine-Tuning API?
No. OpenAI says the fine-tuning platform is winding down and is no longer accessible to new users. Existing users can create training jobs for the coming months. Fine-tuned models remain available for inference until their base models are deprecated, so check OpenAI's current deprecation guidance before planning around an existing model.
What is supervised fine-tuning for?
Supervised fine-tuning trains a supported OpenAI base model on example inputs and desired outputs. OpenAI lists uses such as classification, nuanced translation, a specific output format, and correcting instruction-following failures. It requires a defined task and a way to measure whether the trained model is actually better.
Should I fine-tune a model for changing support policies?
Usually start by testing the policy as current context and clear instructions. A changing policy needs a controlled source of truth, access rules, and representative tests. Fine-tuning can change behavior for a defined task, but it is not a replacement for checking which current policy applies to a particular customer.
How much fine-tuning data does OpenAI recommend?
The minimum training file has 10 examples. OpenAI reports that 50 to 100 examples can improve results and recommends starting with 50 well-crafted demonstrations, then evaluating before adding more. The useful number depends on the task and the quality of the examples.
Why run evals before fine-tuning?
An eval gives you a baseline against a representative test set. Without it, a team cannot show whether a changed prompt, current knowledge source, or fine-tuned model improved the behavior that matters. Include normal cases, exceptions, and unsafe actions that must be held or handed to a person.
Where does eesel CLI fit with fine-tuning work?
eesel CLI does not create OpenAI training jobs or operate a fine-tuned model. It operates an eesel teammate in the same workspace as the dashboard. A person, script, or coding agent can inspect the teammate's sources and instructions, make an approved change, then test a redacted support scenario before customer-facing automation uses it.
Can Claude Code, Codex, or Cursor use eesel CLI?
Yes. The CLI prints JSON, so coding agents can perform a scoped inspection or setup task. Confirm the intended teammate, keep account connections and writes behind human approval, run --dry-run before a write, and check representative results after the change.
Article by
Rama Adi Nugraha
Rama is a software engineer at eesel AI with two years of experience writing about B2B SaaS, AI tools, and customer support technology. Based in Bali, Indonesia, he brings a developer's perspective to product comparisons — cutting through marketing copy to what the integrations and APIs actually do.
