From Notebook to Product: Building and Deploying a Simple NLP App
A model that works in a notebook is a good start, but nobody can use it. Sooner or later you want a button, a form or another program to send text to your model and get an answer back. That step, turning a trained model into a small service, is what this article is about.
We will keep it deliberately simple. The model is a sentiment classifier, and the goal is not to build the best classifier. The goal is to see every piece that sits between a trained model and a working product: saving it, wrapping it in an API, checking inputs, handling uncertainty, testing, packaging and watching it once it is live.
The Big Picture
Think of the app as a pipeline with two phases. Training happens once, offline. Serving happens every time someone sends a request.
The bridge between the two phases is a single file. You train, save the model to disk, and the service loads that file when it starts. This separation matters because training can be slow and messy, while serving needs to be fast and predictable.
Train and Save the Model
Our model is a classic combination you met earlier in the series: TF-IDF turns text into numbers, and logistic regression turns those numbers into a label. Wrapping both in one pipeline means the same text processing is applied at training time and at serving time, which avoids a very common bug where the two quietly drift apart.
model = Pipeline([
("tfidf", TfidfVectorizer(lowercase=True)),
("clf", LogisticRegression(C=5.0, max_iter=1000)),
])
model.fit(texts, labels)
joblib.dump(model, "model.joblib")One honest note before going further. This toy model is trained on only 12 short sentences. It is enough to demonstrate the engineering, but the confidence numbers you will see below are for illustration only. A real model needs real data, and you would measure it properly, as we covered in Evaluating NLP Systems.
Wrap It in an API
An API is just a doorway. A program sends text in, and the model’s answer comes out. FastAPI makes this short. We expose two endpoints: one to ask whether the service is alive, and one to make a prediction.
@app.get("/health")
def health():
return {"status": "ok"}@app.post("/predict")
def predict(req: PredictRequest):
probs = model.predict_proba([req.text.strip()])[0]
...The health endpoint looks trivial, but it is what hosting platforms and monitoring tools call to check that your service is running. Add it early.
Validate the Input
Users send surprising things: empty strings, whole novels, nothing at all. If you do not decide what to do with them, the model will decide for you, usually badly. Declaring the shape of a valid request lets the framework reject bad input before it ever reaches the model.
class PredictRequest(BaseModel):
text: str = Field(..., min_length=1, max_length=1000)An empty text now returns a clear error (status 422) instead of a confusing prediction. Setting a maximum length also protects your service from huge inputs that would slow it down.
Be Honest About Uncertainty
Here is the idea I think matters most in this article. A classifier always picks a label, even when the text has nothing to do with sentiment. If you only return the winning label, a neutral sentence gets confidently pushed into positive or negative.
The fix is to look at the confidence, which is the probability the model assigns to its top choice. If it falls below a threshold, we say so.
if confidence < 0.60:
label = "uncertain"Here is what the running service returned for three sentences:
Get Tabish alam’s stories in your inbox
Join Medium for free to get updates from this writer.
Text Label Confidence I loved this wonderful movie positive 0.712 Terrible waste of time negative 0.764 The projector was on a table uncertain 0.502
The third sentence has no sentiment, and the model’s confidence sits near a coin flip, so the app admits it does not know. Remember that these values come from a 12-sentence model, so they show the mechanism rather than real-world quality. The threshold itself is a product decision: set it higher and you get fewer wrong answers but more “uncertain” ones.
Test It
Tests are what let you change the app later without fear. We do not need many. Three small ones cover the essentials: the health check responds, a normal request returns a label and a confidence in a valid range, and an empty text is rejected.
def test_empty_text_is_rejected():
r = client.post("/predict", json={"text": ""})
assert r.status_code == 422Notice that the second test checks the shape of the answer, not the exact label. Tests that depend on exact model outputs break every time you retrain.
Package It with a Container
“It works on my machine” is the oldest problem in software. A container solves it by bundling your code, the Python version and the libraries into one unit that runs the same way everywhere.
FROM python:3.11-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY app.py train.py ./
RUN python train.py
EXPOSE 8000
CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8000"]In this version the image trains the tiny model while it builds. For a bigger model you would train elsewhere and copy the saved file in instead.
Deploy It
Once you have a container, deployment is mostly a choice of where to run it. Managed platforms that accept a container and give you a public URL are the easiest starting point for a small project. Whichever you pick, set the health endpoint as the platform’s check, and keep the model file versioned alongside the code so you always know which model is live.
Monitor It
Deployment is not the finish line. Once real users arrive, the text they send will differ from your training sentences, and that gap is where models quietly degrade.
You do not need a heavy setup to begin. Log each request’s length, label and confidence, and watch two things over time. If the share of “uncertain” answers climbs, the incoming text is drifting away from what the model knows. If the confidence distribution shifts, something has changed. Both are cheap signals that tell you when it is time to collect fresh data, retrain and measure again.
Key Takeaways
- Separate training from serving, and connect them with a saved model file.
- Validate input so the model only sees what it can handle.
- Return “uncertain” instead of forcing a confident answer.
- A few small tests protect you every time you change something.
- A container makes the app portable, and monitoring tells you when it stops being right.
This was the capstone of the series: from raw text, through representation, classification, transformers and evaluation, to a working service. There is more to explore, and I will cover further topics in future parts.
Written by Tabish Alam, Proxima.ai
