Serving a Model Is a Contract, Not Just a Container
By Horia Beschea, Smallsteps
A container can be running while your application still cannot make a valid model request. The artifact, input signature, model name, version, and response shape all have to agree.
This lesson exports the previous classifier as a TensorFlow SavedModel and shows how to compare local predictions with TensorFlow Serving. The Smallsteps platform also separates worker code from model serving. This teaching example uses a simpler REST contract and public sample input, rather than reproducing a deployed workload.
Ask the agent for a testable serving boundary
Export the trained Keras classifier as a versioned TensorFlow SavedModel.
Make its serving input shape and output name explicit. Run TensorFlow Serving,
call it with a representative input and a timeout, and compare its output with
local Keras inference. Explain what happens when the model is unavailable.A reasonable output is not proof of parity. Test the actual artifact and the same tensor through both paths.
Export a versioned SavedModel
The included export.py loads classifier.keras, wraps it in a tf.Module, and exports serving_default. Its named input image is a float tensor shaped (batch, 96, 96, 3), and its named output scores contains two model scores. The model still expects RGB values between 0 and 255; its rescaling layer is inside the exported model. TensorFlow's SavedModel guide explains signatures and numeric version directories.
Get Horia Beschea’s stories in your inbox
Join Medium for free to get updates from this writer.
Run the commands in the lesson README. The script refuses to overwrite an existing output path: export a new numbered directory for a new candidate. It does not itself prove that newer models are safe or compatible.
Compare the same input through both paths
The companion check_serving.py loads the Keras model, creates an all-zero test tensor, posts the same input to TensorFlow Serving with a ten-second timeout, and checks numerical agreement within a tolerance. That test detects a number of serialization and wiring errors, but one input is not a representative model-quality evaluation. Expand it to include ordinary images and boundary cases before using a model in a service.
For a deliberately constructed failure, point the client at an unused local port and observe the connection error. A real application should translate timeouts and unavailable backends into a stable error response. It should not quietly return old or fabricated scores.
Decide what to version and monitor
A model-server boundary allows independent deployment and can reduce duplicated model copies, but adds a network dependency and a shared capacity bottleneck. Version the preprocessing contract and label order alongside the model; serving version 1 does not automatically identify a compatible downstream client.
The local export and reload can be verified without Docker. The container exercise additionally needs a running Docker daemon and a compatible TensorFlow Serving image. Serving is a technical delivery step, not proof of model quality or permission to deploy. The official Docker guide describes the REST port and model directory layout.
Previous: Repeatable inference · Next: Feedback and retraining
