You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: content/manuals/ai/model-runner/_index.md
+63-19Lines changed: 63 additions & 19 deletions
Original file line number
Diff line number
Diff line change
@@ -11,24 +11,27 @@ description: Learn how to use Docker Model Runner to manage and run AI models.
11
11
keywords: Docker, ai, model runner, docker desktop, docker engine, llm
12
12
aliases:
13
13
- /desktop/features/model-runner/
14
-
- /ai/model-runner/
14
+
- /model-runner/
15
15
---
16
16
17
17
{{< summary-bar feature_name="Docker Model Runner" >}}
18
18
19
-
The Docker Model Runner plugin lets you:
19
+
## Key features
20
20
21
-
-[Pull models from Docker Hub](https://hub.docker.com/u/ai)
22
-
- Run AI models directly from the command line
23
-
- Manage local models (add, list, remove)
24
-
- Interact with models using a submitted prompt or in chat mode in the CLI or Docker Desktop Dashboard
25
-
- Push models to Docker Hub
21
+
-[Pull and push models to and from Docker Hub](https://hub.docker.com/u/ai)
22
+
- Run and interact with AI models directly from the command line or from the Docker Desktop GUI
23
+
- Manage local models and display logs
24
+
25
+
## How it works
26
26
27
27
Models are pulled from Docker Hub the first time they're used and stored locally. They're loaded into memory only at runtime when a request is made, and unloaded when not in use to optimize resources. Since models can be large, the initial pull may take some time — but after that, they're cached locally for faster access. You can interact with the model using [OpenAI-compatible APIs](#what-api-endpoints-are-available).
28
28
29
29
> [!TIP]
30
30
>
31
-
> Using Testcontainers or Docker Compose? [Testcontainers for Java](https://java.testcontainers.org/modules/docker_model_runner/) and [Go](https://golang.testcontainers.org/modules/dockermodelrunner/), and [Docker Compose](/manuals/compose/how-tos/model-runner.md) now support Docker Model Runner.
31
+
> Using Testcontainers or Docker Compose?
32
+
> [Testcontainers for Java](https://java.testcontainers.org/modules/docker_model_runner/)
33
+
> and [Go](https://golang.testcontainers.org/modules/dockermodelrunner/), and
34
+
> [Docker Compose](/manuals/compose/how-tos/model-runner.md) now support Docker Model Runner.
32
35
33
36
## Enable Docker Model Runner
34
37
@@ -76,7 +79,58 @@ You can now use the `docker model` command in the CLI and view and interact with
76
79
$ docker model run ai/smollm2
77
80
```
78
81
79
-
## Integrate the Docker Model Runner into your software development lifecycle
82
+
## Pull a model
83
+
84
+
Models are cached locally.
85
+
86
+
{{< tabs >}}
87
+
{{< tab name="From Docker Desktop">}}
88
+
89
+
1. Select **Models** and select the **Docker Hub** tab.
90
+
2. Find the model of your choice and select **Pull**.
91
+
92
+
{{< /tab >}}
93
+
{{< tab name="From the Docker CLI">}}
94
+
95
+
Use the [`docker model pull` command](/reference/cli/docker/).
96
+
97
+
{{< /tab >}}
98
+
{{< /tabs >}}
99
+
100
+
## Run a model
101
+
102
+
{{< tabs >}}
103
+
{{< tab name="From Docker Desktop">}}
104
+
105
+
Select **Models** and select the **Local** tab and click the play button.
106
+
The interactive chat screen opens.
107
+
108
+
{{< /tab >}}
109
+
{{< tab name="From the Docker CLI">}}
110
+
111
+
Use the [`docker model run` command](/reference/cli/docker/).
112
+
113
+
{{< /tab >}}
114
+
{{< /tabs >}}
115
+
116
+
## Troubleshooting
117
+
118
+
To troubleshoot potential issues, display the logs:
119
+
120
+
{{< tabs >}}
121
+
{{< tab name="From Docker Desktop">}}
122
+
123
+
Select **Models** and select the **Logs** tab.
124
+
125
+
{{< /tab >}}
126
+
{{< tab name="From the Docker CLI">}}
127
+
128
+
Use the [`docker model log` command](/reference/cli/docker/).
129
+
130
+
{{< /tab >}}
131
+
{{< /tabs >}}
132
+
133
+
## Example: Integrate Docker Model Runner into your software development lifecycle
80
134
81
135
You can now start building your Generative AI application powered by the Docker Model Runner.
82
136
@@ -164,7 +218,6 @@ with `/exp/vDD4.40`.
164
218
> [!NOTE]
165
219
> You can omit `llama.cpp` from the path. For example: `POST /engines/v1/chat/completions`.
166
220
167
-
168
221
### How do I interact through the OpenAI API?
169
222
170
223
#### From within a container
@@ -280,12 +333,3 @@ The Docker Model CLI currently lacks consistent support for specifying models by
280
333
## Share feedback
281
334
282
335
Thanks for trying out Docker Model Runner. Give feedback or report any bugs you may find through the **Give feedback** link next to the **Enable Docker Model Runner** setting.
283
-
284
-
## Disable the feature
285
-
286
-
To disable Docker Model Runner:
287
-
288
-
1. Open the **Settings** view in Docker Desktop.
289
-
2. Navigate to the **Beta** tab in **Features in development**.
290
-
3. Clear the **Enable Docker Model Runner** checkbox.
0 commit comments