Skip to content

publish updates from main #22642

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
May 15, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 6 additions & 4 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

/content/manuals/build/ @crazy-max @aevesdocker

/content/manuals/build-cloud/ @crazy-max @aevesdocker
/content/manuals/build-cloud/ @crazy-max @aevesdocker

/content/manuals/compose/ @aevesdocker

Expand All @@ -19,11 +19,11 @@

/content/manuals/docker-hub/ @craig-osterhout

/content/manuals/engine/ @thaJeztah @aevesdocker
/content/manuals/engine/ @thaJeztah @aevesdocker

/content/reference/api/engine/ @thaJeztah @aevesdocker
/content/reference/api/engine/ @thaJeztah @aevesdocker

/content/reference/cli/ @thaJeztah @aevesdocker
/content/reference/cli/ @thaJeztah @aevesdocker

/content/manuals/subscription/ @sarahsanders-docker

Expand All @@ -41,4 +41,6 @@

/content/manuals/accounts/ @sarahsanders-docker

/content/manuals/ai/ @ArthurFlag

/_vendor @sarahsanders-docker
6 changes: 6 additions & 0 deletions .github/labeler.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
area/ai:
- changed-files:
- any-glob-to-any-file:
- content/manuals/ai/**
- content/reference/cli/model/**

area/release:
- changed-files:
- any-glob-to-any-file:
Expand Down
2 changes: 1 addition & 1 deletion _vendor/modules.txt
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# github.com/moby/moby v28.1.0-rc.2+incompatible
# github.com/moby/buildkit v0.21.1
# github.com/moby/buildkit v0.22.0-rc1
# github.com/docker/buildx v0.23.0
# github.com/docker/cli v28.1.1+incompatible
# github.com/docker/compose/v2 v2.36.0
Expand Down
110 changes: 59 additions & 51 deletions content/manuals/ai/model-runner.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@ params:
text: Beta
group: AI
weight: 20
description: Learn how to use Docker Model Runner to manage and run AI models.
description: Learn how to use Docker Model Runner to manage and run AI models.
keywords: Docker, ai, model runner, docker deskotp, llm
aliases:
aliases:
- /desktop/features/model-runner/
- /ai/model-runner/
---
Expand All @@ -34,8 +34,8 @@ Models are pulled from Docker Hub the first time they're used and stored locally

1. Navigate to the **Features in development** tab in settings.
2. Under the **Experimental features** tab, select **Access experimental features**.
3. Select **Apply and restart**.
4. Quit and reopen Docker Desktop to ensure the changes take effect.
3. Select **Apply and restart**.
4. Quit and reopen Docker Desktop to ensure the changes take effect.
5. Open the **Settings** view in Docker Desktop.
6. Navigate to **Features in development**.
7. From the **Beta** tab, check the **Enable Docker Model Runner** setting.
Expand All @@ -46,7 +46,7 @@ You can now use the `docker model` command in the CLI and view and interact with

### Model runner status

Check whether the Docker Model Runner is active:
Check whether the Docker Model Runner is active and displays the current inference engine:

```console
$ docker model status
Expand All @@ -55,7 +55,7 @@ $ docker model status
### View all commands

Displays help information and a list of available subcommands.

```console
$ docker model help
```
Expand All @@ -74,15 +74,15 @@ Commands:
version Show the current version
```

### Pull a model
### Pull a model

Pulls a model from Docker Hub to your local environment.

```console
$ docker model pull <model>
```

Example:
Example:

```console
$ docker model pull ai/smollm2
Expand Down Expand Up @@ -114,7 +114,13 @@ You will see something similar to:

### Run a model

Run a model and interact with it using a submitted prompt or in chat mode.
Run a model and interact with it using a submitted prompt or in chat mode. When you run a model, Docker
calls an Inference Server API endpoint hosted by the Model Runner through Docker Desktop. The model
stays in memory until another model is requested, or until a pre-defined inactivity timeout is reached (currently 5 minutes).

You do not have to use `Docker model run` before interacting with a specific model from a
host process or from within a container. Model Runner transparently loads the requested model on-demand, assuming it has been
pulled beforehand and is locally available.

#### One-time prompt

Expand Down Expand Up @@ -150,18 +156,18 @@ Chat session ended.

### Push a model to Docker Hub

Use the following command to push your model to Docker Hub:
To push your model to Docker Hub:

```console
$ docker model push <namespace>/<model>
```

### Tag a model

You can specify a particular version or variant of the model:
To specify a particular version or variant of the model:

```console
$ docker model tag
$ docker model tag
```

If no tag is provided, Docker defaults to `latest`.
Expand All @@ -171,7 +177,7 @@ If no tag is provided, Docker defaults to `latest`.
Fetch logs from Docker Model Runner to monitor activity or debug issues.

```console
$ docker model logs
$ docker model logs
```

The following flags are accepted:
Expand Down Expand Up @@ -211,53 +217,54 @@ If you want to try an existing GenAI application, follow these instructions.

4. Open you app in the browser at the addresses specified in the repository [README](https://github.com/docker/hello-genai).

You'll see the GenAI app's interface where you can start typing your prompts.
You'll see the GenAI app's interface where you can start typing your prompts.

You can now interact with your own GenAI app, powered by a local model. Try a few prompts and notice how fast the responses are — all running on your machine with Docker.

## FAQs

### What models are available?

All the available models are hosted in the [public Docker Hub namespace of `ai`](https://hub.docker.com/u/ai).
All the available models are hosted in the [public Docker Hub namespace of `ai`](https://hub.docker.com/u/ai).

### What API endpoints are available?

Once the feature is enabled, the following new APIs are available:
Once the feature is enabled, new API endpoints are available under the following base URLs:

```text
#### Inside containers ####
- From containers: `http://model-runner.docker.internal/`
- From host processes: `http://localhost:12434/`, assuming you have enabled TCP host access on default port 12434.

http://model-runner.docker.internal/
Docker Model management endpoints:

# Docker Model management
POST /models/create
GET /models
GET /models/{namespace}/{name}
DELETE /models/{namespace}/{name}
```text
POST /models/create
GET /models
GET /models/{namespace}/{name}
DELETE /models/{namespace}/{name}
```

# OpenAI endpoints
GET /engines/llama.cpp/v1/models
GET /engines/llama.cpp/v1/models/{namespace}/{name}
POST /engines/llama.cpp/v1/chat/completions
POST /engines/llama.cpp/v1/completions
POST /engines/llama.cpp/v1/embeddings
Note: You can also omit llama.cpp.
E.g., POST /engines/v1/chat/completions.
OpenAI endpoints:

#### Inside or outside containers (host) ####
```text
GET /engines/llama.cpp/v1/models
GET /engines/llama.cpp/v1/models/{namespace}/{name}
POST /engines/llama.cpp/v1/chat/completions
POST /engines/llama.cpp/v1/completions
POST /engines/llama.cpp/v1/embeddings
```

Same endpoints on /var/run/docker.sock
To call these endpoints via a Unix socket (`/var/run/docker.sock`), prefix their path with
with `/exp/vDD4.40`.

> [!NOTE]
> You can omit `llama.cpp` from the path. For example: `POST /engines/v1/chat/completions`.

# While still in Beta
Prefixed with /exp/vDD4.40
```

### How do I interact through the OpenAI API?

#### From within a container

Examples of calling an OpenAI endpoint (`chat/completions`) from within another container using `curl`:
To call the `chat/completions` OpenAI endpoint from within another container using `curl`:

```bash
#!/bin/sh
Expand All @@ -280,15 +287,18 @@ curl http://model-runner.docker.internal/engines/llama.cpp/v1/chat/completions \

```

#### From the host using a Unix socket
#### From the host using TCP

Examples of calling an OpenAI endpoint (`chat/completions`) through the Docker socket from the host using `curl`:
To call the `chat/completions` OpenAI endpoint from the host via TCP:

1. Enable the host-side TCP support from the Docker Desktop GUI, or via the [Docker Desktop CLI](/manuals/desktop/features/desktop-cli.md).
For example: `docker desktop enable model-runner --tcp <port>`.
2. Interact with it as documented in the previous section using `localhost` and the correct port.

```bash
#!/bin/sh

curl --unix-socket $HOME/.docker/run/docker.sock \
localhost/exp/vDD4.40/engines/llama.cpp/v1/chat/completions \
curl http://localhost:12434/engines/llama.cpp/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "ai/smollm2",
Expand All @@ -303,19 +313,17 @@ curl --unix-socket $HOME/.docker/run/docker.sock \
}
]
}'

```

#### From the host using TCP

In case you want to interact with the API from the host, but use TCP instead of a Docker socket, you can enable the host-side TCP support from the Docker Desktop GUI, or via the [Docker Desktop CLI](/manuals/desktop/features/desktop-cli.md). For example, using `docker desktop enable model-runner --tcp <port>`.
#### From the host using a Unix socket

Afterwards, interact with it as previously documented using `localhost` and the chosen, or the default port.
To call the `chat/completions` OpenAI endpoint through the Docker socket from the host using `curl`:

```bash
#!/bin/sh

curl http://localhost:12434/engines/llama.cpp/v1/chat/completions \
curl --unix-socket $HOME/.docker/run/docker.sock \
localhost/exp/vDD4.40/engines/llama.cpp/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "ai/smollm2",
Expand Down Expand Up @@ -354,15 +362,15 @@ Once linked, re-run the command.

### No safeguard for running oversized models

Currently, Docker Model Runner doesn't include safeguards to prevent you from launching models that exceed their systems available resources. Attempting to run a model that is too large for the host machine may result in severe slowdowns or render the system temporarily unusable. This issue is particularly common when running LLMs models without sufficient GPU memory or system RAM.
Currently, Docker Model Runner doesn't include safeguards to prevent you from launching models that exceed their system's available resources. Attempting to run a model that is too large for the host machine may result in severe slowdowns or render the system temporarily unusable. This issue is particularly common when running LLMs models without sufficient GPU memory or system RAM.

### No consistent digest support in Model CLI

The Docker Model CLI currently lacks consistent support for specifying models by image digest. As a temporary workaround, you should refer to models by name instead of digest.

## Share feedback

Thanks for trying out Docker Model Runner. Give feedback or report any bugs you may find through the **Give feedback** link next to the **Enable Docker Model Runner** setting.
Thanks for trying out Docker Model Runner. Give feedback or report any bugs you may find through the **Give feedback** link next to the **Enable Docker Model Runner** setting.

## Disable the feature

Expand All @@ -371,4 +379,4 @@ To disable Docker Model Runner:
1. Open the **Settings** view in Docker Desktop.
2. Navigate to the **Beta** tab in **Features in development**.
3. Clear the **Enable Docker Model Runner** checkbox.
4. Select **Apply & restart**.
4. Select **Apply & restart**.
6 changes: 0 additions & 6 deletions content/manuals/compose/support-and-feedback/feedback.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,6 @@ aliases:

There are many ways you can provide feedback on Docker Compose.

### In-product feedback

If you have obtained Docker Compose through Docker Desktop, you can use the `docker feedback` command to submit feedback directly from the command line.

<script async id="asciicast-KkC0fFrhV8nAzvXUGqay06UXx" src="https://asciinema.org/a/KkC0fFrhV8nAzvXUGqay06UXx.js"></script>

### Report bugs or problems on GitHub

To report bugs or problems, visit [Docker Compose on GitHub](https://github.com/docker/compose/issues)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,6 @@ There are many ways you can provide feedback on Docker Desktop or Docker Desktop

On each Docker Desktop Dashboard view, there is a **Give feedback** link. This opens a feedback form where you can share ideas directly with the Docker team.

You can also use the `docker feedback` command to submit feedback directly from the command line.

<script async id="asciicast-KkC0fFrhV8nAzvXUGqay06UXx" src="https://asciinema.org/a/KkC0fFrhV8nAzvXUGqay06UXx.js"></script>

### Feedback via Docker Community forums

To get help from the community, review current user topics, join or start a
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -343,13 +343,21 @@ Also, the `\` character has a special meaning in Git Bash.

Portability of the scripts is not affected as Linux treats multiple `/` as a single entry.

### Docker Desktop fails due to Virtualization settings
### Docker Desktop fails due to Virtualization not working

#### Error message

A typical error message is "Docker Desktop - Unexpected WSL error" mentioning the error code
`Wsl/Service/RegisterDistro/CreateVm/HCS/HCS_E_HYPERV_NOT_INSTALLED`. Manually executing `wsl` commands
also fails with the same error code.

#### Cause

- Virtualization settings are disabled in the BIOS.
- Windows Hyper-V or WSL 2 components are missing.

Note some third-party software such as Android emulators will disable Hyper-V on install.

#### Solutions

Your machine must have the following features for Docker Desktop to function correctly:
Expand All @@ -364,6 +372,21 @@ Your machine must have the following features for Docker Desktop to function cor

![WSL 2 enabled](../../images/wsl2-enabled.png)

It must be possible to run WSL 2 commands without error, for example:

```console
PS C:\users\> wsl -l -v
NAME STATE VERSION
* Ubuntu Running 2
docker-desktop Stopped 2
PS C:\users\> wsl -d docker-desktop echo WSL 2 is working
WSL 2 is working
```

If the features are enabled but the commands are not working, first check [Virtualization is turned on](#virtualization-must-be-turned-on)
then [enable the Hypervisor at Windows startup](#hypervisor-enabled-at-windows-startup) if required. If running Docker
Desktop in a Virtual Machine, ensure [the hypervisor has nested virtualization enabled](#turn-on-nested-virtualization).

##### Hyper-V

On Windows 10 Pro or Enterprise, you can also use Hyper-V with the following features enabled:
Expand Down
Loading