Skip to content

Commit dccee59

Browse files
authored
Update docs in accordance with the new auto-complete-config behavior (triton-inference-server#174)
* Update docs in accordance with the new auto-complete-config behavior * Address the review comments to fix the language
1 parent 7a6d46d commit dccee59

File tree

5 files changed

+39
-44
lines changed

5 files changed

+39
-44
lines changed

README.md

Lines changed: 13 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -198,14 +198,14 @@ class TritonPythonModel:
198198

199199
@staticmethod
200200
def auto_complete_config(auto_complete_model_config):
201-
"""`auto_complete_config` is called only once when the server is started
202-
with `--strict-model-config=false`. Implementing this function is optional.
203-
A no implementation of `auto_complete_config` will do nothing. This function
204-
can be used to set `max_batch_size`, `input` and `output` properties of the
205-
model using `set_max_batch_size`, `add_input`, and `add_output`.
206-
These properties will allow Triton to load the model with minimal model
207-
configuration in absence of a configuration file. This function returns the
208-
`pb_utils.ModelConfig` object with these properties. You can use `as_dict`
201+
"""`auto_complete_config` is called only once when loading the model assuming
202+
the server was not started with `--disable-auto-complete-config`. Implementing
203+
this function is optional. No implementation of `auto_complete_config` will
204+
do nothing. This function can be used to set `max_batch_size`, `input` and
205+
`output` properties of the model using `set_max_batch_size`, `add_input`, and
206+
`add_output`. These properties will allow Triton to load the model with minimal
207+
model configuration in absence of a configuration file. This function returns
208+
the `pb_utils.ModelConfig` object with these properties. You can use the `as_dict`
209209
function to gain read-only access to the `pb_utils.ModelConfig` object.
210210
The `pb_utils.ModelConfig` object being returned from here will be used as
211211
the final configuration for the model.
@@ -345,10 +345,10 @@ Every Python backend can implement four main functions:
345345

346346
### `auto_complete_config`
347347

348-
`auto_complete_config` is called only once when the server is started
349-
with [`--strict-model-config=false`](
350-
https://github.com/triton-inference-server/server/blob/main/docs/model_configuration.md#auto-generated-model-configuration).
351-
Implementing this function is optional. A no implementation of
348+
`auto_complete_config` is called only once when loading the model assuming
349+
the server was not started with [`--disable-auto-complete-config`](https://github.com/triton-inference-server/server/blob/main/docs/model_configuration.md#auto-generated-model-configuration).
350+
351+
Implementing this function is optional. No implementation of
352352
`auto_complete_config` will do nothing. This function can be used to set
353353
[`max_batch_size`](
354354
https://github.com/triton-inference-server/server/blob/main/docs/model_configuration.md#maximum-batch-size),
@@ -363,7 +363,7 @@ properties of the model using `set_max_batch_size`, `set_dynamic_batching`, `add
363363
[minimal model configuration](
364364
https://github.com/triton-inference-server/server/blob/main/docs/model_configuration.md#minimal-model-configuration)
365365
in absence of a configuration file. This function returns the
366-
`pb_utils.ModelConfig` object with these properties. You can use `as_dict`
366+
`pb_utils.ModelConfig` object with these properties. You can use the `as_dict`
367367
function to gain read-only access to the `pb_utils.ModelConfig` object.
368368
The `pb_utils.ModelConfig` object being returned from here will be used as the
369369
final configuration for the model.

examples/auto_complete/README.md

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -69,11 +69,10 @@ $ cp examples/auto_complete/batch_model.py models/batch_auto_complete/1/model.py
6969
**Note that we don't need a model configuration file since Triton will use the
7070
auto-complete model configuration provided in the Python model.**
7171

72-
2. Start the tritonserver with
73-
[`--strict-model-config=false`](https://github.com/triton-inference-server/server/blob/main/docs/model_configuration.md#auto-generated-model-configuration):
72+
2. Start the tritonserver:
7473

7574
```
76-
tritonserver --model-repository `pwd`/models --strict-model-config=false
75+
tritonserver --model-repository `pwd`/models
7776
```
7877

7978
## Running inferences on Nobatch and Batch models:

examples/auto_complete/batch_model.py

Lines changed: 11 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -40,19 +40,17 @@ class TritonPythonModel:
4040

4141
@staticmethod
4242
def auto_complete_config(auto_complete_model_config):
43-
"""`auto_complete_config` is called only once when the server is
44-
started with `--strict-model-config=false`. Implementing this
45-
function is optional. A no implementation of `auto_complete_config`
46-
will do nothing. This function can be used to set `max_batch_size`,
47-
`input` and `output` properties of the model using
48-
`set_max_batch_size`, `add_input`, and `add_output`. These properties
49-
will allow Triton to load the model with minimal model configuration
50-
in absence of a configuration file. This function returns the
51-
`pb_utils.ModelConfig` object with these properties. You can use
52-
`as_dict` function to gain read-only access to the
53-
`pb_utils.ModelConfig` object. The `pb_utils.ModelConfig` object being
54-
returned from here will be used as the final configuration for the
55-
model.
43+
"""`auto_complete_config` is called only once when loading the model assuming
44+
the server was not started with `--disable-auto-complete-config`. Implementing
45+
this function is optional. No implementation of `auto_complete_config` will
46+
do nothing. This function can be used to set `max_batch_size`, `input` and
47+
`output` properties of the model using `set_max_batch_size`, `add_input`, and
48+
`add_output`. These properties will allow Triton to load the model with minimal
49+
model configuration in absence of a configuration file. This function returns
50+
the `pb_utils.ModelConfig` object with these properties. You can use the `as_dict`
51+
function to gain read-only access to the `pb_utils.ModelConfig` object.
52+
The `pb_utils.ModelConfig` object being returned from here will be used as
53+
the final configuration for the model.
5654
5755
Note: The Python interpreter used to invoke this function will be
5856
destroyed upon returning from this function and as a result none of

examples/auto_complete/nobatch_model.py

Lines changed: 11 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -40,19 +40,17 @@ class TritonPythonModel:
4040

4141
@staticmethod
4242
def auto_complete_config(auto_complete_model_config):
43-
"""`auto_complete_config` is called only once when the server is
44-
started with `--strict-model-config=false`. Implementing this
45-
function is optional. A no implementation of `auto_complete_config`
46-
will do nothing. This function can be used to set `max_batch_size`,
47-
`input` and `output` properties of the model using
48-
`set_max_batch_size`, `add_input`, and `add_output`. These properties
49-
will allow Triton to load the model with minimal model configuration
50-
in absence of a configuration file. This function returns the
51-
`pb_utils.ModelConfig` object with these properties. You can use
52-
`as_dict` function to gain read-only access to the
53-
`pb_utils.ModelConfig` object. The `pb_utils.ModelConfig` object being
54-
returned from here will be used as the final configuration for the
55-
model.
43+
"""`auto_complete_config` is called only once when loading the model assuming
44+
the server was not started with `--disable-auto-complete-config`. Implementing
45+
this function is optional. No implementation of `auto_complete_config` will
46+
do nothing. This function can be used to set `max_batch_size`, `input` and
47+
`output` properties of the model using `set_max_batch_size`, `add_input`, and
48+
`add_output`. These properties will allow Triton to load the model with minimal
49+
model configuration in absence of a configuration file. This function returns
50+
the `pb_utils.ModelConfig` object with these properties. You can use the `as_dict`
51+
function to gain read-only access to the `pb_utils.ModelConfig` object.
52+
The `pb_utils.ModelConfig` object being returned from here will be used as
53+
the final configuration for the model.
5654
5755
Note: The Python interpreter used to invoke this function will be
5856
destroyed upon returning from this function and as a result none of

src/pb_stub.cc

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -197,8 +197,8 @@ Stub::RunCommand()
197197
}
198198
switch (ipc_message->Command()) {
199199
case PYTHONSTUB_CommandType::PYTHONSTUB_AutoCompleteRequest: {
200-
// Only run this case when Triton Server is started with
201-
// '--strict-model-config=false'
200+
// Only run this case when auto complete was requested by
201+
// Triton core.
202202
bool has_exception = false;
203203
std::string error_string;
204204
std::string auto_complete_config;

0 commit comments

Comments
 (0)