Skip to content

Commit 74722ba

Browse files
authored
Add custom parameters documentation (triton-inference-server#288)
1 parent 0476ee4 commit 74722ba

File tree

1 file changed

+25
-0
lines changed

1 file changed

+25
-0
lines changed

README.md

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1609,6 +1609,31 @@ how to adjust them dynamically, please see Triton's
16091609
[logging extension](https://github.com/triton-inference-server/server/blob/main/docs/protocol/extension_logging.md)
16101610
documentation.
16111611

1612+
# Adding Custom Parameters in the Model Configuration
1613+
1614+
If your model requires custom parameters in the configuration, you can specify
1615+
that in the `parameters` section of the model config. For example:
1616+
1617+
```
1618+
parameters {
1619+
key: "custom_key"
1620+
value: {
1621+
string_value: "custom_value"
1622+
}
1623+
}
1624+
```
1625+
1626+
Now you can access this parameter in the `args` argument of the `initialize`
1627+
function:
1628+
1629+
```python
1630+
def initialize(self, args):
1631+
print(json.loads(args['model_config'])['parameters'])
1632+
# Should print {'custom_key': {'string_value': 'custom_value'}}
1633+
```
1634+
1635+
1636+
16121637
# Reporting problems, asking questions
16131638

16141639
We appreciate any feedback, questions or bug reporting regarding this

0 commit comments

Comments
 (0)