Skip to content

Commit ab59a37

Browse files
author
Hemant Jain
authored
Add support for String input/output (triton-inference-server#60)
* x * Fix string input tensor support * Fix string list creation * Add support for String output * cleanup * review edits * Readme fixes * review edits - throw error during model loading * follow up edits * Fix typo in check * Use verbose checking of output type to ensure appropriate error message is returned during model execution * cleanup * Wrap raw pointer to allow auto freeing of memory * follow up review edits
1 parent 510cc49 commit ab59a37

File tree

2 files changed

+471
-68
lines changed

2 files changed

+471
-68
lines changed

README.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -208,3 +208,11 @@ complex execution modes and dynamic shapes. If not specified, all are enabled by
208208
Setting the parameter `DISABLE_OPTIMIZED_EXECUTION` to "true" in the model
209209
configuration may help in some cases to avoid these negative interactions
210210
due to model specific caching and increase multiple instance performance.
211+
212+
* PyTorch does not support Tensor of Strings but it does support models that accept
213+
a List of Strings as input(s) / produces a List of String as output(s). For these models
214+
Triton allows users to pass String input(s)/recieve String output(s) using the String
215+
datatype. As a limitation of using List instead of Tensor for String I/O, only for
216+
1-dimensional input(s)/output(s) are supported for I/O of String type.
217+
Batching is not allowed for PyTorch models with String I/O. For these models,
218+
the user must specify `max_batch_size: 0` in the configuration.

0 commit comments

Comments
 (0)