…r#394)

* Add parameters support to InferResponse

* Infer response to track parameters

* Add parameters to binding infer response

* Rank parameters argument up among InferResponse constructor arguments

* Add setting parameters to Triton response

* Send response parameters only on non-error

* Fix double declaration

* Unify py dictionary parameters to json str

* Add documentation

* Mark response parameters accessor const and JSON serializable

* [Docs] Note BLS response parameters are not populated currently

* [comment] Clarify why PbTensor::LoadFromSharedMemory() requires holding GIL

…iton-inference-server#395)

…el (triton-inference-server#396)

…equest cancellation (triton-inference-server#398)

Adding a cancel() method to the BLS decoupled response iterator, so that it will be able to cancel the Triton server inference request corresponding to the response iterator if the stub process gets the enough response from the response iterator.

Due to each stub InferenceRequest object can create multiple BLS Triton Server inference requests, so adding cancel() to the response iterator would be more feasible to manage cancelling individual request rather than cancelling all requests generated with the stub InferenceRequest object.

More details can be found in the change of the README.md

…ce-server#403)

…server#405)

…rence-server#406)

…ch has support for CUDA new layout (triton-inference-server#407)

…ailures (triton-inference-server#408)

Validate input parameters used within python_backend model load

…+ implementation (triton-inference-server#416)

* perf: optimize string tensor deserialization with high performance c++ implementation

* Address PR comments

---------

Co-authored-by: Wei Chen <[email protected]>