Skip to content

Conversation

brickZA
Copy link

@brickZA brickZA commented Jul 24, 2025

Commit 3d5fecc added partial support for passing request-params through to the AWS Bedrock backend. This commit plumbs it all the way through to gptel--request-data

I tested it with the following config:

  (setq gptel-model   'claude-3-7-sonnet-20250219 ;'claude-sonnet-4-20250514
        gptel-backend
        (gptel-make-bedrock "AWS"
          ;; optionally enable streaming
          :stream t
          :region "eu-west-1"
          ;; subset of gptel--bedrock-models
          :models '(claude-sonnet-4-20250514 claude-3-7-sonnet-20250219)
          ;; Model region for cross-region inference profiles. Required for models such
          ;; as Claude without on-demand throughput support. One of 'apac, 'eu or 'us.
          ;; https://docs.aws.amazon.com/bedrock/latest/userguide/inference-profiles-use.html
          :request-params '(:additionalModelRequestFields (:thinking (:type "enabled" :budget_tokens 2048))
                      :inferenceConfig (:max_tokens 4096))
          :model-region 'eu)

It does not look like the Bedrock parser understands thinking yet, but I can confirm that it worked by looking at the debug output. We note the added fields in the request body:

{
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "text": "Testing!"
        }
      ]
    }
  ],
  "inferenceConfig": {
    "max_tokens": 4096
  },
  "system": [
    {
      "text": "You are a large language model living in Emacs and a helpful assistant. Respond concisely."
    }
  ],
  "temperature": 1.0,
  "additionalModelRequestFields": {
    "thinking": {
      "type": "enabled",
      "budget_tokens": 2048
    }
  }
}

and the bedrock response now contains reasoningContent events (after running through the strings command):

event{"contentBlockIndex":0,"delta":{"reasoningContent":{"text":"The"}},"p":"abcdef"}
...
event{"contentBlockIndex":0,"delta":{"reasoningContent":{"text":" user is just saying \""}},"p":"abcdefghijklmnopqrstuvw"}
...
event{"contentBlockIndex":0,"delta":{"reasoningContent":{"text":"Testing!\". This looks like they"}},"p":"abcdefghijkl"}
...

@brickZA brickZA force-pushed the bedrock-request-parms-fix branch from 44c9eef to f94c01c Compare July 25, 2025 07:14
@karthink karthink force-pushed the bedrock-request-parms-fix branch from f94c01c to bc2bcd5 Compare July 25, 2025 18:58
* gptel-bedrock.el (gptel--request-data): Commit
3d5fecc added partial support for
passing `request-params' through to the AWS Bedrock backend. This
commit plumbs it all the way through to `gptel--request-data'.
@karthink karthink force-pushed the bedrock-request-parms-fix branch from bc2bcd5 to 73ee1f0 Compare July 25, 2025 18:59
@karthink karthink merged commit 73ee1f0 into karthink:master Jul 25, 2025
@karthink
Copy link
Owner

@brickZA Thanks for the PR!

@brickZA
Copy link
Author

brickZA commented Jul 30, 2025

No problem, just helping myself :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants