fix(anthropic): temp disable beta apis instrumentation #3258
Conversation
WalkthroughDisables beta instrumentation entries for Anthropic and Bedrock in the Anthropic OpenTelemetry integration and marks several Bedrock raw-response tests as skipped. Additional non-functional formatting and logging string layout adjustments were made. Changes
Sequence Diagram(s)sequenceDiagram
actor Dev as Developer Code
participant App as Application
participant Inst as OTel Anthropic Instrumentation
participant SDK as Anthropic/Bedrock SDK
Note over App,SDK: Previous behavior (beta enabled)
App->>Inst: Call beta SDK method
Inst->>SDK: Wrapped call (start span, attrs)
SDK-->>Inst: Response
Inst-->>App: Response (span ended)
Note over App,SDK: Current behavior (beta disabled)
App->>SDK: Call beta SDK method (no wrap)
SDK-->>App: Response (no span)
Estimated code review effort🎯 2 (Simple) | ⏱️ ~8 minutes Possibly related PRs
Suggested reviewers
Poem
Tip 🔌 Remote MCP (Model Context Protocol) integration is now available!Pro plan users can now connect to remote MCP servers from the Integrations page. Connect with popular remote MCPs such as Notion and Linear to add more context to your reviews and chats. ✨ Finishing Touches
🧪 Generate unit tests
🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. CodeRabbit Commands (Invoked using PR/Issue comments)Type Other keywords and placeholders
CodeRabbit Configuration File (
|
galkleinman
changed the title
![ellipsis-dev[bot]](https://avatars.githubusercontent.com/in/64358?s=60&v=4){kind=small}
There was a problem hiding this comment.
Caution
Changes requested ❌
Reviewed everything up to 1728094 in 2 minutes and 38 seconds. Click for details.
- Reviewed
286lines of code in2files - Skipped
0files when reviewing. - Skipped posting
6draft comments. View those below. - Modify your settings and rules to customize what types of comments Ellipsis leaves. And don't forget to react with 👍 or 👎 to teach Ellipsis.
1. packages/opentelemetry-instrumentation-anthropic/opentelemetry/instrumentation/anthropic/__init__.py:443
- Draft comment:
The multi-line import syntax (e.g. for aset_response_attributes) improves readability. Good job keeping a consistent import style. - Reason this comment was not posted:
Confidence changes required:0%<= threshold50%None
2. packages/opentelemetry-instrumentation-anthropic/tests/test_bedrock_with_raw_response.py:28
- Draft comment:
Beta API tests are marked as skipped. If beta instrumentation is permanently disabled, consider removing these tests to keep the suite clear and maintainable. - Reason this comment was not posted:
Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 10% vs. threshold = 50% The comment is partially incorrect - only one of the three skipped tests is actually a beta API test. The other two are regular API tests. The comment makes assumptions about why the tests are skipped without clear evidence. We don't know from the diff why these tests were skipped - it could be temporary. The comment raises a valid point about test maintenance, but makes incorrect assumptions about which tests are beta tests. I could be missing context about why these tests were skipped. Given the rules state we should only keep comments with strong evidence they are correct, this comment's incorrect assumptions and speculative nature make it unsuitable. Delete the comment because it makes incorrect assumptions about which tests are beta tests and speculates about the reason for skipping without evidence.
3. packages/opentelemetry-instrumentation-anthropic/tests/test_bedrock_with_raw_response.py:61
- Draft comment:
The multi-line formatting for extracting response content improves readability. Nice cleanup! - Reason this comment was not posted:
Confidence changes required:0%<= threshold50%None
4. packages/opentelemetry-instrumentation-anthropic/opentelemetry/instrumentation/anthropic/__init__.py:84
- Draft comment:
Typographical error: The comment marker has an extra '#' at the beginning. Consider using a single '#' for consistency, e.g. '# Beta API methods (regular Anthropic SDK)'. - Reason this comment was not posted:
Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 0% vs. threshold = 50% The double # appears to be intentional - it's used to comment out an entire block of code while preserving the original comment structure. The first # is part of the block comment-out, while the second # is the original comment marker. This is a common and valid practice when temporarily disabling code blocks. Could I be wrong about the intention? Maybe the double # was accidental. But given the systematic use of double # throughout the commented out block, it seems clearly intentional. The systematic and consistent use of double # throughout multiple blocks strongly suggests this is intentional code commenting, not a typo. The comment should be deleted as it misunderstands intentional code commenting structure.
5. packages/opentelemetry-instrumentation-anthropic/opentelemetry/instrumentation/anthropic/__init__.py:97
- Draft comment:
Typographical error: The comment marker here has an extra '#' at the beginning. Suggest using a single '#' for clarity, e.g. '# Beta API methods (Bedrock SDK)'. - Reason this comment was not posted:
Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 0% vs. threshold = 50% The double # appears to be intentional as it's used consistently throughout multiple blocks of commented code. This is a common Python style to indicate a block of code that is more permanently commented out vs a single # for regular comments. The comment is suggesting a style change that would actually make the code less clear. Could the double # actually be intentional to distinguish these commented blocks from regular comments? Looking at the full context, that seems to be the case. Yes, the double # is clearly intentional and consistent throughout the file. This is not a typographical error but a deliberate style choice. The comment should be deleted as it incorrectly identifies an intentional style choice as an error.
6. packages/opentelemetry-instrumentation-anthropic/opentelemetry/instrumentation/anthropic/__init__.py:125
- Draft comment:
Typographical issue: There appears to be an extra '#' character in the comment on this line ("# # Beta API async methods (regular Anthropic SDK)"). Please remove one '#' if this is unintentional. - Reason this comment was not posted:
Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 0% vs. threshold = 50% The double '#' appears to be intentional formatting to show comment hierarchy. The entire block uses this consistently. The comment is suggesting a change that would actually make the formatting less consistent. Additionally, this is just a style nitpick about comments in commented-out code, which is not particularly important. Could the double '#' be confusing to other developers? Could it violate some Python style guide I'm not aware of? Even if it's not standard Python style, this is clearly an intentional formatting choice made consistently throughout the file. The benefit of consistency outweighs any style guide concerns, especially for commented out code. Delete this comment. It's suggesting a change that would make the code less consistent, and it's a trivial style issue about comments in commented-out code.
Workflow ID: wflow_ub8Y5I4tafbwjIUL
You can customize 
| "method": "stream", | ||
| "span_name": "anthropic.chat", | ||
| }, | ||
| # # Beta API methods (regular Anthropic SDK) |
There was a problem hiding this comment.
Temporarily disabled beta API instrumentation by commenting out blocks. Consider using a feature flag or config option to conditionally enable beta instrumentation instead of leaving commented-out code.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (1)
packages/opentelemetry-instrumentation-anthropic/opentelemetry/instrumentation/anthropic/__init__.py (1)
125-151: Mirror the feature-flag approach for async beta entriesApply the same opt-in gating for async wrappers to keep the behavior consistent.
- # # Beta API async methods (regular Anthropic SDK) - # { - # "package": "anthropic.resources.beta.messages.messages", - # "object": "AsyncMessages", - # "method": "create", - # "span_name": "anthropic.chat", - # }, - # { - # "package": "anthropic.resources.beta.messages.messages", - # "object": "AsyncMessages", - # "method": "stream", - # "span_name": "anthropic.chat", - # }, - # # Beta API async methods (Bedrock SDK) - # { - # "package": "anthropic.lib.bedrock._beta_messages", - # "object": "AsyncMessages", - # "method": "create", - # "span_name": "anthropic.chat", - # }, - # { - # "package": "anthropic.lib.bedrock._beta_messages", - # "object": "AsyncMessages", - # "method": "stream", - # "span_name": "anthropic.chat", - # }, + # Optionally enable beta API instrumentation for async methods + *( + [ + { + "package": "anthropic.resources.beta.messages.messages", + "object": "AsyncMessages", + "method": "create", + "span_name": "anthropic.chat", + }, + { + "package": "anthropic.resources.beta.messages.messages", + "object": "AsyncMessages", + "method": "stream", + "span_name": "anthropic.chat", + }, + { + "package": "anthropic.lib.bedrock._beta_messages", + "object": "AsyncMessages", + "method": "create", + "span_name": "anthropic.chat", + }, + { + "package": "anthropic.lib.bedrock._beta_messages", + "object": "AsyncMessages", + "method": "stream", + "span_name": "anthropic.chat", + }, + ] + if os.getenv("TRACELOOP_ENABLE_ANTHROPIC_BETA", "false").lower() == "true" + else [] + ),
🧹 Nitpick comments (5)
packages/opentelemetry-instrumentation-anthropic/opentelemetry/instrumentation/anthropic/__init__.py (2)
84-110: Prefer feature-flag gating over commented-out beta entriesCommenting out list entries is brittle. Gate the beta wrap entries behind an env/config flag so they can be re-enabled without code churn and to keep the list consistent.
Apply this diff to make beta entries opt-in via a flag (keeps them disabled by default):
- # # Beta API methods (regular Anthropic SDK) - # { - # "package": "anthropic.resources.beta.messages.messages", - # "object": "Messages", - # "method": "create", - # "span_name": "anthropic.chat", - # }, - # { - # "package": "anthropic.resources.beta.messages.messages", - # "object": "Messages", - # "method": "stream", - # "span_name": "anthropic.chat", - # }, - # # Beta API methods (Bedrock SDK) - # { - # "package": "anthropic.lib.bedrock._beta_messages", - # "object": "Messages", - # "method": "create", - # "span_name": "anthropic.chat", - # }, - # { - # "package": "anthropic.lib.bedrock._beta_messages", - # "object": "Messages", - # "method": "stream", - # "span_name": "anthropic.chat", - # }, + # Optionally enable beta API instrumentation if explicitly requested + *( + [ + { + "package": "anthropic.resources.beta.messages.messages", + "object": "Messages", + "method": "create", + "span_name": "anthropic.chat", + }, + { + "package": "anthropic.resources.beta.messages.messages", + "object": "Messages", + "method": "stream", + "span_name": "anthropic.chat", + }, + { + "package": "anthropic.lib.bedrock._beta_messages", + "object": "Messages", + "method": "create", + "span_name": "anthropic.chat", + }, + { + "package": "anthropic.lib.bedrock._beta_messages", + "object": "Messages", + "method": "stream", + "span_name": "anthropic.chat", + }, + ] + if os.getenv("TRACELOOP_ENABLE_ANTHROPIC_BETA", "false").lower() == "true" + else [] + ),Follow-up: If you prefer a code-level toggle, consider a Config flag instead of env.
785-791: Avoid f-strings in logging to preserve lazy interpolationUsing f-strings eagerly formats the message even when DEBUG is disabled. Prefer
%s-style arguments to leverage the logger’s lazy formatting.- logger.debug( - f"Successfully wrapped {wrap_package}.{wrap_object}.{wrap_method}" - ) + logger.debug( + "Successfully wrapped %s.%s.%s", + wrap_package, + wrap_object, + wrap_method, + ) except Exception as e: - logger.debug( - f"Failed to wrap {wrap_package}.{wrap_object}.{wrap_method}: {e}" - ) + logger.debug( + "Failed to wrap %s.%s.%s: %s", + wrap_package, + wrap_object, + wrap_method, + e, + )packages/opentelemetry-instrumentation-anthropic/tests/test_bedrock_with_raw_response.py (3)
28-28: Add an explicit skip reason for traceabilityProvide context on why this test is skipped and when to re-enable.
-@pytest.mark.skip +@pytest.mark.skip(reason="Temporarily disabled: Bedrock/raw-response instrumentation is off in PR #3258")
83-83: Add an explicit skip reason for this test as wellHelps future maintainers understand why it's skipped.
-@pytest.mark.skip +@pytest.mark.skip(reason="Temporarily disabled: Bedrock instrumentation paths are not wrapped in PR #3258")
132-132: Add an explicit skip reason for the beta-with-raw-response testSame rationale as above.
-@pytest.mark.skip +@pytest.mark.skip(reason="Temporarily disabled: Anthropic beta Bedrock instrumentation disabled in PR #3258")
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these settings in your CodeRabbit configuration.
📒 Files selected for processing (2)
packages/opentelemetry-instrumentation-anthropic/opentelemetry/instrumentation/anthropic/__init__.py(7 hunks)packages/opentelemetry-instrumentation-anthropic/tests/test_bedrock_with_raw_response.py(5 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
**/*.py
📄 CodeRabbit Inference Engine (CLAUDE.md)
**/*.py: Python code must conform to Flake8 linting rules
Do not hardcode API keys in source code; read them from environment variables or a secure vault
Files:
packages/opentelemetry-instrumentation-anthropic/tests/test_bedrock_with_raw_response.pypackages/opentelemetry-instrumentation-anthropic/opentelemetry/instrumentation/anthropic/__init__.py
🧬 Code Graph Analysis (2)
packages/opentelemetry-instrumentation-anthropic/tests/test_bedrock_with_raw_response.py (1)
packages/opentelemetry-instrumentation-milvus/tests/conftest.py (1)
reader(37-41)
packages/opentelemetry-instrumentation-anthropic/opentelemetry/instrumentation/anthropic/__init__.py (2)
packages/opentelemetry-instrumentation-anthropic/opentelemetry/instrumentation/anthropic/span_utils.py (1)
aset_response_attributes(307-328)packages/opentelemetry-instrumentation-anthropic/opentelemetry/instrumentation/anthropic/utils.py (1)
ashared_metrics_attributes(138-147)
🔇 Additional comments (10)
packages/opentelemetry-instrumentation-anthropic/opentelemetry/instrumentation/anthropic/__init__.py (4)
186-186: No-op/whitespace-only changeNo functional impact. Nothing to do.
280-280: No-op/whitespace-only changeNo functional impact. Nothing to do.
448-451: Function-local import to avoid circular dependency is acceptableMoving
aset_response_attributesimport into the coroutine avoids import-time cycles (span_utils imports back into this module). This is a reasonable trade-off; the overhead is negligible on the happy path.
677-680: Deferred import to prevent cycles for async metrics attributes is fineLazy-importing
ashared_metrics_attributeshere is acceptable and keeps module import clean.packages/opentelemetry-instrumentation-anthropic/tests/test_bedrock_with_raw_response.py (6)
24-25: Trailing comma in fixture call is fineNo behavioral change; keeps diffs cleaner on future edits.
32-37: Parameter reflow is fineFormatting-only; no behavior change.
61-65: LGTM: robust raw-response content extractionThe
hasattr(response, "parse")guard is a pragmatic way to deal with raw responses vs parsed objects.
87-92: Confirm intention to skip non-beta Bedrock regular createThis test targets regular
messages.createon Bedrock, not a beta endpoint. Are we disabling Bedrock instrumentation entirely (not only beta)? If not, consider keeping this test active.If the intent is to skip only when Bedrock is not instrumented, convert to a conditional skip:
-@pytest.mark.skip +@pytest.mark.skipif( + os.getenv("TRACELOOP_ENABLE_ANTHROPIC_BEDROCK", "false").lower() != "true", + reason="Bedrock instrumentation disabled", +)Alternatively, if Bedrock instrumentation truly isn’t supported now, keep it skipped but document the reason (see the previous comment).
136-141: Multiline await call improves readabilityParenthesized, multiline style in the await call is fine and keeps arguments diff-friendly.
Also applies to: 143-154
167-171: LGTM: consistent content extraction for raw responsesMatches the earlier test; keeps behavior consistent across raw/parsed responses.
| # # Beta API methods (regular Anthropic SDK) | ||
| # { | ||
| # "package": "anthropic.resources.beta.messages.messages", | ||
| # "object": "Messages", | ||
| # "method": "create", | ||
| # "span_name": "anthropic.chat", | ||
| # }, | ||
| # { | ||
| # "package": "anthropic.resources.beta.messages.messages", | ||
| # "object": "Messages", | ||
| # "method": "stream", | ||
| # "span_name": "anthropic.chat", | ||
| # }, | ||
| # # Beta API methods (Bedrock SDK) | ||
| # { | ||
| # "package": "anthropic.lib.bedrock._beta_messages", | ||
| # "object": "Messages", | ||
| # "method": "create", | ||
| # "span_name": "anthropic.chat", | ||
| # }, | ||
| # { | ||
| # "package": "anthropic.lib.bedrock._beta_messages", | ||
| # "object": "Messages", | ||
| # "method": "stream", | ||
| # "span_name": "anthropic.chat", | ||
| # }, | ||
| ] |
There was a problem hiding this comment.
💡 Verification agent
🧩 Analysis chain
Sanity-check: ensure no remaining references to now-disabled beta wrappers
Since beta wrappers are disabled, verify there are no left-over tests or code paths expecting those spans.
Run this script to locate potential references to disabled beta APIs across the repo:
Also applies to: 125-151
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Find beta messages and bedrock beta wrapper usages
rg -n -A2 -B2 'resources\.beta\.messages|lib\.bedrock\._beta_messages' || true
# Find bedrock async client usages to inspect related tests
rg -n -A3 -B3 'AsyncAnthropicBedrock|with_raw_response' || trueLength of output: 24853
Remove stale beta wrapper references
The tests and commented‐out code for beta wrappers remain in the repo even though beta spans are now disabled. Please clean these up:
• packages/opentelemetry-instrumentation-anthropic/opentelemetry/instrumentation/anthropic/init.py
– Lines 84–110 and 125–151: commented‐out beta API entries for both regular and async methods.
→ Consider deleting these blocks if they’re no longer needed.
• packages/opentelemetry-instrumentation-anthropic/tests/test_bedrock_with_raw_response.py
– Lines 132–147: test_async_anthropic_bedrock_beta_with_raw_response still invokes async_anthropic_bedrock_client.beta.messages.with_raw_response.create.
→ Remove or adapt this test to reflect that beta wrappers have been disabled.
🤖 Prompt for AI Agents
packages/opentelemetry-instrumentation-anthropic/opentelemetry/instrumentation/anthropic/__init__.py
lines 84–110 and 125–151: the file contains stale, commented-out beta API
wrapper entries for regular and async methods; delete these commented blocks to
remove dead code and keep the file clean.
packages/opentelemetry-instrumentation-anthropic/tests/test_bedrock_with_raw_response.py
lines 132–147: the test calls
async_anthropic_bedrock_client.beta.messages.with_raw_response.create which
relies on disabled beta wrappers; remove or modify this test to call the
supported non-beta API (or delete the test if no equivalent exists) and update
any assertions accordingly.
2 participants
Important
Disable beta API instrumentation in
__init__.pyand skip related tests intest_bedrock_with_raw_response.py.__init__.py, disabling their instrumentation.@pytest.mark.skiptotest_async_anthropic_bedrock_with_raw_response,test_async_anthropic_bedrock_regular_create, andtest_async_anthropic_bedrock_beta_with_raw_responseintest_bedrock_with_raw_response.py, skipping these tests.__init__.pyandtest_bedrock_with_raw_response.py.This description was created by
for 1728094. You can customize this summary. It will automatically update as commits are pushed.
Summary by CodeRabbit