Bug in API OpenAI Compatibility Mode: "Request contains an invalid argument."

[svilupp]

This is a bug in the Gemini API OpenAI Compatibility mode -- I wasn't sure where else to file it. Please let me know if there is a better place to file this.

Situation: If you provide response_format (tool) with a union type of str|int, you'll get the following error:
[{'error': {'code': 400, 'message': 'Request contains an invalid argument.', 'status': 'INVALID_ARGUMENT'}}]

It happens only to Gemini API, it doesn't happen to other APIs (together, OpenAI, etc.)
It happens with all models I've tried, it's model agnostic.

Environment details

• OS type and version: MacOS Sonoma 14.5
• Python version: 3.13
• pip version: 23.3
• google-api-python-client version: --

Steps to reproduce

1. Run the following script
2. If you remove the Union from id_number, it will work fine. It's most likely missing support for the AnyOf operator in JSON schema

Code example

Focus on the id_number type below! Change to type str and it will work.

import httpx
import os
import json
from pydantic import BaseModel
from typing import Union


# Define a Pydantic model for the structured output
class Person(BaseModel):
    name: str
    # The crucial Union field that can be either int or str
    id_number: Union[int, str]


def parse_data(resume_text: str) -> Person:
    try:
        api_key = os.environ.get("GOOGLE_API_KEY")
        if not api_key:
            raise ValueError("GOOGLE_API_KEY environment variable not set")

        # Prepare the request payload
        def add_additional_properties_false(obj):
            if isinstance(obj, dict):
                if "properties" in obj:
                    obj["additionalProperties"] = False
                for value in obj.values():
                    if isinstance(value, dict):
                        add_additional_properties_false(value)

        schema = Person.model_json_schema()
        add_additional_properties_false(schema)
        payload = {
            "model": "gemini-2.0-flash",
            "messages": [
                {
                    "role": "system",
                    "content": "Extract structured information from the provided text. Return a JSON object.",
                },
                {"role": "user", "content": resume_text},
            ],
            "response_format": {
                "type": "json_schema",
                "json_schema": {
                    "name": "person_data",
                    "schema": schema,
                    "strict": True,
                },
            },
        }

        # Make the API request
        url = "https://generativelanguage.googleapis.com/v1beta/openai/chat/completions"
        headers = {
            "Content-Type": "application/json",
            "Authorization": f"Bearer {api_key}",
        }

        with httpx.Client(timeout=30.0) as client:
            response = client.post(
                url.format(api_key=api_key), json=payload, headers=headers
            )
            response.raise_for_status()

            # Parse the response
            response_data = response.json()

            # Extract the content from the response
            content = (
                response_data.get("choices", [{}])[0]
                .get("message", {})
                .get("content", "{}")
            )

            # Parse the JSON string into a dictionary
            person_dict = json.loads(content)

            # Create and return a Person object
            return Person(**person_dict)

    except httpx.HTTPStatusError as e:
        print(f"HTTP error occurred: {e.response.status_code} - {e.response.text}")
        raise
    except Exception as e:
        print(f"Error parsing resume data: {e}")
        raise


if __name__ == "__main__":
    sample_data = """
    John Doe
    ID: ABC123
    """

    person_data = parse_data(sample_data)
    print(f"Name: {person_data.name}, ID: {person_data.id_number}")

Stack trace

HTTP error occurred: 400 - [{
  "error": {
    "code": 400,
    "message": "Request contains an invalid argument.",
    "status": "INVALID_ARGUMENT"
  }
}
]
Traceback (most recent call last):

[logankilpatrick]

Good flag, making a bug to track on our side.

[erik-reges]

This is also something i very much would like to get fixed. having a union or any type in a zod schema for a tool causes an error like this: Invalid JSON payload received. Unknown name "type" at 'tools.function_declarations[189].parameters.properties[0].value.items': Proto field is not repeating, cannot start list.

It would be very appreciated!

[heyjohnlim]

+1. I have to write code to strip off the AnyOf for gemini using openai compat api. AnyOf works fine with anthropic api and openai models running openai api.

[afonsomatos]

Yes, would love to get this fixed... anyOf is critical cc @logankilpatrick

[neerajprad]

I just discovered this bug where the compatibility wrapper is unable to handle anyOf and the error message isn't clear as to what the issue is. anyOf is generated by Pydantic all the time to handle optional fields so not having support for this makes it really hard to swap from OpenAI.