What happened?

Description

The order parameter in litellm_params does not appear to prioritize deployments as documented. According to the official documentation, lower order values should have higher priority, and requests should only fall back to higher order values when the primary deployment is unavailable.

However, in my testing, both deployments (order: 1 and order: 2) receive nearly equal traffic, even when order: 1 deployment is fully available and healthy.

Expected Behavior

• All requests should be routed to the order: 1 deployment first
• order: 2 deployment should only receive traffic when order: 1 fails or becomes unavailable

Actual Behavior

• Both deployments receive approximately 50% of the traffic
• order parameter appears to be ignored
• Traffic distribution is similar to simple-shuffle behavior

LiteLLM Version

litellm version: [v1.80.11]

Configuration

Model List Configuration:

model_list:
  - model_name: gpt-5.2
    litellm_params:
      model: openai/gpt-5.2
      api_key: os.environ/OPENAI_API_KEY_1
      api_base: https://endpoint-1.example.com
      order: 1  # Primary deployment

  - model_name: gpt-5.2
    litellm_params:
      model: openai/gpt-5.2
      api_key: os.environ/OPENAI_API_KEY_2
      api_base: https://endpoint-2.example.com
      order: 2  # Fallback deployment

router_settings:
  num_retries: 3
  # No routing_strategy specified (using default)

Relevant log output

What part of LiteLLM is this about?

Proxy

What LiteLLM version are you on ?

v1.80.11

Twitter / LinkedIn details

No response