Responses with no content cause timeout error and crash

[AlecIsaacson]

I'm using CircuitPython v 9.2.6 on an Adafruit M4 Express Airlift Lite. The Adafruit_Requests module version 4.0.0 crashes with 116: ETIMEOUT when it receives a response with no content (or alternatively, when it receives a response with HTTP response code 204 - no content).

To reproduce, run the following code, adapted from the ESP32SPI example found here.

Note how when we get a 200 response, things are fine, but when we get the 204 (with a no content response) we generate a timeout error. I'd expect adafruit_response to handle no content responses rather than timeout.

# SPDX-FileCopyrightText: 2021 ladyada for Adafruit Industries
# SPDX-License-Identifier: MIT

import os

import adafruit_connection_manager
import board
import busio
from adafruit_esp32spi import adafruit_esp32spi
from digitalio import DigitalInOut

import adafruit_requests

# Get WiFi details, ensure these are setup in settings.toml
ssid = os.getenv("CIRCUITPY_WIFI_SSID")
password = os.getenv("CIRCUITPY_WIFI_PASSWORD")

# If you are using a board with pre-defined ESP32 Pins:
esp32_cs = DigitalInOut(board.ESP_CS)
esp32_ready = DigitalInOut(board.ESP_BUSY)
esp32_reset = DigitalInOut(board.ESP_RESET)

spi = busio.SPI(board.SCK, board.MOSI, board.MISO)
radio = adafruit_esp32spi.ESP_SPIcontrol(spi, esp32_cs, esp32_ready, esp32_reset)

print("Connecting to AP...")
while not radio.is_connected:
    try:
        radio.connect_AP(ssid, password)
    except RuntimeError as e:
        print("could not connect to AP, retrying: ", e)
        continue
print("Connected to", str(radio.ap_info.ssid, "utf-8"), "\tRSSI:", radio.ap_info.rssi)

# Initialize a requests session
pool = adafruit_connection_manager.get_radio_socketpool(radio)
ssl_context = adafruit_connection_manager.get_radio_ssl_context(radio)
requests = adafruit_requests.Session(pool, ssl_context)

# The POST URL uses httpbin's standard post interface, which normally returns a 200.
# The 204 URL will cause httpbin to always return a 204.
JSON_POST_URL = "https://httpbin.org/post"
JSON_204_URL = "https://httpbin.org/status/204"

data = "31F"

# This code will post data to httpbin, which will return a 200 and everything works fine.
print(f"POSTing data to {JSON_POST_URL}: {data}")
with requests.post(JSON_POST_URL, data=data) as response:
    print("-" * 40)
    json_resp = response.json()
    # Parse out the 'data' key from json_resp dict.
    print("Data received from server:", json_resp["data"])
    print("Response code " + str(response.status_code))
    print("-" * 40)

# This code will post data to httpbin, which will return a 204, causing requests to crash.
print(f"POSTing data to {JSON_204_URL}: {data}")
with requests.post(JSON_204_URL, data=data) as response:
    print("-" * 40)
    json_resp = response.json()
    # Parse out the 'data' key from json_resp dict.
    print("Data received from server:", json_resp["data"])
    print("Response code " + str(response.status_code))
    print("-" * 40)

[FoamyGuy]

I have submitted a PR that resolves this for the 204 status case.

I think it may still stall and timeout in the event of a 200 or other status code with no content and without a Content-Length: 0 header. I'm not sure the best way to handle that scenario, I didn't test CPython behavior of this scenario, but it seems reasonable to me that it would keep trying to read until a timeout occurs, user code should probably set a reasonable timeout on the request and catch the timout exception if it is going to request from a server that returns empty responses with statuses other than 204 and without the length header.

[AlecIsaacson]

@FoamyGuy - Thanks for looking at this.

Someone over on the Adafruit forums suggested I work-around the problem like this:

resp = requests.post(JSON_204_URL, data=data)
print("Status code:", resp.status_code)
if resp.status_code == 204:
    # nothing to read!
    resp.close()
    print("No content returned")
else:
    with resp:
        data = resp.json()
        print("Data:", data)

That workaround does prevent the timeout error. Hopefully, this, plus your fixes, will help anyone in the future who runs into the problem and finds this issue.

[FoamyGuy]

This is resolved by #213. However note that the reproducer code here calls response.json() which will still lead to an exception because it will attempt to JSON parse an empty string which does raise an exception on both CPython and CircuitPython. You would need to either access content, or text instead or else add try/except to wrap the json() call.