loop.create_server does not detect if the interface is IPv6 enabled

[cecton]

BPO	30945
Nosy	@pitrou, @1st1, @asottile, @jnwatson, @cecton
PRs	gh-75128: Fix create_server to handle the case when iface isn't IPv6 enabled. #7207
Files	test_ipv6.py: Example file to reproduce the issue eaddrnotavail_asyncio.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2017-07-17.10:04:34.577>
labels = ['type-bug', '3.7', 'expert-asyncio']
title = 'loop.create_server does not detect if the interface is IPv6 enabled'
updated_at = <Date 2018-10-19.21:14:34.454>
user = 'https://github.com/cecton'

bugs.python.org fields:

activity = <Date 2018-10-19.21:14:34.454>
actor = 'jnwatson'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['asyncio']
creation = <Date 2017-07-17.10:04:34.577>
creator = 'cecton'
dependencies = []
files = ['47019', '47026']
hgrepos = []
issue_num = 30945
keywords = ['patch']
message_count = 12.0
messages = ['298474', '298683', '298684', '298714', '298716', '298718', '311915', '311916', '311917', '311942', '317895', '318045']
nosy_count = 5.0
nosy_names = ['pitrou', 'yselivanov', 'Anthony Sottile', 'jnwatson', 'cecton']
pr_nums = ['7207']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue30945'
versions = ['Python 3.5', 'Python 3.6', 'Python 3.7']

Linked PRs

• gh-75128: Ignore EADDRNOTAVAIL error in asyncio.BaseEventLoop.create_server() #114420
• [3.12] gh-75128: Ignore EADDRNOTAVAIL error in asyncio.BaseEventLoop.create_server() (GH-114420) #114441
• [3.11] gh-75128: Ignore EADDRNOTAVAIL error in asyncio.BaseEventLoop.create_server() (GH-114420) #114442

[cecton]

The IPv6 detection in asyncio.base_events.create_server only detect if IPv6 is available instead of checking if the interface can actually support it.

I noticed that by using Python in a Docker container (example code to reproduce in attachment):

docker run -it --rm -v /tmp/test_ipv6.py:/src/test_ipv6.py python:3.6 python /src/test_ipv6.py

Will result in:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/asyncio/base_events.py", line 1043, in create_server
    sock.bind(sa)
OSError: [Errno 99] Cannot assign requested address

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/src/test_ipv6.py", line 11, in <module>
    server = loop.run_until_complete(server_creation)
  File "/usr/local/lib/python3.6/asyncio/base_events.py", line 466, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.6/asyncio/base_events.py", line 1047, in create_server
    % (sa, err.strerror.lower()))
OSError: [Errno 99] error while attempting to bind on address ('::1', 27015, 0, 0): cannot assign requested address

By default Docker containers have only IPv4 enabled:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
38: eth0@if39: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether 02:42:ac:11:00:02 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.2/16 scope global eth0
valid_lft forever preferred_lft forever

I believe this detection mechanism should rely on the interface requested. I found this on the web for Python 2 that manage to get the info per interface: https://pastebin.com/VEnhF1Ht but it's using an external library.

However if you change the hostname to 127.0.0.1 it works normally.

[pitrou]

Better than trying to detect IPv6 compatibility beforehand would probably to recognize the error and simply ignore it.

Note: errno 99 is EADDRNOTAVAIL.

Something like this could work (untested):

diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py
index 33b8f48..413161a 100644
--- a/Lib/asyncio/base_events.py
+++ b/Lib/asyncio/base_events.py
@@ -1038,6 +1038,11 @@ class BaseEventLoop(events.AbstractEventLoop):
                     try:
                         sock.bind(sa)
                     except OSError as err:
+                        if err.errno == errno.EADDRNOTAVAIL:
+                            # See bpo-30945
+                            sockets.pop()
+                            sock.close()
+                            continue
                         raise OSError(err.errno, 'error while attempting '
                                       'to bind on address %r: %s'
                                       % (sa, err.strerror.lower())) from None

[1st1]

Better than trying to detect IPv6 compatibility beforehand would probably to recognize the error and simply ignore it.

+1

[pitrou]

Cécile, could you try the following patch? I have no easy way to test here.

[cecton]

Sure! It seems to work, the process returns an exit code of 0 and I see no traceback but the message is still displayed in the terminal.

(Also I did something weird because your patch applies on branch master and I ran it with Python 3.6... I suppose it shouldn't be a problem)

[0] [11:54:13] /tmp > d run -it --rm -v /tmp:/tmp:ro -v ~/repos/cpython/Lib/asyncio:/usr/local/lib/python3.6/asyncio:ro python:3.6 python /tmp/test_ipv6.py
error while attempting to bind on address ('::1', 27015, 0, 0): cannot assign requested address
[0] [11:54:19] /tmp > d run -it --rm -v /tmp:/tmp:ro python:3.6 python /tmp/test_ipv6.py
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/asyncio/base_events.py", line 1043, in create_server
    sock.bind(sa)
OSError: [Errno 99] Cannot assign requested address

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/tmp/test_ipv6.py", line 11, in <module>
    server = loop.run_until_complete(server_creation)
  File "/usr/local/lib/python3.6/asyncio/base_events.py", line 466, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.6/asyncio/base_events.py", line 1047, in create_server
    % (sa, err.strerror.lower()))
OSError: [Errno 99] error while attempting to bind on address ('::1', 27015, 0, 0): cannot assign requested address
[1] [11:54:52] /tmp >

[pitrou]

Cécile, thank you. The reason the message is still displayed is that I turned the error into a warning (in other words, it is expected).

[asottile]

Seeing this as well when running the cpython test suite in docker:

$ ./python -m test.test_asyncio

...

[18 similar traces omitted]
======================================================================
ERROR: test_sock_sendfile_zero_size (test.test_asyncio.test_unix_events.SelectorEventLoopUnixSockSendfileTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/code/Lib/test/test_asyncio/test_unix_events.py", line 559, in test_sock_sendfile_zero_size
    sock, proto = self.prepare()
  File "/code/Lib/test/test_asyncio/test_unix_events.py", line 483, in prepare
    lambda: proto, support.HOST, port))
  File "/code/Lib/test/test_asyncio/test_unix_events.py", line 476, in run_loop
    return self.loop.run_until_complete(coro)
  File "/code/Lib/asyncio/base_events.py", line 566, in run_until_complete
    return future.result()
  File "/code/Lib/asyncio/base_events.py", line 1346, in create_server
    % (sa, err.strerror.lower())) from None
OSError: [Errno 99] error while attempting to bind on address ('::1', 39527, 0, 0): cannot assign requested address

----------------------------------------------------------------------

I'm going to try and write a patch to skip these tests (there's already a helper)

[asottile]

Actually, my issue seems to be something more strange.

The host being passed in is localhost which resolves to:

>>> pprint.pprint(socket.getaddrinfo('localhost', 80))
[(<AddressFamily.AF_INET: 2>,
  <SocketKind.SOCK_STREAM: 1>,
  6,
  '',
  ('127.0.0.1', 80)),
 (<AddressFamily.AF_INET: 2>,
  <SocketKind.SOCK_DGRAM: 2>,
  17,
  '',
  ('127.0.0.1', 80)),
 (<AddressFamily.AF_INET: 2>,
  <SocketKind.SOCK_RAW: 3>,
  0,
  '',
  ('127.0.0.1', 80)),
 (<AddressFamily.AF_INET6: 10>,
  <SocketKind.SOCK_STREAM: 1>,
  6,
  '',
  ('::1', 80, 0, 0)),
 (<AddressFamily.AF_INET6: 10>,
  <SocketKind.SOCK_DGRAM: 2>,
  17,
  '',
  ('::1', 80, 0, 0)),
 (<AddressFamily.AF_INET6: 10>,
  <SocketKind.SOCK_RAW: 3>,
  0,
  '',
  ('::1', 80, 0, 0))]

asyncio is picking ipv6 because of this code:

cpython/Lib/asyncio/base_events.py

Lines 1334 to 1340 in a445feb

	# Disable IPv4/IPv6 dual stack support (enabled by
	# default on Linux) which makes a single socket
	# listen on both address families.
	if af == AF_INET6 and hasattr(socket, 'IPPROTO_IPV6'):
	sock.setsockopt(socket.IPPROTO_IPV6,
	socket.IPV6_V6ONLY,
	True)

despite my host not actually having an ipv6 network hooked up.

[asottile]

Applying this patch makes the tests pass for me, but I don't think the patch is appropriate (just hides the bug):

$ git diff
diff --git a/Lib/test/test_asyncio/test_unix_events.py b/Lib/test/test_asyncio/test_unix_events.py
index 5bd76d3..ff6c4e1 100644
--- a/Lib/test/test_asyncio/test_unix_events.py
+++ b/Lib/test/test_asyncio/test_unix_events.py
@@ -480,7 +480,7 @@ class SelectorEventLoopUnixSockSendfileTests(test_utils.TestCase):
         proto = self.MyProto(self.loop)
         port = support.find_unused_port()
         server = self.run_loop(self.loop.create_server(
-            lambda: proto, support.HOST, port))
+            lambda: proto, support.HOSTv4, port))
         self.run_loop(self.loop.sock_connect(sock, (support.HOST, port)))
 
         def cleanup():

[cecton]

I see that the patch hasn't been applied to master on GitHub. Is there anything else expected from me on this ticket?

[1st1]

Does anybody wants to make a PR to fix this?

[1st1]

asyncio is picking ipv6 because of this code:

cpython/Lib/asyncio/base_events.py

Lines 1334 to 1340 in a445feb

	# Disable IPv4/IPv6 dual stack support (enabled by
	# default on Linux) which makes a single socket
	# listen on both address families.
	if af == AF_INET6 and hasattr(socket, 'IPPROTO_IPV6'):
	sock.setsockopt(socket.IPPROTO_IPV6,
	socket.IPV6_V6ONLY,
	True)

despite my host not actually having an ipv6 network hooked up.

Which should be fine; create_server enumerates all addresses and tries to connect to each one. IPV6_V6ONLY is only applied to one socket for one IPv6 address. I think Antoine's patch is OK.

[serhiy-storchaka]

Thank you for your patch Antoine. I am surprised that it was not merged at its time. It fixes a constant failure on some buildbots. Buildbot failure could perhaps be fixed by changing the configuration (I think that I encountered such errors on different buildbots in the past), but it is better to fix it in the core.