doctest fails to collect tests from a C function that has been wrapped

[lpsinger]

Bug report

Bug description:

doctest.DocTestFinder is now failing to collect examples from functions that are defined in C and then wrapped. It still works just fine with functions that are defined in C but that are not wrapped.

This bug was introduced by #115440. It breaks doctests for Numpy ufuncs in pytest-doctestplus (see scientific-python/pytest-doctestplus#248).

I have placed reproducer code in this Gist: https://gist.github.com/lpsinger/65e59728555dc2096af88d394e2d4a6b. To reproduce, retrieve the code and run the following commands:

pip install -e .
python test.py

The script test.py fails with this error message:

$ python test.py 
Traceback (most recent call last):
  File "/Users/lpsinger/src/doctest-func-without-code/test.py", line 14, in <module>
    assert len(finder.find(bar.hello)[0].examples) == 1
               ^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/doctest.py", line 942, in find
    self._find(tests, obj, name, module, source_lines, globs, {})
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/doctest.py", line 1004, in _find
    test = self._get_test(obj, name, module, globs, source_lines)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/doctest.py", line 1072, in _get_test
    lineno = self._find_lineno(obj, source_lines)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/doctest.py", line 1121, in _find_lineno
    obj = inspect.unwrap(obj).__code__
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'builtin_function_or_method' object has no attribute '__code__'. Did you mean: '__call__'?

CPython versions tested on:

3.9

Operating systems tested on:

macOS

Linked PRs

• gh-117692: Fix AttributeError in DocTestFinder on wrapped builtin_or_method #117699
• [3.12] gh-117692: Fix AttributeError in DocTestFinder on wrapped builtin_or_method (GH-117699) #117708

[sobolevn]

Shorter repro on main:

>>> import binascii, doctest, functools
>>> def dec(func):
...    @functools.wraps(func)
...    def wrapper(*args, **kwargs):
...         return func(*args, **kwargs)
...    return wrapper
... 
>>> b = dec(binascii.b2a_hex)
>>> finder = doctest.DocTestFinder()
>>> finder.find(b)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
    finder.find(b)
    ~~~~~~~~~~~^^^
  File "/Users/sobolev/Desktop/cpython2/Lib/doctest.py", line 964, in find
    self._find(tests, obj, name, module, source_lines, globs, {})
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/sobolev/Desktop/cpython2/Lib/doctest.py", line 1026, in _find
    test = self._get_test(obj, name, module, globs, source_lines)
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/sobolev/Desktop/cpython2/Lib/doctest.py", line 1094, in _get_test
    lineno = self._find_lineno(obj, source_lines)
             ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/Users/sobolev/Desktop/cpython2/Lib/doctest.py", line 1143, in _find_lineno
    obj = inspect.unwrap(obj).__code__
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'builtin_function_or_method' object has no attribute '__code__'. Did you mean: '__call__'?

[sobolevn]

Looks like I might have an idea on how to fix it! Thanks a lot for the report! 👍

[AlexWaygood]

A fix has been merged, and backported to 3.12. Unfortunately, I think it probably can't be backported to 3.11 due to 3.11 now being in security-only mode; I'll let @pablogsal reopen if he thinks this warrants an exception

[jaraco]

@pablogsal The regression introduced in #115440 was released in the final bugfix of Python 3.11, meaning this bug will remain present indefinitely on that version of Python. I'm unsure if that's happened before, but it means that bugs like jaraco/path#231 have no proper fix until Python 3.11 is sunset. I can't even think of a good workaround, short of disabling doctests on all Python versions or finding some way to disable doctests in pytest for a specific Python version. Do you know if it's happened before that a regression has been introduced in the last bugfix release for a minor version?

I think this bug deserves consideration for an extra bugfix patch release (maybe a selective one against 3.11.9 with only this bugfix).

[lpsinger]

I think this bug deserves consideration for an extra bugfix patch release (maybe a selective one against 3.11.9 with only this bugfix).

That would certainly be very helpful to me. I maintain several domain-specific scientific Python packages with doctests affected by this, and I have had to skip the doctests for Python 3.11, which makes me nervous given that I still claim that I support that version.

[pablogsal]

I think this bug deserves consideration for an extra bugfix patch release (maybe a selective one against 3.11.9 with only this bugfix).

I think it makes sense and I am happy to back port and batch this into the next security update of 3.11.

[jaraco]

I think it makes sense and I am happy to back port and batch this into the next security update of 3.11.

Works for me.

The problem with only rolling the fix into a security-only release is that the installers (and others) don't get updated with security updates, so users of the installers will be perpetually stuck with the bug. I worry that things like GitHub CI on Windows may just use the latest installer and thus may not pick up the fix. I guess we can see how many environments that leaves without a solution. Hopefully the intersection of that with projects that rely on doctests is exceedingly small.

[lpsinger]

That would be fine with me. My affected packages don't even build on Windows!

[lpsinger]

Any update on backporting this?

[jaraco]

I think this bug deserves consideration for an extra bugfix patch release (maybe a selective one against 3.11.9 with only this bugfix).

I think it makes sense and I am happy to back port and batch this into the next security update of 3.11.

@pablogsal I was making the case for making another non-security-only release with this fix, so that the most recent packaged release will include the bugfix, even if it's 3.11.9.1 or 3.11.9+1 or similar). I realize this falls out of the standard operating procedure for Python releases, but without some special handling, this bug will exist essentially in perpetuity in the Python 3.9 line. Since the issue was introduced in the latest non-security release, I think that deserves another non-security release to address the regression.