Closed
Description
Bug report
When running ./python.exe Lib/test/test_doctest.py
you get:
» ./python.exe Lib/test/test_doctest.py
.....................................F................FF.........
======================================================================
FAIL: basics (__main__.test_DocTestFinder)
Doctest: __main__.test_DocTestFinder.basics
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/sobolev/Desktop/cpython/Lib/doctest.py", line 2353, in runTest
raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for __main__.test_DocTestFinder.basics
File "/Users/sobolev/Desktop/cpython/Lib/test/test_doctest.py", line 445, in basics
----------------------------------------------------------------------
File "/Users/sobolev/Desktop/cpython/Lib/test/test_doctest.py", line 469, in __main__.test_DocTestFinder.basics
Failed example:
print(tests) # doctest: +ELLIPSIS
Expected:
[<DocTest sample_func from test_doctest.py:32 (1 example)>]
Got:
[<DocTest sample_func from /Users/sobolev/Desktop/cpython/Lib/test/test_doctest.py:32 (1 example)>]
----------------------------------------------------------------------
File "/Users/sobolev/Desktop/cpython/Lib/test/test_doctest.py", line 576, in __main__.test_DocTestFinder.basics
Failed example:
for t in tests:
print('%2s %s' % (len(t.examples), t.name))
Expected:
1 some_module
3 some_module.SampleClass
3 some_module.SampleClass.NestedClass
1 some_module.SampleClass.NestedClass.__init__
1 some_module.SampleClass.__init__
1 some_module.SampleClass.a_cached_property
2 some_module.SampleClass.a_classmethod
1 some_module.SampleClass.a_classmethod_property
1 some_module.SampleClass.a_property
1 some_module.SampleClass.a_staticmethod
1 some_module.SampleClass.double
1 some_module.SampleClass.get
1 some_module.__test__.c
2 some_module.__test__.d
1 some_module.sample_func
Got:
1 some_module
1 some_module.__test__.c
2 some_module.__test__.d
======================================================================
FAIL: test_pdb_set_trace (__main__)
Doctest: __main__.test_pdb_set_trace
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/sobolev/Desktop/cpython/Lib/doctest.py", line 2353, in runTest
raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for __main__.test_pdb_set_trace
File "/Users/sobolev/Desktop/cpython/Lib/test/test_doctest.py", line 1984, in test_pdb_set_trace
----------------------------------------------------------------------
File "/Users/sobolev/Desktop/cpython/Lib/test/test_doctest.py", line 2043, in __main__.test_pdb_set_trace
Failed example:
try:
runner.run(test)
finally:
sys.stdin = real_stdin
Expected:
--Return--
> <doctest test.test_doctest.test_pdb_set_trace[7]>(3)calls_set_trace()->None
-> import pdb; pdb.set_trace()
(Pdb) print(y)
2
(Pdb) up
> <doctest foo-bar@baz[1]>(1)<module>()
-> calls_set_trace()
(Pdb) print(x)
1
(Pdb) continue
TestResults(failed=0, attempted=2)
Got:
--Return--
> <doctest __main__.test_pdb_set_trace[7]>(3)calls_set_trace()->None
-> import pdb; pdb.set_trace()
(Pdb) print(y)
2
(Pdb) up
> <doctest foo-bar@baz[1]>(1)<module>()
-> calls_set_trace()
(Pdb) print(x)
1
(Pdb) continue
TestResults(failed=0, attempted=2)
======================================================================
FAIL: test_pdb_set_trace_nested (__main__)
Doctest: __main__.test_pdb_set_trace_nested
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/sobolev/Desktop/cpython/Lib/doctest.py", line 2353, in runTest
raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for __main__.test_pdb_set_trace_nested
File "/Users/sobolev/Desktop/cpython/Lib/test/test_doctest.py", line 2118, in test_pdb_set_trace_nested
----------------------------------------------------------------------
File "/Users/sobolev/Desktop/cpython/Lib/test/test_doctest.py", line 2154, in __main__.test_pdb_set_trace_nested
Failed example:
try:
runner.run(test)
finally:
sys.stdin = real_stdin
# doctest: +REPORT_NDIFF
Differences (ndiff with -expected +actual):
- > <doctest test.test_doctest.test_pdb_set_trace_nested[0]>(5)calls_set_trace()
? --------- ^^^^^^^
+ > <doctest __main__.test_pdb_set_trace_nested[0]>(5)calls_set_trace()
? ^^^^^^^
-> self.f1()
(Pdb) print(y)
1
(Pdb) step
--Call--
- > <doctest test.test_doctest.test_pdb_set_trace_nested[0]>(7)f1()
? --------- ^^^^^^^
+ > <doctest __main__.test_pdb_set_trace_nested[0]>(7)f1()
? ^^^^^^^
-> def f1(self):
(Pdb) step
- > <doctest test.test_doctest.test_pdb_set_trace_nested[0]>(8)f1()
? --------- ^^^^^^^
+ > <doctest __main__.test_pdb_set_trace_nested[0]>(8)f1()
? ^^^^^^^
-> x = 1
(Pdb) step
- > <doctest test.test_doctest.test_pdb_set_trace_nested[0]>(9)f1()
? --------- ^^^^^^^
+ > <doctest __main__.test_pdb_set_trace_nested[0]>(9)f1()
? ^^^^^^^
-> self.f2()
(Pdb) step
--Call--
- > <doctest test.test_doctest.test_pdb_set_trace_nested[0]>(11)f2()
? --------- ^^^^^^^
+ > <doctest __main__.test_pdb_set_trace_nested[0]>(11)f2()
? ^^^^^^^
-> def f2(self):
(Pdb) step
- > <doctest test.test_doctest.test_pdb_set_trace_nested[0]>(12)f2()
? --------- ^^^^^^^
+ > <doctest __main__.test_pdb_set_trace_nested[0]>(12)f2()
? ^^^^^^^
-> z = 1
(Pdb) step
- > <doctest test.test_doctest.test_pdb_set_trace_nested[0]>(13)f2()
? --------- ^^^^^^^
+ > <doctest __main__.test_pdb_set_trace_nested[0]>(13)f2()
? ^^^^^^^
-> z = 2
(Pdb) print(z)
1
(Pdb) up
- > <doctest test.test_doctest.test_pdb_set_trace_nested[0]>(9)f1()
? --------- ^^^^^^^
+ > <doctest __main__.test_pdb_set_trace_nested[0]>(9)f1()
? ^^^^^^^
-> self.f2()
(Pdb) print(x)
1
(Pdb) up
- > <doctest test.test_doctest.test_pdb_set_trace_nested[0]>(5)calls_set_trace()
? --------- ^^^^^^^
+ > <doctest __main__.test_pdb_set_trace_nested[0]>(5)calls_set_trace()
? ^^^^^^^
-> self.f1()
(Pdb) print(y)
1
(Pdb) up
> <doctest foo-bar@baz[1]>(1)<module>()
-> calls_set_trace()
(Pdb) print(foo)
*** NameError: name 'foo' is not defined
(Pdb) continue
TestResults(failed=0, attempted=2)
----------------------------------------------------------------------
Ran 65 tests in 0.942s
FAILED (failures=3)
These issues are not easy to fix. Moreover, since they are literally written in docstrings, it is not very readable.
The second problem is that -c
option is claimed to be supported:
cpython/Lib/test/test_doctest.py
Lines 3365 to 3380 in 78e6d72
However:
test_main
is not defined, so it fails- When changed to
unittest.main()
(with all of the other test failures fixed), it still does not work on macos sonoma, no file is created, nothing
I propose to:
- raise an exception that running
test_doctest
directly is not supported - remove
test_coverage
Linked PRs
- gh-111348: Fix direct invocation of
test_doctest
; removetest_doctest.test_coverage
#111349 - [3.12] gh-111348: Fix direct invocation of
test_doctest
; removetest_doctest.test_coverage
(GH-111349) #111359 - [3.11] gh-111348: Fix direct invocation of
test_doctest
; removetest_doctest.test_coverage
(GH-111349) #111360