Pathlib join pure windows path and pure posix path changes it behaviour on 3.13

[pohmelie]

Bug report

Bug description:

3.12.7

Python 3.12.7 (main, Oct  8 2024, 00:39:14) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pathlib; pathlib.PureWindowsPath("C:\\A") / pathlib.PurePosixPath("/foo/C:\\B").relative_to("/")
PureWindowsPath('C:/B')

3.13.0rc3

Python 3.13.0rc3 (main, Oct  7 2024, 23:18:57) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pathlib; pathlib.PureWindowsPath("C:\\A") / pathlib.PurePosixPath("/foo/C:\\B").relative_to("/")
PureWindowsPath('C:/A/foo/C:/B')

Not sure if it is a bug, or a fixed behaviour

CPython versions tested on:

3.12, 3.13

Operating systems tested on:

Linux

Linked PRs

• GH-125069: Fix inconsistent joining in WindowsPath(PosixPath(...)) #125156
• [3.13] GH-125069: Fix inconsistent joining in WindowsPath(PosixPath(...)) (GH-125156) #125409
• [3.12] GH-125069: Fix inconsistent joining in WindowsPath(PosixPath(...)) (GH-125156) #125410

[AA-Turner]

cc @barneygale, though the reproducer does look a bit off, with multiple drive roots.

@pohmelie can you reproduce this using only os.path?

A

[pohmelie]

@AA-Turner
If you provide a code with os.path module, then I can test it. But, AFAIK, os.path have no ability to manage windows and posix paths.

[terryjreedy]

Checking 3.13 What's New or the linked log file might reveal if intentional change.

[ImFeH2]

It happens after this change.
#111431

[hacscred]

Classic that an optimization changes semantics. I'd rip out that optimization or make sure that the new behavior matches the specification.

[ZeroIntensity]

cc @barneygale, apparently that PR introduced a breaking change.

Edit: oh wait, Adam already pinged, sorry!

[barneygale]

Thanks for reporting this! There's a few things going on here. I think it comes down to pathlib's special handling of PurePath(PurePath(...)) and our efforts to delay joining of *pathsegments. I believe the 3.13 behaviour is correct, and that we should fix 3.12. PR coming up...

[barneygale]

Another repro, this one works in 3.13 and 3.14:

>>> a = PureWindowsPath('a')
>>> b1 = PurePosixPath('b', 'c:\\d')
>>> b2 = PurePosixPath('b/c:\\d')
>>> b1 == b2
True
>>> a / b1
PureWindowsPath('c:/d')
>>> a / b2
PureWindowsPath('a/b/c:/d')

[barneygale]

PR available, if anyone has a mo to review: #125156

[barneygale]

I've pushed a fix to 3.12, 3.13 and 3.14 (main).

In the original repro case, the 3.12 behaviour was buggy and the 3.13 behaviour was correct. The optimisation in #111431 (3.13+) happened to fix the issue for paths generated by relative_to(). The change I've applied here should fix the problem at its source.

Thanks for reporting :)

[pohmelie]

@barneygale
So, what should I do in my library code? Should I dispatch behaviour by python version? Will the fix lands on 3.12.8 and buggy behaviour still presents in 3.12.0 to 3.12.7 inclusive? 🤔 I want equal behaviour of my function on all supported python versions from 3.8 to 3.13.

[barneygale]

@barneygale So, what should I do in my library code? Should I dispatch behaviour by python version? Will the fix lands on 3.12.8 and buggy behaviour still presents in 3.12.0 to 3.12.7 inclusive? 🤔 I want equal behaviour of my function on all supported python versions from 3.8 to 3.13.

Untested, but I think this will work in all version:

diff --git a/src/aioftp/server.py b/src/aioftp/server.py
index 1830d78..d31f40a 100644
--- a/src/aioftp/server.py
+++ b/src/aioftp/server.py
@@ -1031,7 +1031,7 @@ class Server:
             else:
                 resolved_virtual_path /= part
         base_path = connection.user.base_path
-        real_path = base_path / resolved_virtual_path.relative_to("/")
+        real_path = base_path / str(resolved_virtual_path.relative_to("/"))
         # replace with `is_relative_to` check after 3.9+ requirements lands
         try:
             real_path.relative_to(base_path)

[hacscred]

(Pathlib code is atrocious, btw. )

(I think for something as big as Python, getting rid of informal specifications would be a good idea, but I doubt the people in the project could ever accept it.

The specification of many pathlib functions is completely underspecified.)

[barneygale]

Please use https://discuss.python.org/ for general discussion. If you're looking to convince other people to do work on your behalf (e.g. improving the pathlib codebase) then I suggest you strike a more constructive note. If you're planning to put in your own issues and PRs I'd be happy to review.

[pohmelie]

@barneygale yep, looks like this str(...) trick works, thank you!

[hacscred]

Please use https://discuss.python.org/ for general discussion. If you're looking to convince other people to do work on your behalf (e.g. improving the pathlib codebase) then I suggest you strike a more constructive note. If you're planning to put in your own issues and PRs I'd be happy to review.

I don't particularly care that much about Python. I looked at the code for this issue and I just wanted to leave my opinion on the Internet, which could be used by someone as an argument to fix pathlib.

I was just being nice in sharing my (highly valuable) opinion.

I don't particularly care if anyone does something with it. I certainly am not forcing anyone to do anything. Not sure where you got that idea.

I am also not interested in discussion, because that would suggest I care about the opinion of others, which I don't, since I already am an expert.

So, ignore my advice all you want, but that doesn't change the reality of the quality of that code. The code generator is even worse, for those interested.

I will say it again: I am not asking for your opinion. I am giving you mine.

[barneygale]

Wow what valuable feedback, thanks buddy! :P