Improve `pathname2url()` and `url2pathname()` docs #127125

barneygale · 2024-11-22T01:09:51Z

These functions have long sown confusion among Python developers. The existing documentation says they deal with URL path components, but that doesn't fit the evidence on Windows:

>>> pathname2url(r'C:\foo')
'///C:/foo'
>>> pathname2url(r'\\server\share')
'////server/share'  # or '//server/share' as of quite recently

If these were URL path components, they would imply complete URLs like file://///C:/foo and file://////server/share. Clearly this isn't right. Yet the implementation in nturl2path is deliberate, and the url2pathname() function correctly inverts it.

On non-Windows platforms, the behaviour until quite recently is to simply quote/unquote the path without adding or removing any leading slashes. This behaviour is compatible with both interpretations -- 1) the value is a URL path component (existing docs), and 2) the value is everything following file: (this PR)

The conclusion I draw is that these functions operate on everything after the file: prefix, which may include an authority section. This is the only explanation that fits both the Windows and non-Windows behaviour. It's also a better match for the function names.

📚 Documentation preview 📚: https://cpython-previews--127125.org.readthedocs.build/

These functions have long sown confusion among Python developers. Even in the urllib implementation and tests, they seem to be used in contradictory ways. A test helper named `sanepathname2url()` has been with us since 2004! The existing documentation says that these functions deal with URL path components. But that doesn't fit the evidence on Windows: >>> pathname2url(r'C:\foo') '///C:/foo' >>> pathname2url(r'\\server\share') '////server/share' # or '//server/share' as of quite recently If these were URL path components, they would imply complete URLs like `file://///C:/foo` and `file://////server/share`. Clearly this isn't right. The conclusion I draw is that these functions operate on everything after the `file:` prefix, which may include an authority section.

barneygale · 2024-11-23T12:56:34Z

I think the confusion came about because the original urllib author understood that file:/etc/hosts was the correct way to express /etc/hosts as a file URI. They weren't wrong per se, and it's still an acceptable form. But later on, it became much more prevalent to express that path as file:///etc/hosts, and some folks began to use pathname2url() and url2pathname() in a slightly different way. But the new definition never gelled with the Windows implementation, and traces of the old understanding can be found in the codebase and tests.

edit: actually, it might be due to a 90s-era misunderstanding between two devs

Doc/library/urllib.request.rst

serhiy-storchaka

LGTM.

miss-islington-app · 2024-11-24T17:33:50Z

Thanks @barneygale for the PR 🌮🎉.. I'm working now to backport this PR to: 3.12, 3.13.
🐍🍒⛏🤖

These functions have long sown confusion among Python developers. The existing documentation says they deal with URL path components, but that doesn't fit the evidence on Windows: >>> pathname2url(r'C:\foo') '///C:/foo' >>> pathname2url(r'\\server\share') '////server/share' # or '//server/share' as of quite recently If these were URL path components, they would imply complete URLs like `file://///C:/foo` and `file://////server/share`. Clearly this isn't right. Yet the implementation in `nturl2path` is deliberate, and the `url2pathname()` function correctly inverts it. On non-Windows platforms, the behaviour until quite recently is to simply quote/unquote the path without adding or removing any leading slashes. This behaviour is compatible with *both* interpretations -- 1) the value is a URL path component (existing docs), and 2) the value is everything following `file:` (this commit) The conclusion I draw is that these functions operate on everything after the `file:` prefix, which may include an authority section. This is the only explanation that fits both the Windows and non-Windows behaviour. It's also a better match for the function names. (cherry picked from commit 307c633) Co-authored-by: Barney Gale <[email protected]>

bedevere-app · 2024-11-24T17:34:00Z

GH-127232 is a backport of this pull request to the 3.13 branch.

bedevere-app · 2024-11-24T17:34:03Z

GH-127233 is a backport of this pull request to the 3.12 branch.

…#127232) Improve `pathname2url()` and `url2pathname()` docs (GH-127125) These functions have long sown confusion among Python developers. The existing documentation says they deal with URL path components, but that doesn't fit the evidence on Windows: >>> pathname2url(r'C:\foo') '///C:/foo' >>> pathname2url(r'\\server\share') '////server/share' # or '//server/share' as of quite recently If these were URL path components, they would imply complete URLs like `file://///C:/foo` and `file://////server/share`. Clearly this isn't right. Yet the implementation in `nturl2path` is deliberate, and the `url2pathname()` function correctly inverts it. On non-Windows platforms, the behaviour until quite recently is to simply quote/unquote the path without adding or removing any leading slashes. This behaviour is compatible with *both* interpretations -- 1) the value is a URL path component (existing docs), and 2) the value is everything following `file:` (this commit) The conclusion I draw is that these functions operate on everything after the `file:` prefix, which may include an authority section. This is the only explanation that fits both the Windows and non-Windows behaviour. It's also a better match for the function names. (cherry picked from commit 307c633) Co-authored-by: Barney Gale <[email protected]>

…#127233) Improve `pathname2url()` and `url2pathname()` docs (GH-127125) These functions have long sown confusion among Python developers. The existing documentation says they deal with URL path components, but that doesn't fit the evidence on Windows: >>> pathname2url(r'C:\foo') '///C:/foo' >>> pathname2url(r'\\server\share') '////server/share' # or '//server/share' as of quite recently If these were URL path components, they would imply complete URLs like `file://///C:/foo` and `file://////server/share`. Clearly this isn't right. Yet the implementation in `nturl2path` is deliberate, and the `url2pathname()` function correctly inverts it. On non-Windows platforms, the behaviour until quite recently is to simply quote/unquote the path without adding or removing any leading slashes. This behaviour is compatible with *both* interpretations -- 1) the value is a URL path component (existing docs), and 2) the value is everything following `file:` (this commit) The conclusion I draw is that these functions operate on everything after the `file:` prefix, which may include an authority section. This is the only explanation that fits both the Windows and non-Windows behaviour. It's also a better match for the function names. (cherry picked from commit 307c633) Co-authored-by: Barney Gale <[email protected]>

barneygale added docs Documentation in the Doc dir skip issue skip news needs backport to 3.12 bug and security fixes needs backport to 3.13 bugs and security fixes labels Nov 22, 2024

bedevere-app bot added the awaiting core review label Nov 22, 2024

barneygale mentioned this pull request Nov 23, 2024

GH-127236: pathname2url(): generate RFC 1738 URL for absolute POSIX path #127194

Merged

barneygale mentioned this pull request Nov 23, 2024

GH-127090: Fix urllib.response.addinfourl.url value for opened file: URIs #127091

Open

Add examples

efc7c84

barneygale requested a review from serhiy-storchaka November 23, 2024 15:40

Merge branch 'main' into pythongh-125866-docs

cea4967

serhiy-storchaka reviewed Nov 24, 2024

View reviewed changes

Doc/library/urllib.request.rst Outdated Show resolved Hide resolved

Doc/library/urllib.request.rst Outdated Show resolved Hide resolved

Address review feedback

4ae0e22

barneygale requested a review from serhiy-storchaka November 24, 2024 17:08

serhiy-storchaka approved these changes Nov 24, 2024

View reviewed changes

bedevere-app bot added awaiting merge and removed awaiting core review labels Nov 24, 2024

barneygale merged commit 307c633 into python:main Nov 24, 2024
29 checks passed

bedevere-app bot removed the awaiting merge label Nov 24, 2024

bedevere-app bot removed the needs backport to 3.13 bugs and security fixes label Nov 24, 2024

bedevere-app bot removed the needs backport to 3.12 bug and security fixes label Nov 24, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve `pathname2url()` and `url2pathname()` docs #127125

Improve `pathname2url()` and `url2pathname()` docs #127125

barneygale commented Nov 22, 2024 •

edited

Loading

barneygale commented Nov 23, 2024 •

edited

Loading

serhiy-storchaka left a comment

miss-islington-app bot commented Nov 24, 2024

bedevere-app bot commented Nov 24, 2024

bedevere-app bot commented Nov 24, 2024

Improve pathname2url() and url2pathname() docs #127125

Improve pathname2url() and url2pathname() docs #127125

Conversation

barneygale commented Nov 22, 2024 • edited Loading

barneygale commented Nov 23, 2024 • edited Loading

serhiy-storchaka left a comment

Choose a reason for hiding this comment

miss-islington-app bot commented Nov 24, 2024

bedevere-app bot commented Nov 24, 2024

bedevere-app bot commented Nov 24, 2024

Improve `pathname2url()` and `url2pathname()` docs #127125

Improve `pathname2url()` and `url2pathname()` docs #127125

barneygale commented Nov 22, 2024 •

edited

Loading

barneygale commented Nov 23, 2024 •

edited

Loading