Skip to content

Report every generated __text_signature__, and keep it out of __doc__ - #8614

Merged
youknowone merged 7 commits into
RustPython:mainfrom
leehanjeong:8383-method-text-signature
Aug 31, 2026
Merged

youknowone merged 7 commits into
RustPython:mainfrom
leehanjeong:8383-method-text-signature

Conversation

@leehanjeong

@leehanjeong leehanjeong commented Aug 30, 2026

Copy link
Copy Markdown
Contributor

Summary

A generated signature and a function's documentation share one string, separated by \n--\n\n, as in CPython. Defects in writing and reading that string left __doc__ carrying the signature and nearly every method carrying none.

before after
builtin functions whose __doc__ still has the signature line 44 / 46 0 / 46
functions of the native modules reporting a __text_signature__ 630 / 791 775 / 791
methods and classmethods on the builtin types reporting one 1 / 1409 1316 / 1409
of those, matching CPython exactly where it describes them too 0 / 1030 713 / 1030

What changed

  • __doc__ returned the whole string while __text_signature__ read only the front of it, so len.__doc__ was 'len(obj, /)\n--\n\nReturn the number of items in a container.'. get_doc_from_internal_doc already stripped the prefix but was reachable only from PyType.__doc__; split it the way CPython does, into doc_without_signature (_PyType_DocWithoutSignature) and a wrapper mapping an empty result to None (_PyType_GetDocFromInternalDoc), and call it from PyNativeFunction and PyMethodDescriptor as well.
  • An undocumented #[pyfunction] stored the bare signature, leaving no -- marker for a reader to find: time.clock_getres.__doc__ was 'clock_getres(clk_id, /)' and its __text_signature__ was None. Emit the marker in that case too, as Argument Clinic does for an undocumented function.
  • A #[pymethod] built its doc only when the method carried a doc comment, dropping the generated signature along with it. The signature comes from the Rust parameter list, not from the comment, so it was already being built and then thrown away. Build the doc from either part, as #[pyfunction] does.
  • The receiver needs marking. A method taking it as an ordinary first argument rather than &self would otherwise report it as a plain parameter, and binding could not drop it. Mark it $self for 310 method descriptors and $type for 211 classmethod entries, as CPython does, so inspect.signature([].__dir__) is () rather than (zelf, /). A method taking FuncArgs receives the receiver inside that bundle, so it reports both: object.__subclasshook__ is ($type, *args, **kwargs).

Left for #8383

317 of the 1030 methods CPython also describes still differ, in two groups needing different work.

  • The parameter is a #[derive(FromArgs)] struct or FuncArgs the generator cannot see into, so int.from_bytes reports ($type, args, /) where CPython reports ($type, /, bytes, byteorder='big', *, signed=False). ArgAttribute already parses each field's name, kind and default for argument binding; teaching FromArgs to report them would close this.
  • The parameter names differ, as list.append reporting x where CPython reports object. Mechanical, and independent of the above.

Notes

  • Lib/test/test_pydoc/test_pydoc.py and Lib/test/test_pyrepl/test_pyrepl.py drop three expectedFailure markers that pass now. rlcompleter closes the parenthesis for a callable whose signature takes no parameters, so completing os.getpid yields os.getpid().
  • Slot wrappers keep no signature: PyWrapper has no __text_signature__ getter, and its docs are hand-written rather than generated, so there is nothing to strip.
  • #[pymethod] receivers are now uniformly named zelf, and func_sig's name check for it is gone. Both are cosmetic: the receiver is marked by position, so every generated signature is byte for byte identical without them.

Summary by CodeRabbit

  • New Features

    • Improved signatures shown for built-in functions and methods, including bound and unbound methods.
    • Improved generated documentation for native functions, methods, and types.
    • Empty documentation is now represented consistently as None.
  • Bug Fixes

    • Corrected displayed method signatures when implicit receivers are involved.
    • Normalized documentation so internal signature details are no longer exposed unnecessarily.
  • Tests

    • Added coverage for built-in signatures, including zero-argument and bound methods.

A function's signature and its documentation share one string, separated
by `\n--\n\n`, as in CPython. __text_signature__ read the front of it,
but __doc__ returned the whole thing:

    >>> len.__doc__
    'len(obj, /)\n--\n\nReturn the number of items in a container.'   # was
    'Return the number of items in a container.'                      # now

44 of the 46 builtin functions leaked the prefix.

get_doc_from_internal_doc already stripped it but was reachable only from
PyType.__doc__. Split it the way CPython does, into doc_without_signature
(_PyType_DocWithoutSignature) and a wrapper mapping an empty result to
None (_PyType_GetDocFromInternalDoc), and call it from PyNativeFunction
and PyMethodDescriptor as well.

Assisted-by: Claude Code:claude-opus-5
A signature and its documentation share one string, separated by
`\n--\n\n`, and readers find the signature by searching for that marker.
A #[pyfunction] with no doc comment stored the bare signature, which no
reader could parse:

    >>> time.clock_getres.__doc__
    'clock_getres(clk_id, /)'
    >>> time.clock_getres.__text_signature__
    None

Emit the marker in that case too, as Argument Clinic does for an
undocumented function. 136 of the 915 functions reachable from the
importable modules were affected.

__doc__ now reports None for them, matching CPython, because nothing
follows the marker.

Assisted-by: Claude Code:claude-opus-5
A #[pymethod] built its doc only when the method carried a doc comment,
so one without a comment dropped the generated signature too:

    >>> list.append.__text_signature__
    None

Build it from either part, as #[pyfunction] does. All 1101 method
descriptors reachable from the builtin types now report a signature, up
from 1, and 713 of the 1030 that CPython also describes match it exactly.

A method that takes its receiver as an ordinary first argument instead of
`&self` reported that argument by name, so binding the method could not
drop it:

    >>> inspect.signature([].__dir__)
    (obj, /)     # was
    ()           # now

Mark it $self, or $type for a classmethod, as CPython does. 310 methods
were affected.

test_unbound_builtin_method_noargs and test_bound_builtin_method_noargs
pass now.

Assisted-by: Claude Code:claude-opus-5
@coderabbitai

coderabbitai Bot commented Aug 30, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro Plus

Run ID: 686745a1-1526-472a-8eb9-56f5a707df1e

📥 Commits

Reviewing files that changed from the base of the PR and between 1e3573c and 5d4257b.

⛔ Files ignored due to path filters (2)
  • Lib/test/test_pydoc/test_pydoc.py is excluded by !Lib/**
  • Lib/test/test_pyrepl/test_pyrepl.py is excluded by !Lib/**
📒 Files selected for processing (12)
  • crates/derive-impl/src/pyclass.rs
  • crates/derive-impl/src/pymodule.rs
  • crates/derive-impl/src/util.rs
  • crates/stdlib/src/ssl/error.rs
  • crates/vm/src/builtins/bool.rs
  • crates/vm/src/builtins/builtin_func.rs
  • crates/vm/src/builtins/descriptor.rs
  • crates/vm/src/builtins/object.rs
  • crates/vm/src/builtins/type.rs
  • crates/vm/src/exceptions.rs
  • crates/vm/src/stdlib/_io.rs
  • extra_tests/snippets/builtin_signature.py

Included review availability: Your plan provides up to 10 included reviews per hour; 9 remain after this review.


📝 Walkthrough

Walkthrough

Changes

Built-in signature and documentation handling

Layer / File(s) Summary
Implicit receiver signature generation
crates/derive-impl/src/util.rs, crates/derive-impl/src/pyclass.rs, crates/derive-impl/src/pymodule.rs, crates/vm/..., crates/stdlib/..., extra_tests/snippets/builtin_signature.py
Generated signatures now accept explicit $self or $type markers. Method documentation combines generated signatures with attribute documentation. Receiver names are aligned to zelf. Signature tests cover zero-argument and bound built-ins.
Internal documentation normalization
crates/vm/src/builtins/type.rs, crates/vm/src/builtins/builtin_func.rs, crates/vm/src/builtins/descriptor.rs
Internal documentation handling now returns None for empty documentation. Native functions, method descriptors, and non-heap types use the shared normalized result.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk: ⚪ Minimal · up to 5d425

This change corrects built-in documentation and signature metadata without introducing an actionable merge-blocking risk; it is merge-ready after normal checks and review.

Suggested reviewers: youknowone

Sequence Diagram(s)

sequenceDiagram
  participant MethodItem
  participant text_signature
  participant func_sig
  participant SignatureTests
  MethodItem->>text_signature: pass implicit_self marker
  text_signature->>func_sig: generate signature arguments
  func_sig-->>MethodItem: return formatted signature
  MethodItem->>SignatureTests: expose generated signature
  SignatureTests->>SignatureTests: assert bound and unbound signatures
Loading
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the primary changes: reporting generated text_signature values and preventing signatures from appearing in doc.
Docstring Coverage ✅ Passed Docstring coverage is 85.37% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 41 functions across 12 files.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added the z-ca-2026 Tag to track Contribution Academy 2026 label Aug 30, 2026
@github-actions

github-actions Bot commented Aug 30, 2026

Copy link
Copy Markdown
Contributor

📦 Library Dependencies

The following Lib/ modules were modified. Here are their dependencies:

[x] lib: cpython/Lib/pydoc.py
[x] lib: cpython/Lib/pydoc_data
[ ] test: cpython/Lib/test/test_pydoc (TODO: 29)

dependencies:

  • pydoc

dependent tests: (5 tests)

  • pydoc: test_enum test_pydoc
    • pdb: test_pdb
    • xmlrpc.server: test_docxmlrpc test_xmlrpc

[ ] lib: cpython/Lib/concurrent
[ ] test: cpython/Lib/test/test_concurrent_futures (TODO: 4)
[ ] test: cpython/Lib/test/test_interpreters
[ ] test: cpython/Lib/test/test__interpreters.py
[ ] test: cpython/Lib/test/test__interpchannels.py
[ ] test: cpython/Lib/test/test_crossinterp.py

dependencies:

  • concurrent (native: _crossinterp, _interpqueues, _interpreters, _queues, concurrent.futures, concurrent.futures._base, interpreter, itertools, multiprocessing.connection, multiprocessing.queues, multiprocessing.synchronize, process, sys, thread, time)
    • logging (native: atexit, collections.abc, email.message, email.utils, errno, http.client, logging.handlers, multiprocessing.queues, select, sys, time, urllib.parse, win32evtlog, win32evtlogutil)
    • multiprocessing (native: _multiprocessing, _posixshmem, _posixsubprocess, _winapi, array, atexit, collections.abc, connection, context, dummy, errno, forkserver, heap, itertools, managers, mmap, msvcrt, multiprocessing.connection, pool, popen_fork, popen_forkserver, popen_spawn_posix, popen_spawn_win32, queues, resource_sharer, resource_tracker, sharedctypes, spawn, synchronize, sys, time, util, xmlrpc.client)
    • pickle (native: _pickle, itertools, sys)
    • collections, functools, os, queue, threading, traceback, types, weakref

dependent tests: (17 tests)

  • concurrent: test_asyncio test_compileall test_concurrent_futures test_context test_genericalias test_inspect test_struct test_sys test_threading test_types test_wmi
    • asyncio: test_asyncio test_external_inspection test_logging test_os test_pdb test_unittest

[x] test: cpython/Lib/test/test_descr.py (TODO: 31)
[ ] test: cpython/Lib/test/test_descrtut.py (TODO: 3)

dependencies:

dependent tests: (no tests depend on descr)

[ ] test: cpython/Lib/test/test_import (TODO: 3)

dependencies:

dependent tests: (no tests depend on import)

[ ] test: cpython/Lib/test/test_class.py (TODO: 13)
[x] test: cpython/Lib/test/test_genericclass.py
[x] test: cpython/Lib/test/test_subclassinit.py

dependencies:

dependent tests: (no tests depend on class)

[x] test: cpython/Lib/test/test_cmd_line_script.py (TODO: 14)

dependencies:

dependent tests: (no tests depend on cmd_line_script)

[ ] test: cpython/Lib/test/test_pyrepl (TODO: 21)
[ ] test: cpython/Lib/test/test_repl.py (TODO: 7)

dependencies:

dependent tests: (no tests depend on pyrepl)

Legend:

  • [+] path exists in CPython
  • [x] up-to-date, [ ] outdated

@youknowone

Copy link
Copy Markdown
Member
======================================================================
UNEXPECTED SUCCESS: test_simple_completion (test.test_pyrepl.test_pyrepl.TestPyReplCompleter.test_simple_completion)
---------------------------------

rlcompleter closes the parenthesis for a callable whose signature takes
no parameters, and leaves it open when the signature cannot be read:

    def _callable_postfix(self, val, word):
        if callable(val):
            word += "("
            try:
                if not inspect.signature(val).parameters:
                    word += ")"
            except ValueError:
                pass

os.getpid reports `()` since the generated signature carries the `--`
terminator, so completing `os.getpid` yields `os.getpid()` and the test
passes.

Assisted-by: Claude Code:claude-opus-5
A method that needs an owned handle cannot take `&self`, so it takes the
receiver as its first argument and names it `zelf`, `self` being a Rust
keyword. 24 of the 163 such methods named it something else: `instance`
(11), `obj` (4), `exc` (3), `_self` (3), `_instance` (2).

Rename them. The generated signature is unaffected either way, since the
macro marks the receiver by position rather than by name.

Assisted-by: Claude Code:claude-opus-5
The receiver is now marked by position, so the name it carries no longer
matters, and no path reaches this check with an argument named zelf:

- a #[pymethod] or #[pyclassmethod] without a `&self` receiver has its
  first argument replaced by the marker before the name is read
- a `&self` receiver is handled as syn::FnArg::Receiver
- a #[pymethod(raw)] is passed to static_raw_func, whose PyNativeFn bound
  fixes its signature to (&VirtualMachine, FuncArgs)
- no #[pystaticmethod] or #[pyfunction] names an argument zelf
- #[pymember], #[pygetset] and #[pyslot] never reach func_sig

Every generated signature is byte for byte identical without it.

Assisted-by: Claude Code:claude-opus-5
A #[pyclassmethod] that takes FuncArgs receives the class inside the
bundle along with every other argument, but the marker was spent on the
bundle and the arguments went unreported:

    >>> object.__subclasshook__.__text_signature__
    '($type, /)'                    # was, claims it takes nothing
    '($type, *args, **kwargs)'      # now

Report both, the shape CPython uses for __new__. Two methods take their
arguments this way, __subclasshook__ on object and on type.

Assisted-by: Claude Code:claude-opus-5
@codspeed

codspeed Bot commented Aug 30, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 36 untouched benchmarks


Comparing leehanjeong:8383-method-text-signature (5d4257b) with main (4bd9545)

Open in CodSpeed

@leehanjeong leehanjeong changed the title Report __text_signature__ for methods, and keep it out of __doc__ Report every generated __text_signature__, and keep it out of __doc__ Aug 30, 2026
@leehanjeong
leehanjeong marked this pull request as ready for review August 30, 2026 18:55

@youknowone youknowone left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks great, thank you so much! 👍

@youknowone
youknowone merged commit 6cf0321 into RustPython:main Aug 31, 2026
30 checks passed
youknowone pushed a commit that referenced this pull request Sep 16, 2026
…c__` (#8614)

* Strip the signature prefix from builtin __doc__

A function's signature and its documentation share one string, separated
by `\n--\n\n`, as in CPython. __text_signature__ read the front of it,
but __doc__ returned the whole thing:

    >>> len.__doc__
    'len(obj, /)\n--\n\nReturn the number of items in a container.'   # was
    'Return the number of items in a container.'                      # now

44 of the 46 builtin functions leaked the prefix.

get_doc_from_internal_doc already stripped it but was reachable only from
PyType.__doc__. Split it the way CPython does, into doc_without_signature
(_PyType_DocWithoutSignature) and a wrapper mapping an empty result to
None (_PyType_GetDocFromInternalDoc), and call it from PyNativeFunction
and PyMethodDescriptor as well.

Assisted-by: Claude Code:claude-opus-5

* Terminate a generated __text_signature__ that carries no documentation

A signature and its documentation share one string, separated by
`\n--\n\n`, and readers find the signature by searching for that marker.
A #[pyfunction] with no doc comment stored the bare signature, which no
reader could parse:

    >>> time.clock_getres.__doc__
    'clock_getres(clk_id, /)'
    >>> time.clock_getres.__text_signature__
    None

Emit the marker in that case too, as Argument Clinic does for an
undocumented function. 136 of the 915 functions reachable from the
importable modules were affected.

__doc__ now reports None for them, matching CPython, because nothing
follows the marker.

Assisted-by: Claude Code:claude-opus-5

* Report a __text_signature__ for every method

A #[pymethod] built its doc only when the method carried a doc comment,
so one without a comment dropped the generated signature too:

    >>> list.append.__text_signature__
    None

Build it from either part, as #[pyfunction] does. All 1101 method
descriptors reachable from the builtin types now report a signature, up
from 1, and 713 of the 1030 that CPython also describes match it exactly.

A method that takes its receiver as an ordinary first argument instead of
`&self` reported that argument by name, so binding the method could not
drop it:

    >>> inspect.signature([].__dir__)
    (obj, /)     # was
    ()           # now

Mark it $self, or $type for a classmethod, as CPython does. 310 methods
were affected.

test_unbound_builtin_method_noargs and test_bound_builtin_method_noargs
pass now.

Assisted-by: Claude Code:claude-opus-5

* Drop the expectedFailure on test_simple_completion

rlcompleter closes the parenthesis for a callable whose signature takes
no parameters, and leaves it open when the signature cannot be read:

    def _callable_postfix(self, val, word):
        if callable(val):
            word += "("
            try:
                if not inspect.signature(val).parameters:
                    word += ")"
            except ValueError:
                pass

os.getpid reports `()` since the generated signature carries the `--`
terminator, so completing `os.getpid` yields `os.getpid()` and the test
passes.

Assisted-by: Claude Code:claude-opus-5

* Name every #[pymethod] receiver zelf

A method that needs an owned handle cannot take `&self`, so it takes the
receiver as its first argument and names it `zelf`, `self` being a Rust
keyword. 24 of the 163 such methods named it something else: `instance`
(11), `obj` (4), `exc` (3), `_self` (3), `_instance` (2).

Rename them. The generated signature is unaffected either way, since the
macro marks the receiver by position rather than by name.

Assisted-by: Claude Code:claude-opus-5

* Drop the unreachable zelf name check in func_sig

The receiver is now marked by position, so the name it carries no longer
matters, and no path reaches this check with an argument named zelf:

- a #[pymethod] or #[pyclassmethod] without a `&self` receiver has its
  first argument replaced by the marker before the name is read
- a `&self` receiver is handled as syn::FnArg::Receiver
- a #[pymethod(raw)] is passed to static_raw_func, whose PyNativeFn bound
  fixes its signature to (&VirtualMachine, FuncArgs)
- no #[pystaticmethod] or #[pyfunction] names an argument zelf
- #[pymember], #[pygetset] and #[pyslot] never reach func_sig

Every generated signature is byte for byte identical without it.

Assisted-by: Claude Code:claude-opus-5

* Keep the argument bundle when marking the receiver

A #[pyclassmethod] that takes FuncArgs receives the class inside the
bundle along with every other argument, but the marker was spent on the
bundle and the arguments went unreported:

    >>> object.__subclasshook__.__text_signature__
    '($type, /)'                    # was, claims it takes nothing
    '($type, *args, **kwargs)'      # now

Report both, the shape CPython uses for __new__. Two methods take their
arguments this way, __subclasshook__ on object and on type.

Assisted-by: Claude Code:claude-opus-5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

z-ca-2026 Tag to track Contribution Academy 2026

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants