ctest: add support for modules - #5459
Draft
dybucc wants to merge 9 commits into
Draft
Conversation
dybucc
force-pushed
the
feature/ctest-specific-modules
branch
9 times, most recently
from
September 5, 2026 11:20
060d0cc to
2e69844
Compare
Add support for parsing modules in the input Rust crate to `ctest`. This should allow more easily implementing support for a number of recent feature additions that have been needed in `libc`.
dybucc
force-pushed
the
feature/ctest-specific-modules
branch
2 times, most recently
from
September 6, 2026 13:35
30b4a8a to
f845200
Compare
Add support in `TranslationHelper` to filter out modules based off of a new type of skip accepted in `TestGenerator`'s public API. Filtering containing items is done by filtering on stringified paths (e.g. `foo::bar` to skip function `bar` inside top-level module `crate::foo`.)
Trim comment lines and strings to not surpass an 80-character mark from the very first character after a newline in source. This has been applied automatically through relevant unstable options in `rustfmt`.
- Rename `ident` function on all parsed items to `path`. After the changes in the last few patches, this function was returning the cached string we keep in each item representative of the stringified token stream from each item's `path` field. Thus, speaking of an identifier here is a bit misleading, and considering it a path to the item is more correct. - Add `ident` function to return the last segment of each parsed item's `path` field. This corresponds now with the previous semantics of `ident`, now `path`, but breaks the API because it returns a fully-owned `String` and not a `&str`. It also incurrs an additional allocation on each call, but that can be easily fixed by also "caching" this last segment of the item's path in one of the item's fields.
- Replace and tweak call sites where there were uses of the `ident` function exposed by parsed items into using a combination of both the new `ident` (which replicates the former's semantics,) `path` (which provides the full path to the item from the crate root,) and a new routine; `escape_item_path`. The latter is used to replace the default path separator `::` with `_`. Use of the these three is necessary to get the tests to both refer to the right Rust items in the parsed crate submodules, while keeping the same (single-segment) identifiers for C symbols. - Add `rust_ty` and `rust_val` fields to some of the types gathering data for the test templates. This is also necessary for keeping track of the actual paths to the types and symbols exposed by the crate, as modules introduce the possibility for items to be referred to by a path with more than one segment. - Tweak uses of Rust to C `MapInput` type variants for remapping into specifically only returning `CEnum`s in the current module. Before modules were supported, checking straight with the list of skips in the running `TestGenerator` was enough. This is not the case anymore, as those skips are provided by the user with respect to full paths (i.e. they apply globally across parsed modules.) The change in this patch ensures that when returning a `MapInput` variant that yields a type and not an item, the `CEnumType` variant is returned only when the current module being parsed contains an alias with the passed identifier, and further checks with the `TestGenerator` if the full path to that alias (if found) has a `CEnum` remapping set. - Tweak the Rust test template to reflect the changes made to the `template` module. This uses the new fields introduced in this same patch, and changes some uses of the `id` field to either one of the `rust_ty` or `rust_val` fields, as those keep the full path to the item type or item, respectively.
- Tweak one of the tests to adjust to the way module support has been implemented. Previously, items in nested modules would be expected to surface at the top-level. Now they are meant to be part of the parsed nested module within the initial `FfiItems` instance. - Tweak Rust test template to avoid `unused` lints against some of the utility functions. I am still looking through some stuff in the tests, but thus far these functions sometimes simply don't get used because certain askama loops never iterate when the tests that use them are wholly skipped.
dybucc
force-pushed
the
feature/ctest-specific-modules
branch
from
September 6, 2026 16:20
f845200 to
3f32191
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Add support for (currently) parsing and filtering Rust modules found in
the target crate to test. This is supposed to provide scaffolding for
multiple open feature requests that depend on ctest having support for
modules.
I decided to open this before finishing up work on generating proper
tests on both C and Rust sides, because I am not sure if I should just
let the user deal with including a cohesive set of modules that do not
cause item resolution conflicts, or if I should instead generate
separate tests for each parsed module.
Edit: the above paragraph really goes to say that I am not sure
whether I should generate multiple test templates while recursing
through the modules in
TestGenerator::generate_files, or if I shouldinstead recurse through them in
TestTemplate::new(with the passedFfiItems) and return a collection of test templates for each parsedmodule that has not been skipped.
Edit 2: I decided to go for generating multiple test templates for
each parsed module.
Edit 3: I completely forgot about
usestatements, so most of thisis currently useless.
cc @tgross35 @mbyx
TODO
Look into parsing
usestatements, as those are the one thingsmissing from getting modules to work.
Update the docs to the public API of
TestGeneratorto mentionthat the name remapping ought return the C identifiers without
worrying about potential item resolution conflicts.
Look through the
populate_roundtrip_testsfunction, as that oneI skipped to go first to
populate_field_ptr_testsbecause of theorder these tests appear in the test template files.
Look into properly resolving paths to the types of type aliases,
constants and statics. These are handled poorly now. An example is
the check for arrays that is made in
template::TestTemplate::populate_roundtrip_tests, which does notconsider whether the type of the alias is itself an alias to be
resolved recursively (and potentially be an array.)
Update 1
I have changed the way we parse item identifiers, and more specifically,
their absolute paths without
crateat the start (e.g.foo::bar::ctime()instead ofcrate::foo::bar::ctime().)This should make filtering continue working as-is right now for items,
but allow filtering on items in nested submodules by using an
appropriate string matching something like the above example.
Update 2
I think the initial set of changes is done. I have gone down the route
of potentially generating multiple templates for each parsed module,
such that more than a single test is generated. I have not yet
implemented the multi-template generation stuff, but that should be
fairly straightforward. What I believe to be done is the logic for
generating test to the right Rust paths while also keeping the single
segment paths on the C side of things.
I am currently working through the tests, to see what is missing and
broken from the prior
ctestinterface. At this point, I am only tryingto work through the prior tests to see if they continue working and
generating the same set of tests (barring nested modules.)
Checklist
ctest/testsandctest/src/tests.rshave beenupdated
LIBC_BLESS=1 cargo test -p ctest);@rustbot label +stable-nominated