Thanks to visit codestin.com
Credit goes to github.com

Skip to content

duckdb_base_std:: plus compile time test on discontinued functions#17866

Merged
Mytherin merged 16 commits intoduckdb:v1.3-ossivalisfrom
carlopi:wrapped_std_forbidden_std
Jun 12, 2025
Merged

duckdb_base_std:: plus compile time test on discontinued functions#17866
Mytherin merged 16 commits intoduckdb:v1.3-ossivalisfrom
carlopi:wrapped_std_forbidden_std

Conversation

@carlopi
Copy link
Contributor

@carlopi carlopi commented Jun 10, 2025 •
edited
Loading

Core constraints are as follow:

  • we want to deprecate some std:: namespace functions (e.g.: unique_ptr or isspace)
  • we want to provide a duckdb level replacement
  • implementing duckdb level replacement might need to access the original std code
  • there should be a mode to trigger compile time errors

Design:

  1. ::std namespace is wrapped in ::duckdb_base_std. This means all original functions are still available, if needed during migration, if needed to implement wrapped classes, or for boundaries with other libraries. This is currently done only for certain classes or functions, but more can be exposed, it's a matter of adding to the headers in src/include/duckdb/original/std/*.hpp or adding a new header
  2. duckdb level equivalent are implemented, possibly including the wrapped header and using such as duckdb_base_std::unique_ptr instead of std::unique_ptr
  3. in src/include/duckdb/common/shadow_forbidden_functions.hpp functions or classes to be forbidden are re-declared in std namespace

Now when explicitly including shadow_forbidden_functions.hpp (that can be injected at compile time, see changes in CMakeLists.txt) the compiled codebase will be checked for absence of those functions or classes due to errors being throws connected to multiple declarations (but only on usage).

How to add more functions:
  1. Add relevant header in src/include/duckdb/original, adding using ::std::some_func within the namespace duckdb_base_std
  2. Move explicit usage of std::some_func to duckdb_base_std::some_func
  3. Add some_func to shadow_forbidden_functions.hpp
  4. Compile with SHADOW_FORBIDDEN_FUNCTIONS=1 GEN=ninja make, now any other usage will be flagged as compiler error. If so, iterate

How to expand to more extensions

Currently this is checked only for a very limited set of source files (src/, third_party/, and some in-tree extensions).
This can be expanded for relevant extensions like:

VCPKG_... CORE_FUNCTIONS="a;b;c" SHADOW_FORBIDDEN_FUNCTIONS=1 GEN=ninja make

Future work

  1. Add CI testing that enables SHADOW_FORBIDDEN_FUNCTIONS on a set of known to be working extensions
  2. Add std's *stream to the group of to be avoided functions

@carlopi
Copy link
Contributor Author

carlopi commented Jun 10, 2025

For the reviewers, main question are:

  • do the changes to third_party/ and src/ look to make sense? it's this too invasive
  • usability / how to roll this in
  • bikeshedding
  • else

@carlopi carlopi requested a review from Mytherin June 10, 2025 14:31
@carlopi

This comment was marked as outdated.

Sign in to view
@carlopi carlopi force-pushed the wrapped_std_forbidden_std branch from f680b79 to c3758f8 Compare June 10, 2025 20:45
@duckdb-draftbot duckdb-draftbot marked this pull request as draft June 10, 2025 20:45
@carlopi carlopi marked this pull request as ready for review June 10, 2025 20:46
Mytherin
Mytherin reviewed Jun 11, 2025
View reviewed changes
Copy link
Collaborator

@Mytherin Mytherin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR! Looks good - some comments

@carlopi carlopi force-pushed the wrapped_std_forbidden_std branch from e029ace to 5ab81c8 Compare June 11, 2025 12:46
@carlopi carlopi marked this pull request as draft June 11, 2025 12:51
@carlopi carlopi marked this pull request as ready for review June 11, 2025 13:19
@carlopi carlopi changed the title duckdb_wrapped::std plus compile time test on discontinued functions duckdb_base_std:: plus compile time test on discontinued functions Jun 11, 2025
@duckdb-draftbot duckdb-draftbot marked this pull request as draft June 11, 2025 15:06
@carlopi carlopi marked this pull request as ready for review June 11, 2025 17:42
@carlopi
Copy link
Contributor Author

carlopi commented Jun 12, 2025 •
edited
Loading

Thanks for the review comments, adapted, I think this is ready for another go

Mytherin
Mytherin reviewed Jun 12, 2025
View reviewed changes
Copy link
Collaborator

@Mytherin Mytherin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! I think this looks good - one minor comment

@Mytherin Mytherin merged commit f79594b into duckdb:v1.3-ossivalis Jun 12, 2025
93 of 99 checks passed
@carlopi carlopi deleted the wrapped_std_forbidden_std branch June 12, 2025 06:36
github-actions bot pushed a commit to duckdb/duckdb-r that referenced this pull request Jun 12, 2025
@krlmlr
vendor: Update vendored sources to duckdb/duckdb@f79594b
17a8a08 
`duckdb_base_std::` plus compile time test on discontinued functions (duckdb/duckdb#17866)
@github-actions github-actions bot mentioned this pull request Jun 12, 2025
Merged
@carlopi carlopi mentioned this pull request Jun 12, 2025
Merged
github-actions bot added a commit to duckdb/duckdb-r that referenced this pull request Jun 12, 2025
@github-actions @krlmlr
vendor: Update vendored sources to duckdb/duckdb@f79594b (#1232)
9cc4bb4 
`duckdb_base_std::` plus compile time test on discontinued functions (duckdb/duckdb#17866)

Co-authored-by: krlmlr <[email protected]>
Mytherin added a commit that referenced this pull request Jun 12, 2025
@Mytherin
Deprecated std::stringstream (#17894)
dd2f747 
Improved version of #17807, using
infrastructure from #17866 to check
`std::stringstream` are not used anymore at compile time.

Original problem I had bumped in duckdb-wasm, where there would be some
yet to be properly understood factor that would change the global static
default locale to an unsupported one that would, via a SQL call to
`duckdb_tables` end up calling a locale-influenced stringstream that
would then throw.

Testing can also be done on native applying the following diff, yet to
be added as testing mode
```diff
diff --git a/test/unittest.cpp b/test/unittest.cpp
index 1c04f98..a70847a2a6 100644
--- a/test/unittest.cpp
+++ b/test/unittest.cpp
@@ -30,6 +30,8 @@ int main(int argc, char *argv[]) {
        duckdb::unique_ptr<FileSystem> fs = FileSystem::CreateLocal();
        string test_directory = DUCKDB_ROOT_DIRECTORY;
 
+       std::locale::global(std::locale("ja_JP.SJIS"));
+
        int new_argc = 0;
        auto new_argv = duckdb::unique_ptr<char *[]>(new char *[argc]);
        for (int i = 0; i < argc; i++) {
```

Also cleans up comment from previous PR review at
#17866 (comment)

Note that other streams might (and will) have the same problems, so it
would be handy to expand this further, here taking it one step further.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Mytherin Mytherin Mytherin left review comments

Assignees

No one assigned

Labels

Ready For Review

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@carlopi @Mytherin

Comments