Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Share unified UTF-8/16 parsing with BigInteger #95402

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 6 commits into from
Feb 1, 2024
Merged

Share unified UTF-8/16 parsing with BigInteger #95402

merged 6 commits into from
Feb 1, 2024

Conversation

huoyaoyuan
Copy link
Member

Follow up of #85978 . Should really be in same PR.

Copied latest CoreLib implementation and polyfill internal interfaces.
This allows sharing UTF unified parsing and formatting code.

Originally I tried this with heavy ifdefs and tricks like using TChar = System.Char;. Then I realized that all the internal members can be polyfilled in S.R.Numerics, with very little overhead for UTF-16. This also unblocks UTF-8 adoption for BigInteger.

Wanna check with @stephentoub for whether this design is desired.

@ghost ghost added community-contribution Indicates that the PR has been added by a community member area-System.Numerics labels Nov 29, 2023
@ghost
Copy link

ghost commented Nov 29, 2023

Tagging subscribers to this area: @dotnet/area-system-numerics
See info in area-owners.md if you want to be subscribed.

Issue Details

Follow up of #85978 . Should really be in same PR.

Copied latest CoreLib implementation and polyfill internal interfaces.
This allows sharing UTF unified parsing and formatting code.

Originally I tried this with heavy ifdefs and tricks like using TChar = System.Char;. Then I realized that all the internal members can be polyfilled in S.R.Numerics, with very little overhead for UTF-16. This also unblocks UTF-8 adoption for BigInteger.

Wanna check with @stephentoub for whether this design is desired.

Author: huoyaoyuan
Assignees: -
Labels:

area-System.Numerics, community-contribution
Milestone: -

This was referenced Nov 29, 2023
Open
Closed
Closed
Closed
@huoyaoyuan
Copy link
Member Author

Test failures look unrelated. No integer/biginteger related.

@huoyaoyuan huoyaoyuan mentioned this pull request Dec 7, 2023
Merged
@huoyaoyuan huoyaoyuan mentioned this pull request Jan 13, 2024
Merged
This was referenced Jan 29, 2024
Closed
Closed
Open
@tannergooding tannergooding mentioned this pull request Feb 1, 2024
Merged
@tannergooding
Copy link
Member

CC. @vargaz, another LLVM OOM. It's not clear to me which line indicates the failing assembly so I can know what needs to be skipped until the final PR can get merged?

C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(818,5): error : Failed to compile C:\helix\work\workitem\e\wasm_build\obj\wasm\for-build\aot-instances.dll.bc -> C:\helix\work\workitem\e\wasm_build\obj\wasm\for-build\aot-instances.dll.o [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(818,5): error : C:\helix\work\workitem\e\publish>C:\Windows\System32\chcp.com 65001 1>nul  [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(818,5): error : C:\helix\work\workitem\e\publish>setlocal [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(818,5): error : C:\helix\work\workitem\e\publish>set errorlevel=dummy  [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(818,5): error : C:\helix\work\workitem\e\publish>set errorlevel=  [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(818,5): error : C:\helix\work\workitem\e\publish>"C:\helix\work\correlation\build\\wasi-sdk\\bin\clang.exe" "@C:\helix\work\correlation\build\microsoft.netcore.app.runtime.wasi-wasm\runtimes\wasi-wasm\native\src\wasi-default.rsp" "@C:\helix\work\workitem\e\wasm_build\obj\wasm\for-build\wasi-compile-bc.rsp" -c -o "C:\Users\ContainerAdministrator\AppData\Local\Temp\tmpa2qi3t.tmp" "C:\helix\work\workitem\e\wasm_build\obj\wasm\for-build\aot-instances.dll.bc"  [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(818,5): error : clang version 16.0.0 [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(818,5): error : Target: wasm32-unknown-wasi [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(818,5): error : Thread model: posix [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(818,5): error : InstalledDir: C:/helix/work/correlation/build//wasi-sdk//bin [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(818,5): error :  (in-process) [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(818,5): error :  "C:/helix/work/correlation/build/wasi-sdk/bin/clang.exe" -cc1 -triple wasm32-unknown-wasi -emit-obj -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name aot-instances.dll.bc -mrelocation-model static -mframe-pointer=none -ffp-contract=on -fno-rounding-math -mconstructor-aliases -target-cpu generic -fvisibility=hidden -mllvm -treat-scalable-fixed-error-as-warning -debug-info-kind=constructor -dwarf-version=4 -debugger-tuning=gdb -v -fcoverage-compilation-dir=C:/helix/work/workitem/e/publish -resource-dir C:/helix/work/correlation/build/wasi-sdk/lib/clang/16 -Oz -fdebug-compilation-dir=C:/helix/work/workitem/e/publish -ferror-limit 19 -fgnuc-version=4.2.1 -vectorize-slp -o "C:\\Users\\ContainerAdministrator\\AppData\\Local\\Temp\\tmpa2qi3t.tmp" -x ir "C:\\helix\\work\\workitem\\e\\wasm_build\\obj\\wasm\\for-build\\aot-instances.dll.bc" [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(818,5): error : clang -cc1 version 16.0.0 based upon LLVM 16.0.0 default target wasm32-wasi [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(818,5): error : LLVM ERROR: out of memory [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(818,5): error : Allocation failed [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(818,5): error : PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace, preprocessed source, and associated run script. [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(818,5): error : Stack dump: [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(818,5): error : 0.	Program arguments: C:\\helix\\work\\correlation\\build\\\\wasi-sdk\\\\bin\\clang.exe @C:\\helix\\work\\correlation\\build\\microsoft.netcore.app.runtime.wasi-wasm\\runtimes\\wasi-wasm\\native\\src\\wasi-default.rsp @C:\\helix\\work\\workitem\\e\\wasm_build\\obj\\wasm\\for-build\\wasi-compile-bc.rsp -c -o C:\\Users\\ContainerAdministrator\\AppData\\Local\\Temp\\tmpa2qi3t.tmp C:\\helix\\work\\workitem\\e\\wasm_build\\obj\\wasm\\for-build\\aot-instances.dll.bc [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(818,5): error : 1.	Code generation [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(818,5): error : 2.	Running pass 'Function Pass Manager' on module 'C:\helix\work\workitem\e\wasm_build\obj\wasm\for-build\aot-instances.dll.bc'. [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(818,5): error : 3.	Running pass 'WebAssembly Instruction Selection' on function '@aot_instances_System_SZGenericArrayEnumerator_1_T_SINGLE__cctor' [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]

@vargaz
Copy link
Contributor

vargaz commented Feb 1, 2024

Those failures are happening on other PRs as well, so i think this is ok to merge.

@tannergooding tannergooding merged commit 9e302a5 into dotnet:main Feb 1, 2024
@huoyaoyuan huoyaoyuan deleted the numerics-format-1-polifill branch February 2, 2024 02:32
@huoyaoyuan huoyaoyuan mentioned this pull request Feb 2, 2024
Merged
@matouskozak matouskozak mentioned this pull request Feb 14, 2024
Closed
@github-actions github-actions bot locked and limited conversation to collaborators Mar 3, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@tannergooding tannergooding tannergooding approved these changes

Assignees
No one assigned
Labels
area-System.Numerics community-contribution Indicates that the PR has been added by a community member
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@huoyaoyuan @tannergooding @vargaz