Fix segfault and invalid cast in UntypedStorage#186207
Open
vishalgoyal316 wants to merge 1 commit into
Open
Conversation
Passing tensor.data_ptr() as allocator= to UntypedStorage() caused a segfault because the integer was cast to c10::Allocator* and used to call allocate(). Fix: add c10::isKnownAllocator() which validates the pointer against PyTorch's registered allocators before use, raising RuntimeError instead of crashing. Also guard TypedStorage._to() against quantized dtype casts (qint8, qint32, quint8, etc.) which are semantically invalid without scale and zero_point; users should dequantize first. Added test to verify both changes.
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/186207
Note: Links to docs will display an error until the docs builds have been completed. ⏳ No Failures, 50 PendingAs of commit 3828d2c with merge base 35e0ad6 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
This PR needs a
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes two error-checking gaps reported in #169210 where invalid usage of
UntypedStoragecaused a segfault rather than a clear Python error.Fix 1 — Segfault on invalid
allocator=argument (c10/,torch/csrc/)Passing
tensor.data_ptr()asallocator=toUntypedStorage()caused a segfault because the integer was blindly cast toc10::Allocator*andallocator->allocate()was called on it. Addsc10::isKnownAllocator()which validates the pointer against PyTorch's registered allocators (CPU default, all device slots viaallocator_array, CUDA caching allocator) before use.THPStorage_pynewnow callsTORCH_CHECK(isKnownAllocator(allocator))and raisesRuntimeErrorwith an actionable message instead of crashing.Fix 2 — Silent invalid cast on quantized storage (
torch/storage.py)TypedStorage._to()silently attempted dtype conversion for quantized storage types (qint8,qint32,quint8,quint4x2,quint2x4), which is semantically invalid because quantized values requirescaleandzero_pointto be meaningful. Adds
_raise_if_quantized_storage_cast()guard inTypedStorage._to()which raisesRuntimeErrordirecting users to dequantize first.Also fixes two pre-existing clang-tidy warnings in
Storage.cpp(unmatchedNOLINTENDandPyGetSetDefdesignated initializer warnings) that were surfaced when the file was touched.Test plan
New tests in
test/test_torch.py:test_untyped_storage_invalid_allocator_raises— verifiesRuntimeErroris raised whendata_ptr()is passed asallocator=test_quantized_storage_dtype_cast_raises— verifies all 12 dtype cast methods raiseRuntimeErroracross all 5 quantized storage typesCo-authored-with: AI assistant (Cursor/Claude Sonnet)
Fixes: #169210