Thanks to visit codestin.com
Credit goes to github.com

Skip to content

BUG: Fix byteswapping for complex scalars#2886

Merged
charris merged 1 commit into
numpy:masterfrom
seberg:complex256-byteswap
Jan 5, 2013
Merged

BUG: Fix byteswapping for complex scalars#2886
charris merged 1 commit into
numpy:masterfrom
seberg:complex256-byteswap

Conversation

@seberg
Copy link
Copy Markdown
Member

@seberg seberg commented Jan 5, 2013

During a cleanup, the fast paths were invalidated because SIZEOF_LONGDOUBLE
was not defined anymore and needs to be replaced with NPY_SIZEOF_LONGDOUBLE.
The other SIZEOF macros still existed however so only complex long double
broke because it switched to the already broken fast path.

This commit fixes the fast path, and replaces all SIZEOF_ macros within
arraytypes.c.src with their corresponding NPY_SIZEOF_ macros.


This probably fixes issue gh-411 though that still needs checking. Note that there are still many occurrences where these macros need to be replaced. Also I noticed that the noprefix.h does not include SIZEOF_COMPLEX_*.

During a cleanup, the fast paths were invalidated because SIZEOF_LONGDOUBLE
was not defined anymore and needs to be replaced with NPY_SIZEOF_LONGDOUBLE.
The other SIZEOF macros still existed however so only complex long double
broke because it switched to the already broken fast path.

This commit fixes the fast path, and replaces all SIZEOF_ macros within
arraytypes.c.src with their corresponding NPY_SIZEOF_ macros.
@njsmith
Copy link
Copy Markdown
Member

njsmith commented Jan 5, 2013

This change looks good to me.

If there are still lots of places in the code that are using invalid macros, maybe we should just run a global copy/replace now while we're thinking about it?

@charris
Copy link
Copy Markdown
Member

charris commented Jan 5, 2013

Looks good to me also. The unprefixed macro should be put into the noprefix.h header. You can probably look for other spots using a modification of the replace_old_macros.sed script in tools/ I run those scripts with find. There are lot of files to look at...

@certik These sort of fixes probably need a back port as 1.7 is supposed to be 'macro clean' and clearly there are some subtle problems that can appear when that isn't done. But I hate to suggest such intrusive changes after the first rc.

charris added a commit that referenced this pull request Jan 5, 2013
BUG: Fix byteswapping for complex scalars
@charris charris merged commit 05dde0f into numpy:master Jan 5, 2013
@seberg
Copy link
Copy Markdown
Member Author

seberg commented Jan 5, 2013

On a different note though, maybe someone can just tell me that this is actually plausible:

In [7]: def tohex(num, swap=False):
    return '-'.join(c.encode('hex') for c in (num if not swap else num.byteswap()).tostring()[::1 if swap else -1])
   ...: 

In [8]: tohex(np.float128(1))
Out[8]: '83-00-01-83-00-00-3f-ff-80-00-00-00-00-00-00-00'

In [9]: tohex(np.float128(1))
Out[9]: '00-00-00-00-03-74-3f-ff-80-00-00-00-00-00-00-00'

the underlying data is different, but even when double checking the value is identical also == works, so I guess maybe my float128 is not a real float128? Of course the 3f-ff and some more seem to be always there.

@njsmith
Copy link
Copy Markdown
Member

njsmith commented Jan 5, 2013

Intel long doubles are 96 bits padded to 128 bit alignment, so it's
possible that what you're seeing are differences in the contents of the
padding. (Not sure which bits are ignored.)
On 5 Jan 2013 15:43, "seberg" [email protected] wrote:

On a different note though, maybe someone can just tell me that this is
actually plausible:

In [7]: def tohex(num, swap=False):
return '-'.join(c.encode('hex') for c in (num if not swap else num.byteswap()).tostring()[::1 if swap else -1])
...:
In [8]: tohex(np.float128(1))Out[8]: '83-00-01-83-00-00-3f-ff-80-00-00-00-00-00-00-00'
In [9]: tohex(np.float128(1))Out[9]: '00-00-00-00-03-74-3f-ff-80-00-00-00-00-00-00-00'

the underlying data is different, but even when double checking the value
is identical also == works, so I guess maybe my float128 is not a real
float128? Of course the 3f-ff and some more seem to be always there.


Reply to this email directly or view it on GitHubhttps://github.com//pull/2886#issuecomment-11915405.

@charris
Copy link
Copy Markdown
Member

charris commented Jan 5, 2013

This shows some more

charris@f16 [numpy.git (master)]$ grep -r '[^_]SIZEOF_' numpy/

@charris
Copy link
Copy Markdown
Member

charris commented Jan 5, 2013

@seberg Do you want to make those fixes or should I?

@seberg
Copy link
Copy Markdown
Member Author

seberg commented Jan 5, 2013

Please go ahead, this macro stuff is confusing me anyway...

@seberg seberg deleted the complex256-byteswap branch January 5, 2013 16:39
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, t is one of:

In [2]: np.typecodes['Complex']                                                
Out[2]: 'FDG'

but the original test was testing with t="c8" (effectively). So why was testing for "c8" removed?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its not removed np.dtype('F') == np.dtype('c8').

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I see. All is ok then. Thanks!

@certik
Copy link
Copy Markdown
Contributor

certik commented Jan 5, 2013

During a cleanup, the fast paths were invalidated because SIZEOF_LONGDOUBLE
was not defined anymore and needs to be replaced with NPY_SIZEOF_LONGDOUBLE.
The other SIZEOF macros still existed however so only complex long double
broke because it switched to the already broken fast path.

I don't understand the paragraph. So SIZEOF_LONGDOUBLE is still defined, and that's the reason why the old code compiled, correct? What is the difference between NPY_SIZEOF_LONGDOUBLE and SIZEOF_LONGDOUBLE?

@certik
Copy link
Copy Markdown
Contributor

certik commented Jan 5, 2013

@charris, the change looks ok to me, I only have one question above in the test ("c8"...) and then about the macros in my previous comment. After I understand the answers to those two questions, I'll see if we should backport it.

@seberg
Copy link
Copy Markdown
Member Author

seberg commented Jan 5, 2013

I may have misinterpreted it (I don't know macro stuff), but my guess was, it is not defined anymore, but since it was only inside an #if ... preprocessor statement that statement that #if simply evaluated false dropping through to the general not unrolled for loop case (which was buggy prior to this, but since no compiler has a floating with more then 16 bytes it never mattered).

@certik
Copy link
Copy Markdown
Contributor

certik commented Jan 5, 2013

Ah, I see how it works. Yes, in this case I think we should backport it. @charris, @njsmith, let me know if you agree.

@njsmith
Copy link
Copy Markdown
Member

njsmith commented Jan 5, 2013

@certik: Are you planning to do an rc2?

@charris
Copy link
Copy Markdown
Member

charris commented Jan 5, 2013

@certik None of the uprefixed macros are defined for numpy sources, so any use of them is inviting a bug. Or may expose bugs ;) As Sebastian has pointed out, the compiler will not raise any errors if an undefined macro is used in a logic statement, it just evaluates to false. We will need to do another release candidate though.

@certik
Copy link
Copy Markdown
Contributor

certik commented Jan 5, 2013

@njsmith, I was planning to just do the final release, but given this, I can do rc2. It's not a problem at all --- I have it all scripted now. I am now updating my Fabric fab files for Mac, so that they are fully automated too. So either is fine with me.

@certik certik mentioned this pull request Jan 5, 2013
@njsmith
Copy link
Copy Markdown
Member

njsmith commented Jan 6, 2013

Yeah, I guess it's a good idea to both backport those and do an rc2 then.

@certik
Copy link
Copy Markdown
Contributor

certik commented Jan 6, 2013

OK.

@certik certik mentioned this pull request Jan 10, 2013
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants