Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@jchelly
Copy link
Collaborator

@jchelly jchelly commented Jan 20, 2023

For each halo the code needs to sum the X-ray luminosities of the relevant particles. These luminosities are recorded in multiple bands so they're stored in a 2D array indexed as [particle_index, band_index]. The code was calling np.sum() on this array which returns a single, total luminosity. Numpy then broadcasts this scalar back to an array when we store the result.

This pull request adds some asserts to catch this type of bug in future and corrects the calculation to only sum over the particle axis of the 2D luminosity array.

This should detect the bug that's causing all xray bands to
contain the same value in the output.
@jchelly
Copy link
Collaborator Author

jchelly commented Jan 20, 2023

The affected properties were Xraylum , Xrayphlum , Xraylum_no_agn and Xrayphlum_no_agn.

@jchelly jchelly self-assigned this Jan 20, 2023
@jchelly jchelly added the bug Something isn't working label Jan 20, 2023
@MatthieuSchaller MatthieuSchaller self-requested a review January 20, 2023 11:24
Copy link
Member

@MatthieuSchaller MatthieuSchaller left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All looks good to me. The fix matches what is done for other similar quantities, like the luminosities.

As a quick check, could @wullm or @jemme07 try on a snapshot of the recent runs?

@jemme07
Copy link

jemme07 commented Jan 30, 2023

All looks good to me. The fix matches what is done for other similar quantities, like the luminosities.

As a quick check, could @wullm or @jemme07 try on a snapshot of the recent runs?

I am runnig the z=0 Jets box atm.

We also need to merge the chages in this other branch: https://github.com/jchelly/SOAP/compare/master...SWIFTSIM:SOAP:flamingo-v2

@jemme07
Copy link

jemme07 commented Jan 30, 2023

@jchelly @MatthieuSchaller. Runnig the script halo_properties_L1000N1800.sh, which calls compute_halo_properties.py failed for the Jets run with the following message:

*** EXCEPTION ***
'NoneType' object has no attribute 'shape' on rank 126

Traceback (most recent call last):
  File "/snap8/scratch/dp004/fkgm22/SOAP//compute_halo_properties.py", line 477, in <module>
    compute_halo_properties()
  File "/snap8/scratch/dp004/fkgm22/SOAP//compute_halo_properties.py", line 414, in compute_halo_properties
    metadata = task_queue.execute_tasks(
  File "/snap8/scratch/dp004/fkgm22/SOAP/task_queue.py", line 195, in execute_tasks
    result.append(task(*args))
  File "/snap8/scratch/dp004/fkgm22/SOAP/chunk_tasks.py", line 265, in __call__
    total_time, task_time, nr_left, nr_done = process_halos(comm, cellgrid.snap_unit_registry, data, mesh,
  File "/snap8/scratch/dp004/fkgm22/SOAP/halo_tasks.py", line 273, in process_halos
    halo_result = process_single_halo(
  File "/snap8/scratch/dp004/fkgm22/SOAP/halo_tasks.py", line 118, in process_single_halo
    halo_prop.calculate(
  File "/snap8/scratch/dp004/fkgm22/SOAP/subhalo_properties.py", line 942, in calculate
    assert subhalo[name].shape == val.shape, f"Attempting to store {name} with wrong dimensions"
AttributeError: 'NoneType' object has no attribute 'shape'

@jchelly
Copy link
Collaborator Author

jchelly commented Jan 30, 2023

That has been caused by the check I put in to prevent further bugs like this one. The calculation can fail if it turns out too small a region was read in, in which case it returns None and it shouldn't try to verify the dimensions of the result. I thought I dealt with that case, but apparently not. I'll take another look at it.

@jchelly
Copy link
Collaborator Author

jchelly commented Jan 30, 2023

I think I just missed a "if val is not None" in the subhalo properties case.

@MatthieuSchaller
Copy link
Member

Thanks John. Jaime, could you test it once more?

@MatthieuSchaller
Copy link
Member

Ah wait, looks like no changes were actually made in the branch. Just an analysis of what went wrong.

@MatthieuSchaller
Copy link
Member

@jemme07 could you open a separate pull request with the changes you mentioned above?

@jchelly
Copy link
Collaborator Author

jchelly commented Feb 1, 2023

Yes, sorry, I looked at it a bit more and wasn't 100% sure where those None values were coming from so I haven't pushed a fix yet. I'll take another look at it now.

@jchelly
Copy link
Collaborator Author

jchelly commented Feb 1, 2023

I think the Nones actually come from halos which have no particles of some type. That doesn't change the fix though.

@jemme07
Copy link

jemme07 commented Feb 5, 2023

@jemme07 could you open a separate pull request with the changes you mentioned above?

I alread have. It is here: https://github.com/jchelly/SOAP/pull/19

@jemme07
Copy link

jemme07 commented Feb 5, 2023

It is all working now. I tested it with the HYDRO_STRONG_JETS model. The z=0 SOAP catalogue is temporarily stored here:
/cosma8/data/dp004/fkgm22/FLAMINGO_SOAP/ScienceRuns/L1000N1800/HYDRO_STRONG_JETS/SOAP/

Once I have write permissions in the FLAMINGO data directory I'll copy it over.

@MatthieuSchaller
Copy link
Member

@jemme07 could you open a separate pull request with the changes you mentioned above?

I alread have. It is here: jchelly#19

I don't understand. If this is missing in the main version then we should merge here not in John's version. We should really try to keep a single maintained official version otherwise we gonna have confusion about what to run.

@MatthieuSchaller
Copy link
Member

Thanks for checking it all works.

I'll accept.

@MatthieuSchaller MatthieuSchaller merged commit 0aec352 into master Feb 6, 2023
@MatthieuSchaller MatthieuSchaller deleted the xray_fix branch February 6, 2023 09:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants