Loading Generic Particle Data¶
This example creates a fake in-memory particle dataset and then loads it as a yt dataset using the load_particles function.
Our “fake” dataset will be numpy arrays filled with normally distributed randoml particle positions and uniform particle masses. Since real data is often scaled, I arbitrarily multiply by 1e6 to show how to deal with scaled data.
[1]:
import numpy as np
rng = np.random.default_rng()
n_particles = 5_000_000
ppx, ppy, ppz = 1e6 * rng.normal(size=(3, n_particles))
ppm = np.ones(n_particles)
The load_particles function accepts a dictionary populated with particle data fields loaded in memory as numpy arrays or python lists:
[2]:
data = {
("io", "particle_position_x"): ppx,
("io", "particle_position_y"): ppy,
("io", "particle_position_z"): ppz,
("io", "particle_mass"): ppm,
}
To hook up with yt’s internal field system, the dictionary keys must be ‘particle_position_x’, ‘particle_position_y’, ‘particle_position_z’, and ‘particle_mass’, as well as any other particle field provided by one of the particle frontends.
The load_particles function transforms the data dictionary into an in-memory yt Dataset object, providing an interface for further analysis with yt. The example below illustrates how to load the data dictionary we created above.
[3]:
import yt
from yt.units import Msun, parsec
bbox = 1.1 * np.array(
[[min(ppx), max(ppx)], [min(ppy), max(ppy)], [min(ppz), max(ppz)]]
)
ds = yt.load_particles(data, length_unit=1.0 * parsec, mass_unit=1e8 * Msun, bbox=bbox)
The length_unit and mass_unit are the conversion from the units used in the data dictionary to CGS. I’ve arbitrarily chosen one parsec and 10^8 Msun for this example.
The n_ref parameter controls how many particle it takes to accumulate in an oct-tree cell to trigger refinement. Larger n_ref will decrease poisson noise at the cost of resolution in the octree.
Finally, the bbox parameter is a bounding box in the units of the dataset that contains all of the particles. This is used to set the size of the base octree block.
This new dataset acts like any other yt Dataset object, and can be used to create data objects and query for yt fields.
[4]:
ad = ds.all_data()
print(ad.mean(("io", "particle_position_x")))
print(ad.sum(("io", "particle_mass")))
-582.08457330033 code_length
9.9420793e+47 g
We can project the particle mass field like so:
[5]:
prj = yt.ParticleProjectionPlot(ds, "z", ("io", "particle_mass"))
prj.set_width((8, "Mpc"))
[5]:
Finally, one can specify multiple particle types in the data directory by setting the field names to be field tuples (the default field type for particles is "io") if one is not specified:
[6]:
n_gas_particles = 1_000_000
n_star_particles = 1_000_000
n_dm_particles = 2_000_000
ppxg, ppyg, ppzg = 1e6 * rng.normal(size=(3, n_gas_particles))
ppmg = np.ones(n_gas_particles)
hsml = 10000 * np.ones(n_gas_particles)
dens = 2.0e-4 * np.ones(n_gas_particles)
ppxd, ppyd, ppzd = 1e6 * rng.normal(size=(3, n_dm_particles))
ppmd = np.ones(n_dm_particles)
ppxs, ppys, ppzs = 5e5 * rng.normal(size=(3, n_star_particles))
ppms = 0.1 * np.ones(n_star_particles)
bbox = 1.1 * np.array(
[
[
min(ppxg.min(), ppxd.min(), ppxs.min()),
max(ppxg.max(), ppxd.max(), ppxs.max()),
],
[
min(ppyg.min(), ppyd.min(), ppys.min()),
max(ppyg.max(), ppyd.max(), ppys.max()),
],
[
min(ppzg.min(), ppzd.min(), ppzs.min()),
max(ppzg.max(), ppzd.max(), ppzs.max()),
],
]
)
data2 = {
("gas", "particle_position_x"): ppxg,
("gas", "particle_position_y"): ppyg,
("gas", "particle_position_z"): ppzg,
("gas", "particle_mass"): ppmg,
("gas", "smoothing_length"): hsml,
("gas", "density"): dens,
("dm", "particle_position_x"): ppxd,
("dm", "particle_position_y"): ppyd,
("dm", "particle_position_z"): ppzd,
("dm", "particle_mass"): ppmd,
("star", "particle_position_x"): ppxs,
("star", "particle_position_y"): ppys,
("star", "particle_position_z"): ppzs,
("star", "particle_mass"): ppms,
}
ds2 = yt.load_particles(
data2, length_unit=1.0 * parsec, mass_unit=1e8 * Msun, bbox=bbox
)
We now have separate "gas", "dm", and "star" particles. Since the "gas" particles have "density" and "smoothing_length" fields, they are recognized as SPH particles:
[7]:
ad = ds2.all_data()
c = np.array([ad.mean(("gas", ax)).to("code_length") for ax in "xyz"])
[8]:
slc = yt.SlicePlot(ds2, "z", ("gas", "density"), center=c)
slc.set_zlim(("gas", "density"), 1e-19, 2.0e-18)
slc.set_width((4, "Mpc"))
slc.show()
---------------------------------------------------------------------------
UnitConversionError Traceback (most recent call last)
Cell In[8], line 1
----> 1 slc = yt.SlicePlot(ds2, "z", ("gas", "density"), center=c)
2 slc.set_zlim(("gas", "density"), 1e-19, 2.0e-18)
3 slc.set_width((4, "Mpc"))
File /var/jenkins_home/workspace/yt_docs/yt/visualization/plot_window.py:1856, in AxisAlignedSlicePlot.__init__(self, ds, normal, fields, center, width, axes_unit, origin, fontsize, field_parameters, window_size, aspect, data_source, buff_size, north_vector)
1854 slc.get_data(fields)
1855 validate_mesh_fields(slc, fields)
-> 1856 PWViewerMPL.__init__(
1857 self,
1858 slc,
1859 bounds,
1860 origin=origin,
1861 fontsize=fontsize,
1862 fields=fields,
1863 window_size=window_size,
1864 aspect=aspect,
1865 buff_size=buff_size,
1866 geometry=ds.geometry,
1867 )
1868 if axes_unit is None:
1869 axes_unit = get_axes_unit(width, ds)
File /var/jenkins_home/workspace/yt_docs/yt/visualization/plot_window.py:876, in PWViewerMPL.__init__(self, *args, **kwargs)
874 self._plot_type = kwargs.pop("plot_type")
875 self._splat_color = kwargs.pop("splat_color", None)
--> 876 PlotWindow.__init__(self, *args, **kwargs)
878 # import type here to avoid import cycles
879 # note that this import statement is actually crucial at runtime:
880 # the filter methods for the present class are defined only when
881 # fixed_resolution_filters is imported, so we need to guarantee
882 # that it happens no later than instantiation
884 self._callbacks: list[PlotCallback] = []
File /var/jenkins_home/workspace/yt_docs/yt/visualization/plot_window.py:264, in PlotWindow.__init__(self, data_source, bounds, buff_size, antialias, periodic, origin, oblique, window_size, fields, fontsize, aspect, setup, geometry)
261 self._projection = get_mpl_transform(projection)
262 self._transform = get_mpl_transform(transform)
--> 264 self._setup_plots()
266 for field in self.data_source._determine_fields(self.fields):
267 finfo = self.data_source.ds._get_field_info(field)
File /var/jenkins_home/workspace/yt_docs/yt/visualization/plot_window.py:1077, in PWViewerMPL._setup_plots(self)
1073 extenty.convert_to_units(unit_y)
1075 extent = [*extentx, *extenty]
-> 1077 image = self.frb.get_image(f)
1078 mask = self.frb.get_mask(f)
1079 assert mask is None or mask.dtype == bool
File /var/jenkins_home/workspace/yt_docs/yt/visualization/fixed_resolution.py:213, in FixedResolutionBuffer.get_image(self, key)
211 def get_image(self, key, /) -> ImageArray:
212 if not (key in self.data and self._data_valid):
--> 213 self._generate_image_and_mask(key)
214 return self.data[key]
File /var/jenkins_home/workspace/yt_docs/yt/visualization/fixed_resolution.py:179, in FixedResolutionBuffer._generate_image_and_mask(self, item)
176 b = float(b.in_units("code_length"))
177 bounds.append(b)
--> 179 buff, mask = self.ds.coordinates.pixelize(
180 self.data_source.axis,
181 self.data_source,
182 item,
183 bounds,
184 self.buff_size,
185 int(self.antialias),
186 return_mask=True,
187 )
189 buff = self._apply_filters(buff)
191 # FIXME FIXME FIXME we shouldn't need to do this for projections
192 # but that will require fixing data object access for particle
193 # projections
File /var/jenkins_home/workspace/yt_docs/yt/geometry/coordinates/cartesian_coordinates.py:238, in CartesianCoordinateHandler.pixelize(self, dimension, data_source, field, bounds, size, antialias, periodic, return_mask)
235 mask = np.squeeze(np.transpose(mask, (yax, xax, ax)))
237 elif self.axis_id.get(dimension, dimension) is not None:
--> 238 buff, mask = self._ortho_pixelize(
239 data_source, field, bounds, size, antialias, dimension, periodic
240 )
241 else:
242 buff, mask = self._oblique_pixelize(
243 data_source, field, bounds, size, antialias
244 )
File /var/jenkins_home/workspace/yt_docs/yt/geometry/coordinates/cartesian_coordinates.py:551, in CartesianCoordinateHandler._ortho_pixelize(self, data_source, field, bounds, size, antialias, dim, periodic)
541 for chunk in data_source.chunks([], "io"):
542 hsmlname = "smoothing_length"
543 pixelize_sph_kernel_slice(
544 buff,
545 mask_uint8,
546 chunk[ptype, px_name].to("code_length").v,
547 chunk[ptype, py_name].to("code_length").v,
548 chunk[ptype, pz_name].to("code_length").v,
549 chunk[ptype, hsmlname].to("code_length").v,
550 chunk[ptype, "mass"].to("code_mass").v,
--> 551 chunk[ptype, "density"].to("code_density").v,
552 chunk[field].in_units(ounits).v,
553 bnds,
554 data_source.coord.to("code_length").v,
555 _check_period=_periodic.astype("int"),
556 period=period3,
557 kernel_name=kernel_name,
558 )
559 if normalize:
560 pixelize_sph_kernel_slice(
561 buff_den,
562 mask_uint8,
(...) 574 kernel_name=kernel_name,
575 )
File /var/jenkins_home/workspace/yt_docs/.venv/lib/python3.11/site-packages/unyt/array.py:979, in unyt_array.to(self, units, equivalence, **kwargs)
937 def to(self, units, equivalence=None, **kwargs):
938 """
939 Creates a copy of this array with the data converted to the
940 supplied units, and returns it.
(...) 977 898755178736817.6 J
978 """
--> 979 return self.in_units(units, equivalence=equivalence, **kwargs)
File /var/jenkins_home/workspace/yt_docs/.venv/lib/python3.11/site-packages/unyt/array.py:907, in unyt_array.in_units(self, units, equivalence, **kwargs)
905 else:
906 new_units = units
--> 907 (conversion_factor, offset) = self.units.get_conversion_factor(
908 new_units, self.dtype
909 )
910 dsize = max(2, self.dtype.itemsize)
911 if self.dtype.kind in ("u", "i"):
File /var/jenkins_home/workspace/yt_docs/.venv/lib/python3.11/site-packages/unyt/unit_object.py:675, in Unit.get_conversion_factor(self, other_units, dtype)
648 def get_conversion_factor(self, other_units, dtype=None):
649 """Get the conversion factor and offset (if any) from one unit
650 to another
651
(...) 673 (1.7999999999999998, -31.999999999999886)
674 """
--> 675 return _get_conversion_factor(self, other_units, dtype)
File /var/jenkins_home/workspace/yt_docs/.venv/lib/python3.11/site-packages/unyt/unit_object.py:920, in _get_conversion_factor(old_units, new_units, dtype)
897 """
898 Get the conversion factor between two units of equivalent dimensions. This
899 is the number you multiply data by to convert from values in `old_units` to
(...) 917
918 """
919 if old_units.dimensions != new_units.dimensions:
--> 920 raise UnitConversionError(
921 old_units, old_units.dimensions, new_units, new_units.dimensions
922 )
923 old_basevalue = old_units.base_value
924 old_baseoffset = old_units.base_offset
UnitConversionError: Cannot convert between 'dimensionless' (dim '1') and 'code_density' (dim '(mass)/(length)**3').