A recursive subclass of collections.ChainMap.
python -m pip install deep-chainmapThe canonical use case for collections.ChainMap is to aggregate configuration
data from layered mapping (basically dictionaries) sources. However, it is not
suited for non-flat (nested) mappings, since the lookup mechanism only works for
the top level of a mapping.
deep_chainmap.DeepChainMap provides a simple solution to this problem by making
recurive lookups in arbitrarily deeply nested mappings. Let's illustrate this
with a simple example. We will simulate 3 layers of mapping, and pretend they
were obtained from different sources (a default configuration, a configuration
file and parameters configured at runtime).
from deep_chainmap import DeepChainMap
default_layer = {
"architecture": "gpu",
"logging_level": "warning",
"solver": "RK4",
"database": {
"url": "unset",
"keep_in_sync": False,
},
"mesh": {
"type": "rectangular",
"resolution": {
"x": {
"npoints": 100,
"spacing": "linear",
},
"y": {
"npoints": 100,
"spacing": "linear",
},
"z": {
"npoints": 100,
"spacing": "linear",
},
},
},
}
config_file_layer = {
"architecture": "cpu",
"mesh": {
"resolution": {
"x": {
"spacing": "log",
},
"z": {
"npoints": 1,
},
},
},
}
runtime_layer = {
"logging_level": "debug",
"database": {
"url": "https://my.database.api",
"keep_in_sync": True
},
}
# now building a DeepChainMap
cm = DeepChainMap(runtime_layer, config_file_layer, default_layer)Now when a single parameter is requested, it is looked up in each layer until a
value is found, by order of insertion. Here the runtime_layer takes priority
over the config_file_layer, which in turns takes priority over the
default_layer.
>>> cm["logging_level"]
'debug'
>>> cm["mesh"]["resolution"]["x"]["spacing"]
'log'
>>> cm["mesh"]["resolution"]["x"]["npoints"]
100Note that submappings at any level can be retrieved as new
DeepChainMap instances
>>> cm["mesh"]
DeepChainMap({'resolution': {'x': {'spacing': 'log'}, 'z': {'npoints': 1}}},
{'resolution': {'x': {'npoints': 100, 'spacing': 'linear'},
'y': {'npoints': 100, 'spacing': 'linear'},
'z': {'npoints': 100, 'spacing': 'linear'}},
'type': 'rectangular'})The other important feature is the to_dict method, which constructs a builtin
dict from a DeepChainMap
>>> cm.to_dict()
{
'architecture': 'cpu',
'logging_level': 'debug',
'solver': 'RK4',
'database': {
'url': 'https://my.database.api',
'keep_in_sync': True
},
'mesh': {
'type': 'rectangular',
'resolution': {
'x': {'npoints': 100, 'spacing': 'log'},
'y': {'npoints': 100, 'spacing': 'linear'},
'z': {'npoints': 1, 'spacing': 'linear'}
}
}
}An important implication is that the DeepChainMap class enables a very simple,
functional implementation of a depth-first dict-merge algorithm as
from deep_chainmap import DeepChainMap
def depth_first_merge(*mappings) -> dict:
return DeepChainMap(*mappings).to_dict()As the standard collections.ChainMap class, DeepChainMap does not, by
design, perform any kind of data validation. Rather, it is assumed that the
input mappings are similar in structure, meaning that a key which maps to a dict
in one of the input mappings is assumed to map to dict instances as well in
every other input mapping. Use the excellent
schema library or similar projects for this
task.
collections.ChainMap is that, when
setting a (key, value) pair in a DeepChainMap instance, the new value is
stored in the first mapping which already contains the parent map. For example
if we run
>>> cm["mesh"]["resolution"]["x"]["spacing"] = "exp"The affected layer is config_file_layer rather than runtime_layer, as one
can see
>>> config_file_layer
{
'architecture': 'cpu',
'mesh': {
'resolution': {
'x': {'spacing': 'exp'},
'z': {'npoints': 1}
}
}
}
>>> runtime_layer
{
'logging_level': 'debug',
'database': {
'url': 'https://my.database.api',
'keep_in_sync': True
}
}This behaviour is a side effect on an implementation detail and subject to change in a future version. Please do not rely on it.