Tags: keras-team/keras
Tags
Patch release 3.14.1 (#22846) * Fix `L1L2` regularizer. (#22629) This is a partial revert of #22438 The `if self.l1` and `if self.l2` statement render this code non-compilable as the value is a tensor. Converting to a tensor early also causes other issue. This is not a full revert and the `cast`s remain. * Define contiguous replica-group data shard ID convention in ModelParallel (#22179) * Define contiguous replica-group data shard ID convention in ModelParallel * Define contiguous replica-group data shard ID convention in ModelParallel * Fix review comments * Refactor multi-device test * Refactor multi-device test * Fix regression with metrics passed to `compile`. (#22663) #22308 introduced a regression. This changed assumed that `y_pred` and `y_true` have the same structure, but it turns out it's allowed to have `y_pred` be a list and `y_true` be a dict with the keys matching the Functional model output names. The regression was that `y_pred` and `y_true` were not ordered identically and could be paired incorrectly. This changes re-introduces some of the concepts removed in #22308 in particular: - `self._resolved_output_names` is back to control the flattening order of `self._flatten_y` - `flatten_y` is back Also: - added support for output names for deeply nested structures - made tests more robust by having multiple `y_pred` and `y_true` have different shapes so that we detect if they are not paired correctly - removed `test_dict_outputs_uses_output_names` test which was an incorrect scenario - added `test_list_outputs_dict_labels_uses_output_names` Fixes #22625 * Improve validation and error reporting in `Functional.from_config`. (#22800) Also allow many of the tests in `functional_test.py` to run even with non-trainable backends because they're not using `fit`. * Refactor and share H5 validation code between legacy and new format. (#22801) * Improve sub path resolution in `file_utils`. (#22839) - Replaced `is_path_in_dir` with `resolve_sub_path` which resolves a relative path and makes sure it stays within the `base_dir`. - Fixed `is_link_in_dir` to correctly resolve the paths depending on whether it is a hard link or a symlink. - Use `resolve_sub_path` in `saving_lib._write_nested_dict_to_dir` which is used by the Orbax reloading. * Update patch release version and cherry-pick commits * Fix test broken with the release of JAX 0.10.0 (#22694) `PartitionSpec` objects can no longer be directly compared with tuples. - Fixed all verifications of `PartitionSpec` - Import `jax.sharding.PartitionSpec` as `P` as is customary --------- Co-authored-by: hertschuh <[email protected]> Co-authored-by: Malyala Karthik <[email protected]>
Patch fix 3.12.2 (#22850) * Don't let unit test modify the source folder. (#22194) Changed some unit tests to use a temporary folder instead of the current one. Also fixed `file_utils.filter_safe_zipinfos` and `filter_safe_tarinfos` to resolve links within the context of the destination folder instead of hardcoding `"."` as the destination path. * Improve validation and error reporting in `Functional.from_config`. (#22800) Also allow many of the tests in `functional_test.py` to run even with non-trainable backends because they're not using `fit`. * Refactor and share H5 validation code between legacy and new format. (#22801) * Improve sub path resolution in `file_utils`. (#22839) - Replaced `is_path_in_dir` with `resolve_sub_path` which resolves a relative path and makes sure it stays within the `base_dir`. - Fixed `is_link_in_dir` to correctly resolve the paths depending on whether it is a hard link or a symlink. - Use `resolve_sub_path` in `saving_lib._write_nested_dict_to_dir` which is used by the Orbax reloading. * version update and cherry pick fixes * fix lazy module import for h5py * rm temp dir * [OpenVINO backend] Remove deprecated openvino.runtime import (#21826) * Remove openvino.runtime import * Fix inconsistent import style * remove unused test --------- Co-authored-by: hertschuh <[email protected]> Co-authored-by: Alicja Miloszewska <[email protected]>
Version bump and cherry picks for 3.13.2 (#22080) * Fix DoS via malicious HDF5 dataset metadata in KerasFileEditor (#21880) * Fix DoS via malicious HDF5 dataset metadata in KerasFileEditor * Refactor: move MAX_BYTES constant outside loop per review feedback * Fix: harden HDF5 dataset metadata validation in KerasFileEditor * Do not allow external links in HDF5 files. (#22057) Keras never uses this feature. - verify that we get H5 Groups when expected, otherwise, merely by doing `[key]` we may be loading an external Dataset. - verify that the H5 Datasets are not external links and fail if they are. - remove unused methods `items` and `values` in `H5IOStore` and `ShardedH5IOStore`. They are not used, the implementation of `MutableMapping` was incomplete anyway and these methods we return unverified Datasets. - fixed logic related to `failed_saveables` in `load_state`. - preserve the order of keys in the implementation of `ShardedH5IOStore.keys()`. * Set mutable to True by default in nnx_metadata (#22074) * Disallow TFSMLayer deserialization in safe_mode to prevent external SavedModel execution (#22035) * Implement safe mode checks in TFSMLayer Added safe mode checks for loading TFSMLayer from external SavedModels. * Update keras/src/export/tfsm_layer.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Align logic with __init__ method for robust checks Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Fix indentation and formatting in tfsm_layer.py * Add setup method to enable unsafe deserialization Enable unsafe deserialization for TFSM Layer tests. * Update TFSMLayer initialization in tests * Fix import for TFSMLayer in tfsm_layer_test.py * Remove safe_mode check from TFSMLayer.__init__() The safe_mode check should only be in from_config(), not __init__(). Direct instantiation (TFSMLayer(filepath=...)) is a legitimate use case where the user explicitly creates the layer. The security concern is only during deserialization of untrusted .keras files, which goes through from_config(). This allows attackers to create malicious .keras files while still blocking victims from loading them with safe_mode=True. * Implement tests for TFSMLayer safe mode functionality Add comprehensive tests for TFSMLayer safe_mode behavior: - test_safe_mode_direct_instantiation_allowed: Verifies direct TFSMLayer instantiation works as expected - test_safe_mode_from_config_blocked: Verifies from_config() raises ValueError when safe_mode=True - test_safe_mode_from_config_allowed_when_disabled: Verifies from_config() works with safe_mode=False - test_safe_mode_model_loading_blocked: Tests the full attack scenario where loading a .keras file with safe_mode=True is blocked * Clarify test docstrings in tfsm_layer_test.py Updated test docstrings for clarity on instantiation and loading behavior. * Invoke model with random input in tfsm_layer tests Added model invocation with random input to tests for TFSMLayer. * Set safe_mode default to True in from_config method * Update tfsm_layer_test.py * Update tfsm_layer_test.py * Update tfsm_layer_test.py to original * New test case tfsm_layer_test.py * Update Comments tfsm_layer.py * Update tfsm_layer_test.py * Update tfsm_layer.py * Update tfsm_layer.py to remove ruff errors * Update tfsm_layer.py * Update tfsm_layer_test.py * Update tfsm_layer.py * Update tfsm_layer.py * Update tfsm_layer_test.py * Update tfsm_layer.py format fix Changes in format * Update tfsm_layer.py * Update tfsm_layer_test.py * Update tfsm_layer.py * Update tfsm_layer_test.py * Fixes unnecessary changes tfsm_layer.py * Added new test case tfsm_layer_test.py * Set `safe_mode=None` in `from_config`, which fixes the unit tests. Also re-added empty lines. * Remove unneeded `custom_objects` in unit tests. --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Fabien Hertschuh <[email protected]> * patch release 3.12.2 changes --------- Co-authored-by: sarvesh patil <[email protected]> Co-authored-by: hertschuh <[email protected]> Co-authored-by: Divyashree Sreepathihalli <[email protected]> Co-authored-by: Manan Patel <[email protected]> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
3.12.1 cherry pick changes for patch release. (#22081) * Fix DoS via malicious HDF5 dataset metadata in KerasFileEditor (#21880) * Fix DoS via malicious HDF5 dataset metadata in KerasFileEditor * Refactor: move MAX_BYTES constant outside loop per review feedback * Fix: harden HDF5 dataset metadata validation in KerasFileEditor * Do not allow external links in HDF5 files. (#22057) Keras never uses this feature. - verify that we get H5 Groups when expected, otherwise, merely by doing `[key]` we may be loading an external Dataset. - verify that the H5 Datasets are not external links and fail if they are. - remove unused methods `items` and `values` in `H5IOStore` and `ShardedH5IOStore`. They are not used, the implementation of `MutableMapping` was incomplete anyway and these methods we return unverified Datasets. - fixed logic related to `failed_saveables` in `load_state`. - preserve the order of keys in the implementation of `ShardedH5IOStore.keys()`. * Disallow TFSMLayer deserialization in safe_mode to prevent external SavedModel execution (#22035) * Implement safe mode checks in TFSMLayer Added safe mode checks for loading TFSMLayer from external SavedModels. * Update keras/src/export/tfsm_layer.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Align logic with __init__ method for robust checks Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Fix indentation and formatting in tfsm_layer.py * Add setup method to enable unsafe deserialization Enable unsafe deserialization for TFSM Layer tests. * Update TFSMLayer initialization in tests * Fix import for TFSMLayer in tfsm_layer_test.py * Remove safe_mode check from TFSMLayer.__init__() The safe_mode check should only be in from_config(), not __init__(). Direct instantiation (TFSMLayer(filepath=...)) is a legitimate use case where the user explicitly creates the layer. The security concern is only during deserialization of untrusted .keras files, which goes through from_config(). This allows attackers to create malicious .keras files while still blocking victims from loading them with safe_mode=True. * Implement tests for TFSMLayer safe mode functionality Add comprehensive tests for TFSMLayer safe_mode behavior: - test_safe_mode_direct_instantiation_allowed: Verifies direct TFSMLayer instantiation works as expected - test_safe_mode_from_config_blocked: Verifies from_config() raises ValueError when safe_mode=True - test_safe_mode_from_config_allowed_when_disabled: Verifies from_config() works with safe_mode=False - test_safe_mode_model_loading_blocked: Tests the full attack scenario where loading a .keras file with safe_mode=True is blocked * Clarify test docstrings in tfsm_layer_test.py Updated test docstrings for clarity on instantiation and loading behavior. * Invoke model with random input in tfsm_layer tests Added model invocation with random input to tests for TFSMLayer. * Set safe_mode default to True in from_config method * Update tfsm_layer_test.py * Update tfsm_layer_test.py * Update tfsm_layer_test.py to original * New test case tfsm_layer_test.py * Update Comments tfsm_layer.py * Update tfsm_layer_test.py * Update tfsm_layer.py * Update tfsm_layer.py to remove ruff errors * Update tfsm_layer.py * Update tfsm_layer_test.py * Update tfsm_layer.py * Update tfsm_layer.py * Update tfsm_layer_test.py * Update tfsm_layer.py format fix Changes in format * Update tfsm_layer.py * Update tfsm_layer_test.py * Update tfsm_layer.py * Update tfsm_layer_test.py * Fixes unnecessary changes tfsm_layer.py * Added new test case tfsm_layer_test.py * Set `safe_mode=None` in `from_config`, which fixes the unit tests. Also re-added empty lines. * Remove unneeded `custom_objects` in unit tests. --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Fabien Hertschuh <[email protected]> * commit 3.12.1 cherry-pick changes --------- Co-authored-by: sarvesh patil <[email protected]> Co-authored-by: hertschuh <[email protected]> Co-authored-by: Manan Patel <[email protected]> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Patch release commits for 3.13.1 (#22005) * Remove NumPy warning with NumPy >= 2. (#21949) Merely importing keras currently triggers this warning with NumPy 2. ``` keras/src/export/tf2onnx_lib.py:8: FutureWarning: In the future `np.object` will be defined as the corresponding NumPy scalar. ``` Only patch NumPy if and when needed. * Fix CUDNN flash attention for JAX > 0.6.2. (#21970) The signature of `check_is_flash_attention` changed with JAX 0.7.0. In addition to `query` and `key`, a positional argument of `value` was added. This was not caught as our kokoro tests use JAX 0.6.2 because it's the last version that supports Python 3.10. This change was tested here: #21957 * Do no always make batch size dynamic during export. (#21944) This is a follow-up of #21674 This PR changed the signature of `make_tf_tensor_spec` from `(x)` to `(x, dynamic_batch=True)`, thereby adding the ability to make the batch size dynamic. This PR also adds `_get_save_spec(self, dynamic_batch=True)` which uses `make_tf_tensor_spec` and forwards the `dynamic_batch` argument. However, the default before this change for other export (SavedModel, ONNX) was to keep the batch size untouched. In particular, when a user manually provides an `input_signature` to [`ExportArchive.add_endpoint`](https://github.com/keras-team/keras/blob/master/keras/src/export/saved_model.py#L362), we should honor. The user controls whether the batch size is dynamic or not in the `input_signature`. This PR changes the default of `make_tf_tensor_spec` back to `dynamic_batch=False` to revert SavedModel and ONNX exports to the previous behavior. Also removed call to `return super()._get_save_spec(dynamic_batch)` which can never succeed as `TFLayer` is a top level class (ignoring the auto-tracking stuff). * Cherry pick & patch release --------- Co-authored-by: hertschuh <[email protected]>
Version bump to 3.11.3 (#21607) * Disable `torch.load` in `TorchModuleWrapper` when in safe mode. (#21575) Raise an exception and explain the user about the risks. * Propagate `safe_mode` flag to legacy h5 loading code. (#21602) Also: - made various error messages related to `safe_mode` more consistent - removed no-op renaming code in legacy saving - uncommented unit tests in `serialization_lib_test.py` * Fix GRU with return_state=True on tf backend with cuda (#21603) * Version bump to 3.11.3 --------- Co-authored-by: hertschuh <[email protected]> Co-authored-by: Matt Watson <[email protected]>
Version bump 3.11.2 and nnx fix #21565 (#21570) * Fix nnx object state (#21565) * Update operation.py * Update actions.yml * Update operation.py * Update actions.yml * Update operation.py * Update operation.py * Update operation.py * fix test * code reformat * Version bump to 3.11.2 --------- Co-authored-by: Divyashree Sreepathihalli <[email protected]>
PreviousNext