Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Clarification: new mean option is synonym with sum_over_batch_size in loss function base class #20352

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Oct 15, 2024
Merged

Clarification: new mean option is synonym with sum_over_batch_size in loss function base class #20352

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
4074af1
mean as a synonym of sum_over_batch_size
jm-willy Oct 14, 2024
1be6c56
mean synonym with sum_over_batch_size
jm-willy Oct 14, 2024
4d03167
typo fix
jm-willy Oct 14, 2024
19933d9
Update loss.py
jm-willy Oct 14, 2024
f81287f
reduction in ("mean", "sum_over_batch_size") loss.py
jm-willy Oct 14, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions keras/src/losses/loss.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,8 @@ class Loss(KerasSaveable):
Args:
reduction: Type of reduction to apply to the loss. In almost all cases
this should be `"sum_over_batch_size"`.
Supported options are `"sum"`, `"sum_over_batch_size"` or `None`.
Supported options are `"sum"`, `"sum_over_batch_size"`, `"mean"`
or `None`.
name: Optional name for the loss instance.
dtype: The dtype of the loss's computations. Defaults to `None`, which
means using `keras.backend.floatx()`. `keras.backend.floatx()` is a
Expand Down Expand Up @@ -92,7 +93,7 @@ def _obj_type(self):


def standardize_reduction(reduction):
allowed = {"sum_over_batch_size", "sum", None, "none"}
allowed = {"sum_over_batch_size", "sum", None, "none", "mean"}
if reduction not in allowed:
raise ValueError(
"Invalid value for argument `reduction`. "
Expand Down Expand Up @@ -132,7 +133,7 @@ def reduce_values(values, reduction="sum_over_batch_size"):
):
return values
loss = ops.sum(values)
if reduction == "sum_over_batch_size":
if reduction in ("mean", "sum_over_batch_size"):
loss /= ops.cast(
ops.prod(ops.convert_to_tensor(ops.shape(values), dtype="int32")),
loss.dtype,
Expand Down Expand Up @@ -177,7 +178,7 @@ def apply_mask(sample_weight, mask, dtype, reduction):
"""Applies any mask on predictions to sample weights."""
if mask is not None:
mask = ops.cast(mask, dtype=dtype)
if reduction == "sum_over_batch_size":
if reduction in ("mean", "sum_over_batch_size"):
# Valid entries have weight `total/valid`, while invalid ones
# have 0. When summed over batch, they will be reduced to:
#
Expand Down
2 changes: 1 addition & 1 deletion keras/src/losses/loss_test.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ def test_reduction(self):
self.assertEqual(backend.standardize_dtype(loss.dtype), "float32")
self.assertAllClose(np.sum((y_true - y_pred) ** 2), loss)

# sum_over_batch_size
# sum_over_batch_size or mean
loss_fn = ExampleLoss(reduction="sum_over_batch_size")
loss = loss_fn(y_true, y_pred)
self.assertEqual(backend.standardize_dtype(loss.dtype), "float32")
Expand Down