Codestin Search App

li-roy · 2018-03-09T22:33:16Z

Implement TripletMarginLoss as a native function
add size_average arg
add reduce arg as per losses per-batch-element #264

Benchmarks run on inputs of size (5,500) x 5000 iterations.

old:
forward [1.0334448860958219, 1.0698160571046174, 1.054812144022435]
backward [1.6976923400070518, 1.6186234278138727, 1.617640192154795]
double backward [3.2021269120741636, 3.165953448973596, 3.24088541790843]

new:
forward [0.8313469190616161, 0.8255189431365579, 0.8157028129789978]
backward [1.5887232730165124, 1.6056769208516926, 1.5178304109722376]
double backward [3.154403690015897, 3.1905114939436316, 3.2450901730917394]

Sign in to view


-    def __init__(self, margin=1.0, p=2, eps=1e-6, swap=False):
-        super(TripletMarginLoss, self).__init__()
+    def __init__(self, margin=1.0, p=2, eps=1e-6, swap=False, size_average=True, reduce=True):


Sign in to view

+        constructor_args=(torch.rand(10),),
+        input_fn=lambda: torch.randn(5, 10),
+        target_fn=lambda: torch.rand(5, 10).mul(2).floor(),
+        reference_fn=lambda i, t, m: -((t * i.sigmoid().log() + (1 - t) * (-i).sigmoid().log()) * get_weight(m)).sum() /


Sign in to view

+Tensor pairwise_distance(const Tensor& x1, const Tensor& x2, double p, double eps) {
+  auto diff = abs(x1 - x2);
+  auto out = pow(diff + eps, p).sum(1);
+  return pow(out, 1 / p);


goldsborough · 2018-03-14T04:17:53Z

@pytorchbot retest this please

zou3519

Code looks good to me. I had a few minor comments and a concern that we're subtly changing the behavior of pairwise_distance

Sign in to view

+        margin (float, optional): Default: `1`.
+        p (int, optional): The norm degree for pairwise distance. Default: `2`.
+        swap (float, optional): The distance swap is described in detail in the paper
+            `Learning shallow convolutional feature descriptors with triplet losses` by


Sign in to view


-    def __init__(self, margin=1.0, p=2, eps=1e-6, swap=False):
-        super(TripletMarginLoss, self).__init__()
+    def __init__(self, margin=1.0, p=2, eps=1e-6, swap=False, size_average=True, reduce=True):


Sign in to view

 - func: ones_like(Tensor self, *, Type dtype) -> Tensor
  variants: function

+- func: pairwise_distance(Tensor x1, Tensor x2, double p, double eps) -> Tensor


Sign in to view

 - func: transpose_(Tensor self, int64_t dim0, int64_t dim1) -> Tensor
  variants: method

+- func: triplet_margin_loss(Tensor anchor, Tensor positive, Tensor negative, double margin, double p, double eps, bool swap, bool size_average, bool reduce) -> Tensor


Sign in to view

+
+Tensor pairwise_distance(const Tensor& x1, const Tensor& x2, double p, double eps) {
+  auto diff = abs(x1 - x2 + eps);
+  return norm(diff, p, 1);


Sign in to view

+namespace at { namespace native {
+
+Tensor pairwise_distance(const Tensor& x1, const Tensor& x2, double p, double eps) {
+  auto diff = abs(x1 - x2 + eps);


Sign in to view

-    assert x1.size() == x2.size(), "Input sizes must be equal."
-    assert x1.dim() == 2, "Input must be a 2D matrix."
-    diff = torch.abs(x1 - x2)
-    out = torch.pow(diff + eps, p).sum(dim=1, keepdim=True)


Sign in to view

@@ -1960,11 +1960,7 @@ def pairwise_distance(x1, x2, p=2, eps=1e-6):
        >>> output = F.pairwise_distance(input1, input2, p=2)


zou3519

LGTM. Minor nit below

Sign in to view

@@ -5,7 +5,6 @@
 namespace at { namespace native {

 Tensor pairwise_distance(const Tensor& x1, const Tensor& x2, double p, double eps) {


zou3519

Yay! Approve approve approve

* implement TripletMarginLoss as a native function * implement TripletMarginLoss as native function * fix compile error * address comments * address comments * Add keepdim arg to pairwise distance

Roy Li added 3 commits March 9, 2018 14:27

implement TripletMarginLoss as a native function

fc9a4f8

implement TripletMarginLoss as native function

2913cec

fix compile error

cb69c3a

zou3519 reviewed Mar 12, 2018

View reviewed changes

Comment thread aten/src/ATen/native/Distance.cpp Outdated

Tensor pairwise_distance(const Tensor& x1, const Tensor& x2, double p, double eps) {

auto diff = abs(x1 - x2);

auto out = pow(diff + eps, p).sum(1);

return pow(out, 1 / p);

This comment was marked as off-topic.

Sign in to view

address comments

4d23df1

zou3519 reviewed Mar 14, 2018

View reviewed changes

address comments

7354bb7

zou3519 approved these changes Mar 15, 2018

View reviewed changes

Add keepdim arg to pairwise distance

1f0560d

zou3519 approved these changes Mar 16, 2018

View reviewed changes

soumith merged commit e876b5d into pytorch:master Mar 17, 2018

li-roy deleted the triplet branch March 20, 2018 14:13

		@@ -1960,11 +1960,7 @@ def pairwise_distance(x1, x2, p=2, eps=1e-6):
		>>> output = F.pairwise_distance(input1, input2, p=2)

		@@ -5,7 +5,6 @@
		namespace at { namespace native {

		Tensor pairwise_distance(const Tensor& x1, const Tensor& x2, double p, double eps) {

Conversation

li-roy commented Mar 9, 2018

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

goldsborough commented Mar 14, 2018

Uh oh!

zou3519 left a comment

Choose a reason for hiding this comment

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

zou3519 left a comment

Choose a reason for hiding this comment

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

zou3519 left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants