-
Notifications
You must be signed in to change notification settings - Fork 22
Description
Okay, so this might not exactly be a "good first issue" - it is a little more advanced, but is still very much accessible to newcomers.
Similar to the mygrad.nnet.max_pool function, I would like there to be a mean-pooling layer. That is, a convolution-style windows is strided over the input, and the mean is computed for each window. E.g. the following is shows how mean-pooling should work on a shape-(3, 3) tensor, using a shape-(2, 2) pooling window strided with a step-size of 1 (both along the rows and the columns.
>>> import mygrad as mg
>>> x = mg.Tensor([[0., 1., 2.],
... [3., 4., 5.],
... [6., 7., 8.]])
# Forward Pass
>>> out = mean_pool(x, pool=(2, 2), stride=1)
>>> out
Tensor([[2., 3.],
[5., 6.]])
# Backprop
>>> out.sum().backward() # must backprop from a scalar, thus we sum `out`
>>> x.grad
array([[0.25, 0.5 , 0.25],
[0.5 , 1. , 0.5 ],
[0.25, 0.5 , 0.25]])Like max_pool, this function should accommodate N-dimensional tensors. mygrad.sliding_window_view makes short work of this. This function basically boils down to taking the appropriate sliding-window view of the underlying numpy array of the input tensor, and using numpy.mean to take the average over the trailing N dimensions that you want to pool over. This is much easier than doing max-pooling, since numpy.mean is able to accept multiple axes .
Try starting with the forward pass for the 1D and 2D cases only. I can help you generalize to N-dimensions if you get stuck. I am also happy to help derive the proper back-propagation for this.