Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[MRG+2] faster way of computing means across each group #10020

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 16 commits into from
Nov 7, 2017
Merged

[MRG+2] faster way of computing means across each group #10020

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
9807ef0
faster way of computing means across each group
sergulaydore Oct 26, 2017
dfcd422
fixed bug
sergulaydore Oct 26, 2017
b36422c
fixed the bug with paranthesis
sergulaydore Oct 26, 2017
534f757
fixed PEP8 issues
sergulaydore Oct 26, 2017
f37287a
Benchmarking np.bincount vs np.mean for feature agglomeration
sergulaydore Oct 26, 2017
593c8ed
fixed PEP8 issues for benchmark file
sergulaydore Oct 26, 2017
94ad19a
removed pooling_func as it was depreciated
sergulaydore Oct 27, 2017
3defcd6
test file added,bencmark removed
sergulaydore Oct 29, 2017
7b5de28
special case was added back
sergulaydore Oct 29, 2017
5818abf
after jnothman's review
sergulaydore Oct 30, 2017
ef786c6
flake8 error fixed
sergulaydore Oct 31, 2017
fba7691
sparse condition added
sergulaydore Nov 2, 2017
2f9846c
fixed 'and' and paranthesis
sergulaydore Nov 2, 2017
72caa77
more tests added, reshaped input
sergulaydore Nov 2, 2017
a35000b
added spaces before and after div operator
sergulaydore Nov 4, 2017
9684ebe
added spaces before after div op - 2
sergulaydore Nov 5, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 14 additions & 5 deletions sklearn/cluster/_feature_agglomeration.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,12 @@
from ..base import TransformerMixin
from ..utils import check_array
from ..utils.validation import check_is_fitted
from scipy.sparse import issparse

###############################################################################
# Mixin class for feature agglomeration.


class AgglomerationTransform(TransformerMixin):
"""
A class for feature agglomeration via the transform interface
Expand All @@ -40,14 +42,21 @@ def transform(self, X):

pooling_func = self.pooling_func
X = check_array(X)
nX = []
if len(self.labels_) != X.shape[1]:
raise ValueError("X has a different number of features than "
"during fitting.")

for l in np.unique(self.labels_):
nX.append(pooling_func(X[:, self.labels_ == l], axis=1))
return np.array(nX).T
if pooling_func == np.mean and not issparse(X):
size = np.bincount(self.labels_)
n_samples = X.shape[0]
# a fast way to compute the mean of grouped features
nX = np.array([np.bincount(self.labels_, X[i, :]) / size
for i in range(n_samples)])
else:
nX = []
for l in np.unique(self.labels_):
nX.append(pooling_func(X[:, self.labels_ == l], axis=1))
nX = np.array(nX).T
return nX

def inverse_transform(self, Xred):
"""
Expand Down
43 changes: 43 additions & 0 deletions sklearn/cluster/tests/test_feature_agglomeration.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
"""
Tests for sklearn.cluster._feature_agglomeration
"""
# Authors: Sergul Aydore 2017
import numpy as np
from sklearn.cluster import FeatureAgglomeration
from sklearn.utils.testing import assert_true
from sklearn.utils.testing import assert_array_almost_equal


def test_feature_agglomeration():
n_clusters = 1
X = np.array([0, 0, 1]).reshape(1, 3) # (n_samples, n_features)

agglo_mean = FeatureAgglomeration(n_clusters=n_clusters,
pooling_func=np.mean)
agglo_median = FeatureAgglomeration(n_clusters=n_clusters,
pooling_func=np.median)
agglo_mean.fit(X)
agglo_median.fit(X)
assert_true(np.size(np.unique(agglo_mean.labels_)) == n_clusters)
assert_true(np.size(np.unique(agglo_median.labels_)) == n_clusters)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should test the size of labels_ too

assert_true(np.size(agglo_mean.labels_) == X.shape[1])
assert_true(np.size(agglo_median.labels_) == X.shape[1])

# Test transform
Xt_mean = agglo_mean.transform(X)
Xt_median = agglo_median.transform(X)
assert_true(Xt_mean.shape[1] == n_clusters)
assert_true(Xt_median.shape[1] == n_clusters)
assert_true(Xt_mean == np.array([1 / 3.]))
assert_true(Xt_median == np.array([0.]))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did we stop using assert_equal with the switch towards pytest?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I used test_ward_agglomeration function in test_hierarchical.py as a guideline for this test. However, for the rest of test_hierarchical.py, assert_equal and assert_true have been used interchangeably.


# Test inverse transform
X_full_mean = agglo_mean.inverse_transform(Xt_mean)
X_full_median = agglo_median.inverse_transform(Xt_median)
assert_true(np.unique(X_full_mean[0]).size == n_clusters)
assert_true(np.unique(X_full_median[0]).size == n_clusters)

assert_array_almost_equal(agglo_mean.transform(X_full_mean),
Xt_mean)
assert_array_almost_equal(agglo_median.transform(X_full_median),
Xt_median)