Unsupervised ML SOM
and how to choose a
Data Science method
Quick Review of the methods we learned
> Statistical analysis
> Supervised ML
– Linear regression
– NN,
– KNN,
– Decision Tree
– SVM
> Unsupervised ML
– K-Means Clustering
– PCA
– ….. Why another one,
Why another method, SOM
> Demonstrate some data science methods that are not
widely used or well known, but also can be very useful
for material informatic study
> Introduce a method I have used, and feel is adequate
to the uniqueness of many materials study
applications.
> Demonstrate how various data science methods can
be used together to drive improved results
> Demonstrate a few projects using the same methods
so that we can understand a methods from user point
of view
What is Self-Organizing Map (SOM)
> An Unsupervised ML method
> Dimensional reduction, enabling powerful
visualizations of the data:
– K-Means does clustering, but neither dimensionality
reduction nor visualization
– PCA does dimensionality reduction, enabling visualization to
certain level (not applicable if the first 3 principal
components won’t represent the data well), however, it
does not perform clustering. Besides, the visualization does
not keep the original topographic information.
> Give some insights into how data is clustered in high
dimensions
What is SOM
> You can think of SOM as an artificial neural network
with a single neuronal layer, whose neurons are
arranged in a two-dimensional matrix.
– The 2D matrix can been seen as a position map that
captures the characteristics of the data
> Merits of SOM
– Effective in training big datasets
– Since this is a 2D matrix, visualization of the resulting map
is possible
– kept the topography of the original data,
– Possible to present the Euclidean distance between data
points
Algorithm of SOM
– Normalization of the input data, all features will be distributed more
balancely
– Initialization: each (x,y) position in the map is assigned a weight for each
input neuron, thus associating a weight vector for each map position.
– Iteration:
> Choose a sample from dataset
> Calculate Euclidean distance between that sample and each weight vector
> The (x,y) position ”closest” to the sample is declared the Best Matching Unit
> The weights vector for the BMU get adjusted to more closely match the sample.
Amount of adjustment (learning) decreases as we go through iterations
> The weights vector for neighbors of the BMU also get adjusted, to a lesser extent.
The number of neighbors and how much they get adjusted also depends on
hyperparameters and the number of iterations.
– Convergence:
> Max number of iterations
> Monitoring of topological error
– Reference: https://link.springer.com/article/10.1007/BF00337288
Self-Organizing Map (SOM)
How does it work?
𝑎!
𝑏!
𝑐!
𝑥! = 𝑑
!
𝑒!
𝑓!
Two Dimensional Mesh structure
Each connection can deform
a
1 11 12
b 6 10
f
2
4 8
3
9
5 7
c
e
d
a
1 11 12
b 6 10
f
2
4 8
3
9
5 7
c
e
d
a
1 11 12
b 6 10
f
2
4 8
3
9
5 7
c
e
d
a
1 11 12
b 6 10
f
2
4 8
3
9
5 7
c
e
d
a
1 11 12
b 6 10
f
2
4 8
3
9
5 7
c
e
d
a
1 11 12
b 6 10
f
2
4 8
3
9
5 7
c
e
d
a
1 11 12
b 6 10
f
2
4 8
3
9
5 7
c
e
d
a
1 11 12
b 6 10
f
2
4 8
3
9
5 7
c
e
d
a
1 11 12
b 6 10
f
2
4 8
3
9
5 7
c
e
d
a
1 11 12
b 6 10
f
2
4 8
3
9
5 7
c
e
d
a
1 11 12
b 6 10
f
2
4 8
3
9
5 7
c
e
d
a
1 11 12
b 6 10
f
2
4 8
3
9
5 7
c
e
d
a
1 11 12
b 6 10
f
2
4 8
3
9
5 7
c
e
d
Self-Organizing Map (SOM) Algorithm
> Dragging Nodes
> “Flattening a crumpled paper”
U-matrix and how to use it to get insights for
clustering
> After training, the nodes in
the 2D key map are not
evenly distributed. The
adjacent data point might
not be similar to each
other in the higher
dimension space.
> U-matrix use the concept
of the heatmap to
illustrate the distance in
Euclidean space
Using SOM in conjunction with other methods
> Since this is a dimensionality
reduction method, for smaller
dataset, you can initialize your
SOM map using the first 2
Principal components,
essentially the 2D PCA map
> K-means can also be run on the
same dataset, and
corresponding clusters can be
visualized on SOM map.
K-Means clustering and U-Matrix
They can be compared to validate the results!
> SOM can provide a means to visualize K-Means!
> If the boundary matches well, then the training is
successful
Different Implementations of SOM
> SOM is just an algorithm, there are many
packages you can use that implement it
> We will introduce
– An augmented version of SOMPY, a version our group has
contributions on
– MiniSOM
The uniqueness and functions of augmented
SOMPY
https://github.com/DataScienceUWMSE/SOM
> Utilizes PCA for initialization, and include K-Means
Clustering overlay
> “Heat maps” provide a way to visualize each
feature after training
> Projection function helps users find additional
correlations or patterns among features,
including for categorical data
“heatmap” concept
> Map each node’s
weight onto the 2D
map
> Number of heat maps
equals to number of
input variables
Example of utilizing the
heatmap on materials research
Example 1 Granta Data Set: Experimental Commercial Materials
Property Dataset
> Training data set
contains 398 commercial
materials and 21
numerical properties
Example of utilizing the heatmap on materials
research
Example 1 Granta Data Set: Experimental Commercial Materials
Property Dataset (continue)
Project information concept
> Overlay one specific data
property onto SOM, can
use even categorical
values
> Easily identify patterns
Example of utilizing the project function on
materials research
Example 1 Granta Data Set: Experimental Commercial Materials
Property Dataset (continue), finding the outliers’ uniqueness
Example of utilizing the projection function on
materials research
Example 2 OPV materials study using an experimental dataset
Reference Y.Huang, J. Phys. Chem. C 2020, 124, 12871−12882
> Dataset includes 1203 donor
polymers of Donor-Acceptor
pairs, with properties
related to the proficiency of
the charge transfer.
Molecular Descriptors
Python package of Molecular Descriptor
> There are Python tools to extract molecular
structural or geometrical information from
notation of molecule, such as SMILES (Simplified
molecular-input line-entry system)
> We will introduce Mordred, (covered in the Hands-
on session)
The advantage of using MiniSOM
> SOMPY is not as easy to use as the other packages
introduced in this class.
– The Augmented SOMPY has contribution from a few
Materials Science researchers in our group, including
your TA Jimin, Qian
> MiniSOM is relatively easier to use, well
documented and constantly maintained, and
have the basic implementation of the SOM
algorithm
What MiniSOM provides
> It has :
– The core implementation of SOM
– Visualization
– U-Matrix (“distance map” in MiniSOM)
– Project certain feature onto SOM
> Doesn’t have:
– PCA initialization
– Cannot generate heatmap for each features
– K-Means clustering,
Hyperparameters of SOM
> Length of input vectors (the number of properties)
> Map size, the most important one
> Map topology – rectangular or hexagonal
– Important in defining the notion of “neighbors”
> Sigma – spread of the neighborhood function
> Learning Rate – initial learning rate, decreases with the
number of iterations
> Decay function – defines how much learning rate and sigma
decrease with the number of iterations
> Neighborhood function – defines how much neighbors of
the BMU get impacted at each iteration (eg gaussian,
bubble,…)
> Activation distance function (eg Euclidean distance)
> Initialization method – random or PCA
Hands-on session and HW for this week
