Introduction
POSTGRESQL SUMMARY STATS AND WINDOW FUNCTIONS
Michel Semaan
Data Scientist
Motivation
USA total and running total of Summer Discus throw reigning champion status
Olympics gold medals since 2004
| Year | Champion | Last_Champion | Reigning_Champion |
|------ |----------|--------------- |------------------- |
| Year | Medals | Medals_RT |
| 1996 | GER | null | false |
|------|--------|----------- |
| 2000 | LTU | GER | false |
| 2004 | 116 | 116 |
| 2004 | LTU | LTU | true |
| 2008 | 125 | 241 |
| 2008 | EST | LTU | false |
| 2012 | 147 | 388 |
| 2012 | GER | EST | false |
POSTGRESQL SUMMARY STATS AND WINDOW FUNCTIONS
Course outline
1. Introduction to window functions
2. Fetching, ranking, and paging
3. Aggregate window functions and frames
4. Beyond window functions
POSTGRESQL SUMMARY STATS AND WINDOW FUNCTIONS
Summer olympics dataset
Each row represents a medal awarded in the Summer Olympics games
Columns
Year , City
Sport , Discipline , Event
Athlete , Country , Gender
Medal
POSTGRESQL SUMMARY STATS AND WINDOW FUNCTIONS
Window functions
Perform an operation across a set of rows that are somehow related to the current row
Similar to GROUP BY aggregate functions, but all rows remain in the output
Uses
Fetching values from preceding or following rows (e.g. fetching the previous row's value)
Determining reigning champion status
Calculating growth over time
Assigning ordinal ranks (1st, 2nd, etc.) to rows based on their values' positions in a sorted list
Running totals, moving averages
POSTGRESQL SUMMARY STATS AND WINDOW FUNCTIONS
Row numbers
Query Result
SELECT | Year | Event | Country |
Year, Event, Country |------ |---------------------------- |--------- |
FROM Summer_Medals | 1896 | 100M Freestyle | HUN |
WHERE | 1896 | 100M Freestyle For Sailors | GRE |
Medal = 'Gold'; | 1896 | 1200M Freestyle | HUN |
| ... | ... | ... |
POSTGRESQL SUMMARY STATS AND WINDOW FUNCTIONS
Enter ROW_NUMBER
Query Result
SELECT | Year | Event | Country | Row_N |
Year, Event, Country, |------ |---------------------------- |---------|------- |
ROW_NUMBER() OVER () AS Row_N | 1896 | 100M Freestyle | HUN | 1 |
FROM Summer_Medals | 1896 | 100M Freestyle For Sailors | GRE | 2 |
WHERE | 1896 | 1200M Freestyle | HUN | 3 |
Medal = 'Gold'; | ... | ... | ... | ... |
POSTGRESQL SUMMARY STATS AND WINDOW FUNCTIONS
Anatomy of a window function
Query
SELECT
Year, Event, Country,
ROW_NUMBER() OVER () AS Row_N
FROM Summer_Medals
WHERE
Medal = 'Gold';
FUNCTION_NAME() OVER (...)
ORDER BY
PARTITION BY
ROWS/RANGE PRECEDING/FOLLOWING/UNBOUNDED
POSTGRESQL SUMMARY STATS AND WINDOW FUNCTIONS
Let's practice!
POSTGRESQL SUMMARY STATS AND WINDOW FUNCTIONS
ORDER BY
POSTGRESQL SUMMARY STATS AND WINDOW FUNCTIONS
Michel Semaan
Data Scientist
Row numbers
Query Result*
SELECT | Year | Event | Country | Row_N |
Year, Event, Country, |------ |---------------------------- |---------|------- |
ROW_NUMBER() OVER () AS Row_N | 1896 | 100M Freestyle | HUN | 1 |
FROM Summer_Medals | 1896 | 100M Freestyle For Sailors | GRE | 2 |
WHERE | 1896 | 1200M Freestyle | HUN | 3 |
Medal = 'Gold'; | ... | ... | ... | ... |
POSTGRESQL SUMMARY STATS AND WINDOW FUNCTIONS
Enter ORDER BY
ORDER BY in OVER orders the rows related to the current row
Example: Ordering by year in descending order in ROW_NUMBER 's OVER clause will assign 1
to the most recent year's rows
POSTGRESQL SUMMARY STATS AND WINDOW FUNCTIONS
Ordering by Year in descending order
Query Result
SELECT | Year | Event | Country | Row_N |
Year, Event, Country, |------|--------------- |--------- |------- |
ROW_NUMBER() OVER (ORDER BY Year DESC) AS Row_N | 2012 | Wg 96 KG | IRI | 1 |
FROM Summer_Medals | 2012 | 4X100M Medley | USA | 2 |
WHERE | 2012 | Wg 84 KG | RUS | 3 |
Medal = 'Gold'; | ... | ... | ... | ... |
| 2008 | 50M Freestyle | BRA | 637 |
| 2008 | 96 - 120KG | CUB | 638 |
| ... | ... | ... | ... |
POSTGRESQL SUMMARY STATS AND WINDOW FUNCTIONS
Ordering by multiple columns
Query Result
SELECT | Year | Event | Country | Row_N |
Year, Event, Country, |------|--------- |--------- |------- |
ROW_NUMBER() OVER | 2012 | + 100KG | FRA | 1 |
(ORDER BY Year DESC, Event ASC) AS Row_N | 2012 |+67KG | SRB | 2 |
FROM Summer_Medals | 2012 | + 78KG | CUB | 3 |
WHERE | ... | ... | ... | ... |
Medal = 'Gold';
POSTGRESQL SUMMARY STATS AND WINDOW FUNCTIONS
Ordering in- and outside OVER
Query Result
SELECT | Year | Event | Country | Row_N |
Year, Event, Country, |------|--------- |--------- |------- |
ROW_NUMBER() OVER | 2012 | 1500M | ALG | 36 |
(ORDER BY Year DESC, Event ASC) AS Row_N | 2000 | 1500M | ALG | 1998 |
FROM Summer_Medals | 1996 | 1500M | ALG | 2662 |
WHERE | ... | ... | ... | ... |
Medal = 'Gold'
ORDER BY Country ASC, Row_N ASC; ORDER BY inside OVER takes effect before
ORDER BY outside OVER
POSTGRESQL SUMMARY STATS AND WINDOW FUNCTIONS
Reigning champion
A reigning champion is a champion who's won both the previous and current years'
competitions
The previous and current year's champions need to be in the same row (in two different
columns)
Enter LAG
LAG(column, n) OVER (...) returns column 's value at the row n rows before the current row
LAG(column, 1) OVER (...) returns the previous row's value
POSTGRESQL SUMMARY STATS AND WINDOW FUNCTIONS
Current champions
Query Result
SELECT | Year | Champion |
Year, Country AS Champion |------|----------|
FROM Summer_Medals | 1996 | GER |
WHERE | 2000 | LTU |
Year IN (1996, 2000, 2004, 2008, 2012) | 2004 | LTU |
AND Gender = 'Men' AND Medal = 'Gold' | 2008 | EST |
AND Event = 'Discus Throw'; | 2012 | GER |
POSTGRESQL SUMMARY STATS AND WINDOW FUNCTIONS
Current and last champions
Query Result
WITH Discus_Gold AS ( | Year | Champion | Last_Champion |
SELECT |------ |----------|--------------- |
Year, Country AS Champion | 1996 | GER | null |
FROM Summer_Medals | 2000 | LTU | GER |
WHERE | 2004 | LTU | LTU |
Year IN (1996, 2000, 2004, 2008, 2012) | 2008 | EST | LTU |
AND Gender = 'Men' AND Medal = 'Gold' | 2012 | GER | EST |
AND Event = 'Discus Throw')
SELECT
Year, Champion,
LAG(Champion, 1) OVER
(ORDER BY Year ASC) AS Last_Champion
FROM Discus_Gold
ORDER BY Year ASC;
POSTGRESQL SUMMARY STATS AND WINDOW FUNCTIONS
Let's practice!
POSTGRESQL SUMMARY STATS AND WINDOW FUNCTIONS
PARTITION BY
POSTGRESQL SUMMARY STATS AND WINDOW FUNCTIONS
Michel Semaan
Data Scientist
Motivation
Query Result
WITH Discus_Gold AS ( | Year | Event | Champion | Last_Champion |
SELECT |------|-------------- |---------- |--------------- |
Year, Event, Country AS Champion | 2004 | Discus Throw | LTU | null |
FROM Summer_Medals | 2008 | Discus Throw | EST | LTU |
WHERE | 2012 | Discus Throw | GER | EST |
Year IN (2004, 2008, 2012) | 2004 | Triple Jump | SWE | GER |
AND Gender = 'Men' AND Medal = 'Gold' | 2008 | Triple Jump | POR | SWE |
AND Event IN ('Discus Throw', 'Triple Jump') | 2012 | Triple Jump | USA | POR |
AND Gender = 'Men')
SELECT
When Event changes from Discus Throw
Year, Event, Champion, to Triple Jump , LAG fetched
LAG(Champion) OVER
(ORDER BY Event ASC, Year ASC) AS Last_Champion
Discus Throw 's last champion as opposed
FROM Discus_Gold to a null
ORDER BY Event ASC, Year ASC;
POSTGRESQL SUMMARY STATS AND WINDOW FUNCTIONS
Enter PARTITION BY
PARTITION BY splits the table into partitions based on a column's unique values
The results aren't rolled into one column
Operated on separately by the window function
ROW_NUMBER will reset for each partition
LAG will only fetch a row's previous value if its previous row is in the same partition
POSTGRESQL SUMMARY STATS AND WINDOW FUNCTIONS
Partitioning by one column
Query Result
WITH Discus_Gold AS (...) | Year | Event | Champion | Last_Champion |
|------|-------------- |---------- |--------------- |
SELECT | 2004 | Discus Throw | LTU | null |
Year, Event, Champion, | 2008 | Discus Throw | EST | LTU |
LAG(Champion) OVER | 2012 | Discus Throw | GER | EST |
(PARTITION BY Event | 2004 | Triple Jump | SWE | null |
ORDER BY Event ASC, Year ASC) AS Last_Champion | 2008 | Triple Jump | POR | SWE |
FROM Discus_Gold | 2012 | Triple Jump | USA | POR |
ORDER BY Event ASC, Year ASC;
POSTGRESQL SUMMARY STATS AND WINDOW FUNCTIONS
More complex partitioning
| Year | Country | Event | Row_N |
|------|--------- |---------------------- |------- |
| 2008 | CHN | + 78KG (Heavyweight) | 1 |
| 2008 | CHN |-49KG | 2 |
| ... | ... | ... | ... |
| 2008 | JPN | 48 - 55KG | 27 |
| 2008 | JPN | 48 - 55KG | 28 |
| ... | ... | ... | ... |
| 2012 | CHN | +75KG | 32 |
| 2012 | CHN |-49KG | 33 |
| ... | ... | ... | ... |
| 2012 | JPN | +75KG | 51 |
| 2012 | JPN |-49KG | 52 |
| ... | ... | ... | ... |
Row number should reset per Year and Country
POSTGRESQL SUMMARY STATS AND WINDOW FUNCTIONS
Partitioning by multiple columns
Query Result
WITH Country_Gold AS ( | Year | Country | Event | Row_N |
SELECT |------|--------- |---------------------- |------- |
DISTINCT Year, Country, Event | 2008 | CHN | + 78KG (Heavyweight) | 1 |
FROM Summer_Medals | 2008 | CHN |-49KG | 2 |
WHERE | ... | ... | ... | ... |
Year IN (2008, 2012) | 2008 | JPN | 48 - 55KG | 1 |
AND Country IN ('CHN', 'JPN') | 2008 | JPN | 48 - 55KG | 2 |
AND Gender = 'Women' AND Medal = 'Gold') | ... | ... | ... | ... |
| 2012 | CHN | +75KG | 1 |
SELECT | 2012 | CHN |-49KG | 2 |
Year, Country, Event, | ... | ... | ... | ... |
ROW_NUMBER() OVER (PARTITION BY Year, Country) | 2012 | JPN | +75KG | 1 |
FROM Country_Gold; | 2012 | JPN |-49KG | 2 |
| ... | ... | ... | ... |
POSTGRESQL SUMMARY STATS AND WINDOW FUNCTIONS
Let's practice!
POSTGRESQL SUMMARY STATS AND WINDOW FUNCTIONS
