TDT4259 – Applied Data Science
Lecture 5: Lifecycle of a data science project
Nisha Dalal
Adj. Associate Professor
[email protected]
2
But first,
• Check your groups
• Contact your team members
• Decide datasets
• Discuss contributions
• Email from TAs
• Language preferences
• Schedule group meeting with TAs (preferably after)
• Some changes in the scoring scheme for group assignment
• Questions on Slack
3
Reference groups and feedback
We are looking for 5-8 students to comprise the reference group. The purpose of the reference group is to provide
constructive feedback about the course through an ongoing open dialogue with other students throughout the semester.
You can read more about task of the reference group in this link.
If you want to sign up to be a member of the reference group, use this link.
A survey will be sent out to all to evaluate the course during the last week.
CRISP-DM: with a use case
5
Aneo: Grid loss data
• Grid load
• Total amount of electric energy in the grid
• Grid loss
• Difference in electricity between what has been
produced by the power plants (or load) and what has
been sold to the customers
• Grid load = consumption by customers + grid loss
6
What is CRISP-DM
Cross-industry standard process for data
mining - CRISP-DM
An open standard developed in 1996 by leading
companies in data analysis
It is still the most popular methodology for data-centric
projects
It is an agile method that introduces almost no
overhead and emphasizes adaptive transitions between
project phases
Source
7
What is CRISP-DM
Cross-industry standard process for data
mining - CRISP-DM
Maintenace and
An open standard developed in 1996 by leading monitoring
companies in data analysis
It is still the most popular methodology for data-centric
projects
It is an agile method that introduces almost no
overhead and emphasizes adaptive transitions between
project phases
8
What is CRISP-DM
Maintenace and
monitoring
9
Business Understanding
• Initially, it is vital to understand the problem to be solved
• This may seem obvious, but business projects seldom
come pre-packaged as clear and unambiguous data Maintenace and
monitoring
science problems
• The design team should think carefully about the problem
to be solved and about the use scenario
• Learn to concretize or even reduce the scope of the initial
idea
10
Documentation
• One Pager
• Design document
• High level documents explaining the overall goal
• Quick feedback from the stakeholders and data scientists/engineers
• Different document for different audience
• Enough information to make decisions and provide feedback
• Everyone on the same page
• Easier to scope the project
• Provide clarity and avoids getting into the perfection rabbit hole
• Make project planning easier
11
Data analytics
Examining data to answer questions, identify trends, and extract insights.
12
Types of data analytics
13
Descriptive analysis
• Pull trends from raw data and succinctly describe it.
• Focus on What happened or is currently happening ?
14
Descriptive analysis
15
Descriptive analysis
16
Descriptive analysis
17
Diagnostic analysis
• Comparing coexisting trends or movement, uncovering correlations between
variables, and determining causal relationships where possible.
• Focus on: Why did it happen?
18
Descriptive analysis
19
Descriptive analysis
• Comparing coexisting trends or movement, uncovering correlations between
variables, and determining causal relationships where possible.
• Focus on: Why did it happen?
• Correlation versus Causation
20
Correlation versus Causation
21
Spurious Correlations
22
Correlation versus Causation
23
Predictive analysis
• Predict the future trends and events, using the data at hand.
• Focus on: What might happen in future?
24
Predictive analysis
25
Predictive analysis
Source
26
Prescriptive analysis
• Suggests actionable takeaways considering all possible factors in a scenario
• Focus on: What should we do next?
27
Prescriptive analysis
28
Prescriptive analysis
29
Business Understanding
• Initially, it is vital to understand the problem to be solved
• This may seem obvious, but business projects seldom
come pre-packaged as clear and unambiguous data Maintenace and
monitoring
science problems
• The design team should think carefully about the problem
to be solved and about the use scenario
• Learn to concretize or even reduce the scope of the initial
idea
30
Aneo: Grid loss data
• Grid load
• Total amount of electric energy in the grid
• Grid loss
• Difference in electricity between what has been
produced by the power plants (or load) and what has
been sold to the customers
• Grid load = consumption by customers + grid loss
• Estimated grid loss = idle loss + k*(expected power consumption)2
31
Aneo: Grid loss data
• Grid load
• Total amount of electric energy in the grid
• Grid loss
• Difference in electricity between what has been
produced by the power plants (or load) and what has
been sold to the customers
• Grid load = consumption by customers + grid loss
• Estimated grid loss = idle loss + k*(expected power consumption)2
32
Grid loss data: Problems
• Not grid-specific
• Manual retraining
• Manual and subjective alterations
• Lack of monitoring infrastructure
• Poor scalability
33
Data Understanding
• If solving the business problem is the goal, the data
comprise the available raw material from which the
solution will be built
• Collect initial data Maintenace and
o Existing data monitoring
o Purchased data
o Additional data
• Describe data
o Amount of data
o Value types
o Coding schemes
• Explore data
• Verify data quality
o Missing data
o Data errors
o Coding inconsistencies
o Bad metadata
34
Data Understanding
Maintenace and
monitoring
35
Data Understanding
Maintenace and
monitoring
36
Aneo: Grid loss data
•https://www.kaggle.com/trnderenergikraft/grid-loss-time-series-dataset
37
Aneo: Grid loss data
1. Grid load:
• Grid load = consumption by customers + grid loss
• Estimated grid loss = idle loss + k*(expected power consumption)2
2. Calendar features
3. Weather forecasts
4. Estimated demand in the area
38
Data Understanding
Dos and DON’T’S
• Do not economize on this phase
o The earlier you discover issues with your data the better
o Data understanding leads to domain understanding Maintenace and
monitoring
• Verify as far as you can, if your data is correct,
complete, coherent, deduplicated, representative,
independent and up-to-date
• Investigate what sort of processing was applied to the
raw data
• Understand anomalies and outliers
39
Grid loss data: Problems
• Delayed measurements
• Missing values
• Incorrect values
• Changing values
• Small datasets
• Missing features
40
Data Preparation
1. Select data
• Select features
2. Clean data
• Correct data errors
Maintenace and
• Make coding consistent monitoring
• Fill in or infer missing data
3. Construct data
• Generate derived attributes
4. Integrate data
• Merge information from different sources
5. Format data
• Convert to format convenient for modelling
41
Data Preparation
Maintenace and
monitoring
42
Day in the life of a Data Scientist
43
Day in the life of a Data Scientist
44
Aneo: Grid loss data
1. Grid load
• Is it possible to predict grid load?
2. Calendar features
• Categorical features
• Encoding?
3. Time series decomposed features
• Prophet based features?
45
Modelling
1. Select modelling techniques
• Select an algorithm or a model
Maintenace and
2. Build the model monitoring
• Feature selection
• Hyperparameter optimization
• Training and validation
3. Assess model
• Model performance on test dataset
• Time
• Other Key Performance Indicators (KPIs)
46
Aneo: Grid loss data
1. Select modelling techniques
• Select an algorithm or a model
2. Build the model
• Feature selection
• Hyperparameter optimization
• Training and validation
3. Assess model
• Model performance on test dataset
• Time
• Other KPIs
47
Aneo: Grid loss data
1. Select modelling techniques
• Select an algorithm or a model
2. Build the model
• Feature selection
• Hyperparameter optimization
• Training and validation
3. Assess model
• Model performance on test dataset
• Time
• Other KPIs
48
Aneo: Grid loss data
Which models to use
• Multi-layer perceptron
• Decision tree regressor
• Gradient boosting regressor ensemble
• CatBoost
What baselines to compare to
• Manual method
• Last week
How much training data to use
Which features to use
49
Aneo: Grid loss data
Which models to use
• Multi-layer perceptron
• Decision tree regressor
• Gradient boosting regressor ensemble
• CatBoost
What baselines to compare to
• Manual method
• Last week
How much training data to use
Which features to use
50
Traditional software testing
Breck, Eric, et al. "The ML test score: A rubric for ML production readiness and technical debt
reduction." 2017 IEEE International Conference on Big Data (Big Data). IEEE, 2017.
51
Data Science pipeline testing
Breck, Eric, et al. "The ML test score: A rubric for ML production readiness and technical debt
reduction." 2017 IEEE International Conference on Big Data (Big Data). IEEE, 2017.
52
Important Deadlines
When you will need to deliver or complete a task
1 20/9 Register yourself/group and the company/dataset for group assignment
2 30/10 Deliver individual assignment
3 27/11 Deliver presentation and report for group assignment
53
Lecture Plan
Unpacking the course syllabus
1 23/8 Lecture 1: Introduction [Nisha Dalal] 8 11/10 Lecture 7: Data Visualization & Storytelling
[Manos Papagiannidis]
2 30/8 Lecture 2: Presentation of datasets [Nisha Dalal]
9 18/10 Lecture 8: Data Science in the time of Chat-
GPT [Pikakshi Manchanda]
3 6/9 Lecture 3: Crash course in machine learning
[Kshitij Sharma]
10 25/10 No lecture
13/9 Lecture 4: Data analysis with low or no-code
4
tools [Nisha Dalal] 1/11 Lecture 9: Experiences from Industry [Thomas
11
Thorensen]
5 20/9 No lecture
8/11 Lecture 10: Decision making with data science
12
[Nisha Dalal]
6 27/9 Lecture 5: Lifecycle of a Data Science project I
[Nisha Dalal]
4/10 Lecture 6: Lifecycle of a Data Science project II 13 15/11 Course finish
7
[Nisha Dalal]
Summer Internship 2024
in the AI and Product Development department of Aneo
Rea d more a nd a pply here:
We are looking for you who want to contribute to a sustainable future by
applying AI and/or software development in the renewable energy sector!
Where: Trondheim
When: Summer 2024 (7 weeks, dates TBA)
Deadline: November 12th, 2023
https ://tinyurl.com/2xxh5uhx
55
Nisha Dalal
Questions & Discussion [email protected]
