Business Intelligence Solutions, Purpose and

Architecture
Advanced Certificate in Spreadsheet Analytics (ACSA)
Module 1 - Deck 1
About the instructor Domain Expertise
Data Governance, Engineering, Enterprise Design Thinking,
Business Development, Project Management

Current Appointment
Data Governance Lead
Infineum

Past Appointments
• Data Governance & Data Quality Analyst, Business Development /
Sales & Marketing / Project Management / Product Development /
Systems Integration
• Certis Technology Master Planning, 3M, Agilent Technologies
Education & Certifications
• M.Sc. In Engineering
• Certified Data Management Professional
• ITL4 Certified
• Enterprise Architecture Practitioner
• Advanced Certificate in Learning and Performance (ACLP)
• Certified Scrum Master
• Enterprise Design Thinking Practitioner
EVELYN WONG • Certified in Lean Six Sigma (i.e., Green Belt)

Interests

3
Meet and Greet: Discover Our Participants

4
Course materials
• SMU Academy Learning Management System (LMS)
▪ https://academylearn.smu.edu.sg/

• Updated files (if any)

▪ https://bit.ly/acsa-r3-content
▪ Will let you know which files are updated

• Slack
▪ For communications outside class
▪ https://bit.ly/acsa-slack-invite

5
 ACSA course objectives
• Upskill your spreadsheet skills
▪ Uncover Microsoft Excel features that will improve your productivity

• Learn about business analytics

• High-level intro to core analytics skills & techniques
• Introduce helpful frameworks and cover the key stages of the data
workflows

Understand Data Data Data Data Data

Automation
business case Collection Storage Cleaning analytics Visualization

6
What is ACSA not about
• Not full-fledged data analytics course
▪ Less math (statistics), more on applications
▪ Not fully focused on predictive analysis

• Not on programming
▪ But in module 6 (Automation), may need to introduce VBA syntax

7
Why Microsoft Excel for analytics?
• Often the first tool users encounter when dealing with data
• Widely available in organizations
▪ Since 1992, reached 30 million users by 1996, and by 2015 having 1.1 billion
users (1/8 of Earth’s population)
▪ Survey of 1,000 office workers found
–Office users spend 38% of time using Excel
–But 50% of people have no formal training
• Many features in Excel are not known and underused
▪ Knowing them opens up productivity gains
▪ And lessens opportunities for errors

• Using Excel means you not have to learn a completely new product /
language to dive into analytics
8
Learning and assessment approach
• Introducing concepts via seminar
• Hands-on case guided by instructor
• Assessment designed to fit individual modules
▪ MCQs
▪ Individual assignment
▪ Group assignment

9
 Overview of ACSA’s 6 modules
M1 Business Intelligence Solutions, Purpose and Architecture
• Excel data dashboards
• Primer on Analytics, data collection techniques, regression using Excel

M2 Querying and Wrangling Data

• Power Query
• Extract Transform Load (ETL), data wrangling

M3 Creating and Using Databases

• Power Pivot
• Data tables, data models, data governance

M4 Advanced Analytics
• Power Pivot, DAX
• Measures, KPIs

M5 Creating and Using Dashboard Visualizations

• Power BI Desktop
• Visualization principles

M6 Automating Tasks
• Macros, VBA, task scheduler
• Scheduling, automation
10
Software required
• Microsoft Excel
▪ Excel 365 (SAAS version, Office365, Microsoft365) desktop version
▪ Excel 2016 onwards (best to be 2019 onwards)
▪ Mac versions are alright to use but some features may not be available
–Except for M5: Power BI Desktop only available in Windows

11
Business Analytics Primer
Data/Business Analytics
Data and Information
• Data does not simply exist; it has to be created

• Knowledge is required to create data in the first place

• Data is a form of information and information is a form

of data

• Organizations may differentiate between information

and data for communication between different
stakeholders

Fill in the blanks with data / information:

“Here is a sales report for the last quarter [ ]. It is

based on data from our data warehouse [ ]. Next
quarter, these results [ ] will be used to generate our
quarter-over-quarter performance measures [ ] “

14
Data Management Principles

16
The Data Lifecycle
• Data has a lifecycle.
* Creation and usage are most critical points.
* Data quality must be managed throughout lifecycle

• Data may be:

* Cleansed
* Transformed
* Merged
* Enhanced
* Aggregated

• Data has lineage (Data Chain)

17
Data Management Frameworks:
• Strategic Alignment Model

18
Data Management Framework:
• The DAMA Wheel defines the Data Management knowledge areas.

19
Evolved DAMA Wheel
• Core activities in center

• Surrounded by lifecycle and usage activities

• Lifecycle management: Plan and Enable

• Uses come from lifecycle activities

• Data Governance provides oversight and

containment

20
Data Management and Data Governance

21
Data Management Function Framework:
• Guiding purpose of Data Management

• Deriving value requires lifecycle

management – center of diagram

• Foundational activities support the

lifecycle

• A data governance program enables the

organization to be data driven

22
Data-centric Organization
• A data-centric organization values data as an asset and manages data through all phases
of its lifecycle, including project development and ongoing operations.

• Principles:
* Manage data as a corporate asset

* Incentivize Data Management best practices across the organization

* Enterprise data strategy is aligned to overall business strategy

* Continually improve data management processes

23
Generic Data Governance Organization

24
Why Analytics?
THE WORLD RUNS ON DATA
▪ Data is everywhere, but without analytics, it’s just numbers and noise
▪ Analyst’s role is to translate raw information into insights and outcomes

Reasons to work in Data/Business Analytics:

High average salary and job satisfaction

Strong and growing demand for talent

Highly versatile and transferrable skills

Wide range of roles and specialties

Unique blend of creative & analytical thinking

Opportunity to make a real impact, at any level

Business vs Data Analytics
Aspect Business Analytics Data Analytics

Focus Application of data analysis to Extracting, processing and analysing

business problems raw data for insights
Scope Financial analysis, market Data mining, statistical analysis,
analysis, performance analysis, machine learning, predictive
strategic planning modelling
Tools & Business intelligence tools like Programming languages/tools like
Techniques Excel, Power BI, Tableau, Python, R, SQL, Hadoop, Spark.
Qilkview. Use methodologies like Techniques include regression
SWOT analysis, PEST analysis, analysis, clustering, classification,
financial modelling natural language processing
Outcome Provides actionable business Uncovers deeper insights and
insights and recommendations. knowledge from data. Focuses on
Focuses on strategic and technical and analytical aspects of
operational business decision- data processing and interpretation
making
Application Drives business decisions, Applicable to various fields such as
improves processes, achieves healthcare, finance, marketing etc.
strategic goals
 Components of Data Analysis

• Descriptive • Predictive
▪ Historical ▪ Usinghistory to predict the future
▪ Summary ▪ Machine learning
▪ Simple KPIs • Prescriptive
• Diagnostic ▪ Business actions to achieve business
▪ Why events happen goals
▪ Anomalies ▪ Data driven decisions
▪ Collecting further data related to
anomalies
▪ Statistics to explain anomalies

27
Before we get there..
• Many organizations are seeing Digital Transformation (DX) as the new
objective
▪ Use of Business Analytics (BA), Data Science (DS), AI to transform their organization
▪ Longer term process

• Business Analytics (BA) / Business Intelligence (BI)

▪ Before Data Analytics (DA), DS, AI, DX terms became popular
▪ Analysis of mainly business data (sometimes alternative data as well)
▪ Applicable to day-to-day organizational issues
–Sales
–Customer
–Operation
–HR
–Finance
▪ Leads to tactical and strategic decisions to support DX

29
So many terms!
Business Analytics (BA) / Use of data to monitor, analyze, report and inform
Business Intelligence (BI) /
Data Analytics (DA)
Artificial Intelligence (AI) Umbrella term of the use of machines to learn and apply
human-level intelligence to various use-cases with little to
no human intervention
Machine Learning (ML) Specific execution of using computing power and
algorithms to learn relationships within data and predict
with reasonable levels of accuracies
Data Science (DS) Combining domain knowledge, user needs, and data, with
the abovementioned techniques to solve business
problems
Digital Transformation The use of data science to transform a business from
(DX) within

30
Data science and AI overhyped
• What are you currently doing with business data?
▪ Open a spreadsheet, how to get value from it?
▪ Boss tells you to use Analytics, data science, AI

• But hype on analytics has been overshadowing their actual

execution

• Only some organizations are succeeding with data science and AI

• What is your experience with your organization implementation

of analytics, data science and AI?
31
Disillusion vs Productive Applications

32
How are companies doing BA wrong
• Even though BA has been implemented within companies for a
long time
• Not seem to be the magic bullet that contributes to strategic
objectives

• Applications of BA in organizations
▪ Separated from business strategy
–Ad-hoc reporting, boss says so
▪ Sometimes Coordinated
–To monitor performance of strategic choice
▪ Better ones see BA as a Dialogue
–Use of collected data to support strategic innovation
▪ The best use BA as a Holistic strategic resource
–Data as a weapon, achieving DX status
Laursen, & Thorlund, J. (2017). Business Analytics for Managers: Taking Business Intelligence Beyond Reporting, Second
Edition (Second Edition). John Wiley & Sons (US).
33
Applying analytics as a coordinated resource
• Holiday Inn Singapore Orchard City Centre
▪ Wanted a way to make better and faster decisions based on data
▪ Previously collate information manually from various sources

• Then consolidated all information onto one platform

(Qlikview)
▪ Hotel previously can only see non-performing sources at country level
–Now can see specific cities and marketing channels (OTA, Google, FB, etc)
▪ Keeping track of performance (occupancy rates, daily room rates, revenue
per available room) to optimize pricing

• Productivity gains
▪ Improve team morale (less mundane work, more analysis and planning)
▪ Improve revenue, rooms, average rate
▪ Cut manpower by 12 hours per day

34
Using analytics as a dialogue
• Domino’s Pizza – traditional pizzas to data-driven
organization
▪ Opened up multiple channels for customer ordering
–Google Home, Amazon Alexa
–Slack, FB Messenger, Twitter
–Smartphones, smartwatches, smart TVs
–“Easy Order”, one click, tap, tweet, voice

▪ Created holistic view of customer behaviour

–To understand their needs
–To target effectively for marketing and loyalty rewards

▪ Using AI to serve customer needs

–Hit 95% accuracy for predicting when order will be ready
–Worked data into operations – achieved order to delivery in 10
min or less
–‘Points for Pie’ campaign – rewards customers for snapping
picture of pizza eating
• Used deep learning to build image recognition application
35
Applying analytics as a holistic strategic resource
• The Weather Company – Traditional TV channel to selling data
▪ Only had and provided weather data and analysis as core competency
▪ Based on core competency, started forecasting division
–Provided weather-related analytics for different sectors
–For Aviation: Improving airline operational efficiency and safety
–For Energy: Forecast energy load based on weather predictions
▪ Transformed company to monetize its data assets
▪ Sold to IBM in 2016
▪ Developed new business model targeted at consumers
–Free weather forecast app
–User can pay IBM $30/year to remove ads, get more detailed radar and weather
forecasts
–With every major weather emergencies, subscription rate went up
–As of Apr 2021, has over 900,000 paying subscribers - $27 million revenue per year

36
Data Science Value Framework

User Value Business What can What do

Proposition Objectives DS do? you need?
(UVP)
• Rule-based • Domain
• Who is the • Revenue • Analytics experts
user? • Margins • SML
• USML • Data
• What are • Valuation • Human
• Deep learning
their pains • Reinforcement Capital
and gains? learning

Asking the right question Finding the right solution

Modified by Jinghao Ke, PhD for Data Science processes

Original Source: British Design Council (2005). Double Diamond (design process model)
37
Discover: User Value Propositions (UVP)
• Defining the user • What are their pains?
▪ Externalvs Internal ▪ Undesired outcomes / Problems
▪ Consumer vs Business
▪ Obstacles
▪ Stakeholders: Shareholders,
▪ Risks
Employees, Suppliers,
Customers, Government • What are their gains?
▪ Focus on one group of users ▪ Required
• What are the jobs they ▪ Desired
want to accomplish? ▪ Expected vs Unexpected
▪ Functional
▪ Social
▪ Personal / Emotional

38
Define: Business / Policy Objectives
• How do you want to add value?
• What are the constraints?

Revenue Valuation
(Improve Market Share) (Stakeholder expectations)

Margins
(Cost Efficiency)

39
Company’s vision and mission
• Senior management sets the direction
▪ Apple: To bring the best personal computing products and support to students,
educators, designers, scientists, engineers, businesspersons and consumers in over 140
countries around the world.

▪ Google: To organize the world’s information and make it universally accessible and
useful.

▪ Microsoft: To empower every person and every organization on the planet to

achieve more.

40
Translates to Long term plan (LTP) and annual
business plans
• How to measure and quantify the mission? Finance
translate into numbers
▪ Predict revenue, costs, profits
▪ Prepare LTP (3 to 5 years plan)
–Need input from strategy, sales, marketing, HR, legal, supply chain,
admin and other departments
–All of them have expectations, finance translate to budget / target

▪ Annual business plan

–Detailed to capture expectations of business stakeholders
–Record actuals vs budget / expectations
–Update expectations
41
Class Activity
• Open https://bit.ly/acsa-m1-d1-activity
• Think about the main (or one) process of what you do in your work

• User Value Proposition (UVP)

▪ Who is the end-user?
▪ What do they want to accomplish?
▪ What is their current workflow? What about the proposed future?
–What are their current pains and gains?
▪ How will they benefit from the application?

• Business Objectives
▪ Which business objectives (Revenue, Margins, Valuation) does the application enables?
▪ Does it gel with the corporate direction?
▪ Is the value sufficient for the estimated effort?

42
Develop: What can Data Science do?
• Rule-based
▪ Mainly used for automation
▪ Or to escalate or to focus on cases when it
matters (Anomaly Detection)
▪ Includes optimization techniques like linear
programming, constraint programming

• Data Analytics
▪ Extract patterns from data to help with decision
making
▪ Descriptive statistics
▪ Association rules
▪ Reporting / Monitoring
▪ Internal or Client facing

43
Develop: What can Data Science do?
• Supervised Machine Learning (SML)
▪ Requires labelled outcomes to “teach” machine
▪ Traditional Statistical modelling (e.g. regressions)
also included
▪ Most useful when outcome is clear with historical
data

• Unsupervised Machine Learning (USML)

▪ Extracting previously unseen patterns from data
▪ Often used to group data into segments
▪ Advanced Anomaly Detection
▪ Or used as additional features in SML

44
Develop: What can Data Science do?
• Deep Learning
▪ Use of layers to create intermediate inputs into final prediction
models
▪ Very useful in image recognition / natural language applications
–Tesla’s Autopilot: recognizing objects on road without LIDAR
–OpenAI’s GPT: enabling conversational AI

• Reinforcement Learning
▪ Create an environment and place an agent
▪ Assign rewards and penalty to different moves
▪ Let the agent run millions of times (trial and error)
▪ Recommender systems
–Google’s AlphaGo Zero: reduced data centres cooling costs by
40%
–Spotify’s Personalization: exploitation (similarity) vs exploration
(difference)

45
Deliver: What do you need?
• Domain experts
▪ The end user is sometimes the expert
▪ Or talk to, read from experts in the field to look at how they solve problems

• Data
▪ Internal data
▪ External data

• Human Capital
▪ Empowering leadership
–Structure to remove mundane, manual processes
–Give your team time to tinker with data
–Processes to record and reward data initiatives
▪ Data Champions
–Curious about how is data
• Collected, Stored, Processed, and Delivered
▪ Data Translators
–Technical users to help business users to translate business requirements into technical
solutions
–Pass to dedicated tech team to execute

46
Common problems faced in execution
• Planning Stage
▪ Not sure where/how to start
–Lack of clear question to answer
–Lack of talent
▪ Too ambitious
–Visions can be grand, but initial execution
should be small and manageable
▪ Weak management support and coordination
–Translating requirements
–Data identification and ownership
–Process of automation

47
Common problems faced in execution
• Execution stage
▪ Treating analytics as a one-off exercise
–Buy a technology product and magic
happens?
▪ Data Issues
–Lack of data
–Dirty data
▪ AI and analytics are probabilistic
–Accuracy is not 100%
▪ Not refining processes, just stacking on more
work!
–End product / results not used by actual
decision makers on the ground
If Planning is structured well, Execution will have less problems!
48
 Solutions to those problems
• Talk to your team!
▪ Check if any mundane processes should be automated
▪ Check for any critical decisions to be made frequently

• Pareto Principle 80/20

▪ Refine and focus on small bite-size projects that has highest impact
▪ Inject confidence and motivates team to continue DX journey

• Encourage collaboration
▪ Your team champions to work with other team champions
▪ Send them for data translator courses

• Keep working on it
▪ Small improvements to be demonstrated regularly
▪ Agile principles

• Reward!
▪ After project, don’t assign new responsibilities immediately

49
 Typical business analytics process in organization

Clean &
Identify Connect Model Visualize
Transform

• What are the • Flat files (csv, • Missing • Linking • Using the
business excel, txt) values datasets / right chart for
requirements • SQL • Transposing tables the right
? databases • Dropping together dataset
• Build a (MySQL, MS values / • Creating • Tracking
measurement SQL, columns measures / progress
plan PostgreSQL) • Duplicates KPIs • Feedback
• What data is • HTML pages • Machine loop
required? • Others learning

51
Understand The Business Requirements
• Before start thinking like an analyst, think like a business owner.

• Understand which specific outcomes you are trying to impact, who are the
key stakeholders are and what motivates them, and how your analysis fits
into the bigger picture.

• This will help to align on requirements, project scope and desired outcomes
from day one.

52
Build A Measurement Plan
• Think of a measurement plan like a roadmap for success.

• Measurement planning is about defining what a successful outcome looks

like for the business, determining which KPIs align with that outcome, and
identifying the data need to capture, track and optimize those metrics.

• If the goal is to deliver data-driven insights and outcomes, do not skip this
step!

53
Identify: Measurement Planning
IF WE CANNOT MEASURE IT, WE CANNOT OPTIMIZE IT

Measurement planning is about defining exactly what a successful outcome looks like
for the business, and building a framework to identify, track and optimize key metrics.

54
Case Study: Measurement Planning
• The Scenario
You have just landed a job as the Business Analyst working in the call
centre for the largest telecom operator. The call centre handles customer
service inquires, new service sign-ups and technical support.

• The Assignment
Your first assignment is to build a detailed measurement plan for the VP
who runs the call centre. He reports to the COO and indirectly to the CEO,
and oversees 8 Managers who each lead a team of call centre reps.

• The Objectives
1. Consider the goals of the business and needs of the stakeholders

2. Identify and prioritize Key Performance Indicators (KPIs) and supporting metrics

3. Make plan to gather the data and address any gaps

55
 Identify: Measurement Roadmap

One of the most common pitfalls for Analysts is jumping into the data too quickly,
before you start thinking like an analyst, think like a business owner first!

Key questions to ask:

• What are the key business goals and objectives?

• Where does this analysis fit into the overall business?
• What are the most important questions that stakeholders need answer to?
• What types of actions do you want them to take after seeing your analysis?

56
Step 1: Think Business First
1. What are the high-level goals of the largest telecom operator?

2. How does the call centre support those business goals?

3. What are the most important goals for the call centre?

4. What questions should you try to answer for call centre leadership?

5. What actions can you imagine leadership taking based on your data?

57
 Identify: Measurement Roadmap

After thinking about the business impact, focus on the key stakeholders next; who are
they, what do they need, and how will your analysis support them?

Key questions to ask:

• Who is the primary audience you are designing the measurement plan for?
• What are their goals and incentives? What do they care most about?
• Are there multiple stakeholders who will be impacted by your analysis?
• What types of information would inspire them to take action?

59
Step 2: Know Your Audience
1. What is your primary stakeholder most interested in? What are his goals?

2. What information should you provide to help him to do his job well?

3. What are the COO and CEO most interested in?

4. What are the call centre managers most interested in?

5. What information do managers need to make their teams more effective?

60
 Identify: Measurement Roadmap

Once you understand the business impact and stakeholder needs, you are ready to
start identifying Key Performance Indicators (KPIs) and supporting metrics.

Key questions to ask:

• Which business goals have you already identified?

• For each goal, which metrics can be used to accurately measure it?
• Which of those are most important for your stakeholders?
• Thinking one level deeper, what other supporting metrics might help you
optimize or make decisions to improve your KPIs?

62
Step 3: Define The KPIs

63
 Identify: Measurement Roadmap

After identifying your KPIs and supporting metrics, it is time to think about what data
you need, where to source it and how to prepare it for analysis.

Key questions to ask:

• What sources provide the data your need to track each metric?
• Who owns or manages each data source?
• How frequently is each data source refreshed?
• Can you automate or streamline the data collection process?

65
Common Business Data
Customer Marketing
•Demographics (age, gender, •Search trends
race) •Market research
•Personal particulars (contact •Competitive intelligence
details, identification)
•Behavior (spending, locations,
mobile use, social media)
•Qualitative (interests,
opinions, reviews)

Finance and accounting Product and services

•Transactions •Locations
•Costs •Product specifications
•Price •Guides and tutorials

What other business data can

you think of? Logistics
•Inventory
•Processes
•Weather 69
 Identify Data Sources – Where
• Where are the sources of data to collect?
▪ Internal
–Sales & marketing (CRM systems, e.g. Salesforce, Hubspot, MS Dynamics, SAP)
–Transactions (POS, invoicing systems)
–Mobile apps (loyalty apps)
–Product movement (inventory or production systems)
–Finance (accounting systems)

▪ External – mostly to validate hypothesis then start collecting internally

–Opensource datasets
• Kaggle, data.gov.sg, Google public data, awesome public datasets on Github
–Competitors’ data
• Ecommerce, Taobao
–Web scraping
• Scrapehero
–Public APIs
• Onemap, Any-Api, RapidAPI
–Private / Paid APIs
• Quandl

70
 Identify Data Sources – How
• How to collect – the collection strategy?
▪ Initially
–Review past transactions / data
–Discover insights and gaps (compare to wish list in What)
–Use external data to validate hypothesis
▪ Subsequent
–Set up processes to capture data
• Not always about full automation
• Human processes are crucial and unavoidable (especially customer touchpoints)
• Support human processes without interference
–If possible, to capture at source
• Most timely
• Reduces human and machine errors – improves accuracy

71
 Identify Data Sources – How
• Common problems in data collection
▪ Red tape
–E.g. Marketing not talking to accounting
–Geographical segments not collaborative
▪ Seen as additional process
–No staff motivation to do extra work without reward
–Work the collection into the processes to support humans
▪ Lack of monetary and time budget
–“I want to know each and every customer’s preferences”
–“I am able to know the preferences of customers who are willing to
reveal their likes and dislikes”
–“I know the preferences of customers who always patronize my
services”
72
 Identify Data Sources – How

• Logistical issues
▪ Do current systems already capture what we need?
–If not, can we recalibrate them to capture new variables?
–Try not to use new systems just for purpose of capturing data
▪ If manual processes, how do we ensure that
–Collection is done consistently for every transaction
–While keeping accuracy
▪ Where and how to store the data?

73
 Data Collection Framework – Frequency
• How frequently should you collect the data?
▪ Every transaction? Daily, weekly, monthly?
▪ Keep in mind budget and process constraints
• Collection and analysis frequency can be different
▪ E.g.
collect every transaction but group data together to analyze on a
weekly basis
• How long do you need to collect the data for before analysis -
duration?
▪ Lookat the frequency of transactions
–E.g. if only 1 transaction per day, need 1 month to get 30 datapoints
(barely enough for statistical tests)
–Usually will collect 3 to 6 months of data
• To analyze
• To revisit collection process

74
Step 4: Identify Data Sources

75
Key Takeaways
• Before you think like an analyst, think like a business owner
 Focus on the specific business outcomes you want to impact, and the role your analysis will play.
• Understand who the stakeholders are, and what motivates them
▪ Think about your analysis will impact key decision makers, and what will inspire them to act.
• Define clear, measurable KPIs tied to key outcomes
▪ Take time to identify the most important metrics to help you track and optimize performance
• Document your data requirements to stay organized
▪ Create a shared document to outline your KPIs, supporting metrics, data sources and requirements.

77
Data Storage
Collect Data
• An analysis is only as strong as the data supporting it.

• Data prep tends to be one of the most challenging and time-consuming

stages in the workflow, often involving a mix of quality assurance (QA),
data profiling, feature engineering and ETL automation.

• This creates a strong foundation for analysis and ensures that we are
working with clean, high-quality data.

79
Typical data storage process

Raw Data Data Lake Data BI platforms /

Collection Warehouse Spreadsheets

• From • Storing raw • Cleaned and • Serving end

internal data as is organized users
and/or • File-based / data • Self-service
external Dictionary • Relational or via data
sources (NoSQL) database tech team
• AWS s3, (RDBMS) -
Azure Blob SQL
• MSSQL,
MySQL,
Postgresql

81
Data lakes vs Data warehouses
• Data lakes are supposed to store raw data
• While data warehouses contains the clean and organized
data for different functions / departments
• We store raw data because we may find a need for them in
the future

82
Common database types
• Flat files
▪ Csv, xlsx,txt
▪ On local or cloud drives (dropbox, onedrive, google drive)
• SQL database
▪ Structured, well-definedtables
▪ Needs effort to maintain and extend
▪ MSSQL, MySQL, Postgressql
• NoSQL
▪ Dictionary based, less structured
▪ Easy to store and maintain, but can get very messy quickly
▪ MongoDB, Apache Cassandra

83
End-to-end Processes in Organizations
Why look at business processes?
• Before we dive into analytics
• We look at the business processes first
▪ In many cases, just refining the business process yield good results
▪ And set up for data collection, storage and analytics later on

85
Common processes in Sales
Sales Sales Delivery
Lead
Sales order
generation

Invoice
Pitch
customer

Product
Closing
delivery

Collect
Follow up
payments
86
Common processes in Procurement
Suppliers Procurement
Demand Production or
Procurement
planning

Requisition
Vendor sourcing
Track
Inventory
Supplier
negotiations Purchase
order

Select &
approve supliers Delivery

87
Common processes in Human Resources
Hiring Onboarding Retirement
Headcount approval Induction Resignation / Voluntary
retirement / retirement

Job ads Deployment

Handover

Receive
Setting goals
applications
Exit interview
Filter & select Training

Outstanding leaves
Transfers /
Interviews
relocation

Rewards / Final settlement

Hire
promotion
88
Common processes in Finance
Reconcillation
Actuals from sub-ledgers,
AP, AR, Inventory

General ledger

Prepare P&L, BS

Dashboarding

Reporting

89
 Data Architecture – How

• Map out the dataflow

▪ Flowchart of systems and data
▪ Typically at the table level
–But should be as detailed at the column level

90
Thank you