Prepared by Mrs Kerina Blessmore Mukavaidzi

Big Data Everywhere!

BIG
DATA
Shares traded on US
Stock Markets each
day:

7 Billion
Data generated in
one flight from NY
to London:

10 Terabytes

Number of tweets Number of ‘Likes’

Data that is TOO LARGE & TOO each day on
per day on Twitter:
COMPLEX for conventional data tools Facebook:
to capture, store and analyze.
400 Million 3 Billion
The 3V’s of Big Data

VOLUME VARIETY VELOCITY

90 % OF THE WORLD’S
DATA WAS
GENERATED IN THE
LAST TWO YEARS

2
What is Analytics?

Data on its own is useless unless you can make sense of it!

WHAT IS ANALYTICS?
The scientific process of transforming data into insight for making
better decisions, offering new opportunities for a competitive
advantage
 Types of Analytics

1 Prescriptive Analytics
Enabling smart decisions
based on data
What should we do?

Analytics

2 3

Predictive analytics Descriptive analytics

Predicting the future based Mining data to provide
on historical patterns business insights
What could happen? What has happened?
4
Types of Analytics

Why do airline prices

change every hour?

Prescriptive How do grocery cashiers

Analytics know to hand you coupons
advice on possible outcomes you might actually use?
Predictive How does Netflix
Analytics frequently recommend just
understanding the future the right movie?

Descriptive
Analytics
insight into the past

5
 What is Big Data?
• Massive sets of unstructured/semi-structured data from Web traffic,
social media, sensors, etc
• Petabytes, exabytes of data
• Volumes too great for typical DBMS
• Information from multiple internal and external sources:
• Transactions
• Social media
• Enterprise content
• Sensors
• Mobile devices

• In the last minute there were …….

• 204 million emails sent • 100,000 tweets
• 61,000 hours of music • 6 million views and 277,000 Facebook Logins
listened to on Pandora • 2+ million Google searches
• 20 million photo views • 3 million uploads on Flickr
 What is Big Data? continued
• Companies leverage data to adapt products and
services to:
• Meet customer needs
• Optimize operations
• Optimize infrastructure
• Find new sources of revenue
• Reveal more patterns and anomalies
Why Big Data Analytics?

Why is Big Data Analytics important?

Big data analytics helps organizations
harness their data and use it to identify
new opportunities. That, in turn, leads to
smarter business moves, more efficient
operations, higher profits and happier
customers.

8
 Big Data EveryWhere!
• Lots of data is being collected
and warehoused
– Web data, e-commerce
– purchases at department/
grocery stores
– Bank/Credit Card
transactions
– Social Networks
 Big Data Characteristics
Growing quantity of data

VOLUME
e.g. social media, behavioral, video
T Y
R IE
VA

Increase in types of data

e.g. app data, unstructured data VELOCITY

Gartner, Feb 2001

Quickening speed of data

e.g. smart meters, process monitoring
Volume Variety
The amount The types
of data of data

Velocity The 4 V’s

of
Big Data
The Veracity
frequency of The quality
data of data
 Which Big Data characteristic is the biggest
issue for your organization?

Velocity
of data
16%

Variety of data
48%
Volume of data
35%

Source: Getting Value from Big Data, Gartner Webinar, May 2012
Volume

• Petabytes,
exabytes of data
• Volumes too
great for typical
DBMS
 Volume - Bytes Defined

eBay data warehouse

(2010) = 10 PB

eBay will increase this

2.5 times by 2011

Teradata > 10 PB

Megabyte: 220 bytes or, loosely, one Gigabyte: 230 bytes or, loosely one billion
million
5-15 bytes bytes
Volume: scale of data
 4.6
30 billion RFID billion
tags today
12+ TBs (1.3B in 2005)
camera
of tweet data phones
every day world wide

100s of
millions
of GPS
data every day
? TBs of

enabled
devices sold
annually

25+ TBs of
log data 2+
every day billion
people on
the Web
76 million smart meters by end
in 2009… 2011
200M by 2014
Velocity
Massive
amount of
streaming
data
Velocity: analysis of streaming data
 Velocity (Speed)

• Data is being generated fast and need to be

processed fast
• Online Data Analytics
• Late decisions  missing opportunities
• Examples
– E-Promotions: Based on your current location, your purchase history, what
you like  send promotions right now for the store next to you

– Healthcare monitoring: sensors monitoring your activities and body  any

abnormal measurements require immediate reaction

20
 Real-time/Fast Data

Mobile devices
(tracking all objects all the time)

Social media and networks Scientific instruments

(all of us are generating data) (collecting all sorts of data)

Sensor technology and networks

(measuring all kinds of data)

• The progress and innovation is no longer hindered by the ability to collect data
but by the ability to manage, analyze, summarize, visualize, and discover
knowledge from the collected data in a timely manner and in a scalable fashion

21
Variety • Massive sets of
unstructured/se
mi-structured
data from Web
traffic, social
media, sensors,
and so on
 Variety (Complexity)
• Relational Data (Tables/Transaction/Legacy Data)
• Text Data (Web)
• Semi-structured Data (XML)
• Graph Data
– Social Network, Semantic Web (RDF), …

• Streaming Data
– You can only scan the data once

• A single application can be generating/collecting

many types of data

• Big Public Data (online, weather, finance, etc)

To extract knowledge all these types of data

need to linked together 24
 A Single View to the Customer

Social Banking
Media Finance

Our
Gaming
Customer Known
History

Entertain Purchase
Variety: different forms of data
 Veracity: trustworthiness of data

• Origin
• Authenticity
• Trustworthiness
• Completeness
• Integrity
Veracity
Summary of 4V’s

29
Big Data Opportunities
Making better informed decisions
e.g. strategies, recommendations

Discovering hidden insights

e.g. anomalies forensics, patterns,
trends

Automating business processes

e.g. complex events, translation
 Auto Insurance

Identifying Insurance Fraud

• Opportunity
• Save and make money by reducing fraudulent auto insurance claims
• Data & Analytics
• Predictive analytics against years of historical claims and coverage data
• Text mining adjuster reports for hidden clues, e.g. missing facts, inconsistencies,
changed stories
• Results
• Improved success rate in pursuing fraudulent claims from 50% to 88%; reduced
fraudulent claim investigation time by 95%
• Marketing to individuals with low propensity for fraud

What **“dark data” is just laying around that can transform business
processes?
**Operational data that is not being used. Consulting and market research company Gartner
Inc. describes dark data as "information assets that organizations collect, process and store
in the course of their regular business activity, but generally fail to use for other purposes."
31
 Quality Improvement

• Opportunity
– Move from manual to automated inspection of burger bun
production to ensure and improve quality
• Data & Analytics
– Photo-analyze over 1000 buns-per-minute for color, shape
and seed distribution
– Continually adjust ovens and process automatically
• Result
– Eliminate 1000s of pounds of wasted product per year;
speed production; save energy; Reduce manual labor costs

Is the company using all of its “senses” to observe, measure

and optimize business processes?
 Improving Corporate Image
• Opportunity
• Improve reputation, brand and buzz by tapping social media
• Data & Analytics
• Continually scanning twitterverse for mentions of their
business
• Integrating tweeters with their robust customer management
system
• Results
• Saw tweet from a top customer lamenting late flight—no time
to dine at Morton’s
• Tuxedo-clad waiter waiting for him when he landed with a bag
containing his favorite steak, prepared the way he normally
likes it with all the fixin’s

How can the company listen, analyze and respond in real-time?

 The Model Has Changed…
• The Model of Generating/Consuming Data has Changed

Old Model: Few companies are generating data, all others are consuming data

New Model: all of us are generating data, and all of us are consuming data

34
 Big data technologies and tools
• R
• Python
• Hadoop (HDFS and Mapreduce)
• Tableau (visualisation)

• And many other

Application perspective of BDA
Big data applications
 Big Data in banking
• The amount of data in the banking sector is
skyrocketing every second.
• Proper study and analysis of this data can help
detect any and all illegal activities that are being
carried out such as:
– Misuse of credit/debit cards
– Business clarity
– Customer statistics alteration
– Money laundering
– Risk mitigation
 Cont…
Example
• Various anti-money laundering software such
as SAS AML use Data Analytics for the purpose
of detecting suspicious transactions and
analyzing customer data.
• Bank of America has been a SAS AML
customer for more than 25 years.
Big data in healthcare
 Big Data in Health Care
Healthcare is yet another industry which is bound to
generate a huge amount of data. Following are some of the
ways in which big data has contributed to healthcare:
• Big data reduces costs of treatment since there are less
chances of having to perform unnecessary diagnosis.
• It helps in predicting outbreaks of epidemics and also in
deciding what preventive measures could be taken to
minimize the effects of the same.
• It helps avoid preventable diseases by detecting them in
early stages. It prevents them from getting any worse
which in turn makes their treatment easy and effective.
 Cont…

Example
• Wearable devices and sensors have been introduced
in the healthcare industry which can provide real-
time feed to the electronic health record of a
patient. One such technology is from Apple.
• Apple has come up with Apple HealthKit, CareKit,
and ResearchKit. The main goal is to empower the
iPhone users to store and access their real-time
health records on their phones.
•
Sensing devices
Smartwatches
• Smart jewelery
• Fitness trackers
• Sport watches
• Smart glasses
• Smart clothing…
 Fraud detection

• For businesses whose operations involve any type of claims

or transaction processing, fraud detection is one of the
most compelling Big Data application examples.
• In most cases, fraud is discovered long after the fact, at
which point the damage has been done and all that’s left is
to minimize the harm and adjust policies to prevent it from
happening again.
• Big Data platforms that can analyze claims and transactions
in real time, identifying large-scale patterns across many
transactions or detecting anomalous behavior from an
individual user, can change the fraud detection game.
 Big data in electronic commerce
• Where is big data fitting in
 Enhanced shopping experience
• Data mining with Big Data helps in understanding a shopper
very closely.
• Every shopper is being monitored continuously. Right from
entry to exit, whatever a shopper explores is recorded.
• Based on that, the seller digs out their browsing interest and
purchase history and get a catch of their shopping pattern as in
what their demand is and when do they buy most.
• With Big Data, ecommerce sellers have benefitted in managing
their inventory as well. They are now able to organize
customers based on their shopping pattern, demographic
details and time period and thus offload their excess stock
accordingly.
 Other applications in E.C
• Increased Sales With Customized Offers and
Recommendations
• Better customer service
 Amazon Web Services
• offers software as a service for eCommerce brands to
build data lakes on which they can run AI-based big data
analytics processes, including predictive analytics.
• eCommerce companies can import their data using AWS
Direct Connect, which purportedly allows for a secure and
reliable connection between the system from which the
client is uploading their data and the newly built data lake.
• The process of uploading the kind of volumes of data
most eCommerce brands have can be lengthy, and any
disruptions in the process could result in missing data on
transfer
Making Smarter and More Efficient
Organisations
 Cont..
• The NYPD brilliantly uses BDA to detect and identify
crimes before they occur. They analyse historical arrest
patterns and then maps them with events such as
federal holidays.
• This aids them in analyzing the information immediately
by utilizing these data patterns.
• BDA strategy helps them identify crime locations,
through which they deploy their officers to these
locations. Thus by reaching these locations before the
crimes were committed, they prevent the occurrence of
crime.
Optimize Business Operations by Analysing
Customer Behaviour
 Cont..
• Most organisations use behavioural analytics of
customers in order to provide customer satisfaction
and hence, increase their customer base.
• The best example of this is Amazon. Amazon is one
of the best and most widely used e-
commerce websites with a customer base of about
300 million.
• They use customer click-stream data and historical
purchase data to provide them with customized
results on customized web pages.
 Cont..
• Analysing the clicks of every visitor on their
website aids them in understanding their site-
navigation behaviour, paths the user took to
buy the product, paths that led them to leave
the site and more.
• All this information helps Amazon to improve
their user experience, thereby improving their
sales and marketing.
Cost reduction
 Cont..
• Patients nowadays are using new sensor
devices when at home or outside, which send
constant streams of data that can be
monitored and analysed in real-time to help
patients avoid hospitalization by self-
managing their conditions.
• For hospitalized patients, physicians can use
predictive analytics to optimize outcomes and
reduce readmissions.
 Cont…
• Parkland Hospital uses analytics and predictive
modelling to identify high-risk patients and
predict likely outcomes once patients are sent
home. As a result, Parkland reduced 30-day
readmissions for patients with heart failure, by
31%, saving $500,000 annually.
New generation products
 Cont…
• With the ability to gauge customer needs and
satisfaction through analytics, comes the
power to give customers what they want
 Useful Links
• https://www.edureka.co/blog/big-data-analyti
cs
/
• https://www.datamation.com/big-data/big-da
ta-vs.-
artificial-intelligence.html
• https://
heartbeat.fritz.ai/artificial-intelligence-ai-vs-m
achine-learning-ml-vs-big-data-909906eb6a92