The Weakening and Delayed Effects of Long

Tail Distributions in Big Data Accesses

Xiaodong Zhang
Ohio State University

1
 Big Data and Power Law

# of hits to
each data
object

Popularity ranks
for each data object

To the rights (the yellow region) is the long tail of lower 80%
objects; to the left are the few that dominate (the top 20%
objects). With limited space to store objects and limited search
ability to a large volume of objects, most attentions and hits
have to be in the top 20% objects, ignoring the long tail.
The Change of Time (short search latency) and Space (unlimited storage
capacity) for Big Data Creates Different Data Access Distributions

Traditional long tail distribution

Flattered distribution after the
long tail can be easily accessed

• The head is lowered and the tail is dropped more and more slowly
• If the flattered distribution is not power law anymore, what is it?
Distribution Changes in DVDs in Netflix 2000 to 2011

2011
predicted

• The growth of Netflix selections ( today: 30 million US users, 40 million users total,
1/3 streaming traffic of Internet)
– 2000: 4,500 DVDs, 2005: 18,000 DVDs
– 2011: over 100,000 DVDs (the long tail would be dropped even more slowly for more demands)
– Note: “breaks and mortar retailers”: face-to-face sell shops.
Amazon Case: Growth of Sales from the Changes of Time/Space
We Must Find the New Distribution for Big Data Accesses

• Internet stores all kinds of huge big data sets

– The rapid growth and wide distribution of Internet media
content is a representative case study of big data

– The media content is carried by scalable distributed systems

• We hope distribution model developed is

– General purpose for other applications of big data

– Scalability nature of both data and systems

7
 Zipf distribution is believed the general
model of data access patterns
• Zipf distribution (power law)
logy y
– Characterizes the property of scale
invariance
– Heavy tailed, scale free
slope: -

• 80-20 rule heavy tail

– Income distribution: 80% of social wealth
owned by 20% people (Pareto law) log
i i
– Web traffic: 80% Web requests access
20% pages (Breslau, INFOCOM’99) 
yi  i : 0.6~0.8
• System implications
i : rank of objects
– Objectively caching the working set in
yi : number of references
proxy
– Significantly reduce network traffic
8
Does Internet media traffic follow Zipf’s law?
Web media systems VoD media systems

Chesire, USITS’01: Zipf-like Acharya, MMCN’00: non-Zipf

Cherkasova, NOSSDAV’02: non-Zipf Yu, EUROSYS’06: Zipf-like

P2P media systems Live streaming and IPTV systems

Gummadi, SOSP’03: non-Zipf Veloso, IMW’02: Zipf-like

9
Iamnitchi, INFOCOM’04: Zipf-like Sripanidkulchai, IMC’04: non-Zipf
 Inconsistent media access pattern models
• Still based on the Zipf model
– Zipf with exponential cutoff
– Zipf-Mandelbrot distribution
– Generalized Zipf-like distribution
– Two-mode Zipf distribution heuristic assumptions
– Fetch-at-most-once effect
– Parabolic fractal distribution
– …
• All case studies
– Based on one or two workloads
– Different from or even conflict with each other
• An insightful understanding is essential to
– Content delivery system design
– Internet resource provisioning
– Performance optimization 10
 Research Objectives

• Find a general distribution model of Internet media

access patterns as a case for big data

– Comprehensive measurements and experiments

– Rigorous mathematical analysis and modeling

– Insights into media system designs

11
 Outline
• Motivation and objectives

• Stretched exponential model of Internet media traffic

• Dynamics of access patterns in media systems

• Caching implications and storage requirements

• Summary

• Other newly reported SE distributions in real world

12
 Workload summary

• 16 workloads in different media systems

– Web, VoD, P2P, and live streaming nearly all workloads
available on the Internet
– Both client side and server side
• Different delivery techniques
– Downloading, streaming, pseudo streaming all major delivery
techniques
– Overlay multicast, P2P exchange, P2P swarming
• Data set characteristics
– Workload duration: 5 days - two years
data sets of
– Number of users: 10 - 10
3 5
different scales
– Number of requests: 104 - 108
– Number of objects: 102 - 106
13
Distribution Changes in DVDs in Netflix 2000 to 2011

2011
predicted

• The growth of Netflix selections ( today: 30 million US users, 40 million users total,
1/3 streaming traffic of Internet)
– 2000: 4,500 DVDs, 2005: 18,000 DVDs
– 2011: over 100,000 DVDs (the long tail would be dropped even more slowly for more demands)
– Note: “breaks and mortar retailers”: face-to-face sell shops.
 Stretched exponential distribution
• Media reference rank follows stretched exponential distribution
(passed Chi-square test)
log y
Probability distribution: Weibull fat head
x
P ( X  x) 1  exp[ ( )c ]
x0
c: stretch factor

Rank distribution: thin tail

• fat head and thin tail in log-log scale log i
• straight line in logx-yc scale
yc c: stretch factor
i : rank of media objects (N objects)
y : number of references
i
P ( y  yi )  b slope: -a
N
y ic   a l o g i  b (1  i  N , a  x 0c )
b 1  a log N (assuming y N 1) 15
log i
 Evidences: Web media systems (server
logs)
fat *HPC-98
head (14 MB)thin tail *HPLabs-99 (120 MB) ST-SVR-01 (15 MB)
powered scale yc

c = 0.22 log scale

R2 ~ 1

log scale in x axis

x: rank of media object, y: number of references to the object. Title: workload name (median file size)
data in stretched exponential scale
data in log-log scale
R2: coefficient of determination (1 means a perfect fit)
HPC-98: enterprise streaming media server logs of HP corporation (29 months)
HPLabs: logs of video streaming server for employees in HP Labs (21 months)
ST-SVR-01: an enterprise streaming media server log workload like HPC-98 (4 months)
16
Evidences: Web media systems (req packets)
fat head thin
PS-CLT-04 (1.5 MB)tail ST-CLT-04 (2 ST-CLT-05 (4.5 MB)
MB)
powered scale yc

log scale

log scale in x axis

All collected from a large cable network hosted by a well-known ISP

PS-CLT-04: first IP packets of HTTP requests for media objects (downloading and
pseudo streaming), 9 days
ST-CLT-04: RTSP/MMS streaming requests (on-demand media), 9 days
ST-CLT-05: RTSP/MMS streaming requests (on-demand media), 11 days

17
 Evidences: VoD media systems
*mMoD-98 (125 MB) fat*CTVoD-04
head thin
(300 MB)tail
• mMoD-98: logs of a multicast

powered scale yc
Media-on-Demand video
server, 194 days

log scale
• CTVoD-04: streaming serer
logs of a large VoD system by
China telecom, 219 days,
reported as Zipf in
EUROSYS’06

log scale in x axis • IFILM-06: number of web

page clicks to video clips in
IFILM-06 (2.25 MB) YouTube-06 (3.4 MB) IFILM site, 16 weeks (one
week for the figure)

• YouTube-06: cumulative
number of requests to
YouTube video clips, by
crawling on web pages
publishing the data

18
 Evidences: P2P media systems
*KaZaa-02 (300 MB) *KaZaa-03 (5 MB) BT-03 (636 MB)

KaZaa-02: large video file (> 100 MB. Files smaller than 100 MB are intensively removed)
transferring in KaZaa network, collected in a campus network, 203 days.

KaZaa-03: music files, movie clips, and movie files downloading in KaZaa network, 5 days,
reported as Zipf in INFOCOM’04.

BT-03: 48 days BitTorrent file downloading (large video and DVD images) recorded by
two tracker sites
19
Evidences: Live streaming and other systems
Akamai-03 Movie-02 IMDB-06

Akamai-03: server logs of live streaming media collected from akamai CDN, 3 months,
reported as two-mode Zipf in IMC’04

Movie-02: US movie box office ticket sales of year 2002.

IMDB-06: cumulative number of votes for top 250 movies in Internet Movie Database web site

20
Why Zipf was observed in the past?

ad server

cache proxy

media server
• Media traffic is driven by user requests
• Intermediate systems may affect traffic pattern
– Effect of extraneous traffic
– Filtering effect due to caching
• Biased measurements may cause Zipf observation
21
 Extraneous media traffic
ads clip
flag clip
video prog 1
flag clip
video prog 2

ads server
ads
meta file link ads clip
clip
flag clip
video prog 1
flag clip
web
videoserver
prog 2
flag clip

ad and flag video are pushed streaming

video
program to clients mandatorily media server
22
 Effects of extraneous traffic on
reference rank distributions
• Do not represent user access patterns Reference rates

– High request rate (high popularity)

– High total number of requests
• Not necessary Zipf with extraneous traffic
– Extraneous traffic changes
– Always SE without extraneous traffic
• Small object sizes, small traffic volume prog ads flag

without extraneous
with extraneous traffic
traffic 2004: 2 objects 2005: merged
into 1 object
SE SE
2004
Zipf Non-Zipf
2005

2004 2005
23
Fundamental Differences between Zipf and SE

• “Rich-get-richer” phenomenon 3
10
BitTorrent media file
10 3 Web
Video
------ raw data

CCDF of req (log)

– Pareto, power law, …
------ linear fit

Number of distinct objects

– The structure of WWW 2
10
102

• Web accesses are Zipf

– Popular pages can attract more users
1
10 16
10 1

– Pages update to keep popular

0 100
– Zipf-like for long duration 10 0
10
1 0 100
10
1

Popularity rank
200 10
2

Time after object birth (day)

• Media accesses are big data based
– Popularity decreases with time Number of distinct weekly top N
popular objects in 16 weeks
exponentially
– Media objects are permanantly stored
Top 1 Web object never changes
– Rich-get-richer not present
Top 1 video object changes every
– Non-Zipf in long duration week

26
Dynamics of Access Patterns in Media Systems

• Media reference rank distribution in log-log scale

– Different systems have different access patterns
– The distribution changes over time in a system (NOSSDAV’02)

• All follow stretched exponential distribution

– Stretch factor c
– Minus of slope a yc c: stretch factor

• Physical meanings b slope: -a

– Media file sizes
– Aging effects of media objects
– Deviation from the Zipf model log i
27
 Stretched exponential parameters
• In a media system yc
– Constant request rate
– Constant object birth rate
b slope: -a
– Constant median file size
• Stretch factor c is a time invariant
constant log i
• Parameter a increases with time
c
  req 1 1 
a  N ( t )

  obj 1   obj t  (1  c ) 
1

c
; a  
 req  req
t : y  1
 obj   o b j  (1  1c )  29
Huge capacity for long life-span accesses

50% 50%

200 days 150 days

• Media objects have long lifespan

– Most requested objects are created long time ago
– Most requests are for objects created long time ago
• To achieve maximal concentration
– Very long time (months to years)
– Huge amount of storage
– Only large and scalable systems provide such a huge space with a long time
31
 Summary
• Media access patterns do not fit Zipf model, so doesn’t big data

• We give reasons why previous distributions were confusing

• Media access patterns are stretched exponential

• Our findings imply that

– Client-server proxy systems is not effective to deliver media contents

– Scalable distributed systems are suitable for this purpose

– The storage system in cloud systems must be very scalable

• We believe the stretched exponential model is sufficiently

general for big data accesses. 32
Two Distribution Models are Based on Different
Storage Requirements
• Accesses to big data (e.g. Internet media) follows stretched
exponential distribution
log y
The SE curve implies a wide range access fat head
distribution in long period of time.

Rank distribution:
• fat head and thin tail in log-log scale thin tail
• weak locality needs a huge and log i
distributed storage
log y

The sharp zipf slope implies a concentrated

access distribution on a small number of objects slope: -a
• strong locality makes proxy cache very effective

33
log i
Other Reported Data Access Distributions Fitting SE (after PODC’08)
Internet Video/audio services
– IPTV, user channel selection distribution (SIGMETRICS’09)

– PPLive, P2P streaming request distribution (ICDCS’09)

– Access distribution in PPStream is converting from zipf (2007) to stretched

exponential (2009) (a report from Nanjing Statistical Institute)

– USTC-VOD, Shanghai Jiading TVOD: program request distributions (China National

College Statistical Modeling Competition Outstanding award project, 09)

– User listening behavior of Bugs Music (http://www.bugs.co.kr) in Korea, 72K users,

400K songs, 15M log records (ICIS’10)

– BitTorrent Video-on-demand accesses (NSM 2010)

– VUCLIP (video service to mobile devices): access distributions in servers,

(INFOCOM’12)

– News-on-demand services from 6 Spanish newspapers (IJMA’12)

– Viewer access patterns to a large TV-on-demand system in Sweden (IMC’12)

34
– Mobile viewer access patterns to PPTV VoD system in China (IMC’12)
Other Reported Data Access Distributions Fitting SE (after PODC’08)
Social networks

– Wikipedia, Yahoo answers, social network posting distribution (KDD’09)

– digg.com, (a discussion social network), comment distribution (ICMD’10)

– ireport.com (CNN discussion social network), comment distribution (ICMD’10)

– sina, tudou, i-baidu (social networks in China), access patterns, (ICDCN’10)

– 20minutes.fr (a France news social network), access patterns (U. Paris, 12)

– YouKu (largest user generated video site in China), subscription and access
patterns (ICCCN’12, ICPP’12)

– Yahoo HK blog, posting from SE to power law after being spammed (ICDCS’12)

– Facebook Photo Serving Stack (in the backend Haystack storage), access
patterns show SE distributions (SOSP’13)

Bioinfomatics

– Protein abundances (density vs structure space, Proteome Science, 2013)35

Other Reported Data Access Distributions Fitting SE (after PODC’08)

General data accesses via Internet

– Web access patterns in American University of Nigeria in Africa (AMCIS’09)

– Access patterns to AmazingStore in China (http://www.amazingstore.org), a

P2P storage for college students to access TeraBytes of files (TPDS, 2011)

– FS2You (online storage system in China), file request distribution (INFOCOM’09)

36
Global Research Collaborations Touch the Long Tail
(1998-2013)

37
 Comparing distribution patterns between 1998 and 2013

First excise:
– Finding the collaboration data from the link

• Writing a note on the data availability

– Making the figure as US-centric (US should not be in the figure)

– Plotting the number of collections (vertical bar), sorted by the

collaborating countries with US, comparing the shape of the two figures

– Making a log operation on the data on both vertical and horizontal bars,
and compare the slope.

– Writing notes on your observations.

38
 Another view of Big Data Access Patterns: Gini Coefficient

G = A/(A+B)
Area A shrinks:
•Accesses to big data are less concentrating

•Gap of rich and poor is narrowed

39