Easy Data Parallelism
Richard Warburton
Raoul-Gabriel Urma
Overview
● Why is parallelism Important?
● What is data parallelism?
● Parallelising your Streams
● Performance and Internals
Why is Parallelism important?
source: http://www.gotw.ca/images/CPU.png
Multicore
What is Data Parallelism?
Concurrency is not Parallelism!
● Concurrency
○ At least two threads are making progress
○ May not run at the same time
○ Eg: chrome and eclipse both running
● Parallelism
○ At least two threads are executing simultaneously
○ A specific case of concurrency
○ Eg: servlet container dealing with two users at
once on a multicore machine
Parallelism
● Task
○ Distribute execution processes over processes
○ Threads and Executors in Java
○ Eg: each thread services a user in JEE App
● Data
○ Distribute data over different processes
○ Support built on top of Streams
○ Eg: process a payroll and give each core 100
employee’s salary
What are good data parallel
problems?
● Big Batch Jobs
○ Transaction Processing
○ Analytics/Reporting
● Web crawlers / parsers
● Maths
○ Monte Carlo Simulations
○ Linear Algebra
What’s a good data parallel problem from your
workplace?
Parallelising your Streams
Data Parallelism
● Useful
○ a lot of data
○ want to process in a similar way
● API aims to be explicit, but unobtrusive
○ .parallelStream()
○ .parallel()
● Can flip between sequential and parallel
Data Parallelism
// Replace stream() with parallelStream()
Set<String> origins = musicians
.parallelStream()
.filter(artist -> artist.getName().startsWith("The"))
.map(artist -> artist.getNationality())
.collect(toSet());
Not all serial code works in parallel.
DON’T interfere with data sources
// add double each value into a list.
List<Integer> numbers = getNumbers();
numbers.parallelStream()
.forEach(i -> numbers.add(i * 2));
Referring to data sources fixed
// add double each value into a list.
List<Integer> numbers = getNumbers();
numbers = numbers.parallelStream()
.flatMap(i -> Stream.of(i, i * 2))
.collect(toList());
DON’T misuse reduce
int totalCost(List<Purchase> items) {
return items.parallelStream()
.reduce(DELIVERY_FEE,
(tally, item) -> tally + item.cost());
}
Associativity
“you can flip order around and things still work”
(4 + 2) + 1 = 4 + (2 + 1) = 7
(4 * 2) * 1 = 4 * (2 * 1) = 8
Identity
“the do nothing value”
0 + 5 = 5
1 * 5 = 5
How to fix reduce
int totalCost(List<Purchase> items) {
return DELIVERY_FEE
+ items.parallelStream()
.reduce(0,
(tally, item) -> tally + item.cost());
}
How to fix reduce (2)
int totalCost(List<Purchase> items) {
return DELIVERY_FEE
+ items.parallelStream()
.mapToInt(Purchase::getCost)
.sum();
}
DON’T hold locks
List<Integer> values = getValues();
CountDownLatch latch = new CountDownLatch(values.size());
values.parallelStream()
.forEach(i -> {
try {
doSomething(i);
// Potential Deadlock
latch.countdown();
} catch (Exception e ) {
e.printStackTrace();
}});
No mutable state!
public static long sideEffectParallelSum(long n) {
Accumulator accumulator = new Accumulator();
LongStream.rangeClosed(1,n).parallel()
.forEach(accumulator::add);
return accumulator.total;
}
public static class Accumulator {
private long total = 0;
public void add(long value) {
total += value;
}
}
Parallel Code Summary
● Very easy to make your code parallel,
but …
● Sometimes you can get away with things
sequentially that you can’t in parallel
○ sources
○ reduce
○ locks
○ unprotected mutable data
Performance and Internals
Under the hood
● Work distributed using Fork/Join framework
● Distributed by data
● New abstraction: Spliterator
Parallel Integer Sums
int sum =
values.parallelStream()
.mapToInt(i -> i)
.sum();
Spliterator
public interface Spliterator<T> {
/** Carve off a portion of the data
into a separate Spliterator */
Spliterator<T> trySplit();
/** Iterate the data described by this Spliterator */
void forEachRemaining(Consumer<? super T> action);
/** The size of the data described
by this Spliterator, if known */
long getExactSizeIfKnown();
}
Always a tradeoff ...
● Parallelism eats more CPU time
○ Thread communication
○ Distributing & Decomposing work
○ Potentially increased memory pressure
○ Competing for the CPU with other processes
● It can reduce wall time
○ Time from beginning to end of the processes’
execution
○ Ideally only need to wait for 1/N of the execution
time
Decomposition Performance
● Data Size
● Source Data Structure
● Packing
● Number of Cores
● Cost per Element
Data Structures
● Good
○ ArrayList / Intstream.range / Stream.of
○ Random Access + Easy to balance
● Meh
○ Hashset / Treeset
○ Usually good balance
● Bad
○ LinkedList / BufferedReader.lines() /
Streams.iterate()
○ Unknown length
○ bad random access performance
Stateful Operations
● Stateless
○ no need to keep state when evaluated
○ eg: map, reduce
○ superior parallel decomposition
○ bounded amounts of data
● Stateful
○ accumulate state during evaluation
○ eg: sorted
○ unbounded caching of data
Benchmarking and Testing
● Don’t assume parallel = faster, measure it
● Use jmh:
http://openjdk.java.net/projects/code-tools/jmh/
● Best Practices
○ Warmup
○ Repeatability
○ Evade the JIT
Summary
Lesson Summary
● Easy to obtain Data Parallelism
● Pick your situation well
● A lot of performance influencers
● Benchmark your parallel code
The End
Exercise
In: com.java_8_training.problems.data_parallelism
1. Looks at OptimisationExample
2. Try to improve the performance of this code
3. Measure performance using the benchmark harness
4. Don’t make the code uglier!
Exercise
In: com.java_8_training.problems.data_parallelism
1. Parallelise the sum of squares method
Question1Test
2. Fix the bug in the "multiplyThrough" method
Question2Test
3. Remove the locks and keep the code safe
Question3Test
Amdahl’s Law
● Defines upper bound for parallel speedup
● Time(n) = Time(1) * (s + 1/n * (1 - s))
○ n = number of cores
○ s = proportion of code that is strictly serial
● Speedup(n) = 1 / (s + 1/n * (1 - s))
● Example
○ 1024 cores, 50% serial
○ 1 / (0.5 + 1/1024 * (1 - 0.5)) ~= 2x speedup
