Data Structures
Binary Indexed Trees
TH I N K F A ST
Mostafa Saad Ibrahim
PhD Student @ Simon Fraser University
Contents:-
◼Background
◼Problem Statement
◼Motivation
◼Prefix Sum
◼Upadation
◼References
Problem Statement
◼ We often need some sort of data structure to make our
algorithms faster. The Binary Indexed Trees structure,
proposed by Peter M. Fenwick. This structure was first used
for data compression, Peter M. Fenwick. We begin by
motivating the use of this structure by an example.
◼ Consider the following problem: There are n boxes that
undergo the following queries:
1. add marble to box i
2. sum marbles from box k to box l
◼ Our goal is to implement those two queries.
Problem Statement
The naive solution has time complexity of O(1) for query 1
and O(n) for query 2.
Suppose we make m queries. The worst case (when all the
queries are 2) has time complexity O(n * m).
Using some data structure (i.e. RMQ) we can solve this
problem with the worst case time complexity of O(mlogn)
Binary Indexed Tree data structure, also with the worst time
complexity O(m log n) — but Binary Indexed Trees are
easier to code and require less memory space than RMQ.
Binary Representation
Removing bits from mask
mask = think in integer bits
Assume we have numbers X, Y
If for every position in Y with 1 AND X has 1
X = 10010100
Y = 00010100
X - Y removes all Y 1s from X
Another longer/general way to do so:
X & ~Y
10010100 &
11101011 =
10000000
Least/Most Significant Bit
Src: http://www.electronique-et-informatique.fr/Digit/images/MSBLSB.gif
Src: http://tronixstuff.com/wp-content/uploads/2011/05/binnum.jpg
Two’s Complement Representation
Start to flip AFTER the “last bit”
-number = 2’s complement of number
One way to compute manually:
Get 1’s complement...then add 1
Removing Last Bit
Get last bit using index & -index
+20 = 00010100
-20 = 11101100
20 & -20 = 00000100
Remove last bit
Get it...subtract it
index - (index & -index)
00010100 - 00000100 = 00010000
Integer as sums of powers of 2
= 010010011
Src: http://images.slideplayer.com/5/1546412/slides/slide_22.jpg
Our problem
Let’s move to our problem
Given an array of integer N
Assume index 0 always will be 0 (NOT in use)
2 query types:
Add value val to position index
Sum values from 1 to index
Segment Tree can be used to such problem
O(N) preprocess, O(NlogN) queries, O(nlogn) memory
BIT has a better memory order (shorter code)
O(n) memory + O(NlogN) queries
Problem Example
0 1 2 3 4 5 6 7 8 9 10 11
xx 3 2 -1 6 5 4 -3 3 7 2 3
- Accumulative Sum (1, 3): 3 + 2 - 1 = 4
- Accumulative Sum (1, 5): 3 + 2 - 1 + 6 + 5 = 15
- Add: index 3, value = 4
xx 3 2 4 6 5 4 -3 3 7 2 3
- AccumulativeSum (1, 3): 3 + 2 + 4 = 9
- Accumulative Sum (1, 5): 3 + 2 + 4 + 6 + 5 = 21
Motivation
Integer = Sum of Powers of 2
Accumulative Sum = Sum of Sub sums
Recall: 147 = 128 + 16 + 2 + 1
Think in accumulative sum (1 to 147)
Sum of last 1 number +
Sum of next 2 numbers +
Sum of next 16 numbers +
Sum of next 128 numbers
Sum(1,147) =
Sum(147,147) + Sum(146,145) + Sum(144,129) + Sum(128,1)
147 ⇒ positions {147, 146, 144, 128} with ranges {1, 2, 16,
128}
Motivation
◼ To get starting positions fast? Remove last bit
▪ 147 = 010010011 [remove last 1 bit]
▪ 146 = 010010010 [remove last 1 bit]
▪ 144 = 010010000 [remove last 1 bit]
▪ 128 = 010000000 [remove last 1 bit]
▪ 0 = DONE
◼ How to interpret:
▪ 147 responsible for range 147 to > 146
▪ 146 responsible for range 146 to > 144
▪ 144 responsible for range 144 to > 128
▪ 146 responsible for range 128 to > 0
Binary Indexed Tree
Create a new array of Length N, name it BIT
BIT[position] = sum of its responsible range
Then For each Query
Sum(147)= BIT(147) + BIT(146) + BIT(144) + BIT(128)
That is: 4 steps only to get the answer
Sum(144) = BIT(144) + BIT(128)
Sum(15) = BIT(15) + BIT(14) + BIT(12) + BIT(8)
Recall: 1111 = 1111, 1110, 1100, 1000, 0
Sum(11) = BIT(11) + BIT(10) + BIT(8)
Recall 1011: 1011, 1010, 1000, 0
Sum(7) = BIT(7) + BIT(6) + BIT(4) ⇒ 111, 110, 100, 0
Binary Indexed Tree
Sum(15) = BIT(15) + BIT(14) + BIT(12) + BIT(8)
E.g. node 12 has values: BIT[12] = val[12] + val[11]+val[9]
12 = 1100 ⇒ removing last 1 bit ⇒ 1000 = 8
Then parent of 12 ⇒ 8 (e.g. next closest position 12 is not covering)
Notice: we removed bit at position 2 ⇒ 12 covers 2^2 numbers = 12 - 8 = 4
Binary Indexed Tree
Notice: 8 = 1000 => has 3 trailing zeros. Try to replace each 0 with 1
1001 = 9
1010 = 10
1100 = 12
# of trailing zeros = # children … child remove last bit => go to parent
Get Interval Accumulation
Sum(15) = BIT(15) + BIT(14) + BIT(12) + BIT(8)
= 1111 ⇒ 1110 ⇒ 1100 ⇒ 1000 ⇒ 0 = STOP
15 is responsible for 1 number, 14 for 2, 12 for 4, 8 for 8 numbers
Updating position
Position 1: Covered by 4 intervals ⇒ 1, 2, 4, 8
Add -3 to 1 ⇒ add -3 to these 4 intervals
Given index, how to get smallest position covering it?
E.g. 1 ⇒ 2 6 ⇒ 8 10 ⇒ 12 13 ⇒ 14
Then 1 goes to 2...2 goes to 4..4 goes to 8 [recursive]
Updating position
◼ Recall given number idx it covers 2^r values
▪ r is position of “last bit”
▪ It covers numbers from idx to idx – 2^r + 1
◼ All following numbers cover 8 values
▪ 0001000 ⇒ r = 3 ⇒ 2^3 = 8
▪ 0101000
▪ 1101000
▪ 1111000
▪ 1001000
◼ So our focus on “last bit”, NOT before that
Updating position
◼ 1000 covers 8 numbers
▪ 1000 - 000 = 1000
▪ 1000 - 001 = 0111
▪ 1000 - 010 = 0110
▪ 1000 - 011 = 0101
▪ 1000 - 100 = 0100
▪ 1000 - 101 = 0011
▪ 1000 - 110 = 0010
▪ 1000 - 111 = 0001
◼ Each of these 8 numbers covered by 1000
◼ But 1000 is NOT their smallest cover number
Updating position
◼ Let’s get who covers 4 = 0100
▪ 4 has “last bit” at k = 2
▪ When target number enumerate its 2^r, one contains 100
▪ So we need to go at least 1 bit higher than k
▪ E.g. re-set last bit k = 3 ⇒ 1000 ⇒ first one to cover 0100
◼ Let’s get who covers 5 = 0101
▪ k=0
▪ We need target number to include our 1 at k = 0
▪ The earlier one should exist in smallest coverer number
▪ So again, shift k = 0 1 step to be in its enumeration
▪ E.g. re-set last bit k = 1 ⇒ 110. Note, 1000 also cover 5
Updating position
◼ Let’s get who covers 3 = 0011
▪ “last bit” at k = 0
▪ We need enumeration includes whole 11
▪ So parent need to be a 1 before these 11
▪ E.g. ⇒ 0100
◼ So general rule
▪ 100100001000 100100011100
▪ 100100010000 100100100000
◼ How to get that number easily?
▪ Just add 2^k ⇒ if one or more bits ⇒ shifted
▪ E.g. 100100011100 + 000000000100 = 100100100000
Updating tree
Updating tree
Notice: 8 = 1000 => has 3 trailing zeros. Remove last bit, and add 1, 2, 3...trailing ones
0100 = 4
0110 = 6
0111 = 7
# of trailing zeros = # children
Updating tree
Building initial tree from input: just iterate on input
and add it to its position
References
Article on Prefix Sums and Their Applications by Guy E.
Blelloch School of Computer Science
Carnegie Mellon University Pittsburgh, PA 15213-3890
Efficient Range Minimum Queries using Binary Indexed
Trees :
Mircea Dima and Rodica Ceterchi
Olympiads in Informatics, 2015, Vol. 9, 39–44
DOI:http://dx.doi.org/10.15388/ioi.2015.04
