ِالرحيم
َ ّ حم ِن َ ّ ِبس ِم الل َّ ِه
ٰ الر
Markov Analysis
Dr. Mohammed Shamim Uddin Khan
Professor and Ex-Chairman
Department of Finance
University of Chittagong
INTRODUCTION-1
Markov analysis, like decision analysis is a
probabilistic technique. However, Markov
analysis is different in that it does not provide a
recommended decision.
Instead, Markov analysis provides probabilistic
information about decision situation that can aid
the decision maker in making a decision.
In other words, Markov analysis is not an
optimization technique; it is a descriptive
technique that results in probabilistic information.
INTRODUCTION-2
Markov analysis is a method of analyzing the current
behaviour of some variable in an effort to predict the future
behaviour of that same variable.
This procedure was developed by the Russian mathematician
Andrel A. Markov early in this century. He first used it to
describe and predict the behaviour of practices of gas in a
closed container.
As a management tool, Markov analysis has been
successfully applied to a wide variety of decision situations.
Perhaps it widest use is in examining and predicting the
behaviour of consumers in terms of their brand loyalty and
their switching from one brand to another.
INTRODUCTION-3
Another interesting application if Markov analysis has been to
the study of the life newspaper subscriptions.
A more recent application of this technique has been to the
study of accounts receivable behaviour, that is to the study of
customers as they change from ‘current account’ through ’30
days overdue’, to ’60 days overdue’ and then to ‘bad debt’.
Thus Markov analysis can be useful for describing the
behaviour of a certain class of systems that change from state
to state on a period by period basis according to known
transition probabilities.
Consumer buying patterns, market shares and equipment
breakdowns sometimes lend themselves to description in
Markov terms.
INTRODUCTION-4
Markov analysis uses the following three
approaches:
1. Tree Diagram
2. Matrix Multiplication and
3. Algebraic Approach.
The first two approaches are especially useful for
describing short term system behaviour whereas
the third approach is more appropriate for
describing long or steady-state behaviour.
CHARACTERISTICS OF MARKOV ANALYSIS
1. The transition probabilities for a given beginning
state of the system sum to one.
2. The probabilities apply to all participants in the
system.
3. The transitions probabilities are constant over
time.
4. The states are independent over time.
5. The states are both mutually exclusive and
collective exhaustive.
6. It will operate or exist for a finite number of
periods.
ASSUMPTION OF MARKOV ANALYSIS
Markov analysis is predicted on a number of
important assumptions. These are:
1. The probability that an item in the system either
will change from one state to another or remain
in its current state is a function of the transition
probabilities only.
2. The transition probabilities remain constant.
3. The system is a closed one, there will be no
arrivals to the system or exist from the system.
FUNDAMENTAL MATRIX
If the probability of the process remaining in a particular state at
each transition equals one (1) the state is called an absorbing
state. For a Markov process which has some absorbing and some
non-absorbing states, the transition matrix can be thought of
being comprised of four sub-matrix
The upper left-hand square sub-matrix I, corresponding to the
absorbing states, is an identity matrix. The upper right-hand sub-
matrix, O, is a zero matrix. The lower right hand sub-matrix, Q,
represents the transition probabilities between the non-absorbing
states. The inverse matrix of the difference between the identity
matrix and the lower right hand sub-matrix, N = (I – Q) is called
the fundamental matrix.
Problem 1: Assume the brand switching matrix for
two products C and D is
Requirements:
(a) Compute P2, P3 and P4.
(b) Compute expected brand shares for week 2, 3
and 4 if initial brand shares probabilities are 0.45
and 0.55 for products C and D respectively.
(c) If the market consists of 2000 consumers and
each unit of product C sells for $15 and each unit
of product D sells $10, forecast total (cumulative)
sales volume over the weeks 2, 3 and 4.
PROBLEM 1 CONT.
Solution: (a)
PROBLEM 1 CONT.
(b) We will write S1 and S2 as decimals 0.45 and 0.55
respectively. The expected brand shares for week 2
will be
= =
For weeks 3 will be
= =
For weeks 4 will be
= =
PROBLEM 1 CONT.
(c) Forecast sales volume in week 2:
Total customers 2000:
Brand C sells $15 per unit
Brand D sells $10 per unit
For C:
Brand shares of C in week 2 X Total customers
= 0.212 X 2000 = 424
For D:
Brand shares of D in week 2 X Total customers
= 0.788 X 2000 = 1576
Total sales dollar in week 2 = (424X15+1576X10) = $22,120.
PROBLEM 1 CONT.
Forecast sales volume in week 3:
For C:
Brand shares of C in week 3 X Total customers
= 0.1848 X 2000 = 369.6
For D:
Brand shares of D in week 3 X Total customers
= 0.8152 X 2000 = 1630.4
Total sales dollar in week 3
= (369.6 X15 + 1630.4X10) + 22,120 = $43,968
PROBLEM 1 CONT.
Forecast sales volume in week 4:
For C:
Brand shares of C in week 4 X Total customers
= 0.17392 X 2000 = 347.84
For D:
Brand shares of D in week 4 X Total customers
= 0.82608 X 2000 = 1652.16
Total sales dollar in week 4
= (347.84X15 + 1652.16X10) + 43,968 = $65,707.2
[Note: Assuming each customer makes a purchase of either
brand C or D once in a week.]
Problem 2: Find the steady state probabilities for the
following transition
From/To A
matrix:
B C
A 0.5 0.4 0.1
B 0.2 0.5 0.3
C 0.3 0. 0.6
Solution: The steady equations are derived from column
probabilities of each state. Thus we get,
A = 0.5A + 0.2B + 0.3C............. (i)
B = 0.4A + 0.5B + 0.1C............. (ii)
C = 0.1A + 0.3B + 0.6C............. (iii)
In addition, we that the three probabilities must sum to one (1).
1 = A + B + C............................. (iv)
PROBLEM 2 CONT.
We can use this to solve for one of the three probabilities in
terms of the other two.
For convenience, let’s choose C. Then we have from (iv)
C=1–A–B
With three unknowns, we need only three equations. We can
estimate the steady–state equations for C and then substitute
this last equation for C in the first two steady state equations.
That is
A = 0.5A + 0.2B + 0.3(1 – A – B)
B = 0.4A + 0.5B + 0.1(1 – A – B)
Expanding the first of these yields
A = 0.5A + 0.2B + 0.3 – 0.3A – 0.3B
PROBLEM 2 CONT.
Combining terms and moving the variables to the left side of the
equation yields
0.8A + 0.1B = 0.3
Expanding the steady state equation for B yields
B = 0.4A + 0.5B + 0.1 – 0.1A – 0.1B
Combining terms and moving the variables to the left side of the
equation yields
- 0.3A + 0.6B – 1
Thus two resulting equations are:
0.8A + 0.1B = 0.3
- 0.3A + 0.6B = 1
Solving simultaneously we get A = 0.422 and B = 0.376
Given these values and that 1 = A + B + C we can determine that
C = 0.202.
Problem 3: On January 1, 2020, Bakery A had 40% of its
local market share while the other two bakeries B and
C had 40% and 20% respectively of the market share.
Based upon a study by a market research firm, the
following facts were compiled: Bakery A retains 90%
of its customers while gaining 5% of competitors B’s
customers and 10% of C’s customers. Bakery B retains
85% of its customers while gaining 5% of A’s
customers and 7% of C’s customers. Bakery C retains
83% of its customers and gains 5% of A’s customers as
well as 10% of B’s customers. What will each firm’s
share on January 1, 2021 and what will each firm’s
market share be at equilibrium?
Solution: Using the data of the given problem we
formulate the state-transaction matrix:
PROBLEM 3 CONT.
From/To A B C
A 0.90 0.05 0.05
B 0.05 0.85 0.10
C 0.10 0.07 0.83
On January, 1 2020, the markets hares of the three
bakeries are 40%, 40% and 20% respectively. The
management of three bakeries is interested in knowing
their market shares on January 1, 2021. The expected
market shares for Bakery A, B, and C on January 1,
2021 are computed as below:
(Market share on January 1, 2020) (State transition matrix) =
(Expected market share on January 1, 2021)
PROBLEM 3 CONT.
(0.40 0.40 0.20) = (0.400 0.374 0.226)
Thus, the market shares of Bakery A, B and C on January 1, 2021 will
be 40%, 37.4% and 22.6% respectively.
2nd Part: Equilibrium market shares x, y and z for the three Bakeries
must satisfy the following relation:
(x y z) = (x y z)
where x + y + z = 1 and x0, y0, z0
This gives x = 0.90x + 0.05y + 0.10z
y = 0.05x + 0.85y + 0.07z
z = 0.05x + 0.10y + 0.83z
PROBLEM 3 CONT.
The equations can be written as
0.10x + 0.05y + 0.10z = 0………… ….(i)
0.05x – 0.15y + 0.07z = 0……………….(ii)
0.05x + 0.10y – 0.17z = 0……………….(iii)
x + y + z = 1………………………….. (iv)
x = 1 – y – z…………………….. (v)
Now putting the value of x [from (v)] in equation (ii)
& (iii) and we get
0.05(1-y-z) – 0.15y + 0.07z = 0
-0.20y + 0.02z = -0.05……………….(vi)
PROBLEM 3 CONT.
and 0.05(1-y-z) + 0.10y – 0.17z = 0
0.05y – 0.22z = 0.05……………….(vii)
(vi) + 4(vii) we get, Z = 0.29
Putting the value of z in (vi) we get y = 0.28
Further we put the value of y and z in (iv) and get
x = 0.43
Thus ,solution is x = 0.43, y = 0.28, and z = 0.29
Hence the three Bakeries market shares at equilibrium are:
Bakery A: 43% of the total market share, Bakery B: 28% of
the total market share, and Bakery C: 29% of the total market
share.
Problem 4: A manager has developed the following transition matrix
for a firm’s accounts receivable:
Where p = paid; 1 = 1 to 30 days overdue; 2 = 31 to 60 days
overdue; b = bad debt
Accounts are billed weekly but classified in terms of months
overdue. Consequently, it is possible for an account to remain in
either the 1 or 2 category for several periods. Therefore, there is
non-zero probability of remaining in either 1 or 2.
Requirements:
Obtain the fundamental matrix.
If there is currently $ 10,000 in accounts in the 1 category and $
6000 in the 2 category, determine the expected amount of bad debt.
PROBLEM 4 CONT.
Solution: By arranging the transition matrix so that the
two absorbing states (p and b) listed first, followed
by the two non-absorbing states. This yields the
matrix:
Then, partition the matrix into four parts that are
defined as,1 I,0 O, R,Q
0 0 0.5 0 0.3 0.2
Where, I , O , R , Q
0 1 0 0 0.3 0.3 0 0.4
Now, (I– Q) = –
I Q 0.42 =
PROBLEM 4 CONT.
The co-factor of (I– Q) is (I – Q) C =
(I – Q) –1 = 1 =F
0.42
This is the fundamental matrix F.
First, multiply matrix R by the fundamental matrix:
FR = =
Thus, the expected amount of paid and bad debt can be
calculated as:
( 10,000 6,000) = (11,570 4,430)
Hence, the expected amount of bad debt is $ 4,430.
Problem 5: A pottery maker specializes in the manufacture
of intricately designed handcrafted flower pots and has
hired three apprentices to work in his shop to
manufacture the pots. Each pot is considered to be in one
of four categories during any given day: finished (and
ready for shipment), damaged beyond rework capability,
in the process of manufacture; or being reworked (to
correct defects which occur during the in-process stage).
The transition matrix describing the status of a typical
flower pot is given in the following table (the transition
time period is one day)
Finished Damaged In Process Rework
Finished 1.0 0.0 0.0 0.0
Damaged
0.0 1.0 0.0 0.0
In Process 0.5 0.0 0.25 0.25
Rework 0.5 0.2 0.05 0.25
PROBLEM 5 CONT.
Requirements:
(i) If a flower pot is in process, what is the
probability that it will eventually be finished? That
it will be damaged beyond repair?
(ii) Answer the same question for a flower pot that is
being reworked.
(iii) If there are currently 10 flower pots in process
and 3 flower pots in rework and if each finished
flower pot yields an $75 profit and each damaged
pot results in a $20 loss, what is the expected net
profit on the current work?
PROBLEM 5 CONT.
Solution: We are given
Finished Damage d In Process Rework
Finished
1.0 0.0 0.0 0.0
Damaged
0.0 1.0 0.0 0.0
In Process 0.40 0.25 0.20 0.15
Rework 0.50 0.30 0.0
0.20
Now, (I– Q) = =
--
I Q 0.64
The co-factor o (I– Q) is (I – Q) C =
(I – Q) –1 = 1
0.64
= This is the fundamental matrix F
F;
PROBLEM 5 CONT.
First, multiply matrix R by the fundamental matrix:
FR = =
Requirement (i): The probability of finished pot is 62% and
damaged pot is 38% if the system is in-process.
Requirement (ii): The probability of finished pot is 63% and
damaged pot is 37% if the system is being reworked.
Requirement (iii): We have to determine the number of finished
pot and damaged pot is
Hence the expected profit is $(75X8.09 – 20X4.91)
= $508.55
