Thanks to visit codestin.com
Credit goes to www.scribd.com

Scribd
Upload
149 views1 page

Solution of A Problem in Taylor & Karlin: An Introduction To Stochastic Modeling, 3rd Ed

This document contains a solution to an exercise problem from the textbook "Taylor & Karlin: An Introduction to Stochastic Modeling, 3rd ed". The problem describes a Markov chain with 6 states representing the outcomes of coin tosses. The solution finds the probability uH,TT that a game starting with heads will end with two tails by setting up and solving equations relating the transition probabilities between states. The calculated probability is 1/3.

Uploaded by

Ni'am Passpiss kudu'PissPiss
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
149 views1 page

Solution of A Problem in Taylor & Karlin: An Introduction To Stochastic Modeling, 3rd Ed

This document contains a solution to an exercise problem from the textbook "Taylor & Karlin: An Introduction to Stochastic Modeling, 3rd ed". The problem describes a Markov chain with 6 states representing the outcomes of coin tosses. The solution finds the probability uH,TT that a game starting with heads will end with two tails by setting up and solving equations relating the transition probabilities between states. The calculated probability is 1/3.

Uploaded by

Ni'am Passpiss kudu'PissPiss
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 1

Department of Economics

December 2005
Arne Strøm

Solution of a problem in Taylor & Karlin:


An Introduction to Stochastic Modeling, 3rd ed
Exercise III.4.5, p. 128
The problem describes a Markov chain with 6 states, and the following transition
probability matrix.

H T HH HT TH TT
1 1
0 0 0 0
 
H 2 2
1 1 
T  0 0 0 0 2 2 
 
HH  0 0 1 0 0 0 
P = 
1

1 
HT 
 0 0 0 0 2 2 
 1 1 
TH  0 0 2 2 0 0 
TT 0 0 0 0 0 1

The states H and T can only occur at the very beginning. They represent the
possible outcomes of the first toss, Heads or Tails. The other states represent the
outcomes of the two most recent tosses, e.g., HT indicates heads followed by tails.
HH and TT are absorbing states, signaling the end of the game.
In order to find the probability uH,TT that a game starting with heads will
end with two tails, we do a first step analysis. We find

uH,TT = 21 uHH,TT + 12 uHT,TT = 0 + 21 uHT,TT


uHT,TT = 12 uTH,TT + 21 uTT,TT = 12 uTH,TT + 1
2
uTH,TT = 21 uHH,TT + 12 uHT,TT = 0 + 21 uHT,TT

These equations yield uHT,TT = 32 , uTH,TT = 31 , and uH,TT = 13 .

1
solutions 13.12.2005 1332

You might also like