The Markov C
The Markov Chain with one-step transition matrix is given below.
P =
1. The above transition matrix is right stochastic if the sum of each row is 1 or it can be right stochastic if the sum of each column is 1 or it can be said that the matrix is doubly stochastic if both sum of row and column sums up to 1. Now, a MATLAB script is written to find the sum of each row and column.
MATLAB code:
P = [0.25 0.35 0.15 0 0.25;0.15 0.2 0.4 0.1 0.15;0.1 0.35 0.05 0.3 0.2;0.55 0 0.25 0.15 0.05;0 0.6 0.25 0 0.15];
for i=1:length(P)
scol(i) = sum(P(:,i));
srow(i) = sum(P(i,:));
end
srow
scol = scol’
Output:
sumprob
srow =
1 1 1 1 1
scol =
1.0500
1.5000
1.1000
0.5500
0.8000
Hence, as the row sum is 1 hence the state transition matrix is right stochastic only.
2. The nth step of Markov chain simulation is given by,
Where, is the probability vector after nth step.
= initial state distribution.
n = simulation number.
MATLAB code:
P = [0.25 0.35 0.15 0 0.25;0.15 0.2 0.4 0.1 0.15;0.1 0.35 0.05 0.3 0.2;0.55 0 0.25 0.15 0.05;0 0.6 0.25 0 0.15]; % state transition matrix
pi0 = [3/15 2/15 4/15 5/15 1/15]; % initial state distribution
n = 10000; % number of steps is 10000
fprob = pi0*(P^n); % final state probability matrix
row_sum = sum(fprob);
fprob
row_sum
Output:
EE380_Exp10_A
fprob =
0.1781 0.3050 0.2316 0.1176 0.1676
row_sum =
1.0000
Hence, it can be seen that after 10000 simulation the sum of probabilities in the row is equal to 1. Hence, the final state transition matrix is also right stochastic.
3. Now, the final probability array is generated after 10000 simulations for 20 randomly chosen initial distributions. The initial distributions are chosen from uniform distribution having values between [0,1].
MATLAB code:
P = [0.25 0.35 0.15 0 0.25;0.15 0.2 0.4 0.1 0.15;0.1 0.35 0.05 0.3 0.2;0.55 0 0.25 0.15 0.05;0 0.6 0.25 0 0.15]; % state transition matrix
for i=1:20
pi0 = rand(1,length(P)) % initial state distribution chosen randomly from uniform distribution
n = 10000; % number of steps is 10000
fprob = pi0*(P^n) % final state probability matrix
row_sum = sum(fprob)
end
Output:
EE380_Exp10_B
pi0 =
0.6557 0.0357 0.8491 0.9340 0.6787
fprob =
0.5617 0.9617 0.7303 0.3709 0.5286
row_sum =
3.1533
pi0 =
0.7577 0.7431 0.3922 0.6555 0.1712
fprob =
0.4845 0.8295 0.6299 0.3199 0.4559
row_sum =
2.7198
pi0 =
0.7060 0.0318 0.2769 0.0462 0.0971
fprob =
0.2063 0.3532 0.2682 0.1362 0.1941
row_sum =
1.1581
pi0 =
0.8235 0.6948 0.3171 0.9502 0.0344
fprob =
0.5024 0.8601 0.6532 0.3317 0.4727
row_sum =
2.8201
pi0 =
0.4387 0.3816 0.7655 0.7952 0.1869
fprob =
0.4574 0.7832 0.5948 0.3021 0.4305
row_sum =
2.5679
pi0 =
0.4898 0.4456 0.6463 0.7094 0.7547
fprob =
0.5426 0.9289 0.7054 0.3583 0.5106
row_sum =
3.0457
pi0 =
0.2760 0.6797 0.6551 0.1626 0.1190
fprob =
0.3371 0.5772 0.4383 0.2226 0.3172
row_sum =
1.8924
pi0 =
0.4984 0.9597 0.3404 0.5853 0.2238
fprob =
0.4645 0.7953 0.6039 0.3067 0.4371
row_sum =
2.6076
pi0 =
0.7513 0.2551 0.5060 0.6991 0.8909
fprob =
0.5526 0.9462 0.7185 0.3649 0.5200
row_sum =
3.1023
pi0 =
0.9593 0.5472 0.1386 0.1493 0.2575
fprob =
0.3655 0.6258 0.4753 0.2414 0.3440
row_sum =
2.0519
pi0 =
0.8407 0.2543 0.8143 0.2435 0.9293
fprob =
0.5490 0.9400 0.7138 0.3625 0.5167
row_sum =
3.0821
pi0 =
0.3500 0.1966 0.2511 0.6160 0.4733
fprob =
0.3361 0.5755 0.4371 0.2220 0.3163
row_sum =
1.8870
pi0 =
0.3517 0.8308 0.5853 0.5497 0.9172
fprob =
0.5762 0.9865 0.7492 0.3805 0.5422
row_sum =
3.2347
pi0 =
0.2858 0.7572 0.7537 0.3804 0.5678
fprob =
0.4890 0.8372 0.6358 0.3229 0.4602
row_sum =
2.7450
pi0 =
0.0759 0.0540 0.5308 0.7792 0.9340
fprob =
0.4229 0.7240 0.5498 0.2792 0.3979
row_sum =
2.3738
pi0 =
0.1299 0.5688 0.4694 0.0119 0.3371
fprob =
0.2703 0.4627 0.3514 0.1785 0.2543
row_sum =
1.5171
pi0 =
0.1622 0.7943 0.3112 0.5285 0.1656
fprob =
0.3495 0.5983 0.4544 0.2308 0.3289
row_sum =
1.9619
pi0 =
0.6020 0.2630 0.6541 0.6892 0.7482
fprob =
0.5267 0.9017 0.6847 0.3478 0.4956
row_sum =
2.9564
pi0 =
0.4505 0.0838 0.2290 0.9133 0.1524
fprob =
0.3258 0.5578 0.4236 0.2151 0.3066
row_sum =
1.8291
pi0 =
0.8258 0.5383 0.9961 0.0782 0.4427
fprob =
0.5132 0.8787 0.6673 0.3389 0.4830
row_sum =
2.8811
Hence, ti can be seen from the above simulations with 20 restarts that the sum of row probabilities are not equal to 1. Only, the sum is closest to 1 (1.1581) when the initial distribution is
Hence, if the sum of probabilities in the initial distribution is equal to 1 then the final probabilities after simulation of Markov chain will be equal to 1.
4.
Now, distribution is stationary as the sum of the probabilities of initial distribution is not equal to 1 and the equation for being stationary distribution is not satisfied. The equation for stationary distribution is
hain with one-step transition matrix is given below.
P =
1. The above transition matrix is right stochastic if the sum of each row is 1 or it can be right stochastic if the sum of each column is 1 or it can be said that the matrix is doubly stochastic if both sum of row and column sums up to 1. Now, a MATLAB script is written to find the sum of each row and column.
MATLAB code:
P = [0.25 0.35 0.15 0 0.25;0.15 0.2 0.4 0.1 0.15;0.1 0.35 0.05 0.3 0.2;0.55 0 0.25 0.15 0.05;0 0.6 0.25 0 0.15];
for i=1:length(P)
scol(i) = sum(P(:,i));
srow(i) = sum(P(i,:));
end
srow
scol = scol’
Output:
sumprob
srow =
1 1 1 1 1
scol =
1.0500
1.5000
1.1000
0.5500
0.8000
Hence, as the row sum is 1 hence the state transition matrix is right stochastic only.
2. The nth step of Markov chain simulation is given by,
Where, is the probability vector after nth step.
= initial state distribution.
n = simulation number.
MATLAB code:
P = [0.25 0.35 0.15 0 0.25;0.15 0.2 0.4 0.1 0.15;0.1 0.35 0.05 0.3 0.2;0.55 0 0.25 0.15 0.05;0 0.6 0.25 0 0.15]; % state transition matrix
pi0 = [3/15 2/15 4/15 5/15 1/15]; % initial state distribution
n = 10000; % number of steps is 10000
fprob = pi0*(P^n); % final state probability matrix
row_sum = sum(fprob);
fprob
row_sum
Output:
EE380_Exp10_A
fprob =
0.1781 0.3050 0.2316 0.1176 0.1676
row_sum =
1.0000
Hence, it can be seen that after 10000 simulation the sum of probabilities in the row is equal to 1. Hence, the final state transition matrix is also right stochastic.
3. Now, the final probability array is generated after 10000 simulations for 20 randomly chosen initial distributions. The initial distributions are chosen from uniform distribution having values between [0,1].
MATLAB code:
P = [0.25 0.35 0.15 0 0.25;0.15 0.2 0.4 0.1 0.15;0.1 0.35 0.05 0.3 0.2;0.55 0 0.25 0.15 0.05;0 0.6 0.25 0 0.15]; % state transition matrix
for i=1:20
pi0 = rand(1,length(P)) % initial state distribution chosen randomly from uniform distribution
n = 10000; % number of steps is 10000
fprob = pi0*(P^n) % final state probability matrix
row_sum = sum(fprob)
end
Output:
EE380_Exp10_B
pi0 =
0.6557 0.0357 0.8491 0.9340 0.6787
fprob =
0.5617 0.9617 0.7303 0.3709 0.5286
row_sum =
3.1533
pi0 =
0.7577 0.7431 0.3922 0.6555 0.1712
fprob =
0.4845 0.8295 0.6299 0.3199 0.4559
row_sum =
2.7198
pi0 =
0.7060 0.0318 0.2769 0.0462 0.0971
fprob =
0.2063 0.3532 0.2682 0.1362 0.1941
row_sum =
1.1581
pi0 =
0.8235 0.6948 0.3171 0.9502 0.0344
fprob =
0.5024 0.8601 0.6532 0.3317 0.4727
row_sum =
2.8201
pi0 =
0.4387 0.3816 0.7655 0.7952 0.1869
fprob =
0.4574 0.7832 0.5948 0.3021 0.4305
row_sum =
2.5679
pi0 =
0.4898 0.4456 0.6463 0.7094 0.7547
fprob =
0.5426 0.9289 0.7054 0.3583 0.5106
row_sum =
3.0457
pi0 =
0.2760 0.6797 0.6551 0.1626 0.1190
fprob =
0.3371 0.5772 0.4383 0.2226 0.3172
row_sum =
1.8924
pi0 =
0.4984 0.9597 0.3404 0.5853 0.2238
fprob =
0.4645 0.7953 0.6039 0.3067 0.4371
row_sum =
2.6076
pi0 =
0.7513 0.2551 0.5060 0.6991 0.8909
fprob =
0.5526 0.9462 0.7185 0.3649 0.5200
row_sum =
3.1023
pi0 =
0.9593 0.5472 0.1386 0.1493 0.2575
fprob =
0.3655 0.6258 0.4753 0.2414 0.3440
row_sum =
2.0519
pi0 =
0.8407 0.2543 0.8143 0.2435 0.9293
fprob =
0.5490 0.9400 0.7138 0.3625 0.5167
row_sum =
3.0821
pi0 =
0.3500 0.1966 0.2511 0.6160 0.4733
fprob =
0.3361 0.5755 0.4371 0.2220 0.3163
row_sum =
1.8870
pi0 =
0.3517 0.8308 0.5853 0.5497 0.9172
fprob =
0.5762 0.9865 0.7492 0.3805 0.5422
row_sum =
3.2347
pi0 =
0.2858 0.7572 0.7537 0.3804 0.5678
fprob =
0.4890 0.8372 0.6358 0.3229 0.4602
row_sum =
2.7450
pi0 =
0.0759 0.0540 0.5308 0.7792 0.9340
fprob =
0.4229 0.7240 0.5498 0.2792 0.3979
row_sum =
2.3738
pi0 =
0.1299 0.5688 0.4694 0.0119 0.3371
fprob =
0.2703 0.4627 0.3514 0.1785 0.2543
row_sum =
1.5171
pi0 =
0.1622 0.7943 0.3112 0.5285 0.1656
fprob =
0.3495 0.5983 0.4544 0.2308 0.3289
row_sum =
1.9619
pi0 =
0.6020 0.2630 0.6541 0.6892 0.7482
fprob =
0.5267 0.9017 0.6847 0.3478 0.4956
row_sum =
2.9564
pi0 =
0.4505 0.0838 0.2290 0.9133 0.1524
fprob =
0.3258 0.5578 0.4236 0.2151 0.3066
row_sum =
1.8291
pi0 =
0.8258 0.5383 0.9961 0.0782 0.4427
fprob =
0.5132 0.8787 0.6673 0.3389 0.4830
row_sum =
2.8811
Hence, ti can be seen from the above simulations with 20 restarts that the sum of row probabilities are not equal to 1. Only, the sum is closest to 1 (1.1581) when the initial distribution is
Hence, if the sum of probabilities in the initial distribution is equal to 1 then the final probabilities after simulation of Markov chain will be equal to 1.
4.
Now, distribution is stationary as the sum of the probabilities of initial distribution is not equal to 1 and the equation for being stationary distribution is not satisfied. The equation for stationary distribution is
