Exercise 5.3

- The Cumulative Distribution Function of where is a sequence of interatrial intervals defined as; this represents the total number of events occurring between the time interval [0, y]. Define an inter renewal process as . The counting process which defines the CDF of is given as follows:

The quantity iff or implying that,

=

=; we have

=, where the is a sequence of inner renewal intervals defined by

- A counting process represents the observed total number of events that occur from a specific timepoint up to and including the time t which is defined as . A homogeneous Poisson process, therefore, can be defined as a counting process with rate and having the following properties:

, the independent increment of events and the number of events at a time interval say, , follows a Poisson process with mean . The PMF is given as

- A renewal process defines as; when the values of, the process is said to be an ordinary renewal process and the function represents the generalized renewal process and renewal intervals {
*Yi*;*i*≥ 1} at each epoch.

- When modeling batch arrival in renewal and queuing theory, the occurrence of events together as a batch is defined by a generalized renewal process; since the occurrence (Gallager 2012). When the events occur one at a time within a defined rate and time interval, the ordinary renewal interval is preferred.

Exercise 6.13

- A state of a stochastic process defines the range of possible values assumed by a random process at a specific point in time say time t. the arrivals in an M/M/1 process follow a Poisson and exponential departure. The conditional PMF of the states is given by:

= , for a time interval we have =

Define to represent that a system is at state n at time t. the probability of arrival at a short time interval is defined as. The probability of departure within the same time interval is defined a

The average number of customers in the system with Poisson arrivals and exponential departure at a specific time interval say is defined as:; represent the arrival rate and departure rate. For a time an interval the expected customers in the system can be defined

Exercise 7.11

- Yes. The process is reversible.

- , where is the degree if node I (Cinlar 2013).

All the nodes in the system are of degree 2, because they all connected to two nodes each, thus the average time spent in state i is given by =

- The probability transition matrix for the process is

Representing an identity matrix and a matrix of zeros

- Define as the probability weights associated by the shift from state

then we have; are represented by the transition matrix weights, thu

- A steady-state of a stochastic process is defined as an eigenvector for a transition matrix for a stochastic process such that multiplying the probability vectors by transition probabilities, the state will move out from the initial state with the same probabilities.

The probability that the nest state is 1 can be defined as 3 to 1 and 2 to 1, given by = 0.0666

Multiplying by identity matrix and getting the determinant to obtain 0.08754

A Markov chain process defined such that, every state in the process can be attained from other states in the system is said to be irreducible. Reversible Markov chain is defined as the Markov chain process such that the probabilities of moving from one state to another denoted as for condition are equal to representing the probability of reversal, ie moving from (Gallager 2012).

Exercise 7.21

A stochastic process with two queuing models that following a Poisson process and the services exponentially distributed as follows: And the second queuing process have the function , the services are exponential with the function , since the mean services which follows a first come first served process. To combine the two queuing process, we would consider combining the two independent queuing process defined by the functions And. Since these two functions are Poisson processes, combining them would result in a Poisson process with, this defines a Poisson process with

Exercise 7.31

- Consider a steady-state-vector from a Z-state Markov chain having a matrix of transition denoted by , such a vector is viewed as a row vector denoted by and satisfying the following; , such that and .is a vector of probabilities whose components sum to 1 and are non-negative. Assume that for every state ; then it follows that for

; we know that

Thus

is a matrix of transition probabilities.

- A steady-state of a stochastic process is defined as an eigenvector for a transition matrix for a stochastic process such that multiplying the probability vectors by the transition

The number of customers in a process with Poisson arrivals and exponential distributed services id=s given by , at a specific time therefore, the function is defined with

Exercise 9.25

The general rule applied, is to stop when either the obtained threshold is crossed, the two thresholds can be represented by a Wald identity. If only one threshold exists such that and , a quantity j is defined and holds for the two thresholds. Choose, which defines the positively defined root of , then is simplified to obtain (Paul et al., 2013) For some given integer , and let represent the smallest integer value satisfying for every or . Therefore the limits, , the =. Let us assume that the values of quantity fills the upper bound , ten we obtain;

1= for r ranges from

Exercise 9.33

Consider a stochastic process defined as and a defective stopping quantity trial for the function. The stopped process such that for all of the following conditions are satisfied, is defined as a sub martingale, a martingale, or super martingale.

that represent a martingale for the process.

Let us consider to be a sub martingale, then borrowing the concept from theorem 9.8.2, is also a sub martingale and so for all the values satisfying .

Since the this satisfies the first condition on theory 9.98, for the second case, consider any and to be initial sample segments satisfying Then we obtain . Implying that; and thus is a sub martingale.

which is true for all

References

Cinlar, Erhan. *Introduction to stochastic processes*. Courier Corporation, 2013.

Gallager, Robert G. *Discrete stochastic processes*. Vol. 321. Springer Science & Business Media, 2012.

Jacod, Jean, and Albert Shiryaev. *Limit theorems for stochastic processes*. Vol. 288. Springer Science & Business Media, 2013.

Paul, Wolfgang, and Jörg Baschnagel. *Stochastic processes*. Vol. 1. Springer, 2013.