UMD Probability Qualifying Exams/Aug2010Probability

Solution
(i) The Markov transition matrix will be the 2x2 matrix $$Q=(q_{ij})$$ where $$i,j=1$$ corresponds to a win for Player A and $$i,j=0$$ corresponds to a loss for Player A.  For example, $$q_{11}$$ is the probability that Player A wins after winning in the previous hand; $$q_{01}$$ is the probability that Player A wins after losing in the previous hand; etc.  This will give

$$Q=\left(\begin{array}{cc} 0.7&0.3\\ 0.5&0.5 \end{array}\right)$$

The stationary distribution will be the tuple $$\pi=(a,b)$$ such that $$\pi Q=\pi$$. We can calculate this explicitly:

$$(a,b)\left(\begin{array}{cc} 0.7&0.3\\ 0.5&0.5 \end{array}\right)=(a,b)$$ yields the following system of equations: $$.7a+.5b=a; .3a+.5b=b$$ Using the fact that $$\pi$$ must be a probability (i.e. $$a+b=1$$) we get $$a=\frac{5}{8}, b=\frac{3}{8}$$.

(ii) Since $$Q$$ is positive, and hence ergodic, then any initial probability distribution will converge to the stationary distribution just calculated, $$\pi$$. Thus as $$n\to\infty$$ Player A will win with probability $$5/8$$. Can Player A expect to have more money though? For sufficiently large $$n$$ we can compute Player A's expected winnings in one round:

$$E[\text{winnings}]=5/8*4+3/8*(-5)=5/8>0$$

Thus Player A should expect to have more money than before the game with probability 1.

Solution
(i) $$P(X\geq c)\leq P(|X|\geq c) = P(|X-E[x]|\geq c) \leq P(|X-E[X]|\geq c + \sigma)\leq \frac{\sigma^2}{c^2+2c\sigma+\sigma^2}\leq \frac{\sigma^2}{c^2+\sigma^2}$$ were the second-to-last inequality is the standard Chebyshev's inequality.

Solution
(i) Let $$f(x)=|x+Y|$$. Easy to see that $$f$$ is convex.

Then by Jensen's Inequality we have

$$f(E(X|Y))\leq E(f(X)|Y)$$

$$|E(X|Y)+Y|\leq E(|X+Y||Y)$$. Taking the expectation on both sides gives

$$E(|Y|)\leq E(|X+Y|)$$.