Random Processes in Communication and Control/M-Sep14

Last Time
PMF $$P[X=x]=P[s:X(s)=x] \!\,$$

a) $$P_X(x) \geq 0 \forall x \in S_x \!\,$$

b) $$\sum_{x \in S_X} P_X(x)=1 \!\,$$

c) event $$B \subset S_X \!\,$$

$$P[B]= \sum_{x\in B} P_X(x) \!\,$$

Bernoulli R.V
$$ P_X(x)= \begin{cases} 1-p & x=0 \\ p  & x=1 \\ 0  & o.w. \end{cases}

\!\,$$

$$0 \leq p \leq 1 \!\,$$ success probability

Example
1) Flip a coin $$X= \!\,$$# of H

2) Manufacture a Chip $$X=\!\,$$# of acceptable chips

3) Bits you transmit successfully by a modem

Geometric Random Variable
Number of trials until (and including) a success for an underlying Bernoulli

$$P_X(x)=p(1-p)^{x-1} \quad x=1,2,3,\ldots \!\,$$

Example
1) Repeated coin flips $$X= \!\,$$# of tosses until H

2) Manufacture chips $$X= \!\,$$3 of chips produced until an acceptable time

Binomial R.V
"# of successes in n trials"

$$P_x(x)= {n \choose x} p^x (1-p)^{n-x} \quad x=0,1,\ldots n \!\,$$

Example
1) Flip a coin n times. $$X= \!\,$$ # of heads.

2) Manufacture n chips. $$X=  \!\,$$ # of acceptable chips.

Note: Binomial $$X=Y_1+ Y_2+Y_3 + \ldots+Y_n \!\,$$ where $$Y_1+Y_2+Y_3 \ldots+ Y_n \!\,$$ are independent Bernoulli trials

Note: n=1; Binomial=Bernoulli; $$X=Y_1 \!\,$$

Pascal R.V
"number of trials until (and including) the kth success with an underlying Bernoulli"

$$P_X(x)= {x-1 \choose k-1} p^k (1-p)^{x-k} \quad x=k,k+1,k+2,\ldots \!\,$$

where $${x-1 \choose k-1} \!\,$$ is $$k-1 \!\,$$ successes in $$ x-1\!\,$$ trials

Note: Pascal $$X=Y_1+Y_2+\ldots+Y_k \!\,$$ where $$Y_1,Y_2,\ldots Y_k \!\,$$ are geometric R.V.

Note: K=1 Pascal=Geometric

Example
$$X=\!\,$$# of flips until the kth H

Discrete Uniform R.V.
$$ P_X(x)= \begin{cases} \frac{1}{b-a+1} & x=a,a+1,\ldots,b \\ 0              & \mbox{otherwise}

\end{cases} \!\,$$

Example
1) Rolling a die. $$a=1, b=6 \!\,$$

$$P_X(x)= \begin{cases} \frac16 & x=1,\ldots,6 \\ 0      & \mbox{otherwise} \end{cases} \!\,$$

2) Flip a fair coin. $$X \!\,$$=# of H

$$ P_X(x)= \begin{cases} \frac12 & x=0 \\ \frac12 & x=1 \\ 0      & \mbox{ otherwise } \end{cases} \!\,$$

Poisson R.V.
$$P_X(x)= e^{-\alpha} \frac{\alpha^x}{x!} \quad x=0,1,2\ldots \!\,$$

(Exercise) limiting case of binomial with $$n \rightarrow \infty, p \rightarrow 0, np=\alpha \!\,$$

PMF is a complete model for a random variable

Cumulative Distribution Function
$$F_X(x)= P[X \leq x] = \sum_{x' \leq x} P_X(X=x') \!\,$$

Like PMF, CDF is a complete description of random variable.

Example
Flip the coins $$X= \!\,$$# of H

$$P_X(x)= \begin{cases} \frac14 & x=0 \\ \frac12 & x=1 \\ \frac12 & x=2 \\ 0      & \mbox{otherwise}

\end{cases} \!\,$$

$$ \begin{align} P[X \leq 0]&=P[X=0]\\ P[X \leq 0.5] &= P[X=0] \\ P[X \leq 1 ] &= \underbrace{P[X=0]}_{\frac14} + \underbrace{P[X=1]}_{\frac12} \\

\end{align} \!\,$$

Properties of CDF

 * a) $$F_X(-\infty)=0 \Leftarrow F_X(-\infty)=P[X \leq -\infty] = 0 \!\,$$

$$F_X(\infty)=1 \Leftarrow F_X(\infty)=P[X \leq \infty]=1 \!\,$$

"starts at 0 and ends at 1"


 * b) For all $$x' \geq x \!\,$$, $$F_X(x') \geq F_X(x) \!\,$$

"non-decreasing in x"

$$ \begin{align} F_X(x') &\quad F_X(x) \\ P[X \leq x'] &\quad P[X \leq x ] \\ P[s: X(s) \leq x'] &\quad P[s:X(s) \leq x] \\ \{s: X(s) \leq x \} &\subset \{s: X(s) \leq x' \} \\ P[X \leq x] &\leq P[X \leq x']\\ F_X(x) &\leq F_X(x')

\end{align} \!\,$$


 * c) For all $$x,x' \!\,$$

$$P[x \leq X \leq x'] = F_X(x')-F_X(x) \!\,$$

"probabilities can be found by difference of the CDF"

$$\{ s: X(x) \leq x' \} = \{ s: X(x) \leq x \} \cup \{s: x \leq X(s) \leq x' \} \!\,$$

$$ P[X \leq x' ] = P[X \leq x ] + P [x \leq X \leq x']\!\,$$


 * d) For all $$x \!\,$$,

$$\lim_{\epsilon \rightarrow 0} F_X(x+\epsilon)=F_X(x) \!\,$$

"CDF is right continuous"


 * e) For $$x_i \in S_X \!\,$$

$$F_X(x_i)- F_X(x_i - \epsilon) = P_X(x_i) \!\,$$

"For a discrete random variable, there is a jump (discontinuity) in the CDF at each value $$x_i \in S_X \!\,$$. This jump equals $$P_X(x_i) \!\,$$

$$P[x_i \leq x] - P[x_i \leq x_i - \epsilon]=P[x_i- \epsilon < x < x_i] = P[X=x_i] \!\,$$


 * f) $$F_X(x)=F_X(x_i) \!\,$$ for all $$x_i \leq x \leq x_{i+1} \!\,$$

"Between two jumps the CDF is constant"

$$ \begin{align} P[X \leq x] &= P[X \leq x_i \cup x_i < X \leq x ] \\ &= P[X \leq x_i + \underbrace{P[x_i < X \leq x ]}_0 \\ &= F_X(x_i) \end{align} $$


 * g) $$P[X > x] = 1- \underbrace{F_X(x)}_{P[X \leq x]} \!\,$$

Continuous Random Variables
outcomes uncountable many

Example
T: arrival of a partical

$$S_T= \{t : 0 \leq t < \infty \} \!\,$$

V: voltage

$$S_V=\{v : -\infty < v < \infty \} \!\,$$

$$\theta \!\,$$: angle

$$S_{\theta}=\{ \theta: 0 \leq \theta \leq 2 \pi \} \!\,$$

$$X \!\,$$: distance

$$S_X= \{ X: 0 \leq x \leq 1 \} \!\,$$

$$P[x \in A] = \frac{1}{n } \rightarrow 0\!\,$$

No PMF, $$P[X=x]=0 \forall x \!\,$$

Theorem
For any random variable (continuous or discrete)


 * a) $$F_X(-\infty)=0 \quad F_X(\infty)=1 \!\,$$


 * b) $$F_X(x) \!\,$$ is nondecreasing in $$X \!\,$$


 * c) $$P[x < X \leq x'] = F_X(x')-F_X(x) \!\,$$


 * d) $$F_X(x) \!\,$$ is right continuous

Example
$$S_X=[0,1] \!\,$$

$$P_[x \in A]= P[ x\ in B]\!\,$$ where A, B are intervals of the same length contained in [0,1]

$$P[X \leq 1] =1 \Leftrightarrow F_X(1)=1 \!\,$$

$$P[X \leq 0] =0 \Leftrightarrow F_X(0)=0 \!\,$$

$$P[x_1 < x < x_2]=P[0 < x < x_2-x_1] \!\,$$

(exercise)$$F_X(x_2)-F_X(x_1)=F_X(x_2-x_1) \rightarrow F_x(x)=x \!\,$$

Probability Density Function (PDF)
$$f_X(x)= \frac{dF_x(x)}{dx} \!\,$$

discrete: PMF <--> CDF  (sum/difference)

continuous  <--->  (derivative/integral)

Theorem: Properties of PDF

 * a) $$f_X(x) \geq 0 \!\,$$ ($$F_X(x) \!\,$$ is nondecreasing)


 * b) $$F_X(x) = \int_{\infty}^x f_X(x) dx \quad (F_X(\infty)=0)\!\,$$


 * c) $$\int_{-\infty}^\infty f_X(x)dx=1 \quad (F_x(\infty)=1) \!\,$$

Theorem
$$ \begin{align} P[x_1 \leq X \leq x_2] &= F_X(x_2)-F_X(x_1) \\ &= \int_{-\infty}^{x_2} f_X(x)dx - \int_{-\infty}^{x_1} f_X(x) dx \\ &= \int_{x_1}^{x_2} f_X(x) \end{align} \!\,$$

Uniform R.V
$$f_X(x)= \begin{cases} \frac{1}{b-a} & a \leq x \leq b \\ 0            & \mbox{otherwise} \end{cases}

\!\,$$

Exponential R.V
$$f_X(x)=ae^{-ax} \quad x \geq 0\!\,$$

$$F_x(x)=1-e^{-ax} \!\,$$

Gaussian (Normal) R.V.
$$\mathcal{N} (\mu, \sigma^2) \quad \infty < x< \infty\!\,$$

$$f_X(x)=\frac{1}{\sqrt{2\pi \sigma^2}}e^{-\frac{(x-\mu)^2}{2\sigma^2}} \!\,$$

$$F_X(x) = \int_{-\infty^1}\frac{1}{\sqrt{2 \pi \sigma^2}}e^{-\frac{x-\mu}{2\sigma^2}} \!\,$$