Probability/Properties of Distributions

Introduction
Recall that pdf (or cdf) describes the random behaviour of a random variable. However, we may sometimes find the pdf (or cdf) to be too complicated, and only want to know some about the random variable. In view of this, we study some properties of distributions in this chapter, which provide descriptions of the random behaviour of the random variable.

Some examples of such partial descriptions include
 * location (e.g. pdf is 'located' at left or right?),
 * dispersion (e.g. 'sharp' of 'flat' pdf?),
 * skewness (e.g. pdf is symmetric, skewed to left, or skewed to right?), and
 * tail property (e.g. pdf have 'light' or 'heavy' tails?).

We can describe them, but such descriptions are quite subjective and inaccurate. To give a more objective and accurate measure to such descriptions, we evaluate them using some quantitative measures derived from the pdf (or cdf) of the random variable.

We will discuss some of such quantitative measures in this chapter. Among these, the is the most important one, since many of other properties base upon the concept of.

Expectation
We have different alternative names for expectation, e.g. expected value and mean.

{{colored exercise| In a process, we first toss an {{colored em|unfair}} coin one time, with probability $$p$$ for the head to come up. If head comes up in the first toss, we toss {{colored em|another unfair}} coin one time, with probability $$q$$ for the head to come up. If tails comes up in the first toss instead, We throw an arrow to the ground one time. Let $$X$$ be the number of {{colored em|head}} coming up in all tosses, $$Y$$ be the {{colored em|angle}} from the north direction to the direction pointed by the arrow, measured clockwise and in radian, and $$Z$$ be the number we get from the process finally. Suppose that $$Y\sim \mathcal U[0,2\pi)$$.

{Choose correct expression(s) of $$\mathbb E[X]$$. - $$p$$ - $$q$$ + $$p+q$$ - $$(1-p)(1-q)+p(1-q)+q(1-p)+2pq$$ - $$2p(1-q)+2pq$$ }
 * type="[]"}
 * What is $$X$$?
 * What is $$X$$?
 * $$\mathbb E[X]=\cancel{0[(1-p)(1-q)]}+1[p(1-q)+q(1-p)]+2[pq]=p\cancel{-pq}+q\cancel{-pq}\cancel{+2pq}=p+q$$
 * $$0(1-p)(1-q)=0$$
 * $$p(1-q)\ne q(1-p)$$ in general.

{Choose correct expression(s) of $$\mathbb E[Y]$$. + $$\pi$$ - $$p\pi$$ - $$q\pi$$ - $$(1-p)\pi$$ - $$(1-q)\pi$$
 * type="[]"}
 * $$\mathbb E[Y]=(0+2\pi)/2=\pi$$.
 * Remark: recall from the chapter about random variable that the interval of the support can be closed, open or half-open, without affecting the result.
 * The angle we get is not affected by $$p$$.
 * The angle we get is not affected by $$q$$.
 * The angle we get is not affected by $$p$$.
 * The angle we get is not affected by $$q$$.

{Choose correct expression(s) of $$\mathbb E[Z]$$. + $$pq+(1-p)\pi$$ - $$p(p+q)+(1-p)\pi$$ - $$pq+(1-p)q\pi$$ - $$p(p+q)+(1-p)p\pi$$
 * type="[]"}
 * $$\mathbb E[Z]=p\mathbb E[\text{number of head coming up in the 2nd toss}]+(1-p)\mathbb E[Y]=pq+(1-p)\pi.$$

{If the two coins are, choose correct statement(s). - $$\mathbb E[Z]$$ increases. - $$\mathbb E[Z]$$ decreases. + Change in $$\mathbb E[Z]$$ depends on values of $$p$$ and $$q$$. + $$\mathbb E[Y]$$ remains unchanged. - $$\mathbb E[Z]$$ increases if $$p=q=1/3$$. }} In the following, we introduce a useful result that gives the relationship between expectation and probability, we can use expectation to ease the computation of probability using this result.
 * type="[]"}
 * Since $$\mathbb E[Z]=pq+(1-p)\pi$$
 * If $$p=q=1/3$$, when the coins are still unfair,
 * $$\mathbb E[Z]=(1/3)^2+(1-1/3)\pi\approx 2.205506$$.
 * When the coins become fair,
 * $$\mathbb E[Z]=(1/2)^2+(1-1/2)\pi\approx 1.820796$$.
 * So, $$\mathbb E[Z]$$ actually decreases.

When there are multiple random variables involved, we may derive the joint pmf or pdf first to compute the expectation, but it can be quite difficult and complicated to do so. Practically, we use the following theorem more often.

The proof is quite complicated, and hence we skip it. In the following, we will introduce several properties of expectation that can help us to simplify computations of the expectation.

Mean of some distributions of a continuous random variable
We will introduce the formulas for mean of some distributions of a random variable, which are relatively simpler.

Examples
{{colored exercise| {Choose correct statement(s). - $$\mathbb E[X]\ge 0$$ for each random variable $$X$$. + $$\mathbb E[|X|]\ge 0$$ for each random variable $$X$$. + $$|\mathbb E[X]|\ge 0$$ for each random variable $$X$$. - $$\mathbb E[XYZ]=\mathbb E[X]\mathbb E[Y]\mathbb E[Z]$$ if random variables $$X,Y$$ and $$Z$$ are pairwise independent.
 * type="[]"}
 * This is true for each nonnegative random variable, and $$X$$ may not be nonnegative.
 * Actually, if $$X$$ is negative, then its expected value is nonpositive, by considering $$\mathbb E[-X]$$ and linearity.
 * Since $$|X|\ge 0$$, this holds by nonnegativity.
 * This follows from the nonnegativity of absolute value function.
 * The pairwise independence is not sufficient for this to hold. We need mutual independence of $$X,Y$$ and $$Z$$.

{Given that $$\mathbb E[X]=-k=-\mathbb E[Y]$$, Choose correct expression(s) for $$\mathbb E[aX+bY+c]$$. + $$a\mathbb E[X]+\mathbb E[bY+c]$$ - $$c\mathbb E[(a/c)X+(b/c)Y]+c$$ + $$(b-a)k+c$$ - $$\mathbb E[-ak+bk+c]$$ } }} Let us illustrate the usefulness of fundamental bridge between probability and expectation by giving a proof to inclusion-exclusion using this bridge.
 * type="[]"}
 * We can obtain this by linearity.
 * This expression is undefined if $$c=0$$. However, if $$c=0$$, $$\mathbb E[aX+bY+c]$$ may not be undefined.
 * We can obtain this by linearity.
 * This is wrong since $$\mathbb E[Z]\ne \mathbb E[\mathbb E[Z]]$$ for each random variable $$Z$$ in general.

Probability generating functions
An application of expectation is. As suggested by its name, it can probabilities in some sense.

Variance (and standard deviation)
Indeed, is a special case of , and is related to in some sense.

Since $$(X-\mathbb E[X])^2$$ is the squared deviation of the value of $$X$$ from its mean, in view of the definition of variance, we can see that variance measure the (or ) of distribution, since it is what we would of the squared deviation if we are to take an observation of the random variable.

Another term which is closed related is.

Quantile
Then, we will discuss. In particular, and  range are quite related to.

The following are some terminologies related to.

and measure centrality and dispersion respectively. Recall that and  measure the same things respectively. One advantage of and  is, since they are always defined, while and can be infinite, and they fail to measure centrality and dispersion in those occasions. However, and  also have some disadvantages, e.g. they may be more difficult to be computed, and may not be very accurate.

Mode
Mode is another measure of centrality.

Covariance and correlation coefficients
In this section, we will discuss two important properties of distributions, namely  and. As we will see, covariance is related to variance in some sense, and correlation coefficient is closed related to correlation.

Both and  measure  between $$X$$ and $$Y$$. As we will see, $$\rho(X,Y)\in[-1,1]$$, $$X,Y$$ are more highly correlated as $$|\rho(X,Y)|$$ increases, and $$X$$ has a linear relationship with $$Y$$ if $$|\rho(X,Y)|=1$$.

Then, we will discuss about. The following is the definition of between correlation between two random variables.

Covariance and correlation coefficient are, but they have differences. In particular, $$\operatorname{Cov}(X,Y)$$ depends on of $$X$$ and $$Y$$, not just their relationship. Thus, this number is affected by the variances, and does not measure their relationship accurately. On the other hand, $$\rho(X,Y)$$ for  of $$X$$ and $$Y$$, and therefore measures their relationships more.

The following is one of the most important properties of correlation coefficient.

Then, we will define several terminologies related to correlation coefficient.

Then, we will state an important result that is related to independence and correlation. Intuitively, you may think that 'independent' is the same as 'uncorrelated'. However, this is wrong. Indeed, 'independent' is than 'uncorrelated'.

However, converse is true, as we will see in the following example.