Probability/Combinatorics

We all know how to count. However, counting can actually be quite complicated, and there are many sophisticated techniques related to counting. In this chapter, we will introduce some basic techniques, which are the basis for calculating (discussed in next chapter). Those techniques make use of the heavily, discussed in the following section.

Fundamental counting principles
Just as there are four kinds of basic operations in mathematics: addition, subtraction, multiplication, division, there are four corresponding fundamental principles in counting: addition, subtraction, multiplication, division counting principles.

Addition and subtraction counting principles
More generally, we have the counting principle:

Actually, the previous two principles of counting are just special cases of a more general principle:.

Multiplication counting principle
When we are using the multiplication counting principle to solve some counting problems, we need to be careful about the following:
 * We need to ensure that the number ways of doing each task are, and should not depend on the way of doing the other tasks. (See the following example.)
 * After using the multiplication counting principle to count the number of ways to do the $$n$$ tasks, it is possible that some of the ways (or "decision paths", i.e., the paths that we "go through" the tree diagram) actually correspond to the under the situation in a problem, so the those ways should only be counted as one outcome in that context. (See the following example.)

Division counting principle
Graphically, it looks like $$ \begin{align} \text{Task }\quad&\text{ Procedure}\\ \hline \text{way 1}\leftrightarrow&\begin{cases}\text{way 1}\\\text{way 2}\\\quad\vdots\\\text{way }d\end{cases}\\ \text{way 2}\leftrightarrow&\begin{cases}\text{way 1}\\\text{way 2}\\\quad\vdots\\\text{way }d\end{cases}\\ \vdots\\ \text{way }\frac{n}{d}\leftrightarrow&\begin{cases}\text{way 1}\\\text{way 2}\\\quad\vdots\\\text{way }d\end{cases}\\ \hline \qquad\qquad&\text{Total}:\frac{n}{d}\times d=n\text{ ways for the procedure} \end{align} $$

We should be aware that the division counting principle does not apply if different ways of doing task correspond to number of ways in the procedure. Consider the following examples:

Number of ways for placing r balls into n cells
In this section, we will discuss how to count the number of ways for placing $$r$$ balls into $$n$$ cells. From here, one may then ask why do we consider this particular situation, and it may seem that we rarely encounter this situation in practice. Often, we want to count the number of ways for doing things other than placing balls into cells.

Although many situations encountered may seem to be quite different from this situation, particularly, the background is usually not about placing balls into cells, we may them as if we are placing balls into cells. Indeed, many situations are so-called "abstractly equivalent", in the sense that the underlying process of "generating" the outcomes is the same, but we just have different verbal descriptions.

Let us consider some situations that are abstractly equivalent to "placing $$r$$ balls into $$n$$ cells" (for some $$n$$ and $$r$$):
 * 1) Number of possible birthdays for 10 people: placing 10 "people" (balls) into 365 "birthday dates" (cells). (Here, we assume one year has only 365 days.) Notice that we are  placing 365 "birthday dates" into 10 "people". (If this is this case, some people have more than one birthday!) Notice also that we can place more than one person into the same "birthday date". (Multiple people can share the same birthday!)
 * 2) Setting a 6-digit numeric password: placing 6 "password positions" (balls) into 10 "numbers" (cells) (0,1,...,9). (Likewise, we are  placing "numbers" into "password positions". If this is the case, then some password position contains  number, which is impossible.)
 * 3) Classification of 1000 people into ages 0-10, 11-20, ..., 91-100, 101 or above: placing 1000 "people" (balls) into 11 "age groups" (cells).
 * 4) Elevator with 20 passengers and stopping at 10 floors: placing 20 "passengers" (balls) into 10 "floors" (cells).
 * 5) 3 pigeons flying into 2 pigeonholes: placing 3 "pigeons" (balls) into 2 "pigeonholes" (cells).
 * 6) Rolling 10 dice: placing 10 "dice" (balls) into 6 "outcomes" (cells).
 * 7) Drawing 3 cards from a poker deck: placing 3 "draws" (balls) into 52 "cards" (cells).
 * 8) Tossing 2 coins: placing 2 "coin tosses" (balls) into 2 "outcomes" (cells).
 * 9) Select 5 people from 20 people to form a committee: placing 5 "committee positions" (balls) into 20 "people" (cells).

In this section, we will mainly develop some formulas for calculating the number of ways to select some objects from distinguishable objects, , classified by whether the selection is, and whether the selection is. We can apply the above notion of "abstract equivalence" to convert such selection as a problem of placing some distinguishable or indistinguishable balls into some distinguishable cells with certain capacity, so that we can develop the formula more easily and conveniently. (As we will see, the distinguishability of balls corresponds to whether the selection is ordered or not, and the capacity of the cells correspond to whether the selection is with or without replacement.)

Before discussing these four types of selection, we need to introduce a preliminary mathematical concept which will be used frequently in the following:

Placing r distinguishable balls into n distinguishable cells with capacity one (ordered selection without replacement)
Let us first discuss ordered selection of $$r$$ objects from $$n$$ objects without replacement. We can regard this to be abstractly equivalent to placing $$r$$ distinguishable balls (choice 1,2,...,$$r$$) into $$n$$ distinguishable cells (object 1,2,...,$$n$$) with capacity one. After that, we develop the following formula.
 * The capacity is one, since, we are unable to choose the same object multiple times (unable to put two or more balls (choices) into the same cell (object))

The situation in the example above can actually be interpreted as a special case of.

Partition
For permutating the string "APPLE", we can regard the situation as partitioning 5 (distinguishable and ordered) into 4 groups: "A", "P", "L", and "E", where we require the group "A", "P", "L", "E" to contain 1,2,1,1 letter positions respectively. Similarly, for permutating the string "PEPPER", we can regard the situation as partitioning 5 (distinguishable and ordered) into 3 groups: "P", "E" and "R" where we require the group "P", "E", "R" to contain 3,2,1 letter positions respectively.

Such situation is actually abstractly equivalent to placing distinguishable balls into distinguishable cells, with some restrictions on the number of balls each cell contain. Generally, "partitioning $$n$$ distinguishable objects into $$k$$ groups, where group 1,2,...,$$k$$ must contain $$n_1,n_2,\dotsc,n_k$$ objects respectively" (Merely changing the order of placing the objects into a certain group do not affect the partition, since the group still contains the same thing at the end.) is abstractly equivalent to "placing $$n$$ distinguishable balls into cell 1,2,...,$$k$$, where cell 1,2,...,$$k$$ must contain $$n_1,n_2,\dotsc,n_k$$ balls respectively".

In the following, we will discuss unordered selection without replacement, which can be regarded as a special case of partition.

Placing r indistinguishable balls into n distinguishable cells with capacity one (unordered selection without replacement)
Previously, we have discussed placing $$r$$ balls into $$n$$ distinguishable cells with capacity one. Now, we will consider the case where the $$r$$ balls are. (One may conceive the balls to be (say same color and size), and hence indistinguishalble.) Similarly, such placement is equivalent to selection without replacement (the $$r$$ balls represent the choices, and every cell can still contain at most one ball). However, in this case, the $$r$$ balls (choices) are indistinguishable. So, we only know that which $$r$$ of the $$n$$ cells contain one ball (are selected), but do not know in do they contain one ball, since the balls are indistinguishable. Hence, the order for which the cells contain one ball (are selected) is not important in this case, since we cannot identify the different orders of selection anyways. Hence, we consider the selection process as. Such selection process is quite common since we are often more interested in knowing are selected, and are not quite interested in in  the things are selected.

In this case, we can actually regard this as a special case of partition: So, this means such placement is equivalent to the partition of $$n$$ distinguishable cells into two groups with $$r$$ distinguishable cells and $$n-r$$ distinguishable cells respectively. With such consideration, the following result is transparent:
 * For the cells with one ball ($$r$$ of them), they are put into the group "selected" (they are selected to place one ball).
 * For the cells with zero ball ($$n-r$$ of them), they are put into the group "not selected" (they are not selected to place one ball).

Here, we give an alternative proof to this theorem which uses the previous result for distinguishable balls and the division counting principle:

Placing r distinguishable balls into n distinguishable cells with unlimited capacity (ordered selection with replacement)
So far, we have only considered the selection replacement, or equivalently, cells with. From this section onward, we will discuss the selection replacement, or equivalently, cells with  (same thing can be chosen unlimited times). In this section, let us consider the simpler one: placing $$r$$ distinguishable balls into $$n$$ distinguishable cells with unlimited capacity. This is equivalent to ordered selection of $$r$$ objects from $$n$$ distinguishable objects with replacement, by regarding the ball 1,2,...,$$r$$ as choice 1,2,...,$$r$$, cell 1,2,...,$$n$$ as object 1,2,...,$$n$$. In particular, since every object can be chosen unlimited times, every cell has unlimited capacity. But counting the number of ways of such placement is actually quite simple, since we can just use the multiplication counting principle.

Placing r indistinguishable balls into n distinguishable cells with unlimited capacity (unordered selection with replacement)
In the previous section, the balls are distinguishable. Now, we will discuss a more complicated situation where the balls are indistinguishable.

The challenging part of this situation is that we cannot apply the division counting principle to count the number of ways, as in the previous situation about placing $$r$$ indistinguishable balls into $$n$$ distinguishable cells with capacity one ("task"). In that previous situation, we know that every way in the task corresponds to $$r!$$ ways in the procedure (where we are placing $$r$$ distinguishable balls), by considering the number of ways of permutating the $$r$$ cells occupied by.

However, in this case, every cell can contain more than one ball. This means the number of occupied cells can range from 1 (all $$r$$ balls in one cell) to $$r$$ (one ball each for the $$r$$ cells), depending on how we put the balls into the cells in the task. Hence, different ways in the task correspond to different number of ways in the procedure. Thus, we cannot apply the division counting principle.

Because of this, the method for developing the number of ways in this situation is quite different from the methods discussed previously, and we need some tricks.

Summary
Alternatively and equivalently,

Indistinguishable cells
When the are indistinguishable, there are no simple and nice formulas for counting the number of ways generally. Also, in this case, it may be more difficult to imagine what "indistinguishable cells" look like, since even the cells are identical, their positions should be different so that the balls can be placed in different cells. Then, the cells are still distinguishable. To visualize indistinguishable cells, one may conceive that after the balls are placed into the cells, the cells are "sorted", based on the number of balls they contain, with the cell with least balls placed at leftmost, and the cell with most balls placed at rightmost. (The cells with the same number of balls can just be put together, in any order.) Then, after "throwing" the balls into the cells the cells are sorted, we can look at the cells to observe the results. In this case, the cells can be regarded as indistinguishable, since we only know how many cells contain zero, one, etc. balls, but not precisely which of the original cell 1,2,.. contains how many balls (we cannot tell which cell is the original "cell 1,2,..." after sorting).

When we encounter indistinguishable cells, we may need to consider the ways one by one.

A powerful tool for counting problems: generating functions
The following theorem gives some common generating functions.