User:Dom walden/Bivariate Generating Functions

Introduction
Combinatorial class = C

Combinatorial parameter = \chi or \zeta

P(X = k) = #(y \in C | X(y) = k) / #C

So X, \chi, \zeta is both a variable and function...

Or, P(E) = \mu(E) = \sum_{e \in E} p(e)

f(x) (pdf)

F(x) = P(X \leq x) = \int_{-\infty}^x f(u)du (cdf)

For continuous variable:

Fourier/characteristic: \lambda_X(s) = \int_{\R} e^{sx} f(x) dx

Laplace: \phi_X(t) = \int_{\R} e^{itx} f(x) dx

(I don't necessarily believe the above...)

X* = \zeta - E(\zeta) / \sqrt{V(\zeta)}

If Y = X - \mu / \sigma then

\phi_Y(t) = e^{-\mu it} \phi_X(t/\sigma)

\lambda_Y(s) = e^{-\mu s} \lambda_X(s/\sigma)

Characteristic function of standard normal:

1/\sqrt{2\pi} \int_R e^{itz} e^{-z^2/2} dz

Discrete limit law
This applies when the mean and standard deviation of the random variable remain finite.

Theorem
If $$\Omega$$ is a subset of the unit disc with at least one point of accumulation in the interior of the disc. If $$p_n(u) = \sum_{k \geq 0} p_{n,k} u^k$$, $$q(u) = \sum_{k \geq 0} q_k u^k$$ and pointwise for each $$u \in \Omega$$


 * $$\lim_{n \to +\infty} p_n(u) = q(u)$$

then


 * $$\lim_{n \to +\infty} p_{n,k} = q_k$$ and $$\lim_{n \to +\infty} \sum_{j \leq k} p_{n,j} = \sum_{j \leq k} q_j$$

Proof
Applying Vitali's theorem, we take the sequence to be $$\{p_n(u)\}$$ and $$S$$ to be the unit disc. All $$p_n(u)$$ are analytic and bounded on $$S$$ (due to $$p_n(1) = 1$$). The theorem assumes that $$\{p_n(u)\}$$ converges to $$q(u)$$ in $$\Omega \sub S$$ which has an accumulation point in $$S$$.

Vitali's theorem states $$\{p_n(u)\}$$ uniformly converges to $$q(u)$$ on any compact subset of $$S$$, which we will take as the disc $$|u| \leq 1/2$$ and use Cauchy's coefficient formula:


 * $$q_k = \frac{1}{2i\pi} \int_{|u| = 1/2} q(u) \frac{du}{u^{k+1}} = \lim_{n \to \infty} \frac{1}{2i\pi} \int_{|u| = 1/2} p_n(u) \frac{du}{u^{k+1}} = \lim_{n \to \infty} p_{n,k}$$

Vitali's theorem
Theorem

Let $$\{f_n(z)\}$$ be a sequence of functions, all analytic on an open connected set $$S$$ and $$|f_n(z)| \leq M$$ for all $$n$$ and $$z$$ in $$S$$.

If the sequence $$\{f_n(z)\}$$ converges on a subset of $$S$$ with a point of accumulation in $$S$$, then $$\{f_n(z)\}$$ converges uniformly on every compact subset of $$S$$.

Proof

Let


 * $$f_n(z) = a_{0,n} + a_{1,n}z + \cdots$$

We want to prove that each $$a_{v,n}$$ converges to a limit.

We start with a disc centred at the origin of radius $$R$$ and point of accumulation as the origin. Then


 * $$|f_n(z) - f_n(0)| \leq |f_n(z)| + |f_n(0)| \leq 2M$$

Let $$z_0 \neq 0$$ be a point where the sequence converges. Then


 * $$|f_n(0) - f_{n+m}(0)| \leq |f_n(0) - f_n(z_0)| + |f_n(z_0) - f_{n+m}(z_0)| + |f_{n+m}(z_0) - f_{n+m}(0)| \leq \frac{4M|z_0|}{R} + |f_n(z_0) - f_{n+m}(z_0)|$$

because, by Schwarz's lemma, $$|f_n(0) - f_n(z_0)| \leq 2M|z_0|/R$$ and $$|f_{n+m}(z_0) - f_{n+m}(0)| \leq 2M|z_0|/R$$.

We choose $$z_0$$ such that the first term is arbitrarily small and $$n$$ large enough that the second term is arbitrarily small. Therefore, $$f_n(0) = a_{0,n}$$ converges to a limit.

Next, we define


 * $$g_n(z) = \frac{f_n(z) - a_{0,n}}{z} = a_{1,n} + a_{2,n}z + \cdots \leq \frac{2M}{R}$$

For $$|z| < R$$. $$g_n(z)$$ converges to a limit at $$z_0$$ as both $$f_n(z)$$ and $$a_{0,n}$$ converge to a limit. We repeat the argument above, which proved that $$f_n(0) = a_{0,n}$$ converges to a limit, to prove that $$g_n(0) = a_{1,n}$$ also converges to a limit. We keep repeating this to prove that $$a_{v,n}$$ converges for all $$v$$.

Since each term of $$f_n(z)$$ converges to a limit, $$f_n(z)$$ also converges to a limit for $$z < R$$. If we repeat the argument with another disc centred at any limit point (e.g. $$z_0$$), we can extend to any region bounded by $$S$$.

Schwarz's lemma
Lemma

If $$f(z)$$ is analytic and regular(?) for $$|z| \leq R$$, $$|f(z)| \leq M$$ for $$|z| = R$$ and $$f(0) = 0$$, then


 * $$f(re^{i\theta} \leq \frac{Mr}{R} \quad (0 \leq r \leq R)$$

Proof

$$f(z)/z$$ is regular for $$|z| \leq R$$ and $$|f(z)/z| \leq M/R$$ on the circle $$|z| = R$$. By the maximum modulus theorem(?) this inequality holds also inside the circle. Because $$|f(z)/z| = |f(z)|/r$$ we multiply both sides of the inequality by $$r$$ which gives us our lemma.

Example
The number $$k$$ of singleton cycles in a permutation of length $$n$$ is given by $$P(z, u) = \frac{e^{z (u - 1)}}{1 - z}$$.

For each $$u$$ we have a singularity at $$z = 1$$ of order 1. If we treat $$u$$ as a constant and apply our estimate for meromorphic functions


 * $$p_n(u) = [z^n]P(z, u) \sim \frac{(-1) e^{(u - 1)}}{-1} = e^{u-1}$$ as $$n \to \infty$$.

Therefore,


 * $$p_{n,k} \sim [u^k]e^{u-1} = \frac{1}{k! e}$$ as $$n \to \infty$$.

Continuous limit law
This applies when the mean and standard deviation of the random variable tend to infinity.

There are three cases/theorems:
 * Meromorphic
 * Singularity
 * Saddle

Meromorphic schema
Theorem

Let $$F(z, u)$$ be bivariate analytic at $$(z, u) = (0, 0)$$ with non-negative coefficients. Let $$F(z, 1)$$ be meromorphic in $$z \leq r$$ with only a simple pole $$\rho$$ for positive $$\rho < r$$. Assume:


 * 1) Meromorphic perturbation: there exists $$\epsilon > 0$$ and $$r > \rho$$ such that in the domain $$D = \{|z| \leq r\} \times \{|u - 1| < \epsilon\}$$:
 * $$F(z, u) = \frac{B(z, u)}{C(z, u)}$$
 * where $$B(z, u)$$ and $$C(z, u)$$ are analytic in $$D$$ and $$B(\rho, 1) \neq 0$$ and $$\rho$$ is a simple zero of $$C(z, 1)$$.
 * 1) Non-degeneracy: $$\partial_z C(\rho, 1) \partial_u C(\rho, 1) \neq 0$$.
 * 2) Variability: $$v\left(\frac{\rho(1)}{\rho(u)}\right) \neq 0$$.

Then, the standardised random variable $$X_n$$ converges to the Gaussian variable, with speed of convergence $$O(n^{-1/2})$$ and mean and standard deviation asymptotically linear in $$n$$.

Proof

Construct annular region composed of two concentric circles $$C_s$$ and $$C_r$$ of radius $$s$$ ($$\rho < s < r$$) and $$r$$, respectively. By the global Cauchy formula and residue theorem, for $$z$$ in the annular region and $$u$$ in a small enough domain around 1,


 * $$f_n(u) = \frac{1}{2\pi i} \int_{C_s} \frac{B(z, u)}{C(z, u)} \frac{dz}{z^{n+1}} + \frac{1}{2\pi i} \int_{C_r} F(z, u) \frac{dz}{z^{n+1}} = Res\left(\frac{B(z, u)}{C(z, u) z^{n+1}}; z = \rho(u) \right) + \frac{1}{2\pi i} \int_{C_r} F(z, u) \frac{dz}{z^{n+1}}$$

By Cauchy's inequality,


 * $$\int_{C_r} F(z, u) \frac{dz}{z^{n+1}} \leq \frac{\max_{|z| = r,|u-1|\leq\epsilon} |F(z, u)|}{r^n} = \frac{\sup_{|z| = r,|u-1|\leq\epsilon} |B(z, u)|}{\inf_{|z| = r,|u-1|\leq\epsilon} |C(z, u)| r^n} = \frac{K}{r^n} = O(r^{-n})$$

where $$K$$ is a finite constant, due to the fact that $$C(z, u)$$ is non-zero on $$|z| = r$$ and $$B(z, u)$$ is analytic and therefore bounded...


 * $$Res\left(\frac{B(z, u)}{C(z, u) z^{n+1}}; z = \rho(u) \right) = \frac{B(\rho(u), u)}{C_z^'(\rho(u), u) \rho(u)^{n+1}}$$

Therefore,


 * $$f_n(u) = \frac{B(\rho(u), u)}{C_z^'(\rho(u), u) \rho(u)^{n+1}} + O(r^{-n})$$

meaning it meets the conditions of the Quasi-powers theorem.

Saddle-point
Theorem

Assume


 * $$p_n(u) = e^{h_n(u)} (1 + o(1))$$

uniformly for $$u$$ in a fixed neighbourhood $$\Omega$$ of $$1$$ and each $$h_n(u)$$ analytic in $$\Omega$$. Assume also


 * $$h_n'(1) + h_n''(1) \to \infty$$

and


 * $$\frac{h_n'(u)}{(h_n'(1) + h_n(1))^{3/2}} \to 0$$

uniformly for $$u \in \Omega$$. Then


 * $$X_n^* = \frac{X_n - h_n'(1)}{(h_n'(1) + h_n''(1))^{1/2}}$$

converges in distribution to a Gaussian with mean 0 and variance 1.

Proof

Quasi-powers theorem
Theorem

Let the $$X_n$$ be non-negative discrete random variables and $$p_n(u)$$ be probability generating functions. Assume uniformly in a fixed complex neighbourhood of $$u = 1$$ for $$\beta_n, \kappa_n \to +\infty$$


 * $$p_n(u) = A(u) . B(u)^{\beta_n} \left( 1 + O\left(\frac{1}{\kappa_n}\right) \right)$$

where $$A(u), B(u)$$ are analytic at $$u = 1$$ and $$A(u) = B(u) = 1$$. Assume $$B(u)$$ satisfies the "variability condition"


 * $$v(B(u)) \equiv B''(1) + B'(1) - B'(1)^2 \neq 0$$

Then the mean and variance of $$X_n$$ satisfy


 * $$\mu_n \equiv E(X_n) = \beta_n m(B(u)) + m(A(u)) + O(\kappa_n^{-1})$$
 * $$\sigma_n^2 \equiv V(X_n) = \beta_n v(B(u)) + v(A(u)) + O(\kappa_n^{-1})$$

and


 * $$P\left( \frac{X_n - E(X_n)}{\sqrt{V(X_n)}} \leq x \right) = \Phi(x) + O\left( \frac{1}{\kappa_n} + \frac{1}{\sqrt{\beta_n}} \right)$$

where


 * $$\Phi(x) = \frac{1}{\sqrt{2\pi}} \int_{-\infty}^x e^{-w^2/2} dw$$.

Proof

Convert to a Laplace transform


 * $$\lambda_n(s) = e^{\beta_n \log B(s) + \log A(s)} (1 + O(\kappa_n^{-1}))$$

therefore


 * $$\log \lambda_n(s) = \beta_n \log B(s) + \log A(s) + O(\kappa_n^{-1})$$


 * $$\frac{d}{ds} \log \lambda_n(s) = \beta_n \frac{d}{ds} \log B(s) + \frac{d}{ds} \log A(s) + O(\kappa_n^{-1}) = \beta_n m(B(s)) + m(A(s)) + O(\kappa_n^{-1})$$


 * $$\frac{d^2}{ds^2} \log \lambda_n(s) = \beta_n \frac{d^2}{ds^2} \log B(s) + \frac{d^2}{ds^2} \log A(s) + O(\kappa_n^{-1}) = \beta_n v(B(s)) + v(A(s)) + O(\kappa_n^{-1})$$

Continuity of integral transforms
Theorem from Flajolet and Sedgewick.

Fourier

Also known as Lévy's continuity theorem.

If $$p_n(u)$$ and $$q(u)$$ have Fourier transforms $$\phi_n(t)$$ and $$\phi(t)$$, respectively, and $$q(u)$$ has a continuous distribution function, then


 * $$\lim_{n \to +\infty} p_n(u) = q(u)$$

if and only if, pointwise for each real $$t$$


 * $$\lim_{n \to +\infty} \phi_n(t) = \phi(t)$$

Laplace

Assume $$p_n(u)$$ and $$q(u)$$ have Laplace transforms $$\lambda_n(s)$$ and $$\lambda(s)$$, respectively, defined on $$s \in [-s_0, s_0]$$. If pointwise for each real $$s \in [-s_0, s_0]$$


 * $$\lim_{n \to +\infty} \lambda_n(u) = \lambda(u)$$

then


 * $$\lim_{n \to +\infty} p_n(u) = q(u)$$

Pointwise convergence
A sequence of functions $$\{f_n(z)\}$$ is said to converge pointwise to $$f(z)$$ on a set $$S$$ if for each point $$z \in S$$ and $$\epsilon > 0$$ there exists a number $$N$$ such that


 * $$|f_n(z) - f(z)| < \epsilon$$ for $$n > N$$

Compared to uniform convergence, $$N$$ is dependent on both $$z$$ and $$\epsilon$$. In other words, as the $$z$$ changes the $$N$$ may have to change as well.

Uniform convergence
A sequence of functions $$\{f_n(z)\}$$ is said to converge uniformly to $$f(z)$$ on a set $$S$$ if for any $$\epsilon > 0$$ there exists a number $$N$$ such that


 * $$|f_n(z) - f(z)| < \epsilon$$ for $$n > N$$ and for all $$z \in S$$

Compared to pointwise convergence, $$N$$ depends only on $$\epsilon$$. In other words, the same $$N$$ holds for all $$z$$.