Stat 410 (UIUC) Course Notes (PENDING)

0. Preliminaries (Basic Probability theory)

I will jump around and mentions things without order which I feel might be new or require refreshing.

Say you have a joint distribution on r.v \(X, Y\) and want to find the distribution of say \(U = g(X, Y)\), like \(U = XY\). One way is to draw support of \(X,Y\) distribution and use the CDF method to find CDF of \(U\). Often a cleaner way is to find first the joint distribution of \(X, U\) via \(f_{X, U}(x,u) = f_{X, Y}(x, g^{-1}(x,u)) \vert J \vert\) where \(\vert J \vert\) is the determinant of the jacobian of the map \((X, U) \rightarrow (X, Y)\) (or the reciprocal of the opposite jacobian). Followed by marginilization. This is the generalization of \(Y = g(X)\) then \(f_Y(y) = f_X(g^{-1}(y)) dx/dy\) (given monotonic and same for above the mapping has to be bijection to be safe)

Order Statistics: Let \({x_i} \sim X\) i.i.d, and \(Y_k =\) kth minimum of \(\{x_i\}\). Then \(f_{Y_k}(x) = \frac{n!}{(k-1)!(n-k)!} [F_X(x)]^{k-1} [1 - F_X(x)]^{n-k} f_X(x)\).

Definition (Moment generating function): \(M(t) = \mathbb{E}[e^{tx}]\). Easy to check the nth derivative of MGF evaluated at \(t = 0\) gives \(\mathbb{E}[x^n]\). Also, \(M_{\prod X_i}(t) = \prod M_{X_i}(t)\).

A few probability distribution one should know -

1. Exponential Distribution ($\text{Exp}(\theta)$)
   • Gamma(\(\alpha=1, \theta\))
2. Gamma Distribution ($\text{Gamma}(\alpha, \theta)$)
   • PDF: \(f(x) = \frac{1}{\Gamma(\alpha) \theta^\alpha} x^{\alpha-1} e^{-x/\theta} \text{ for } x > 0\)
   • Mean: \(\mathbb{E}[X] = \alpha \theta\)
   • Variance: \(\text{Var}(X) = \alpha \theta^2\)
   • MGF: \(M_X(t) = (1 - t\theta)^{-\alpha} \quad \text{for } t < 1/\theta\)
   • $n$-th Moment: \(\mathbb{E}[X^n] = \frac{\theta^n \Gamma(\alpha + n)}{\Gamma(\alpha)}\)
3. Chi-Square Distribution ($\chi^2(k)$)
   • PDF: \(f(x) = \frac{1}{2^{k/2} \Gamma(k/2)} x^{k/2 - 1} e^{-x/2} \text{ for } x > 0\)
   • Mean: \(\mathbb{E}[X] = k\)
   • Variance: \(\text{Var}(X) = 2k\)
   • MGF: \(M_X(t) = (1 - 2t)^{-k/2} \quad \text{for } t < \frac{1}{2}\)
   • For large \(k\), approximate by Normal(\(k, 2k\)).

We will require a few probability distribution tricks up our sleeves:

1. If \(W_i \sim\) Gamma(\(\alpha_i , \theta\)), then \(\sum W_i \sim\) Gamma(\(\sum \alpha_i , \theta\)).
2. If \(W \sim\) Gamma(\(\alpha , \theta\)), then \(2W/\theta \sim \chi^2(2\alpha)\).
3. Let \(W \sim\) Gamma(\(\alpha , \theta\)), then Pr(\(W \ge t\)) = Pr(\(Y_t \lt \alpha\)) and Pr(\(W \le t\)) = Pr(\(Y_t \ge \alpha\)), where \(Y_t \sim\) Poisson(\(t/ \theta\)).

Todo:

Convergence in probability, continous mapping theorem, Convergence in distribution, why it is weakest, cts mapping thm and delta method, Convergence in MGF, implies Convergence in distribution, CLT theorem, confidence interval

1. Estimators