Bernoulli trial is an experiment that can result in two outcomes: success (with probability $p$) and failure (with probability $1-p$). A Bernoulli random variable $X$ can be represented as $X \sim Bernoulli(p)$. It’s mean $\mu_X$ and variance $\sigma_X^2$ can be computed as:
$$\mu_X = 0 \times (1-p) + 1 \times p = p$$
$$\sigma_X^2 = (0-p)^2(1-p) + (1-p)^2p = p(1-p)$$
#### The Binomial Distribution :When a set of $n$ independent Bernoulli trials are conducted, each with a success probability of $p$, a random variable $X$ which is equal to the number of success in these trials is said to have the binomial distribution with parameters $n$ and $p$ and is represented as $X \sim Bin(n, p)$. Probability mass function of a binomial distribution can be computed as:
$$P(X=x) = (number \ of \ arrangements \ of \ x \ successes \ in \ n \ trials ) \times p^x(1-p)^{n-x}$$
The total number of arrangements of $x$ successes in $n$ trials is given by $n \choose x$. Hence, the PMF can be given as:
$$p(x) = P(X=x) = {n \choose x }p^x(1-p)^{n-x} = \frac{n!}{x!(n-x)!} p^x(1-p)^{n-x}$$
Let the random variables representing the $n$ Bernoulli trials are $Y_1, Y_2, …, Y_n$. Each of these take a value of either 0 (for failure) or 1 (for success). Hence the binomial RV $X$ representing the number of successes in $n$ trials can be represented as the sum of these $n$ Bernoulli RVs, i.e. $X = Y_1 + Y_2 + … + Y_n$. The mean and variance of the binomial random variable can be calculated by applying the notion of it’s being the linear combination of $n$ independent Bernoulli RVs. They are given as:
$$\mu_X = np$$
$$\sigma_X^2 = np(1-p)$$
The success probability associated with a Bernoulli trial can be estimated as:
$$\widehat{p} = \frac{number \ of \ successes}{number \ of \ trials} = \frac{X}{n}$$
The bias and uncertainty associated with the estimate of the probability can be computed as:
$$Bias = \mu _{\widehat{p}} - p = \mu _{\frac{X}{n}} - p = \frac{\mu_X}{n} - p = \frac{np}{n} - p = 0$$
$$\sigma _{\widehat{p}} = \sigma _{\frac{X}{n}} = \frac{\sigma_X}{n} = \frac{\sqrt{np(1-p)}}{n} = \sqrt{\frac{p(1-p)}{n}}$$
#### The Poisson Distribution :The Poisson distribution is an approximation of the binomial distribution when $n$ is large and $p$ is small. When $n$ is large and $p$ is small, the probability mass funaction of the binomial distribution depends on mean $np$, rather than $n$ and $p$. Let $\lambda = np$, the PMF of a binomial distribution is given as:
$$P(X=x) = \frac{n!}{x!(n-x)!} p^x(1-p)^{n-x}$$
Substituting the value of $p$ as $\frac{\lambda}{n}$, we get
$$P(X=x) = \frac{n!}{x!(n-x)!} \bigg(\frac{\lambda}{n}\bigg)^x \bigg(1-\frac{\lambda}{n}\bigg)^{n-x}$$
To get a PMF of Poisson distribution, we need to evaluate the limit of the above quantity as $n \to \infty$. Evaluating the limit and pulling the constants out, we get
$$lim _{n \to \infty}P(X=x) = lim _{n \to \infty} \frac{n!}{x!(n-x)!} \bigg(\frac{\lambda}{n}\bigg)^x \bigg(1-\frac{\lambda}{n}\bigg)^{n-x} = \bigg( \frac{\lambda^x}{x!}\bigg) lim _{n \to \infty} \frac{n!}{(n-x)!} \bigg(\frac{1}{n^x}\bigg)\bigg(1-\frac{\lambda}{n}\bigg)^{n} \bigg(1-\frac{\lambda}{n}\bigg)^{-x}$$
The first two terms in the limit can be evaluated as:
$$lim _{n \to \infty} \frac{n!}{(n-x)!} \bigg(\frac{1}{n^x}\bigg) = lim _{n \to \infty} \frac{n(n-1)(n-2)…(n-x+1)}{n^x} = 1$$
For the evaluation of the limit of the remaining two terms, we need to use the property $e = lim _{k \to \infty} \bigg( 1+\frac{1}{k} \bigg)^k$. Substituting $k = \frac{-n}{\lambda}$ in the third term, we get
$$lim _{n \to \infty} \bigg(1-\frac{\lambda}{n}\bigg)^{n} = lim _{k \to \infty} \bigg(1 + \frac{1}{k}\bigg)^{-k\lambda} = e^{-\lambda}$$
The limit of the fourth term simply evaluates to 1, as $n \to \infty$, $\frac{\lambda}{n} \to 0$. Hence,
$$lim _{n \to \infty}P(X=x) = \bigg( \frac{\lambda^x}{x!}\bigg) e^{-\lambda}$$
Hence, the probability mass funaction of a Poisson distribution is given as:
$$p(x) = P(X=x) = e^{-\lambda} \frac{\lambda^x}{x!}$$
The mean and variance of the Poisson distribution is $\lambda$. An intutive explanation for this is as follows: As the Poisson distreibution is a binomial distribution when the value of $n$ is sufficiently large and $p$ sufficiently small. The mean of the Poisson distribution is same as that of the binomial distribution , which is $np = \lambda$. The variance of binomial distribution is $np(1-p)$ and as $p$ is small, $(1-p) \to 1$, and hence the variance of the Poisson distribution is $np = \lambda$. i.e. For a Poisson distribution,
$$\mu_X = np = \lambda$$
$$\sigma_X^2 = np = \lambda$$
#### The Multinomial Distribution :A generalization of the Bernoulli trial is the multinomial trial, which is the process that can result in any of the $k$ outcomes, where $k \geq 2$. If $n$ independent multinomial trials, each with the same $k$ possible outcomes, having the probabilities $p_1, p_2, …, p_k$ is conducted, and for each outcome $i$, let $X_i$ denotes the number of trials that result in that outcome, the collection $X_1, X_2, …, X_k$ is said to have a multinomial distribution with the parameters $n, p_1, p_2, …, p_k$, and is written as $X_1, X_2, …, X_k \sim MN(n, p_1, p_2, …, p_k)$. The probability mass funaction of the multinomial trial is given as:
$$p(x_1, x_2, …, x_k) = P(X_1 = x_1, X_2 = x_2, …, X_k = x_k) = \frac{n!}{x_1!x_2!…x_k!} p_1^{x_1} p_2^{x_2} … p_k^{x_k}$$
#### Reference :https://www.mheducation.com/highered/product/statistics-engineers-scientists-navidi/M0073401331.html