Bernoulli Distribution – Definition, Formulas & Examples

Distribution is a core concept in data analytics, data science, and machine learning. It lays the foundation for statistical analysis of a given dataset and provides the basis for certain machine learning models. There are various types of distributions used in statistics such as Normal or Gaussian distribution, Bernoulli Distribution, Binomial distribution, Poisson distribution, Exponential distribution, Gamma distribution, and Weibull distribution.

One of the most simple yet important types of distribution to get to grips with is the Bernoulli distribution, named after the Swiss mathematician Jacob Bernoulli. Let’s understand what is Bernoulli distribution, and the formulas used with examples.

What is Bernoulli Distribution?

Bernoulli Distribution is a special kind of distribution that is used to model real-life examples and can be used in many different types of applications. It is a type of discrete probability distribution where every experiment conducted asks a question that can be answered only with yes/no or true/false. In other words, the random variable can be $1$ with a probability $p$ or it can be $0$ with a probability $(1 – p)$. Such an experiment is called a Bernoulli trial. 

Note: If we have a binomial distribution where $n = 1$ then it becomes a Bernoulli distribution.

Examples of Bernoulli Distribution

The following are some examples of Bernoulli distribution.

• Will a coin land on the head?
• Will you pass or fail a test?
• Will your favorite sports team win or lose their next match?
• Will you score $90\%$ in your next exam?
• Will you roll a six in the opening round of your favorite board game?
• Will you win or lose the lottery?

Properties of Bernoulli Distribution

The following are the distinctive properties of a Bernoulli distribution.

Bernoulli distribution is a discrete probability distribution where the Bernoulli random variable can have only $0$ or $1$ as the outcome.

• $p$ is the probability of success and $1 – p$ is the probability of failure.
• The mean of a Bernoulli distribution is $\text{E}[\text{X}] = p$ and the variance, $\text{Var}[\text{X}] = p(1-p)$.
• Bernoulli distribution is a special case of binomial distribution when only $1$ trial is conducted.

What is the Difference Between a Bernoulli Distribution and Binomial Distribution?

While going through the definitions of Bernoulli distribution and binomial distribution, it seems that the two are the same. But there exists a difference between these two distributions. Let’s look at the difference between these two.

In very simplistic terms, a Bernoulli distribution is a type of binomial distribution. We know that the Bernoulli distribution applies to events that have one trial ($n = 1$) and two possible outcomes; for example, one coin flip and an outcome of either head or tail. When we have more than one trial; say, we flip a coin five times; binomial distribution gives the discrete probability distribution of the number of “successes” in that sequence of independent coin flips (or trials).

So, to continue with the coin flip example, the Bernoulli distribution gives you the probability of “success” (say, landing on heads) when flipping the coin just once (that’s your Bernoulli trial). If you flip the coin five times, binomial distribution will calculate the probability of success (landing on heads) across all five coin flips.

Probability Mass Function for Bernoulli Distribution

We calculate the probability mass function for a Bernoulli distribution. The probability that a discrete random variable will be exactly equal to some value is given by the probability mass function. The formula for pmf, $f$, associated with a Bernoulli random variable over possible outcomes $x$ is given as follows.

$\text{pmf} = f(x, p) = p, \text{if } x = 1, \text{ or } f(x, p) = q, \text{if } x = 0$

We can also express this formula as $f(x, p) = p^x (1 – p)^{1 – x}, x \in  \{0, 1\}$

Mean and Variance of Bernoulli Distribution

The arithmetic mean of a large number of independent realizations of the random variable $\text{X}$ gives us the expected value or mean. The expected value is also known as the weighted average. 

Mean of Bernoulli Distribution

We know that for $\text{X}$,

$\text{P}(\text{X} = 1) = p$

$\text{P}(\text{X} = 0) = q$

Therefore the expected value of $\text{X}$, $\text{E}[\text{X}] = \text{P}(\text{X} = 1) . 1 + \text{P}(\text{X} = 0) . 0$

$=> \text{E}[\text{X}] = p . 1 + q . 0$

$=> \text{E}[\text{X}] = p$

Thus, the mean or expected value of a Bernoulli distribution is given by $\text{E}[\text{X}] = p$.

Variance of Bernoulli Distribution

The variance can be defined as the difference between the mean of $\text{X}^2$ and the square of the mean of $\text{X}$. Mathematically this statement can be written as $\text{Var}[\text{X}] = \text{E}[\text{X}^2] – (\text{E}[\text{X}])^2$

Using the properties of $\text{E}[\text{X}^2]$, we get,

$\text{E}[\text{X}^2] = \sum x^2 \text{P}(\text{X} = x)$

$\text{E}[\text{X}^2] = 1^2 . p + 0^2 . q = p$

Substituting this value in $\text{Var}[\text{X}] = \text{E}[\text{X}^2] – (\text{E}[\text{X}])^2$ we have

$\text{Var}[\text{X}] = p – p^2$

$= p(1 – p)$

$= p . q$

Hence, the variance of a Bernoulli distribution is $\text{Var}[\text{X}] = p(1 – p) = p . q$

Examples on Bernoulli Distribution

Example 1: A football player can shoot a goal with a probability of $0.7$. What is the probability that he misses the shot?

$p = 0.7$

The probability of success is given by $p^x (1 – p)^{1 – x} = 0.7^1 (1 – 0.7)^{1 – 1}$

$= 0.7^1 \times 0.3^{1 – 1}$

$= 0.7 \times 0.3^0$

$= 0.7 \times 1 = 0.7 = 70\%$

Example 2: A marble is drawn from a bag containing $5$ blue and $7$ red marbles. Predict whether the trials are Bernoulli trials if the ball drawn 

1. is replaced 
2. not replaced

1. When a ball is drawn with replacement, the probability of success (say, red marble) is $p=\frac{7}{12}$, which is the same for all seven trials (draws). Hence, the trial involving the drawing of marbles with replacements are said to be Bernoulli trials.

2. For the second case, when a marble is drawn without replacement, the probability of success (say, red marble) varies with the number of trials. For example, for the first trial, the probability of success, $p= \frac{7}{12}$. For the second trial, the probability of success is $p=\frac{6}{11}$, which is not equal to the first trial. Hence, the trials involving the drawing of balls without replacements are not Bernoulli’s trials.

Example 3: If a Bernoulli distribution has a parameter $0.58$ then find its mean.

$p = 0.58$

Bernoulli mean = $p = 0.58$

Example 4: If a Bernoulli distribution has a parameter $0.72$ then find its variance.

$p = 0.72$

$q = 1 – p = 1 – 0.72 = 0.28$

Bernoulli variance = $pq = 0.72 \times 0.28 = 0.2016$ 

Applications of Bernoulli Distribution 

As Bernoulli distribution is a simple distribution it is widely used in many industries. The following are some of the applications of Bernoulli distribution.

• In medicine, Bernoulli distributions are used to model the events experienced by a single patient. These events could be disease, death, and so on.
• Logistic regressions use Bernoulli distribution to model the occurrence of certain events such as the specific outcome of a dice roll.
• Bernoulli distribution is also used as a basis to derive several other probability distributions that have applications in the engineering, aerospace, and medical industries.

Practice Problems

1. If the probability of the bulb being defective is $0.8$, then find the probability of the bulb not being defective.
2. If a Bernoulli distribution has a parameter $0.35$ then find its mean.
3. If a Bernoulli distribution has a parameter $0.92$ then find its variance.

FAQs

Conclusion

Bernoulli distribution is a type of discrete probability distribution where every experiment conducted asks a question that can be answered only with yes/no or true/false. In other words, the random variable can be $1$ with a probability $p$ or it can be $0$ with a probability $(1 – p)$.  The formula used to find the probability in the Bernoulli distribution is $f(x, p) = p^x (1 – p)^{1 – x}, x \in  \{0, 1\}$.

Recommended Reading

• What is Probability – Definition, Terminologies, Uses & Examples
• Probability Rules(With Formulas & Examples)
• What is Experimental Probability – Definition, Approach, Formula & Examples
• What is Theoretical Probability – Definition, Approach, Formula & Examples
• What is Binomial Distribution – Meaning, Formulas & Examples
• What is Poisson Distribution in Probability – Definition, Formulas & Examples