Teacher resources and professional development across the curriculum

Teacher professional development and classroom resources across the curriculum

Monthly Update sign up
Mailing List signup

Unit 7

Making Sense of Randomness

7.4 Law of Large Numbers


• Bernoulli's Law of Large Numbers shifted the thinking about probability from determining short-term payoffs to predicting long-term behavior.

In the preceding section, with the use of the Galton board, we found the theoretical probability that a marble will end up in any specific bin. Now let's turn our attention to what actually happens when we let a marble go through the board; furthermore, let's see what happens when many marbles go through it!

Each path is equally likely, and we have to assume that marbles dropped randomly into our machine are not predestined to follow any particular path. Because the number of paths to each of the bins varies, we should expect that over time, bins that have more paths leading to them will end up with more marbles than bins that have fewer paths leading to them. Thus, the distribution of a large number of marbles through the machine will not be even. The Law of Large Numbers will help us to predict roughly the distribution that we would find were we to run such an experiment.

The Law of Large Numbers says that when a random process, such as dropping marbles through a Galton board, is repeated many times, the frequencies of the observed outcomes get increasingly closer to the theoretical probabilities. Jacob Bernoulli, the man who is credited with discovering this law around the beginning of the 18th century, is said to have claimed that this observation was so simple that even the dullest person knows it to be true. Despite this pronouncement, it took him over 20 years to develop a rigorous mathematical proof of the concept.

Let's look at the Law of Large Numbers in terms of the Galton board:

Galton board Diagram Showing Lots of Balls Going Through the 2 Row Galton Board

Recall that we found the probability of a ball ending up in the middle bin to be 1/2. According to the Law of Large Numbers, if we ran 100 marbles through this setup, about 50 of them would end up in the middle bin. If we ran 1000 marbles, about 500 would end up in the middle. Furthermore, as we run more marbles through the board, the proportion in the middle bin will get closer and closer to1/2.

Bernoulli may have thought that this concept is self-evident, but it nevertheless is striking. Recall that we can't say with any certainty at all where one particular marble will end up. Still, we can say with very high accuracy how 1000 marbles will end up. Better yet, the more marbles we run, the better our prediction will be. The Law of Large Numbers is a powerful tool for taming randomness.

back to top


The notion of expected value, or expectation, codifies the "average behavior" of a random event and is a key concept in the application of probability. For example, imagine that you are a door-to-door salesperson. Your experience tells you that the probability of making a sale, and thus a commission, on each try is as follows: 8/10 that you make no sale and make no commission, 3/20 that you make a small sale that leads to a $100 commission, and 1/20 that you make a large sale that leads to a $500 commission.2 How much can you expect to make, on average, per appointment? That is, what do you "expect" to be the value of the total sales divided by the number of appointments? This will be an expected value.

The expectation or expected value of a random process in which each outcome has a particular payoff is simply the sum of the individual probabilities multiplied by their corresponding payoffs. If P1 is the probability of outcome #1 and V1 is the payoff value of that outcome, and so on (Pj and Vj for the jth outcome and jth payoff value respectively), then the expected value can be represented by the expression:

P1V1 + P2V2 +...+PNVN where n = number of possible outcomes

In our sales example, the individual terms are as follows: for a no-sale, 8/10 $0 = $0; for the small commission, 3/20 $100 = $15; and for the large commission, 1/20 × $500 = $25. Thus, the expected value of the sales call payoff, per appointment, is 0 + $15 + $25 = $40. This, of course, does not mean that you will make $40 for every appointment, but it is what you can "expect" to make on average over a period of time (assuming that your probabilities are correct!).

The Law of Large Numbers ensures that the more sales calls you make, the closer your average payoff, per appointment, will be to $40.

he concept of expected value, in conjunction with the Law of Large Numbers, help form the operating principle of businesses that are based on risk. Two prominent examples are casinos and insurance companies. Let's look a little more closely at each.

Any single casino game carries a certain risk for both the player and the house. A player's loss is the house's gain and vice versa. It would seem that no business could thrive in such a zero-sum situation, yet generally, the casino business is quite lucrative. This is possible because while the individual player's risk is concentrated in a small number of hands or rounds of a game, the casino's risk is spread out among all the games and all the bets going on. In short, casinos have the Law of Large Numbers working in their favor. Owners and managers of casinos know that while the outcome of any single game is unpredictable, the outcome of many rounds of that same game is entirely predictable. For instance, they know that the probability of rolling a seven at the craps table is 1/6. Averaging this over many rolls means that a player will roll a seven 1/6 of the time. In other words, in a group of six players, only one, on average, will be a winner. The casinos then structure their payoffs or "odds" slightly in their favor so that the money paid out to any player who wins will be more than offset by the money taken in from the five players who, on average, don't win. Note that this does not require any sort of rigging or cheating as far as actual game play. Casinos don't need to cheat the individual gambler-as long as they keep their doors open, the odds settle in their favor. They've structured their payoffs to guarantee it in the long run and because they generally have more working capital than any of the players, they can take advantage of the long-term "guarantees."

Insurance companies use similar principles to set premiums. They spend a great deal of effort and resources calculating the odds of certain catastrophes, such as a house fire, then multiply this value by the payoff they would give in such an event. This amount is how much the company can expect to have to pay, on average, for each person that they cover. They then set their rates at levels that cover this "expense" in addition to providing their profit. The policyholder gets peace of mind because the insurance company has effectively mitigated the risk of potential loss in a given catastrophe.

The insurance company gets a flow of regular payments in exchange for a massive payoff in the unlikely event of a big claim.

The Law of Large Numbers is a powerful tool that enables us to say definite things about the real-world results of accumulated instances of unpredictable events. This useful tool represents just one example of how mathematics can be used to deal with randomness. The Law of Large Numbers applies to specific outcomes and their probabilities, but what about the entire range of possible outcomes and their associated probabilities? Just as the frequency of a specific event will tend toward its probability over the long run, the full set of possible outcomes will each tend to their own probabilities. Studying the distribution of possible outcomes and probabilities will give us even more powerful tools with which to predict long-term average behavior.

back to top

Next: 7.5 The Galton Board Revisited


© Annenberg Foundation 2016. All rights reserved. Legal Policy