not predicted before the tossing began.

We could also argue that perhaps the coin is a biased coin but this

would make us predict more heads than tails in the future. Thus the

law of averages, or the law of large numbers, should not give you great

comfort if you have had a series of very bad hands dealt you in your

last 100 poker hands. If the dealing is fair, you have the same chance

as ever of getting a good hand.

Early attempts to de¬ne the probability p that success occurs on

a single experiment sounded like this. If the experiment is repeated

inde¬nitely, the fraction of successes obtained will tend to a number p,

and this number p is called the probability of success on a single exper-

iment. While this fails to be satisfactory as a de¬nition of probability,

the law of large numbers captures the spirit of this frequency concept

of probability.

Exercises

1. If an ordinary die is thrown 20 times, what is the expected number

of times that a six will turn up? What is the standard deviation

for the number of sixes that turn up?

10 5

[Ans. ; .]

33

2. Suppose that an ordinary die is thrown 450 times. What is the

expected number of throws that result in either a three or a four?

What is the standard deviation for the number of such throws?

3. In 16 tosses of an ordinary coin, what is the expected number

of heads that turn up? What is the standard deviation for the

number of heads that occur?

151

4.10. THE LAW OF LARGE NUMBERS

[Ans. 8;2.]

4. In 16 tosses of a coin, ¬nd the exact probability that the number

of heads that turn up di¬ers from the expected number by (a)

as much as one standard deviation, and (b) by more than one

standard deviation. Do the same for the case of two standard

deviations, and for the case of three standard deviations. Show

that the approximations given for large n lie between the values

obtained, but are not very accurate for so small an n.

[Ans. .454; .210; .077; .021; .004; .001.]

5. Consider n independent trials with probability p for success. Let r

and s be numbers such that p < r < s. What does the law of large

x

numbers say about Pr[r < n < s] as we increase n inde¬nitely?

Answer the same question in the case that r < p < s.

6. A drug is known to be e¬ective in 20 per cent of the cases where

it is used. A new agent is introduced, and in the next 900 times

the drug is used it is e¬ective 250 times. What can be said about

the e¬ectiveness of the drug?

7. In a large number of independent trials with probability p for

success, what is the approximate probability that the number of

successes will deviate from the expected number by more than

one standard deviation but less than two standard deviations?

[Ans. .272.]

8. What is the approximate probability that, in 10,000 throws of an

ordinary coin, the number of heads which turn up lies between

4850 and 5150? What is the probability that the number of heads

lies in the same interval, given that in the ¬rst 1900 throws there

were 1600 heads?

9. Suppose that it is desired that the probability be approximately

.95 that the fraction of sixes that turn up when a die is thrown n

1

times does not deviate by more than .01 from the value 6 . How

large should n be?

[Ans. Approximately 5555.]

152 CHAPTER 4. PROBABILITY THEORY

10. Suppose that for each roll of a fair die you lose $1 when an odd

number comes up and win $1 when an even number comes up.

Then after 10,000 rolls you can, with approximately 84 per cent

con¬dence, expect to have lost not more than $(how much?).

11. Assume that 10 per cent of the people in a certain city have

cancer. If 900 people are selected at random from the city, what

is the expected number which will have cancer? What is the

standard deviation? What is the approximate probability that

more than 108 of the 900 chosen have cancer?

[Ans. 90;9;.023.]

12. Suppose that in Exercise 11, the 900 people are chosen at random

from those people in the city who smoke. Under the hypothesis

that smoking has no e¬ect on the incidence of cancer, what is the

expected number in the 900 chosen that have cancer? Suppose

that more than 120 of the 900 chosen have cancer, what might be

said concerning the hypothesis that smoking has no e¬ect on the

incidence of cancer?

13. In Example 4.20, we made the assumption in our calculations

that, if the true proportion of voters in favor of the proposal

were p, then the 900 people chosen at random represented an

independent trials process with probability p for a “yes” answer,

and 1 ’ p for a “no” answer. Give a method for choosing the 900

people which would make this a reasonable assumption. Criticize

the following methods.

(a) Choose the ¬rst 900 people in the list of registered Republi-

cans.

(b) Choose 900 names at random from the telephone book.

(c) Choose 900 houses at random and ask one person from each

house, the houses being visited in the mid-morning.

14. For n throws of an ordinary coin, let tn be such that

x1

’ < tn ] = .997,

Pr[’tn <

n2

where x is the number of heads that turn up. Find tn for n = 104 ,

n = 106 , and n = 1020 .

4.11. INDEPENDENT TRIALS WITH MORE THAN TWO OUTCOMES153

[Ans. .015; .0015; .000,000,000,15.]

15. Assume that a calculating machine carries out a million opera-

tions to solve a certain problem. In each operation the machine

gives the answer 10’5 too small, with probability 1 , and 10’5 too

2

1

large, with probability 2 . Assume that the errors are independent

of one another. What is a reasonable accuracy to attach to the

answer? What if the machine carries out 1010 operations?

[Ans. ±.01; ±1.]

16. A computer tosses a coin 1 million times, and obtains 499,588

heads. Is this number reasonable?

4.11 Independent trials with more than

two outcomes

By extending the results of Section 4.8, we shall study the case of

independent trials in which we allow more than two outcomes. We

assume that we have an independent trials process where the possible

outcomes are a1 , a2 , . . . , ak , occurring with probabilities p1 , p2 , . . . , pk ,

respectively. We denote by

f (r1 , r2 , . . . , rk ; p1 , p2 , . . . , pk )

the probability that, in n = r1 + r2 + . . . + rk such trials, there will be r1

occurrences of a1 , r2 occurrences of a2 , etc. In the case of two outcomes

this notation would be f (r1 , r2 ; p1 , p2 ). In Section 4.8 we wrote this as

f (n, r + 1; p) since r2 and p2 are determined from n, r1 , and p1 . We

shall indicate how this probability is found in general, but carry out

the details only for a special case. We choose k = 3, and n = 5 for

purposes of illustration. We shall ¬nd f (1, 2, 2; p1 , p2 , p3 ).

We show in Figure 4.20 enough of the tree for this process to indicate

the branch probabilities for a path (heavy lined) corresponding to the

outcomes a2 , a3 , a1 , a2 , a3 . The tree measure assigns weight p2 · p3 · p1 ·

p2 · p3 = p1 · p2 · p2 to this path.

2 3

There are, of course, other paths through the tree corresponding to

one occurrence of a1 , two of a2 , and two of a3 . However, they would all

be assigned the same weight p1 · p2 · p2 , by the tree measure. Hence to

2 3

154 CHAPTER 4. PROBABILITY THEORY

Figure 4.20: ™¦

¬nd f (l, 2, 2; p1 , p2 , p3 ) we must multiply this weight by the number of

paths having the speci¬ed number of occurrences of each outcome.

We note that the path a2 , a3 , a1 , a2 , a3 can be speci¬ed by the three-

cell partition [{3}, {1, 4}, {2, 5}] of the numbers from 1 to 5. Here the

¬rst cell shows the experiment which resulted in a1 , the second cell

shows the two that resulted in a2 , and the third shows the two that

resulted in a3 . Conversely, any such partition of the numbers from 1

to 5 with one element in the ¬rst cell, two in the second, and two in

the third corresponds to a unique path of the desired kind. Hence the

number of paths is the number of such partitions. But this is

5 5!

=

1, 2, 2 1!2!2!

(see 3.4), so that the probability of one occurrence of a1 , two of a2 , and

two of a3 is

5

· p1 · p2 · p2 .

2 3

1, 2, 2

The above argument carried out in general leads, for the case of

independent trials with outcomes a1 , a2 , . . . , ak occurring with proba-

bilities p1 , p2 , . . . , pk , to the following.

The probability for r1 occurrences of a1 , r2 occurrences of

a2 , etc., is given by

n

p r1 · p r2 · . . . · p rk .

f (r1 , r2 , . . . , rk ; p1 , p2 , . . . , pk ) =

r1 , r2 , . . . , rk 1 2 k

4.11. INDEPENDENT TRIALS WITH MORE THAN TWO OUTCOMES155

Example 4.22 A die is thrown 12 times. What is the probability that

each number will come up twice? Here there are six outcomes, 1, 2,

3, 4, 5, 6 corresponding to the six sides of the die. We assign each

1

outcome probability 6 . We are then asked for

111111

f (2, 2, 2, 2, 2, 2; , , , , , )

666666

which is

12 111111

( )2 ( )2 ( )2 ( )2 ( )2 ( )2 = .0034.

2, 2, 2, 2, 2, 2 6 6 6 6 6 6

™¦

Example 4.23 Suppose that we have an independent trials process

with four outcomes a1 , a2 , a3 , a4 occurring with probability p1 , p2 . p3 ,

p4 , respectively. It might be that we are interested only in the proba-

bility that r1 occurrences of a1 and r2 occurrences of a2 will take place

with no speci¬cation about the number of each of the other possible

outcomes. To answer this question we simply consider a new exper-

iment where the outcomes are a1 , a2 , a3 . Here a3 corresponds to an

¯ ¯

occurrence of either a3 or a4 in our original experiment. The corre-

sponding probabilities would be p1 , p2 and p3 with p3 = p3 + p4 . Let

¯ ¯

r3 = n ’ (r1 + r2 ) Then our question is answered by ¬nding the prob-

¯

ability in our new experiment for r1 occurrences of a1 , r2 of a2 , and r3 ¯

of a3 , which is

¯

n

pr1 · pr2 · p3 r¯3 .

¯

r1 , r2 , r 3 1 2

¯

™¦

The same procedure can be carried out for experiments with any

number of outcomes where we specify the number of occurrences of

such particular outcomes. For example, if a die is thrown ten times

the probability that a one will occur exactly twice and a three exactly

three times is given by

10 114

( )2 ( )2 ( )5 = .043.

2, 3, 5 6 6 6

156 CHAPTER 4. PROBABILITY THEORY

Exercises

1. Suppose that in a city 60 per cent of the population are Democrats,

30 per cent are Republicans, and 10 per cent are Independents.

What is the probability that if three people are chosen at random

there will be one Republican, one Democrat, and one Independent

voter?

[Ans. .108.]

2. Three horses, A, B, and C, compete in four races. Assuming

that each horse has an equal chance in each race, what is the

probability that A wins two races and B and C win one each?

What is the probability that the same horse wins all four races?

41

[Ans. ; .]

27 27

3. Assume that in a certain large college 40 per cent of the students

are freshmen, 30 per cent are sophomores, 20 per cent are juniors,

and 10 per cent are seniors. A committee of eight is chosen at

random from the student body. What is the probability that

there are equal numbers from each class on the committee?

4. Let us assume that when a batter comes to bat, he or she has

probability .6 of being put out, .1 of getting a walk, .2 of getting

a single, .1 of getting an extra base hit. If he or she comes to bat

¬ve times in a game, what is the probability that

(a) He gets two walks and three singles?

[Ans. .0008.]

(b) He gets a walk, a single, an extra base hit (and is out twice)?

[Ans. .043.]

(c) He has a perfect day (i.e., never out)?

[Ans. .010.]

1

5. Assume that a single torpedo has a probability 2 of sinking a

ship, probability 1 of damaging it, and probability 4 of missing.

1

4

Assume further that two damaging shots sink the ship. What

is the probability that four torpedos will succeed in sinking the

ship?

4.11. INDEPENDENT TRIALS WITH MORE THAN TWO OUTCOMES157

251

[Ans. .]

256

6. Jones, Smith, and Green live in the same house. The mailman has

observed that Jones and Smith receive the same amount of mail

on the average, but that Green receives twice as much as Jones

(and hence also twice as much as Smith). If he or she has four

letters for this house, what is the probability that each resident

receives at least one letter?

7. If three dice are thrown, ¬nd the probability that there is one six

and two ¬ves, given that all the outcomes are greater than three.

1

[Ans. .]

9

8. An athlete plays a tournament consisting of three games. In each

1 1 1

game he or she has probability 2 for a win, 4 for a loss, and 4 for a

draw, independently of the outcomes of other games. To win the

tournament he or she must win more games than he or she loses.

What is the probability that he or she wins the tournament?

9. Assume that in a certain course the probability that a student

chosen at random will get an A is .1, that he or she will get a B

is .2, that he or she will get a C is .4, that he or she will get a D

is .2, and that he or she will get an F is .1. What distribution of

grades is most likely in the case of four students?

[Ans. One B, two C™s, one D.]

10. Let us assume that in a World Series game a batter has probability

1 1 1

of getting no hits, 2 for getting one hit, 4 for getting two hits,

4

assuming that the probability of getting more than two hits is

negligible. In a four-game World Series, ¬nd the probability that

the batter gets

(a) Exactly two hits.

7

[Ans. .]

64

(b) Exactly three hits.

7

[Ans. .]

32

(c) Exactly four hits.

158 CHAPTER 4. PROBABILITY THEORY

35

[Ans. .]

128

(d) Exactly ¬ve hits.

7

[Ans. .]

32

(e) Fewer than two hits or more than ¬ve.

23

[Ans. .]

128

11. Gypsies sometimes toss a thick coin for which heads and tails are

1

equally likely, but which also has probability 5 of standing on

edge (i.e., neither heads nor tails). What is the probability of

exactly one head and four tails in ¬ve tosses of a gypsy coin?

12. A family car is driven by the father, two sons, and the mother.

The fenders have been dented four times, three times while the

mother was driving. Is it fair to say that the mother is a worse

driver than the men?

4.12 Expected value

In this section we shall discuss the concept of expected value. Although

it originated in the study of gambling games, it enters into almost any

detailed probabilistic discussion.

De¬nition. If in an experiment the possible outcomes are numbers,

a1 , a2 , . . . , ak , occurring with probability p1 , p2 , . . . , pk , then the expected

value is de¬ned to be

E = a 1 p1 + a 2 p2 + . . . + a k pk .

The term “expected value” is not to be interpreted as the value that

will necessarily occur on a single experiment. For example, if a person

bets $1 that a head will turn up when a coin is thrown, he or she may

either win $1 or lose $1. His expected value is (1)( 1 ) + (’1)( 1 ) = 0,

2 2

which is not one of the possible outcomes. The term, expected value,

had its origin in the following consideration. If we repeat an experiment

with expected value E a large number of times, and if we expect a1 a

fraction p1 of the time, a2 a fraction p2 of the time, etc., then the average

that we expect per experiment is E. In particular, in a gambling game

E is interpreted as the average winning expected in a large number

of plays. Here the expected value is often taken as the value of the

game to the player. If the game has a positive expected value, the

159

4.12. EXPECTED VALUE

game is said to be favorable; if the game has expected value zero it

is said to be fair; and if it has negative expected value it is described

as unfavorable. These terms are not to be taken too literally, since

many people are quite happy to play games that, in terms of expected