be considered an unfavorable game which most people choose to play.

Example 4.24 For the ¬rst example of the application of expected

value we consider the game of roulette as played at Monte Carlo. There

are several types of bets which the gambler can make, and we consider

two of these.

The wheel has the number 0 and the numbers from 1 to 36 marked

on equally spaced slots. The wheel is spun and a ball comes to rest

in one of these slots. If the player puts a stake, say of $1, on a given

number, and the ball comes to rest in this slot, then he or she receives

from the croupier 36 times the stake, or $36. The player wins $35

with probability 37 and loses $1 with probability 36 . Hence his or her

1

37

expected winnings are

1 36

36 · ’1· = ’.027.

37 37

This can be interpreted to mean that in the long run the player can

expect to lose about 2.7 per cent of his or her stakes.

A second way to play is the following. A player may bet on “red”

or “black”. The numbers from 1 to 36 are evenly divided between the

two colors. If a player bets on “red”, and a red number turns up, the

player receives twice the stake. If a black number turns up, the player

loses the stake. If 0 turns up, then the wheel is spun until it stops on

a number di¬erent from 0. If this is black, the player loses; but if it is

red, the player receives only the original stake, not twice it. For this

type of play, the player wins $1 with probability 18 , breaks even with

37

probability 2 · 37 , and loses $1 with probability 37 + 1 · 37 . Hence his

1 1 18 1

2

or her expected winning is

18 1 37

1· +0· ’1· = ’.0135.

37 74 74

In this case the player can expect to lose about 1.35 per cent of his

or her stakes in the long run. Thus the expected loss in this case is

™¦

only half as great as in the previous case.

160 CHAPTER 4. PROBABILITY THEORY

Example 4.25 A player rolls a die and receives a number of dollars

corresponding to the number of dots on the face which turns up. What

should the player pay for playing, to make this a fair game? To answer

this question, we note that the player wins 1, 2, 3, 4, 5 or 6 dollars,

1

each with probability 6 . Hence, the player™s expected winning is

1 1 1 1 1 1

1( ) + 2( ) + 3( ) + 4( ) + 5( ) + 6( ) = 3.5.

6 6 6 6 6 6

™¦

Thus if the player pays $3.50, the expected winnings will be zero.

Example 4.26 What is the expected number of successes in the case

1

of four independent trials with probability 3 for success? We know that

4

the probability of x successes is x ( 3 )x ( 2 )4’x . Thus

1

3

4 1024 4 1123 4 1222

E = 0· ( ) ( ) +1· ( ) ( ) +2· ( )( ) +

033 133 233

4 1321 4 1420

3· ( ) ( ) +4· ( )( )

333 433

32 48 24 4 108 4

= 0+ + + + = =.

81 81 81 81 81 3

In general, it can be shown that in n trials with probability p for success,

™¦

the expected number of successes is np.

Example 4.27 In the game of craps a pair of dice is rolled by one of

the players. If the sum of the spots shown is 7 or 11, he or she wins.

If it is 2, 3, or 12, he or she loses. If it is another sum, he or she must

continue rolling the dice until he or she either repeats the same sum or

rolls a 7. In the former case he or she wins, in the latter he or she loses.

Let us suppose that he or she wins or loses $1. Then the two possible

outcomes are +1 and ’1. We will compute the expected value of the

game. First we must ¬nd the probability that he or she will win.

We represent the possibilities by a two-stage tree shown in Figure

4.21. While it is theoretically possible for the game to go on inde¬-

nitely, we do not consider this possibility. This means that our analysis

applies only to games which actually stop at some time.

The branch probabilities at the ¬rst stage are determined by think-

ing of the 36 possibilities for the throw of the two dice as being equally

likely and taking in each case the fraction of the possibilities which

161

4.12. EXPECTED VALUE

Figure 4.21: ™¦

correspond to the branch as the branch probability. The probabilities

for the branches at the second level are obtained as follows. If, for

example, the ¬rst outcome was a 4, then when the game ends, a 4 or 7

must have occurred. The possible outcomes for the dice were

{(3, 1), (1, 3), (2, 2), (4, 3), (3, 4), (2, 5), (5, 2), (1, 6), (6, 1)}.

Again we consider these possibilities to be equally likely and assign to

the branch considered the fraction of the outcomes which correspond to

this branch. Thus to the 4 branch we assign a probability 9 = 1 . The

3

3

other branch probabilities are determined in a similar way. Having the

tree measure assigned, to ¬nd the probability of a win we must simply

add the weights of all paths leading to a win. If this is done, we obtain

244

. Thus the player™s expected value is

495

244 251 7

1·( ) + (’1) · ( )=’ = ’.0141.

495 495 495

Hence the player can expect to lose 1.41 per cent of his or her stakes

in the long run. It is interesting to note that this is just slightly less

™¦

favorable than the losses in betting on “red” in roulette.

162 CHAPTER 4. PROBABILITY THEORY

Exercises

1. Suppose that A tosses two coins and receives $2 if two heads

appear, $1 if one head appears, and nothing if no heads appear.

What is the expected value of the game to A?

[Ans. $1.]

2. Smith and Jones are matching coins. If the coins match, Smith

gets $1, and if they do not, Jones get $1.

(a) If the game consists of matching twice, what is the expected

value of the game for Smith?

(b) Suppose that Smith quits if he or she wins the ¬rst round he

or she quits, and plays the second round if he or she loses the

the ¬rst. Jones is not allowed to quit. What is the expected

value of the game for Smith?

3. If ¬ve coins are thrown, what is the expected number of heads

that will turn up?

5

[Ans. .]

2

4. A coin is thrown until the ¬rst time a head comes up or until

three tails in a row occur. Find the expected number of times the

coin is thrown.

5. A customer wishes to purchase a ¬ve cent newspaper. The cus-

tomer has in his or her pocket one dime and ¬ve pennies. The

news agent o¬ers to let the customer have the paper in exchange

for one coin drawn at random from the customer™s pocket.

(a) Is this a fair proposition and, if not, to whom is it favorable?

[Ans. Favorable to customer.]

(b) Answer the same question assuming that the news agent

demands two coins drawn at random from the customer™s

pocket.

[Ans. Fair proposition.]

6. A bets 50 cents against B™s x cents that, if two cards are dealt

from a shu¬„ed pack of ordinary playing cards, both cards will be

of the same color. What value of x will make this bet fair?

163

4.12. EXPECTED VALUE

7. Prove that if the expected value of a given experiment is E, and

if a constant c is added to each of the outcomes, the expected

value of the new experiment is E + c.

8. Prove that, if the expected value of a given experiment is E, and

if each of the possible outcomes is multiplied by a constant k, the

expected value of the new experiment is k · E.

9. A gambler plays the following game: A card is drawn from a

bridge deck; if it is an ace, the gambler wins $5; if it is a jack, a

queen or a king, he or she wins $2; for any other card he or she

loses $1. What is the expected winning per play?

10. An urn contains two black and three white balls. Balls are suc-

cessively drawn from the urn without replacement until a black

ball is obtained. Find the expected number of draws required.

11. Using the result of Exercises 13 and 14 of Section 4.6, ¬nd the

expected number of games in the World Series (a) under the as-

sumption that each team has probability 1 of winning each game

and (b) under the assumption that the stronger team has proba-

bility .6 of winning each game.

[Ans. 5.81; 5.75.]

12. Suppose that we modify the game of craps as follows: On a 7 or

11 the player wins $2, on a 2, 3, or 12 he or she loses $3; otherwise

the game is as usual. Find the expected value of the new game,

and compare it with the old value.

13. Suppose that in roulette at Monte Carlo we place 50 cents on

“red” and 50 cents on “black”. What is the expected value on

the game? Is this better or worse than placing $1 on “red”?

14. Betting on “red” in roulette can be described roughly as follows.

We win with probability .49, get our money back with probability

.01, and lose with probability .50. Draw the tree for three plays

of the game, and compute (to three decimals) the probability of

each path. What is the probability that we are ahead at the end

of three bets?

[Ans. .485.]

164 CHAPTER 4. PROBABILITY THEORY

15. Assume that the odds are r : s that a certain statement will be

true. If a gambler receives s dollars if the statement turns out

to be true, and gives r dollars if not, what is his or her expected

winning?

16. Referring to Exercise 9 of Section 4.3, ¬nd the expected number

of languages that a student chosen at random reads.

17. Referring to Exercise 5 of Section 4.4, ¬nd the expected number

of men who get their own hats.

[Ans. 1.]

18. A pair of dice is rolled. Each die has the number 1 on two opposite

faces, the number 2 on two opposite faces, and the number 3 on

two opposite faces. The “roller” wins a dollar if

(i) the sum of four occurs on the ¬rst roll; or

(ii) the sum of three or ¬ve occurs on the ¬rst roll and the same

sum occurs on a subsequent roll before the sum of four occurs.

Otherwise he or she loses a dollar.

(a) What is the probability that the person rolling the dice wins?

23

[Ans. .]

45

(b) What is the expected value of the game?

1

[Ans. .]

45

4.13 Markov chains

In this section we shall study a more general kind of process than the

ones considered in the last three sections.

We assume that we have a sequence of experiments with the fol-

lowing properties. The outcome of each experiment is one of a ¬nite

number of possible outcomes a1 , a2 , . . . , ar . It is assumed that the prob-

ability of outcome aj on any given experiment is not necessarily inde-

pendent of the outcomes of previous experiments but depends at most

upon the outcome of the immediately preceding experiment. We as-

sume that there are given numbers pij which represent the probability

165

4.13. MARKOV CHAINS

Figure 4.22: ™¦

of outcome aj on any given experiment, given that outcome ai oc-

curred on the preceding experiment. The outcomes a1 , a2 , . . . , ar are

called states, and the numbers pij are called transition probabilities. If

we assume that the process begins in some particular state, then we

have enough information to determine the tree measure for the process

and can calculate probabilities of statements relating to the over-all se-

quence of experiments. A process of the above kind is called a Markov

chain process.

The transition probabilities can be exhibited in two di¬erent ways.

The ¬rst way is that of a square array. For a Markov chain with states

a1 , a2 , and a3 , this array is written as

«

p11 p12 p13

¬ ·

P = p21 p22 p23 .

p31 p32 p33

Such an array is a special case of a matrix. Matrices are of fundamental

importance to the study of Markov chains as well as being important

in the study of other branches of mathematics. They will be studied in

detail in ??.

A second way to show the transition probabilities is by a transition

diagram. Such a diagram is illustrated for a special case in Figure

4.22. The arrows from each state indicate the possible states to which

a process can move from the given state.

The matrix of transition probabilities which corresponds to this

166 CHAPTER 4. PROBABILITY THEORY

diagram is the matrix

a1 a2 a3

«

a1 010

1·.

¬ 1

a2 0 2 2

1

02

a3 3 3

An entry of 0 indicates that the transition is impossible.

Notice that in the matrix P the sum of the elements of each row is

1. This must be true in any matrix of transition probabilities, since the

elements of the ith row represent the probabilities for all possibilities

when the process is in state ai .

The kind of problem in which we are most interested in the study

of Markov chains is the following. Suppose that the process starts in

state i. What is the probability that after n steps it will be in state

(n)

j? We denote this probability by pij . Notice that we do not mean by

this the nth power of the number pij . We are actually interested in this

probability for all possible starting positions i and all possible terminal

positions j. We can represent these numbers conveniently again by a

matrix. For example, for n steps in a three-state Markov chain we write

these probabilities as the matrix

«

(n) (n) (n)

p p12 p13

¬ 11 (n) ·

= ¬ p(n) p(n) p23 · .

P (n) 21

22

(n) (n) (n)

p31 p32 p33

Example 4.28 Let us ¬nd for a Markov chain with transition proba-

bilities indicated in Figure 4.22 the probability of being at the various

possible states after three steps, assuming that the process starts at

state a1 . We ¬nd these probabilities by constructing a tree and a tree

measure as in Figure 4.23.

(3)

The probability p13 , for example, is the sum of the weights assigned

by the tree measure to all paths through our tree which end at state

a3 . That is,

11 12 7

(3)

p13 = 1 · · + 1 · · = .

22 23 12

Similarly

11 1

(3)

p12 = 1 · · =

22 4

and

11 1

(3)

p11 = 1 · · = .

23 6

167

4.13. MARKOV CHAINS

Figure 4.23: ™¦

By constructing a similar tree measure, assuming that we start

(3) (3) (3)

at state a2 , we could ¬nd p21 ,p22 ,and p23 . The same is true for

(3) (3) (3)

p31 ,p32 ,and p33 . If this is carried out (see Exercise 7) we can write

the results in matrix form as follows:

a1 a2 a3

« 1 1 7

a1

P (3) = 6 4 12 .

¬ ·

7 7 37

a2

36 24 72

4 7 25

a3 27 18 54

Again the rows add up to 1, corresponding to the fact that if we start

at a given state we must reach some state after three steps. Notice

now that all the elements of this matrix are positive, showing that it is

possible to reach any state from any state in three steps. In the next

chapter we will develop a simple method of computing P (n) . ™¦

Example 4.29 example:4.13.2 Suppose that we are interested in study-

ing the way in which a given state votes in a series of national elec-

tions. We wish to make long-term predictions and so will not consider

conditions peculiar to a particular election year. We shall base our

predictions only on the past history of the outcomes of the elections,

Republican or Democratic. It is clear that a knowledge of these past

results would in¬‚uence our predictions for the future. As a ¬rst ap-

proximation, we assume that the knowledge of the past beyond the last

election would not cause us to change the probabilities for the outcomes

on the next election. With this assumption we obtain a Markov chain

with two states R and D and matrix of transition probabilities

RD

1’a

R a .

1’b

D b

168 CHAPTER 4. PROBABILITY THEORY

The numbers a and b could be estimated from past results as follows.

We could take for a the fraction of the previous years in which the