<< . .

. 17
( : 20)

. . >>

value, are unfavorable. For instance, the buying of life insurance may
be considered an unfavorable game which most people choose to play.

Example 4.24 For the ¬rst example of the application of expected
value we consider the game of roulette as played at Monte Carlo. There
are several types of bets which the gambler can make, and we consider
two of these.
The wheel has the number 0 and the numbers from 1 to 36 marked
on equally spaced slots. The wheel is spun and a ball comes to rest
in one of these slots. If the player puts a stake, say of $1, on a given
number, and the ball comes to rest in this slot, then he or she receives
from the croupier 36 times the stake, or $36. The player wins $35
with probability 37 and loses $1 with probability 36 . Hence his or her
expected winnings are

1 36
36 · ’1· = ’.027.
37 37
This can be interpreted to mean that in the long run the player can
expect to lose about 2.7 per cent of his or her stakes.
A second way to play is the following. A player may bet on “red”
or “black”. The numbers from 1 to 36 are evenly divided between the
two colors. If a player bets on “red”, and a red number turns up, the
player receives twice the stake. If a black number turns up, the player
loses the stake. If 0 turns up, then the wheel is spun until it stops on
a number di¬erent from 0. If this is black, the player loses; but if it is
red, the player receives only the original stake, not twice it. For this
type of play, the player wins $1 with probability 18 , breaks even with
probability 2 · 37 , and loses $1 with probability 37 + 1 · 37 . Hence his
1 1 18 1
or her expected winning is

18 1 37
1· +0· ’1· = ’.0135.
37 74 74
In this case the player can expect to lose about 1.35 per cent of his
or her stakes in the long run. Thus the expected loss in this case is
only half as great as in the previous case.

Example 4.25 A player rolls a die and receives a number of dollars
corresponding to the number of dots on the face which turns up. What
should the player pay for playing, to make this a fair game? To answer
this question, we note that the player wins 1, 2, 3, 4, 5 or 6 dollars,
each with probability 6 . Hence, the player™s expected winning is

1 1 1 1 1 1
1( ) + 2( ) + 3( ) + 4( ) + 5( ) + 6( ) = 3.5.
6 6 6 6 6 6
Thus if the player pays $3.50, the expected winnings will be zero.

Example 4.26 What is the expected number of successes in the case
of four independent trials with probability 3 for success? We know that
the probability of x successes is x ( 3 )x ( 2 )4’x . Thus

4 1024 4 1123 4 1222
E = 0· ( ) ( ) +1· ( ) ( ) +2· ( )( ) +
033 133 233
4 1321 4 1420
3· ( ) ( ) +4· ( )( )
333 433
32 48 24 4 108 4
= 0+ + + + = =.
81 81 81 81 81 3
In general, it can be shown that in n trials with probability p for success,
the expected number of successes is np.

Example 4.27 In the game of craps a pair of dice is rolled by one of
the players. If the sum of the spots shown is 7 or 11, he or she wins.
If it is 2, 3, or 12, he or she loses. If it is another sum, he or she must
continue rolling the dice until he or she either repeats the same sum or
rolls a 7. In the former case he or she wins, in the latter he or she loses.
Let us suppose that he or she wins or loses $1. Then the two possible
outcomes are +1 and ’1. We will compute the expected value of the
game. First we must ¬nd the probability that he or she will win.
We represent the possibilities by a two-stage tree shown in Figure
4.21. While it is theoretically possible for the game to go on inde¬-
nitely, we do not consider this possibility. This means that our analysis
applies only to games which actually stop at some time.
The branch probabilities at the ¬rst stage are determined by think-
ing of the 36 possibilities for the throw of the two dice as being equally
likely and taking in each case the fraction of the possibilities which

Figure 4.21: ™¦

correspond to the branch as the branch probability. The probabilities
for the branches at the second level are obtained as follows. If, for
example, the ¬rst outcome was a 4, then when the game ends, a 4 or 7
must have occurred. The possible outcomes for the dice were

{(3, 1), (1, 3), (2, 2), (4, 3), (3, 4), (2, 5), (5, 2), (1, 6), (6, 1)}.

Again we consider these possibilities to be equally likely and assign to
the branch considered the fraction of the outcomes which correspond to
this branch. Thus to the 4 branch we assign a probability 9 = 1 . The
other branch probabilities are determined in a similar way. Having the
tree measure assigned, to ¬nd the probability of a win we must simply
add the weights of all paths leading to a win. If this is done, we obtain
. Thus the player™s expected value is

244 251 7
1·( ) + (’1) · ( )=’ = ’.0141.
495 495 495
Hence the player can expect to lose 1.41 per cent of his or her stakes
in the long run. It is interesting to note that this is just slightly less
favorable than the losses in betting on “red” in roulette.

1. Suppose that A tosses two coins and receives $2 if two heads
appear, $1 if one head appears, and nothing if no heads appear.
What is the expected value of the game to A?

[Ans. $1.]

2. Smith and Jones are matching coins. If the coins match, Smith
gets $1, and if they do not, Jones get $1.
(a) If the game consists of matching twice, what is the expected
value of the game for Smith?
(b) Suppose that Smith quits if he or she wins the ¬rst round he
or she quits, and plays the second round if he or she loses the
the ¬rst. Jones is not allowed to quit. What is the expected
value of the game for Smith?
3. If ¬ve coins are thrown, what is the expected number of heads
that will turn up?
[Ans. .]

4. A coin is thrown until the ¬rst time a head comes up or until
three tails in a row occur. Find the expected number of times the
coin is thrown.
5. A customer wishes to purchase a ¬ve cent newspaper. The cus-
tomer has in his or her pocket one dime and ¬ve pennies. The
news agent o¬ers to let the customer have the paper in exchange
for one coin drawn at random from the customer™s pocket.
(a) Is this a fair proposition and, if not, to whom is it favorable?
[Ans. Favorable to customer.]
(b) Answer the same question assuming that the news agent
demands two coins drawn at random from the customer™s
[Ans. Fair proposition.]
6. A bets 50 cents against B™s x cents that, if two cards are dealt
from a shu¬„ed pack of ordinary playing cards, both cards will be
of the same color. What value of x will make this bet fair?

7. Prove that if the expected value of a given experiment is E, and
if a constant c is added to each of the outcomes, the expected
value of the new experiment is E + c.

8. Prove that, if the expected value of a given experiment is E, and
if each of the possible outcomes is multiplied by a constant k, the
expected value of the new experiment is k · E.

9. A gambler plays the following game: A card is drawn from a
bridge deck; if it is an ace, the gambler wins $5; if it is a jack, a
queen or a king, he or she wins $2; for any other card he or she
loses $1. What is the expected winning per play?

10. An urn contains two black and three white balls. Balls are suc-
cessively drawn from the urn without replacement until a black
ball is obtained. Find the expected number of draws required.

11. Using the result of Exercises 13 and 14 of Section 4.6, ¬nd the
expected number of games in the World Series (a) under the as-
sumption that each team has probability 1 of winning each game
and (b) under the assumption that the stronger team has proba-
bility .6 of winning each game.

[Ans. 5.81; 5.75.]

12. Suppose that we modify the game of craps as follows: On a 7 or
11 the player wins $2, on a 2, 3, or 12 he or she loses $3; otherwise
the game is as usual. Find the expected value of the new game,
and compare it with the old value.

13. Suppose that in roulette at Monte Carlo we place 50 cents on
“red” and 50 cents on “black”. What is the expected value on
the game? Is this better or worse than placing $1 on “red”?

14. Betting on “red” in roulette can be described roughly as follows.
We win with probability .49, get our money back with probability
.01, and lose with probability .50. Draw the tree for three plays
of the game, and compute (to three decimals) the probability of
each path. What is the probability that we are ahead at the end
of three bets?

[Ans. .485.]

15. Assume that the odds are r : s that a certain statement will be
true. If a gambler receives s dollars if the statement turns out
to be true, and gives r dollars if not, what is his or her expected

16. Referring to Exercise 9 of Section 4.3, ¬nd the expected number
of languages that a student chosen at random reads.

17. Referring to Exercise 5 of Section 4.4, ¬nd the expected number
of men who get their own hats.

[Ans. 1.]

18. A pair of dice is rolled. Each die has the number 1 on two opposite
faces, the number 2 on two opposite faces, and the number 3 on
two opposite faces. The “roller” wins a dollar if

(i) the sum of four occurs on the ¬rst roll; or

(ii) the sum of three or ¬ve occurs on the ¬rst roll and the same
sum occurs on a subsequent roll before the sum of four occurs.

Otherwise he or she loses a dollar.

(a) What is the probability that the person rolling the dice wins?
[Ans. .]

(b) What is the expected value of the game?
[Ans. .]

4.13 Markov chains
In this section we shall study a more general kind of process than the
ones considered in the last three sections.
We assume that we have a sequence of experiments with the fol-
lowing properties. The outcome of each experiment is one of a ¬nite
number of possible outcomes a1 , a2 , . . . , ar . It is assumed that the prob-
ability of outcome aj on any given experiment is not necessarily inde-
pendent of the outcomes of previous experiments but depends at most
upon the outcome of the immediately preceding experiment. We as-
sume that there are given numbers pij which represent the probability

Figure 4.22: ™¦

of outcome aj on any given experiment, given that outcome ai oc-
curred on the preceding experiment. The outcomes a1 , a2 , . . . , ar are
called states, and the numbers pij are called transition probabilities. If
we assume that the process begins in some particular state, then we
have enough information to determine the tree measure for the process
and can calculate probabilities of statements relating to the over-all se-
quence of experiments. A process of the above kind is called a Markov
chain process.
The transition probabilities can be exhibited in two di¬erent ways.
The ¬rst way is that of a square array. For a Markov chain with states
a1 , a2 , and a3 , this array is written as

« 
p11 p12 p13
¬ ·
P =  p21 p22 p23  .
p31 p32 p33

Such an array is a special case of a matrix. Matrices are of fundamental
importance to the study of Markov chains as well as being important
in the study of other branches of mathematics. They will be studied in
detail in ??.
A second way to show the transition probabilities is by a transition
diagram. Such a diagram is illustrated for a special case in Figure
4.22. The arrows from each state indicate the possible states to which
a process can move from the given state.
The matrix of transition probabilities which corresponds to this

diagram is the matrix
a1 a2 a3
a1 010
¬ 1
a2 0 2 2
a3 3 3

An entry of 0 indicates that the transition is impossible.
Notice that in the matrix P the sum of the elements of each row is
1. This must be true in any matrix of transition probabilities, since the
elements of the ith row represent the probabilities for all possibilities
when the process is in state ai .
The kind of problem in which we are most interested in the study
of Markov chains is the following. Suppose that the process starts in
state i. What is the probability that after n steps it will be in state
j? We denote this probability by pij . Notice that we do not mean by
this the nth power of the number pij . We are actually interested in this
probability for all possible starting positions i and all possible terminal
positions j. We can represent these numbers conveniently again by a
matrix. For example, for n steps in a three-state Markov chain we write
these probabilities as the matrix
« 
(n) (n) (n)
p p12 p13
¬ 11 (n) ·
= ¬ p(n) p(n) p23 · .
P (n)  21 
(n) (n) (n)
p31 p32 p33

Example 4.28 Let us ¬nd for a Markov chain with transition proba-
bilities indicated in Figure 4.22 the probability of being at the various
possible states after three steps, assuming that the process starts at
state a1 . We ¬nd these probabilities by constructing a tree and a tree
measure as in Figure 4.23.
The probability p13 , for example, is the sum of the weights assigned
by the tree measure to all paths through our tree which end at state
a3 . That is,
11 12 7
p13 = 1 · · + 1 · · = .
22 23 12
11 1
p12 = 1 · · =
22 4
11 1
p11 = 1 · · = .
23 6

Figure 4.23: ™¦

By constructing a similar tree measure, assuming that we start
(3) (3) (3)
at state a2 , we could ¬nd p21 ,p22 ,and p23 . The same is true for
(3) (3) (3)
p31 ,p32 ,and p33 . If this is carried out (see Exercise 7) we can write
the results in matrix form as follows:
a1 a2 a3 
« 1 1 7
P (3) = 6 4 12 .
¬ ·
7 7 37
a2  
36 24 72
4 7 25
a3 27 18 54

Again the rows add up to 1, corresponding to the fact that if we start
at a given state we must reach some state after three steps. Notice
now that all the elements of this matrix are positive, showing that it is
possible to reach any state from any state in three steps. In the next
chapter we will develop a simple method of computing P (n) . ™¦

Example 4.29 example:4.13.2 Suppose that we are interested in study-
ing the way in which a given state votes in a series of national elec-
tions. We wish to make long-term predictions and so will not consider
conditions peculiar to a particular election year. We shall base our
predictions only on the past history of the outcomes of the elections,
Republican or Democratic. It is clear that a knowledge of these past
results would in¬‚uence our predictions for the future. As a ¬rst ap-
proximation, we assume that the knowledge of the past beyond the last
election would not cause us to change the probabilities for the outcomes
on the next election. With this assumption we obtain a Markov chain
with two states R and D and matrix of transition probabilities
R a .
D b

The numbers a and b could be estimated from past results as follows.
We could take for a the fraction of the previous years in which the

<< . .

. 17
( : 20)

. . >>