today”, “I have a fair chance of passing this course”, “There is an even

chance that a coin will come up heads”, etc. In each case our statement

refers to a situation in which we are not certain of the outcome, but we

express some degree of con¬dence that our prediction will be veri¬ed.

The theory of probability provides a mathematical framework for such

assertions.

Consider an experiment whose outcome is not known. Suppose that

someone makes an assertion p about the outcome of the experiment,

and we want to assign a probability to p. When statement p is consid-

ered in isolation, we usually ¬nd no natural assignment of probabilities.

Rather, we look for a method of assigning probabilities to all conceiv-

able statements concerning the outcome of the experiment. At ¬rst this

might seem to be a hopeless task, since there is no end to the state-

ments we can make about the experiment. However we are aided by a

basic principle:

Fundamental assumption. Any two equivalent statements will

be assigned the same probability. As long as there are a ¬nite number

of logical possibilities, there are only a ¬nite number of truth sets, and

hence the process of assigning probabilities is a ¬nite one. We proceed

in three steps: (l) we ¬rst determine U , the possibility set, that is, the

set of all logical possibilities, (2) to each subset X of U we assign a

number called the measure m(X), (3) to each statement p we assign

m(P ), the measure of its truth set, as a probability. The probability of

statement p is denoted by Pr[p].

83

84 CHAPTER 4. PROBABILITY THEORY

The ¬rst step, that of determining the set of logical possibilities,

is one that we considered in the previous chapters. It is important to

recall that there is no unique method for analyzing logical possibilities.

In a given problem we may arrive at a very ¬ne or a very rough analysis

of possibilities, causing U to have many or few elements.

Having chosen U , the next step is to assign a number to each sub-

set X of U , which will in turn be taken to be the probability of any

statement having truth set X. We do this in the following way.

Assignment of a measure. Assign a positive number (weight) to

each element of U , so that the sum of the weights assigned is 1. Then

the measure of a set is the sum of the weights of its elements. The

measure of the set … is 0.

In applications of probability to scienti¬c problems, the analysis of

the logical possibilities and the assignment of measures may depend

upon factual information and hence can best be done by the scientist

making the application.

Once the weights are assigned, to ¬nd the probability of a particular

statement we must ¬nd its truth set and ¬nd the sum of the weights

assigned to elements of the truth set. This problem, which might seem

easy, can often involve considerable mathematical di¬culty. The de-

velopment of techniques to solve this kind of problem is the main task

of probability theory.

Example 4.1 An ordinary die is thrown. What is the probability that

the number which turns up is less than four? Here the possibility set is

U = {1, 2, 3, 4, 5, 6}. The symmetry of the die suggests that each face

should have the same probability of turning up. To make this so, we

1

assign weight 6 to each of the outcomes. The truth set of the statement

“The number which turns up is less than four” is {1, 2, 3}. Hence the

probability of this statement is 3 = 1 , the sum of the weights of the

6 2

™¦

elements in its truth set.

Example 4.2 A gambler attends a race involving three horses A, B,

and C. He or she feels that A and B have the same chance of winning

but that A (and hence also B) is twice as likely to win as C is. What is

the probability that A or C wins? We take as U the set {A, B, C}. If we

were to assign weight a to the outcome C, then we would assign weight

2a to each of the outcomes A and B. Since the sum of the weights must

be l, we have 2a + 2a + a = 1, or a = 1 . Hence we assign weights 2 ,

5 5

21

, 5 to the outcomes A, B, and C, respectively. The truth set of the

5

85

4.1. INTRODUCTION

statement “Horse A or C wins” is {A, C}. The sum of the weights of

the elements of this set is 2 + 1 = 5 . Hence the probability that A or

3

5 5

3

™¦

C wins is 5 .

Exercises

1. Assume that there are n possibilities for the outcome of a given

experiment. How should the weights be assigned if it is desired

that all outcomes be assigned the same weight?

2. Let U = {a, b, c}. Assign weights to the three elements so that

no two have the same weight, and ¬nd the measures of the eight

subsets of U .

3. In an election Jones has probability 1 of winning, Smith has prob-

2

1 1

ability 3 , and Black has probability 6 .

(a) Construct U .

(b) Assign weights.

(c) Find the measures of the eight subsets.

(d) Give a pair of nonequivalent predictions which have the same

probability.

4. Give the possibility set U for each of the following experiments.

(a) An election between candidates A and B is to take place.

(b) A number from 1 to 5 is chosen at random.

(c) A two-headed coin is thrown.

(d) A student is asked for the day of the year on which his or

her birthday falls.

5. For which of the cases in Exercise 4 might it be appropriate to

assign the same weight to each outcome?

6. Suppose that the following probabilities have been assigned to the

possible results of putting a penny in a certain defective peanut-

1

vending machine: The probability that nothing comes out is 2 .

The probability that either you get your money back or you get

peanuts (but not both) is 1 .

3

86 CHAPTER 4. PROBABILITY THEORY

(a) What is the probability that you get your money back and

also get peanuts?

1

[Ans. .]

6

(b) From the information given, is it possible to ¬nd the proba-

bility that you get peanuts?

[Ans. No.]

7. A die is loaded in such a way that the probability of each face is

proportional to the number of dots on that face. (For instance, a

6 is three times as probable as a 2.) What is the probability of

getting an even number in one throw?

4

[Ans. .]

7

8. If a coin is thrown three times, list the eight possibilities for the

outcomes of the three successive throws. A typical outcome can

be written (HTH). Determine a probability measure by assigning

an equal weight to each outcome. Find the probabilities of the

following statements.

(a) r: The number of heads that occur is greater than the num-

ber of tails.

1

[Ans. .]

2

(b) s: Exactly two heads occur.

5

[Ans. .]

8

(c) t: The same side turns up on every throw.

1

[Ans. .]

4

9. For the statements given in Exercise 8, which of the following

equalities are true?

(a) Pr[r ∨ s] = Pr[r] + Pr[s]

(b) Pr[s ∨ t] = Pr[s] + Pr[t]

(c) Pr[r ∨ ¬r] = Pr[r] + Pr[¬r]

(d) Pr[r ∨ t] = Pr[r] + Pr[t]

87

4.2. PROPERTIES OF A PROBABILITY MEASURE

10. Which of the following pairs of statements (see Exercise 8) are

inconsistent? (Recall that two statements are inconsistent if their

truth sets have no element in common.)

(a) r, s

[Ans. consistent.]

(b) s, t

[Ans. inconsistent.]

(c) r, ¬r

[Ans. inconsistent.]

(d) r, t

[Ans. consistent.]

11. State a theorem suggested by Exercises 9 and 10.

12. An experiment has three possible outcomes, a, b, and c. Let p be

the statement “the outcome is a or b”, and q be the statement

“the outcome is b or c”. Assume that weights have been assigned

to the three outcomes so that Pr[p] = 2 and Pr[q] = 6 . Find the

5

3

weights.

111

[Ans. , , .]

623

1 3

13. Repeat Exercise 12 if Pr[p] = and Pr[q] = 8 .

2

4.2 Properties of a probability measure

Before studying special probability measures, we shall consider some

general properties of such measures which are useful in computations

and in the general understanding of probability theory.

Three basic properties of a probability measure are

(A) m(X) = 0 if and only if X = ….

(B) 0 < m(X) < 1 for any set X.

(C) For two sets X and Y ,

m(X ∪ Y ) = m(X) + m(Y )

if and only if X and Y are disjoint, i.e., have no elements in common.

88 CHAPTER 4. PROBABILITY THEORY

The proofs of properties (A) and (B) are left as an exercise (see

Exercise 16). We shall prove (C). We observe ¬rst that m(X) + m(Y )

is the sum of the weights of the elements of X added to the sum of the

weights of Y . If X and Y are disjoint, then the weight of every element

of X ∪ Y is added once and only once, and hence m(X) + m(Y ) =

m(X ∪ Y ). Assume now that X and Y are not disjoint. Here the

weight of every element contained in both X and Y , i.e., in X © Y ,

is added twice in the sum m(X) + m(Y ). Thus this sum is greater

than m(X ∪ Y ) by an amount m(X © Y ). By (A) and (B), if X © Y

is not the empty set, then m(X © Y ) > 0. Hence in this case we have

m(X) + m(Y ) > m(X © Y ). Thus if X and Y are not disjoint, the

equality in (C) does not hold. Our proof shows that in general we have

(C ) For any two sets X and Y , m(X ∪Y ) = m(X)+m(Y )’m(X ©Y ).

Since the probabilities for statements are obtained directly from the

probability measure m(X), any property of m(X) can be translated into

a property about the probability of statements. For example, the above

properties become, when expressed in terms of statements,

(a) Pr[p] = 0 if and only if p is logically false.

(b) 0 < Pr[p] < 1 for any statement p.

(c) The equality

Pr[p ∨ q] = Pr[p] + Pr[q]

holds and only if p and q are inconsistent.

(c ) For any two statements p and q,

Pr[p ∨ q] = Pr[p] + Pr[q] ’ Pr[p § q].

Another property of a probability measure which is often useful in

computation is

˜

(D) m(X) = 1 ’ m(X),

or, in the language of statements,

(d) Pr[¬p] = 1 ’ Pr[p].

The proofs of (D) and (d) are left as an exercise (see Exercise 17).

It is important to observe that our probability measure assigns prob-

ability 0 only to statements which are logically false, i.e., which are false

for every logical possibility. Hence, a prediction that such a statement

89

4.2. PROPERTIES OF A PROBABILITY MEASURE

will be true is certain to be wrong. Similarly, a statement is assigned

probability 1 only if it is true in every case, i.e., logically true. Thus

the prediction that a statement of this type will be true is certain to be

correct. (While these properties of a probability measure seem quite

natural, it is necessary, when dealing with in¬nite possibility sets, to

weaken them slightly. We consider in this book only the ¬nite possibil-

ity sets.)

We shall now discuss the interpretation of probabilities that are not

0 or 1. We shall give only some intuitive ideas that are commonly

held concerning probabilities. While these ideas can be made mathe-

matically more precise, we o¬er them here only as a guide to intuitive

thinking.

Suppose that, relative to a given experiment, a statement has been

assigned probability p. From this it is often inferred that if a sequence of

such experiments is performed under identical conditions, the fraction

of experiments which yield outcomes making the statement true would

be approximately p. The mathematical version of this is the “law of

large numbers” of probability theory (which will be treated in Section

4.10). In cases where there is no natural way to assign a probability

measure, the probability of a statement is estimated experimentally.

A sequence of experiments is performed and the fraction of the ex-

periments which make the statement true is taken as the approximate

probability for the statement.

A second and related interpretation of probabilities is concerned

with betting. Suppose that a certain statement has been assigned prob-

ability p. We wish to o¬er a bet that the statement will in fact turn

out to be true. We agree to give r dollars if the statement does not

turn out to be true, provided that we receive s dollars if it does turn

out to be true. What should r and s be to make the bet fair? If it were

true that in a large number of such bets we would win s a fraction p

of the times and lose r a fraction 1 ’ p of the time, then our average

winning per bet would be sp ’ r(1 ’ p). To make the bet fair we should

make this average winning 0. This will be the case if sp = r(1 ’ p)

or if r/s = p/(1 ’ p). Notice that this determines only the ratio of r

and s. Such a ratio, written r : s, is said to give odds in favor of the

statement.

De¬nition. The odds in favor of an outcome are r : s (r to s), if the

probability of the outcome is p, and r/s = p/(1 ’ p). Any two numbers

having the required ratio may be used in place of r and s. Thus 6 : 4

odds are the same as 3 : 2 odds.

90 CHAPTER 4. PROBABILITY THEORY

Example 4.3 Assume that a probability of 3 has been assigned to a

4

certain horse winning a race. Then the odds for a fair bet would be

3 1

: 4 . These odds could be equally well written as 3 : 1, 6 : 2 or 12 : 4,

4

etc. A fair bet would be to agree to pay $3 if the horse loses and receive

$1 if the horse wins. Another fair bet would be to pay $6 if the horse

™¦

loses and win $2 if the horse wins.

Exercises

1 1

1. Let p and q be statements such that Pr[p § q] = 4 , Pr[¬p] = 3 ,

1

and Pr[q] = 2 . What is Pr[p ∨ q]?

11

[Ans. .]

12

2. Using the result of Exercise 1, ¬nd Pr[¬p § ¬q].

1

and Pr[q] = 2 . Are

3. Let p and q be statements such that Pr[p] = 2 3

p and q consistent?

[Ans. Yes.]

4. Show that, if Pr[p] + Pr[q] > 1, then p and q are consistent.

5. A student is worried about his or her grades in English and Art.

The student estimates that the probability of passing English is

.4, the probability of passing at least one course with probability

.6, but that the probability of 1 of passing both courses is only

.1? What is the probability that the student will pass Art?

[Ans. .3.]

6. Given that a school has grades A, B, C, D, and F, and that a

student has probability .9 of passing a course, and .6 of getting a

grade lower than B, what is the probability that the student will

get a C or D?

1

[Ans. .]

2

7. What odds should a person give on a bet that a six will turn up

when a die is thrown?

91

4.2. PROPERTIES OF A PROBABILITY MEASURE

8. Referring to Example 4.2, what odds should the man be willing

to give for a bet that either A or B will come in ¬rst?

9. Prove that if the odds in favor of a given statement are r : s, then

the probability that the statement will be true is r/(r + s).

10. Using the result of Exercise 9 and the de¬nition of “odds”, show

that if the odds are r : s that a statement is true, then the odds

are s : r that it is false.

11. A gambler is willing to give 5 : 4 odds that the Dodgers will win

the World Series. What must the probability of a Dodger victory

be for this to be a fair bet?

5

[Ans. .]

9

12. A statistician has found through long experience that if he or she

washes the car, it rains the next day 85 per cent of the time.

What odds should the statistician give that this will occur next

time?

13. A gambler o¬ers 1 : 3 odds that A will occur, 1 : 2 odds that B

will occur. The gambler knows that A and B cannot both occur.

What odds should he or she give that A or B will occur?

[Ans. 7 : 5.]

14. A gambler o¬ers 3 : 1 odds that A will occur, 2 : 1 odds that B

will occur. The gambler knows that A and B cannot both occur.

What odds should he or she give that A or B will occur?

15. Show from the de¬nition of a probability measure that m(X) = 1

if and only if X = U .

16. Show from the de¬nition of a probability measure that properties

(A), (B) of the text are true.

17. Prove property (D) of the text. Why does property (d) follow

from this property?

18. Prove that if R, S, and T are three sets that have no element in

common,

m(R ∪ S ∪ T ) = m(R) + m(S) + m(T ).

92 CHAPTER 4. PROBABILITY THEORY

19. If X and Y are two sets such that X is a subset of Y , prove that

m(X) ¤ m(Y ).

20. If p and q are two statements such that p implies q, prove that

Pr[p] ¤ Pr[q].

21. Suppose that you are given n statements and each has been as-

signed a probability less than or equal to r. Prove that the prob-

ability of the disjunction of these statements is less than or equal

to nr.

22. The following is an alternative proof of property (C ) of the text.

Give a reason for each step.