<< . .

. 9
( : 20)



. . >>

We often hear statements of the following kind: “It is likely to rain
today”, “I have a fair chance of passing this course”, “There is an even
chance that a coin will come up heads”, etc. In each case our statement
refers to a situation in which we are not certain of the outcome, but we
express some degree of con¬dence that our prediction will be veri¬ed.
The theory of probability provides a mathematical framework for such
assertions.
Consider an experiment whose outcome is not known. Suppose that
someone makes an assertion p about the outcome of the experiment,
and we want to assign a probability to p. When statement p is consid-
ered in isolation, we usually ¬nd no natural assignment of probabilities.
Rather, we look for a method of assigning probabilities to all conceiv-
able statements concerning the outcome of the experiment. At ¬rst this
might seem to be a hopeless task, since there is no end to the state-
ments we can make about the experiment. However we are aided by a
basic principle:
Fundamental assumption. Any two equivalent statements will
be assigned the same probability. As long as there are a ¬nite number
of logical possibilities, there are only a ¬nite number of truth sets, and
hence the process of assigning probabilities is a ¬nite one. We proceed
in three steps: (l) we ¬rst determine U , the possibility set, that is, the
set of all logical possibilities, (2) to each subset X of U we assign a
number called the measure m(X), (3) to each statement p we assign
m(P ), the measure of its truth set, as a probability. The probability of
statement p is denoted by Pr[p].

83
84 CHAPTER 4. PROBABILITY THEORY

The ¬rst step, that of determining the set of logical possibilities,
is one that we considered in the previous chapters. It is important to
recall that there is no unique method for analyzing logical possibilities.
In a given problem we may arrive at a very ¬ne or a very rough analysis
of possibilities, causing U to have many or few elements.
Having chosen U , the next step is to assign a number to each sub-
set X of U , which will in turn be taken to be the probability of any
statement having truth set X. We do this in the following way.
Assignment of a measure. Assign a positive number (weight) to
each element of U , so that the sum of the weights assigned is 1. Then
the measure of a set is the sum of the weights of its elements. The
measure of the set … is 0.
In applications of probability to scienti¬c problems, the analysis of
the logical possibilities and the assignment of measures may depend
upon factual information and hence can best be done by the scientist
making the application.
Once the weights are assigned, to ¬nd the probability of a particular
statement we must ¬nd its truth set and ¬nd the sum of the weights
assigned to elements of the truth set. This problem, which might seem
easy, can often involve considerable mathematical di¬culty. The de-
velopment of techniques to solve this kind of problem is the main task
of probability theory.

Example 4.1 An ordinary die is thrown. What is the probability that
the number which turns up is less than four? Here the possibility set is
U = {1, 2, 3, 4, 5, 6}. The symmetry of the die suggests that each face
should have the same probability of turning up. To make this so, we
1
assign weight 6 to each of the outcomes. The truth set of the statement
“The number which turns up is less than four” is {1, 2, 3}. Hence the
probability of this statement is 3 = 1 , the sum of the weights of the
6 2
™¦
elements in its truth set.


Example 4.2 A gambler attends a race involving three horses A, B,
and C. He or she feels that A and B have the same chance of winning
but that A (and hence also B) is twice as likely to win as C is. What is
the probability that A or C wins? We take as U the set {A, B, C}. If we
were to assign weight a to the outcome C, then we would assign weight
2a to each of the outcomes A and B. Since the sum of the weights must
be l, we have 2a + 2a + a = 1, or a = 1 . Hence we assign weights 2 ,
5 5
21
, 5 to the outcomes A, B, and C, respectively. The truth set of the
5
85
4.1. INTRODUCTION

statement “Horse A or C wins” is {A, C}. The sum of the weights of
the elements of this set is 2 + 1 = 5 . Hence the probability that A or
3
5 5
3
™¦
C wins is 5 .


Exercises
1. Assume that there are n possibilities for the outcome of a given
experiment. How should the weights be assigned if it is desired
that all outcomes be assigned the same weight?

2. Let U = {a, b, c}. Assign weights to the three elements so that
no two have the same weight, and ¬nd the measures of the eight
subsets of U .

3. In an election Jones has probability 1 of winning, Smith has prob-
2
1 1
ability 3 , and Black has probability 6 .

(a) Construct U .
(b) Assign weights.
(c) Find the measures of the eight subsets.
(d) Give a pair of nonequivalent predictions which have the same
probability.

4. Give the possibility set U for each of the following experiments.

(a) An election between candidates A and B is to take place.
(b) A number from 1 to 5 is chosen at random.
(c) A two-headed coin is thrown.
(d) A student is asked for the day of the year on which his or
her birthday falls.

5. For which of the cases in Exercise 4 might it be appropriate to
assign the same weight to each outcome?

6. Suppose that the following probabilities have been assigned to the
possible results of putting a penny in a certain defective peanut-
1
vending machine: The probability that nothing comes out is 2 .
The probability that either you get your money back or you get
peanuts (but not both) is 1 .
3
86 CHAPTER 4. PROBABILITY THEORY

(a) What is the probability that you get your money back and
also get peanuts?
1
[Ans. .]
6

(b) From the information given, is it possible to ¬nd the proba-
bility that you get peanuts?
[Ans. No.]

7. A die is loaded in such a way that the probability of each face is
proportional to the number of dots on that face. (For instance, a
6 is three times as probable as a 2.) What is the probability of
getting an even number in one throw?

4
[Ans. .]
7


8. If a coin is thrown three times, list the eight possibilities for the
outcomes of the three successive throws. A typical outcome can
be written (HTH). Determine a probability measure by assigning
an equal weight to each outcome. Find the probabilities of the
following statements.

(a) r: The number of heads that occur is greater than the num-
ber of tails.
1
[Ans. .]
2

(b) s: Exactly two heads occur.
5
[Ans. .]
8

(c) t: The same side turns up on every throw.
1
[Ans. .]
4

9. For the statements given in Exercise 8, which of the following
equalities are true?

(a) Pr[r ∨ s] = Pr[r] + Pr[s]
(b) Pr[s ∨ t] = Pr[s] + Pr[t]
(c) Pr[r ∨ ¬r] = Pr[r] + Pr[¬r]
(d) Pr[r ∨ t] = Pr[r] + Pr[t]
87
4.2. PROPERTIES OF A PROBABILITY MEASURE

10. Which of the following pairs of statements (see Exercise 8) are
inconsistent? (Recall that two statements are inconsistent if their
truth sets have no element in common.)
(a) r, s
[Ans. consistent.]
(b) s, t
[Ans. inconsistent.]
(c) r, ¬r
[Ans. inconsistent.]
(d) r, t
[Ans. consistent.]
11. State a theorem suggested by Exercises 9 and 10.
12. An experiment has three possible outcomes, a, b, and c. Let p be
the statement “the outcome is a or b”, and q be the statement
“the outcome is b or c”. Assume that weights have been assigned
to the three outcomes so that Pr[p] = 2 and Pr[q] = 6 . Find the
5
3
weights.
111
[Ans. , , .]
623

1 3
13. Repeat Exercise 12 if Pr[p] = and Pr[q] = 8 .
2



4.2 Properties of a probability measure
Before studying special probability measures, we shall consider some
general properties of such measures which are useful in computations
and in the general understanding of probability theory.
Three basic properties of a probability measure are
(A) m(X) = 0 if and only if X = ….
(B) 0 < m(X) < 1 for any set X.
(C) For two sets X and Y ,
m(X ∪ Y ) = m(X) + m(Y )
if and only if X and Y are disjoint, i.e., have no elements in common.
88 CHAPTER 4. PROBABILITY THEORY

The proofs of properties (A) and (B) are left as an exercise (see
Exercise 16). We shall prove (C). We observe ¬rst that m(X) + m(Y )
is the sum of the weights of the elements of X added to the sum of the
weights of Y . If X and Y are disjoint, then the weight of every element
of X ∪ Y is added once and only once, and hence m(X) + m(Y ) =
m(X ∪ Y ). Assume now that X and Y are not disjoint. Here the
weight of every element contained in both X and Y , i.e., in X © Y ,
is added twice in the sum m(X) + m(Y ). Thus this sum is greater
than m(X ∪ Y ) by an amount m(X © Y ). By (A) and (B), if X © Y
is not the empty set, then m(X © Y ) > 0. Hence in this case we have
m(X) + m(Y ) > m(X © Y ). Thus if X and Y are not disjoint, the
equality in (C) does not hold. Our proof shows that in general we have
(C ) For any two sets X and Y , m(X ∪Y ) = m(X)+m(Y )’m(X ©Y ).
Since the probabilities for statements are obtained directly from the
probability measure m(X), any property of m(X) can be translated into
a property about the probability of statements. For example, the above
properties become, when expressed in terms of statements,
(a) Pr[p] = 0 if and only if p is logically false.
(b) 0 < Pr[p] < 1 for any statement p.
(c) The equality
Pr[p ∨ q] = Pr[p] + Pr[q]
holds and only if p and q are inconsistent.
(c ) For any two statements p and q,

Pr[p ∨ q] = Pr[p] + Pr[q] ’ Pr[p § q].

Another property of a probability measure which is often useful in
computation is
˜
(D) m(X) = 1 ’ m(X),
or, in the language of statements,
(d) Pr[¬p] = 1 ’ Pr[p].
The proofs of (D) and (d) are left as an exercise (see Exercise 17).
It is important to observe that our probability measure assigns prob-
ability 0 only to statements which are logically false, i.e., which are false
for every logical possibility. Hence, a prediction that such a statement
89
4.2. PROPERTIES OF A PROBABILITY MEASURE

will be true is certain to be wrong. Similarly, a statement is assigned
probability 1 only if it is true in every case, i.e., logically true. Thus
the prediction that a statement of this type will be true is certain to be
correct. (While these properties of a probability measure seem quite
natural, it is necessary, when dealing with in¬nite possibility sets, to
weaken them slightly. We consider in this book only the ¬nite possibil-
ity sets.)
We shall now discuss the interpretation of probabilities that are not
0 or 1. We shall give only some intuitive ideas that are commonly
held concerning probabilities. While these ideas can be made mathe-
matically more precise, we o¬er them here only as a guide to intuitive
thinking.
Suppose that, relative to a given experiment, a statement has been
assigned probability p. From this it is often inferred that if a sequence of
such experiments is performed under identical conditions, the fraction
of experiments which yield outcomes making the statement true would
be approximately p. The mathematical version of this is the “law of
large numbers” of probability theory (which will be treated in Section
4.10). In cases where there is no natural way to assign a probability
measure, the probability of a statement is estimated experimentally.
A sequence of experiments is performed and the fraction of the ex-
periments which make the statement true is taken as the approximate
probability for the statement.
A second and related interpretation of probabilities is concerned
with betting. Suppose that a certain statement has been assigned prob-
ability p. We wish to o¬er a bet that the statement will in fact turn
out to be true. We agree to give r dollars if the statement does not
turn out to be true, provided that we receive s dollars if it does turn
out to be true. What should r and s be to make the bet fair? If it were
true that in a large number of such bets we would win s a fraction p
of the times and lose r a fraction 1 ’ p of the time, then our average
winning per bet would be sp ’ r(1 ’ p). To make the bet fair we should
make this average winning 0. This will be the case if sp = r(1 ’ p)
or if r/s = p/(1 ’ p). Notice that this determines only the ratio of r
and s. Such a ratio, written r : s, is said to give odds in favor of the
statement.
De¬nition. The odds in favor of an outcome are r : s (r to s), if the
probability of the outcome is p, and r/s = p/(1 ’ p). Any two numbers
having the required ratio may be used in place of r and s. Thus 6 : 4
odds are the same as 3 : 2 odds.
90 CHAPTER 4. PROBABILITY THEORY

Example 4.3 Assume that a probability of 3 has been assigned to a
4
certain horse winning a race. Then the odds for a fair bet would be
3 1
: 4 . These odds could be equally well written as 3 : 1, 6 : 2 or 12 : 4,
4
etc. A fair bet would be to agree to pay $3 if the horse loses and receive
$1 if the horse wins. Another fair bet would be to pay $6 if the horse
™¦
loses and win $2 if the horse wins.


Exercises
1 1
1. Let p and q be statements such that Pr[p § q] = 4 , Pr[¬p] = 3 ,
1
and Pr[q] = 2 . What is Pr[p ∨ q]?

11
[Ans. .]
12


2. Using the result of Exercise 1, ¬nd Pr[¬p § ¬q].
1
and Pr[q] = 2 . Are
3. Let p and q be statements such that Pr[p] = 2 3
p and q consistent?

[Ans. Yes.]

4. Show that, if Pr[p] + Pr[q] > 1, then p and q are consistent.

5. A student is worried about his or her grades in English and Art.
The student estimates that the probability of passing English is
.4, the probability of passing at least one course with probability
.6, but that the probability of 1 of passing both courses is only
.1? What is the probability that the student will pass Art?

[Ans. .3.]

6. Given that a school has grades A, B, C, D, and F, and that a
student has probability .9 of passing a course, and .6 of getting a
grade lower than B, what is the probability that the student will
get a C or D?

1
[Ans. .]
2


7. What odds should a person give on a bet that a six will turn up
when a die is thrown?
91
4.2. PROPERTIES OF A PROBABILITY MEASURE

8. Referring to Example 4.2, what odds should the man be willing
to give for a bet that either A or B will come in ¬rst?
9. Prove that if the odds in favor of a given statement are r : s, then
the probability that the statement will be true is r/(r + s).
10. Using the result of Exercise 9 and the de¬nition of “odds”, show
that if the odds are r : s that a statement is true, then the odds
are s : r that it is false.
11. A gambler is willing to give 5 : 4 odds that the Dodgers will win
the World Series. What must the probability of a Dodger victory
be for this to be a fair bet?
5
[Ans. .]
9

12. A statistician has found through long experience that if he or she
washes the car, it rains the next day 85 per cent of the time.
What odds should the statistician give that this will occur next
time?
13. A gambler o¬ers 1 : 3 odds that A will occur, 1 : 2 odds that B
will occur. The gambler knows that A and B cannot both occur.
What odds should he or she give that A or B will occur?

[Ans. 7 : 5.]

14. A gambler o¬ers 3 : 1 odds that A will occur, 2 : 1 odds that B
will occur. The gambler knows that A and B cannot both occur.
What odds should he or she give that A or B will occur?
15. Show from the de¬nition of a probability measure that m(X) = 1
if and only if X = U .
16. Show from the de¬nition of a probability measure that properties
(A), (B) of the text are true.
17. Prove property (D) of the text. Why does property (d) follow
from this property?
18. Prove that if R, S, and T are three sets that have no element in
common,

m(R ∪ S ∪ T ) = m(R) + m(S) + m(T ).
92 CHAPTER 4. PROBABILITY THEORY

19. If X and Y are two sets such that X is a subset of Y , prove that
m(X) ¤ m(Y ).

20. If p and q are two statements such that p implies q, prove that
Pr[p] ¤ Pr[q].

21. Suppose that you are given n statements and each has been as-
signed a probability less than or equal to r. Prove that the prob-
ability of the disjunction of these statements is less than or equal
to nr.

22. The following is an alternative proof of property (C ) of the text.
Give a reason for each step.

<< . .

. 9
( : 20)



. . >>