(iii) Show by means of examples that, if both ∞ a+ and ∞ a’ di-

j=1 j j=1 j

∞

verge, then j=1 aj may converge or may diverge.

(iv) Show that if ∞ aj is convergent but not absolutely convergent, then

j=1

∞ ∞

both j=1 aj and j=1 a’ diverge.

+

j

3

See [31].

81

Please send corrections however trivial to twk@dpmms.cam.ac.uk

(Exercise K.68 contains some advice on testing for convergence. Like

most such advice it ceases to be helpful at the ¬rst point that you really

need it.)

In his ground-breaking papers on number theory, Dirichlet manipulated

conditionally convergent sums in very imaginative ways. However, he was

always extremely careful to justify such manipulations. To show why he took

such care he gave a very elegant speci¬c counterexample which is reproduced

in Exercise K.72. The following more general and extremely striking result

is due to Riemann.

Theorem 5.2.9. Let aj ∈ R. If ∞ aj is convergent but not absolutely

j=1

convergent, then given any l ∈ R we can ¬nd a permutation σ of N+ = {n ∈

Z : n ≥ 1} (that is a bijection σ : N+ ’ N+ ) with ∞ aσ(j) convergent to

j=1

l.

We give a proof in the next exercise.

∞

Exercise 5.2.10. Suppose that aj is convergent but not absolutely con-

j=1

vergent and that l ∈ R. Set

S(0) = {n ∈ N+ : an < 0}, T (0) = {n ∈ N+ : an ≥ 0}.

We use the following inductive construction.

n’1

At the nth step we proceed as follows. If j=1 aσ(j) < l, let mn =

min T (n) and set σ(n) = mn , T (n+1) = T (n)\{mn } and S(n+1) = S(n). If

n’1

j=1 aσ(j) ≥ l, let mn = min S(n) and set σ(n) = mn , S(n+1) = S(n)\{mn }

and T (n + 1) = T (n).

(i) Describe the construction in your own words. Carry out the ¬rst few

steps with an = (’1)n /n and suitable l (for example l = 2, l = ’1).

(ii) Show that σ is indeed a permutation.

(iii) If κ > |a| for all a ∈ S(n) ∪ T (n), m > n and S(m) = S(n),

T (m) = T (n) show that | m aσ(j) ’ l| < κ. (It may be helpful to consider

j=1

m

what happens when j=1 aσ(j) ’ l changes sign.)

(iv) Conclude that n aσ(j) ’ l as n ’ ∞.

j=1

Let me add that Example 5.2.9 is not intended to dissuade you from using

conditionally convergent sums, but merely to warn you to use them carefully.

As Dirichlet showed, it is possible to be both imaginative and rigorous.

In A Mathematician™s Miscellany [35], Littlewood presents an example

which helps understand what happens in Riemann™s theorem. Slightly dressed

up it runs as follows.

I have an in¬nite supply of gold sovereigns and you have none. At one

minute to noon I give you 10 sovereigns, at 1/2 minutes to noon I take one

82 A COMPANION TO ANALYSIS

sovereign back from you but give you 10 more sovereigns in exchange, at 1/3

minutes to noon I take one sovereign back from you but give you 10 more

sovereigns in exchange and so on (that is, at 1/n minutes to noon I take

one sovereign back from you but give you 10 more sovereigns in exchange

[n ≥ 2]). The process stops at noon. How rich will you be?

As it stands the question is not su¬ciently precise. Here are two refor-

mulations.

Reformulation A I have an in¬nite supply of gold sovereigns labelled s1 , s2 ,

. . . and you have none. At one minute to noon I give you the sovereigns

sj with 1 ¤ j ¤ 10, at 1/2 minutes to noon I take the sovereign s1 back

from you but give you the sovereigns sj with 11 ¤ j ¤ 20, in exchange,

at 1/3 minutes to noon I take the sovereign s2 back from you but give you

the sovereigns sj with 21 ¤ j ¤ 30, in exchange and so on (that is, at 1/n

minutes to noon I take the sovereign sn’1 back from you but give you the

sovereigns sj with 10n ’ 9 ¤ j ¤ 10n, in exchange). The process stops at

noon. How rich will you be? In this case, it is clear that I have taken all my

sovereigns back (I took sovereign sn back at 1/(n + 1) minutes to noon) so

you are no richer than before.

Reformulation B I have an in¬nite supply of gold sovereigns labelled s1 , s2 ,

. . . and you have none. At one minute to noon I give you the sovereigns

sj with 1 ¤ j ¤ 10, at 1/2 minutes to noon I take the sovereign s2 back

from you but give you the sovereigns sj with 11 ¤ j ¤ 20, in exchange, at

1/3 minutes to noon I take the sovereign s4 back from you but give you the

sovereigns the sj with 21 ¤ j ¤ 30, in exchange and so on (that is, at 1/n

minutes to noon I take the sovereign s2(n’1) back from you but give you the

sovereigns sj with 10n ’ 9 ¤ j ¤ 10n, in exchange). The process stops at

noon. How rich will you be? In this case, it is clear that I have given you

all my odd numbered sovereigns and taken none of them back. You are now

in¬nitely rich.

Exercise 5.2.11. Give a reformulation in which you end up with precisely

N sovereigns.

Remark: There is a faint smell of sulphur about Littlewood™s example. (What

happens if all the gold pieces are indistinguishable4 ?) However, most mathe-

maticians would agree that the original problem was not correctly posed and

that reformulations A and B are well de¬ned problems with the answers we

have given.

Littlewood™s paradox depends on my having an in¬nite supply of gold

sovereigns. In the same way Dirichlet showed that the phenomenon described

4

There is a connection with Zeno™s paradoxes. See Chapter II, Section 4 of [19] or in

rather less detail (but accompanied by many other splendid paradoxes) in [41].

83

Please send corrections however trivial to twk@dpmms.cam.ac.uk

in Example 5.2.9 cannot occur if the sum is absolutely convergent. We shall

prove this as Lemma 5.3.4 in the next section.

Interchanging limits ™

5.3

We often ¬nd ourselves wishing to interchange the order of two limiting

processes. Among the examples we shall discuss in the course of this book

are

∞ ∞ ∞ ∞

?

aij = aij (interchanging the order of summation)

i=1 j=1 j=1 i=1

b d d b

?

f (x, y) dx dy = f (x, y) dy dx (interchanging the order of integration)

a c c a

‚2f ? ‚2f

= (changing the order of partial di¬erentiation)

‚x‚y ‚y‚x

b b

?

lim fn (x) dx = lim fn (x) dx (limit of integral is integral of limit)

n’∞ a a n’∞

b b

d ‚f

?

f (x, y) dx = (x, y) dx (di¬erentiation under the integral sign)

dy ‚y

a a

(The list is not exhaustive. It is good practice to make a mental note each

time you meet a theorem dealing with the interchange of limits.)

Unfortunately, it is not always possible to interchange limits. The reader

should study the following example carefully. (A good counterexample may

be as informative as a good theorem.)

Exercise 5.3.1. Take anm = 1 if m ¤ n, anm = 0 otherwise. Write out the

values of anm in matrix form (say for 1 ¤ n, m ¤ 5). Find limn’∞ anm

when m is ¬xed and limm’∞ anm when n is ¬xed. Show that

lim ( lim anm ) = lim ( lim anm )

m’∞ n’∞ n’∞ m’∞

In his Course of Pure Mathematics [23], Hardy writes.

If L and L are two limit operations then the numbers LL z and

L Lz are not generally equal in the strict sense of the word ˜gen-

eral™. We can always, by the exercise of a little ingenuity, ¬nd z

so that LL z and L Lz shall di¬er from one another. But they are

equal generally if we use the word in a more practical sense, viz.

as meaning ˜in the great majority of such cases as are likely to

occur naturally™. In practice, a result obtained by assuming that

84 A COMPANION TO ANALYSIS

two limit operations are commutative is probably true; at any rate

it gives a valuable suggestion of the answer to the problem under

consideration. But an answer thus obtained, must in default of

further study . . . be regarded as suggested only and not proved.

To this I would add that, with experience, analysts learn that some limit

interchanges are much more likely to lead to trouble than others.

One particularly dangerous interchange is illustrated in the next example.

Exercise 5.3.2. Take anm = 1/n if m ¤ n, anm = 0 otherwise. Write out

the values of anm in matrix form (say for 1 ¤ n, m ¤ 5). Show that

∞ ∞

lim anm = lim anm .

n’∞ n’∞

m=1 m=1

We shall see a similar phenomenon in Example 11.4.12, and, in Exer-

cise 11.4.14 following that example, we shall look at a branch of mathematics

in which, contrary to Hardy™s dictum, the failure of limit interchange is the

rule rather than the exception.

Exercise 5.3.2 involves an ˜escape to in¬nity™ which is prevented by the

conditions of the next lemma.

Lemma 5.3.3. (Dominated convergence.) Suppose that cj ≥ 0 and

∞

j=1 cj converges. If aj (n) ∈ R

m

and aj (n) ¤ cj for all n, then, if

aj (n) ’ aj as n ’ ∞, it follows that ∞ aj converges and

j=1

∞ ∞

aj (n) ’ aj

j=1 j=1

as n ’ ∞.

∞

Proof. Let > 0 be given. Since j=1 cj converges, it follows from the

general principle of convergence that we can ¬nd an P ( ) such that

q

cj ¤ /3

j=p

for all q ≥ p ≥ P ( ). It follows that

q q

aj (n) ¤ cj ¤ /3 (1)

j=P ( )+1 j=P ( )+1

85

Please send corrections however trivial to twk@dpmms.cam.ac.uk

for all q ≥ P ( ) and all n. Allowing q to tend to in¬nity, we have

∞

aj (n) ¤ /3 (2)

j=P ( )+1

for all n.

Returning to the inequality (1), but this time letting n tend to in¬nity

whilst keeping q ¬xed, we obtain

q q

aj ¤ cj ¤ /3

j=P ( )+1 j=P ( )+1

for all q ≥ P ( ). Since an increasing sequence bounded above converges,

∞ ∞

this shows that j=P ( )+1 aj . converges. Since j=P ( )+1 aj converges

∞

absolutely it converges and the same must be true of j=P ( )+1 aj (n) We

now allow q to tend to in¬nity to obtain

∞

aj ¤ /3. (3)

j=P ( )+1

Since aj (n) ’ aj , we can ¬nd Mj ( ) such that

aj (n) ’ aj ¤ /(6P ( ))

for all n ≥ Mj ( ). In particular, taking M ( ) = max1¤j¤P ( ) Mj ( ) we have

P( ) P( )

aj (n) ’ aj ¤ /3 (4)

j=1 j=1

for all n ≥ M ( ). Combining the inequalities (2), (3) and (4), we obtain

∞ ∞

aj (n) ’ aj ¤

j=1 j=1

for n ≥ M ( ) so we are done.

If we look at Littlewood™s sovereigns, the hypothesis in the dominated

convergence theorem can be interpreted as saying that we have only ∞ cjj=1

sovereigns to play with. It is thus, perhaps, not surprising that we can use

the dominated convergence theorem to prove that an absolutely convergent

sum can be rearranged in any way we wish without a¬ecting its convergence.

86 A COMPANION TO ANALYSIS

Lemma 5.3.4. We work in Rm . If ∞ aj is absolutely convergent and σ

j=1

is a permutation of N+ , then ∞ aσ(j) is absolutely convergent and

j=1

∞ ∞

aσ(j) = aj .

j=1 j=1

Proof. De¬ne aj (n) = aj if j ∈ {σ(1), σ(2), . . . , σ(n)} and aj (n) = 0

otherwise. Then aj (n) ¤ aj for all j and n and

∞

n

aσ(j) = aj (n),

j=1 j=1

so, by the dominated convergence theorem,

∞

n

aσ(j) ’ aj .

j=1 j=1

To see that ∞ aσ(j) is absolutely convergent, apply the result just obtained

j=1

with aj in place of aj .

The reader may feel that there is only one ˜natural™ method of de¬ning

r≥0 ar , to wit the standard de¬nition

N

ar = lim ar .

N ’∞

r=0

r≥0

I would disagree with her even for this case5 , but it is clear that there are

several ˜natural™ methods of de¬ning r,s≥0 ars . Among the possibilities are

N N N r

lim ars , lim a(r’s) s , and lim ars

N ’∞ N ’∞ N ’∞

r=0 s=0 r=0 s=0 r 2 +s2 ¤N

(that is, considering sums over squares, triangles or quadrants of circles).

Fortunately it is clear that each of these summation methods is a rearrange-

ment of the others and so, provided we have absolute convergence, it does not

matter which we use.

We give the details in the next lemma but the reader who is already

convinced can safely ignore them.

5

Communications engineers sometimes use ˜hard summation™

lim ar .

δ’0+

ar >δ

87

Please send corrections however trivial to twk@dpmms.cam.ac.uk

Lemma 5.3.5. Suppose that E is an in¬nite set and that E1 , E2 , . . . and

F1 , F2 , . . . are ¬nite subsets of E such that

(i) E1 ⊆ E2 ⊆ . . . , ∞ Ek = E,

k=1

(ii) F1 ⊆ F2 ⊆ . . . , ∞ Fk = E.

k=1

Suppose further that ae ∈ Rm for each e ∈ E.

Then, if e∈EN ae tends to a limit as N ’ ∞, it follows that e∈EN ae ,

e∈FN ae tend to a limit as N ’ ∞ and

e∈FN ae and

lim ae = lim ae .

N ’∞ N ’∞

e∈EN e∈FN

Proof. Using condition (i) and the fact that the Xk are ¬nite, we can enu-

merate the elements of X as e1 , e2 , . . . in such a way that

EN = {e1 , e2 , . . . , eM (N ) }

for some M (N ). Thus

M (N )

a ej = ae

j=1 e∈EN

tends to a limit, so M (N ) aej is bounded by some K for all N and, since

j=1

∞

all terms are positive, n j=1 aej is bounded by K for all n. Thus j=1 aej

converges absolutely.

We now observe that there is a permutation σ such that

P (N )

aeσ(j) = ae

j=1 e∈FN

for some integer P (N ). The required results now follow from Theorem 5.3.4.

Exercise 5.3.6. Explain how the previous lemma gives the following result

for ars ∈ Rm .

If any of the three limits

N N N r

lim ars , lim a(r’s)s , or lim ars

N ’∞ N ’∞ N ’∞

r=0 s=0 r=0 s=0 r 2 +s2 ¤N

exist, then so do the other two. Further, under this condition, the three limits

N N N r

lim ars , lim a(r’s)s , and lim ars

N ’∞ N ’∞ N ’∞

r=1 s=0 r=0 s=0 r 2 +s2 ¤N

exist and are equal.

88 A COMPANION TO ANALYSIS

There are two other ˜natural™ methods of de¬ning ars in common

r,s≥0

use, to wit

∞ ∞ ∞ ∞

ars and ars

r=0 s=0 s=0 r=0

or more explicitly

N M M N

lim lim ars and lim lim ars .

N ’∞ M ’∞ M ’∞ N ’∞

r=0 s=0 s=0 r=0

We shall show in the next two lemmas that, provided we have absolute con-

vergence, these summation methods are also equivalent.

Lemma 5.3.7. Let ars ∈ Rm for r, s ≥ 0. The following two statements

are equivalent.

(i) N N

s=0 ars tends to a limit as N ’ ∞.

r=0

(ii) ∞ ars converges for all r ≥ 0 and ∞ ∞

s=0 ars converges.

s=0 r=0

∞

Further, if statements (i) and (ii) hold, then s=0 ars converges for all

r ≥ 0 and ∞ ∞

s=0 ars converges.

r=0

Proof. We ¬rst show that (i) implies (ii). Observe that, if M ≥ r, then

M M M N N

ars ¤ aus ¤ lim aus .

N ’∞

s=0 u=0 s=0 u=0 s=0

∞

Thus, since an increasing sequence bounded above tends to a limit, ars