ńņš. 35 |

quite useful, and they graduate from ātricksā to the status of ātechniques.ā (For

example, several of the macros now in Appendix B started out as suggestions

for Appendix D.) In any case, gurus of a language always like to explore its

limits. The depths of have hardly been plumbed, but this appendix

probably reached a new low at the time it was written.

Acknowledgment: More than half of the ideas in this appendix are due

to John Hobby, who has been a tireless and inspiring co-worker during the entire

development of the new system.

Please donā™t read this material until youā™ve had

plenty of experience with plain .

After you have read and understood the secrets below, youā™ll know all sorts of devi-

ous combinations of commands, and you will often be tempted to write

inscrutable macros. Always remember, however, that thereā™s usually a simpler and

better way to do something than the ļ¬rst way that pops into your head. You may not

have to resort to any subterfuge at all, since is able to do lots of things in

a straightforward way. Try for simple solutions ļ¬rst.

1. Macro madness. If you need to write complicated macros, youā™ll need to be familiar

with the ļ¬ne points in Chapter 20. ā™s symbolic tokens are divided into two

main categories, āexpandableā and āunexpandableā; the former category includes all

macros and if . . . ļ¬ tests and for . . . endfor loops, as well as special operations like

input, while the latter category includes the primitive operators and commands listed

in Chapters 25 and 26. The expansion of expandable tokens takes place in ā™s

āmouth,ā but primitive statements (including equations, declarations, and the various

types of commands) are done in ā™s āstomach.ā Thereā™s a communication

between the two, since the stomach evaluates expressions that are needed as arguments

to the mouthā™s macros; any statement can be embedded in a group expression, so

arbitrarily complicated things can be done as part of the expansion process.

Letā™s begin by considering a toy problem that is treated at the beginning of

Appendix D in The TEXbook, in case some readers are interested in comparing TEX to

. Given a numeric variable n ā„ 0, we wish to deļ¬ne a macro asts whose

replacement text consists of precisely n asterisks. This task is somewhat tricky because

expansion is suppressed when a replacement text is being read; we want to use a for

loop, but loops are special cases of expansion. In other words,

def asts = for x=1 upto n: * endfor enddef

deļ¬nes asts to be a macro with a for loop in its replacement text; in practice, asts

would behave as if it contained n asterisks (using possibly diļ¬erent values of n), but

we have not solved the stated problem. The alternative

def makedef primary n =

def asts = for x=1 upto n: * endfor enddef enddef;

makedef n

āfreezesā the present value of n; but this doesnā™t solve the problem either.

286 Appendix D: Dirty Tricks

One solution is to build up the deļ¬nition by adding one asterisk at a time, expandafter

inaccessible

using expandafter as follows:

ENDFOR

def asts = enddef; outer

forbidden token

for x=1 upto n: inner

expandafter def expandafter asts expandafter = asts * enddef; *

concatenation

endfor.

string

pool size

The three expandafters provide a āļ¬ngerā into the replacement text, before def sup-

buļ¬er size

presses expansion; without them the replacement text would turn out to be ā˜asts *ā™, scantokens

causing inļ¬nite recursion. quote

This solution involves a running time proportional to n2 , so the reader might debugging tricks

wonder why a simpler approach like

expandafter def expandafter asts expandafter =

for x = 1 upto n: * endfor enddef

wasnā™t suggested? The reason is that this doesnā™t work, unless n = 0! A for loop isnā™t

entirely expanded by expandafter; only ā™s ļ¬rst step in loop expansion is

carried out. Namely, the loop text is read, and a special inaccessible token ā˜ENDFORā™

is placed at its end. Later on when ā™s mouth encounters ā˜ENDFORā™ (which

incidentally is an expandable token, but it wasnā™t listed in Chapter 20), the loop text

is re-inserted into the input stream, unless of course the loop has ļ¬nished. The special

ENDFOR is an ā˜outerā™ token, hence it should not appear in replacement texts; -

will therefore stop with a āforbidden tokenā error if you try the above with n ā„ 1.

You might try to defeat the outerness by saying

for x=1: inner endfor;

but wonā™t let you. And even if this had worked, it wouldnā™t have solved the

problem; it would simply have put ENDFOR into the replacement text of ast, because

expansion is inhibited when the replacement text is being read.

Thereā™s another way to solve the problem that seems to have running time

proportional to n rather than n2 :

scantokens("def asts=" for x=1 upto n: & "* " endfor) enddef;

but actually ā™s string concatenation operation takes time proportional to

the length of the strings it deals with, so the running time is still order n2 . Furthermore,

the string operations in are rather primitive, because this isnā™t a major

aspect of the language; so it turns out that this approach uses order n2 storage cells

in the string pool, although they are recycled later. Even if the pool size were inļ¬nite,

ā™s ābuļ¬er sizeā would be exceeded for large n, because scantokens puts

the string into the input buļ¬er before scanning it.

Is there a solution of order n? Yes, of course. For example,

def a=a* enddef;

for x=0 upto n:

if x=n: def a=quote quote def asts = enddef; fi

expandafter endfor a enddef;

showtoken asts.

(The ļ¬rst ā˜quoteā™ is removed by the for, hence one will survive until a is redeļ¬ned.

If you donā™t understand this program, try running it with n = 3; insert an isolated

Appendix D: Dirty Tricks 287

expression ā˜0;ā™ just before the ā˜ifā™, and look at the lines of context that are shown input stack size

Derek

when gives you four error messages.) The only ļ¬‚aw in this method is

logo10.mf

that it uses up n cells of stack space; ā™s input stack size may have to be end

increased, if n is bigger than 25 or so. outer

inner

The asterisk problem is just a puzzle; letā™s turn now to a genuine application. Runaway

File ended...

Suppose we want to deļ¬ne a macro called ā˜ten ā™ whose replacement text is the contents

end of a ļ¬le

of the parameter ļ¬le logo10.mf in Chapter 11, up to but not including the last two endinput

lines of that ļ¬le. Those last two lines say

input logo % now generate the font

end % and stop.

The ten macro will make it possible to set up the 10-point parameters repeatedly

(perhaps alternating with 9-point parameters in a nine macro); Appendix E explains

how to create a meta-design tool via such macros.

One idea would be to try to input the entire ļ¬le logo10.mf as the replacement

text for ten . We could nullify the eļ¬ect of the last three unwanted tokens by saying

save input,logo,end;

forsuffixes s=input,logo,end: let s=\; endfor

just before ten is used. To get the entire ļ¬le as a replacement text, we can try one of

the approaches that worked in the asterisk problem, say

expandafter def expandafter ten expandafter = input logo10 enddef.

But this ļ¬rst attempt runs awry if we havenā™t already redeļ¬ned ā˜endā™; Appendix B

makes ā˜endā™ an ā˜outerā™ token, preventing its appearance in replacement texts. So we

say ā˜inner endā™ and try again, only to discover an unwritten law that somehow never

came up in Chapters 20 or 26:

Runaway definition?

font_size10pt#;ht#:=6pt#;xgap#:=0.6pt#;u#:=4/9pt#;s#:=0;o#:=1/ ETC.

! File ended while scanning the definition of ten.

<inserted text>

enddef

l.2 ...fter ten expandafter = input logo10

enddef;

The end of a ļ¬le is invisible; but itā™s treated like an ā˜outerā™ token, in the sense that a

ļ¬le should never end when is passing rapidly over text.

Therefore this whole approach is doomed to failure. Weā™ll have to ļ¬nd a way

to stop the replacement text before the ļ¬le ends. OK, weā™ll redeļ¬ne ā˜inputā™ so that it

means ā˜enddef ā™, and redeļ¬ne logo so that it means ā˜endinputā™.

let INPUT = input; let input = enddef; let logo = endinput;

expandafter def expandafter ten expandafter = INPUT logo10;

showtoken ten.

It works! By the way, the line with three expandafters can be replaced by a more

elegant construction that uses scantokens as follows:

scantokens "def ten=" INPUT logo10;

288 Appendix D: Dirty Tricks

This does the job because always looks ahead and expands the token im- delimiter

text argument

mediately following an expression that is being evaluated. (The expression in this

argument

case is the string "def ten=", which is an argument to scantokens. The token that tracingall

immediately follows an expression almost always needs to be examined in order to and

conditional and

be sure that the expression has ended, so always examines it.) Curi- conditional or

ously, the expandafter alternative causes ten ā™s replacement text to begin with the

tokens ā˜font_size10pt#;ht#:=...ā™, while the scantokens way makes it start with

ā˜designsize:=(10);ht#:=...ā™. Do you see why? In the second case, expansion con-

tinued until an unexpandable token (ā˜designsizeā™) was found, so the font_size macro

was changed into its replacement text; but expandafter just expanded ā˜INPUTā™.

Now letā™s make the problem a bit harder. Suppose we know that ā˜inputā™ comes

at the end of where we want to read, but we donā™t know that ā˜logoā™ will follow. We

know that some program ļ¬le name will be there, but it might not be for the logo font.

Furthermore, letā™s assume that ā˜endā™ might not be present; therefore we canā™t simply

redeļ¬ne it to be enddef . In this case we can make ā˜inputā™ into a right delimiter,

and read the ļ¬le as a delimited text argument; that will give us enough time to insert

other tokens, which will terminate the input and ļ¬‚ush the unwanted ļ¬le name. But

the construction is more complex:

let INPUT = input; delimiters begintext input;

def makedef(expr name)(text t) =

expandafter def scantokens name = t enddef;

endinput flushfilename enddef;

def flushfilename suffix s = enddef;

makedef("ten") expandafter begintext INPUT logo10;

showtoken ten.

This example merits careful study, perhaps with ā˜tracingallā™ to show exactly how

proceeds. We have assumed that the unknown ļ¬le name can be parsed

as a suļ¬x; this solves the problem that a ļ¬le cannot end inside of a text parameter

or a false condition. (If we knew that ā˜endā™ were present, we could have replaced

ā˜endinput flushfilenameā™ by ā˜if false:ā™ and redeļ¬ned ā˜endā™ to be ā˜fiā™.)

Letā™s turn now to a simpler problem. allows you to consider the

ā˜andā™ of two Boolean expressions, but it always evaluates both expressions. This is

problematical in situations like

if pair x and (x>(0,0)): A else: B fi

because the expression ā˜x>(0,0)ā™ will stop with an error message unless x is of type

pair. The obvious way to avoid this error,

if pair x: if x>(0,0): A else: B fi else: B fi

is cumbersome and requires B to appear twice. What we want is a āconditional andā

operation in which the second Boolean expression is evaluated only if the ļ¬rst one turns

out to be true; then we can safely write

if pair x cand (x>(0,0)): A else: B fi.

Similarly we might want āconditional orā in which the second operand is evaluated

only if the ļ¬rst is false, for situations like

if unknown x cor (x<0): A else: B fi.

Appendix D: Dirty Tricks 289

Such cand and cor macros can be deļ¬ned as follows: cand

cor

def cand(text q) = startif true q else: false fi enddef; text arguments

if

def cor(text q) = startif true true else: q fi enddef;

hierarchy

tertiarydef p startif true = if p: enddef; group delimiters

begingroup

the text arguments are now evaluated only when necessary. We have essentially replaced endgroup

vardef

the original line by

spark

gobbled

if if pair x: x<(0,0) else: false fi: A else: B fi. gobble

vacuous expressions

This construction has one catch; namely, the right-hand operands of cand and cor

must be explicitly enclosed in delimiters. But delimiters are only a minor nuisance,

because the operands to ā˜andā™ and ā˜orā™ usually need them anyway. It would be impos-

sible to make cand and cor obey the normal expression hierarchy; when macros make

primary/secondary/tertiary distinctions, they evaluate their arguments, and such eval-

uation is precisely what cand and cor want to avoid.

If these cand and cor macros were changed so that they took undelimited text

arguments, the text argument wouldnā™t stop at a colon. We could, however, use such

modiļ¬ed macros with group delimiters instead. For example, after

let {{ = begingroup; let }} = endgroup;

def cand text q = startif true q else: false fi enddef

we could write things like

if {{pair x cand x>(0,0)}}: A else: B fi.

(Not that this buys us anything; it just illustrates a property of undelimited text

arguments.) Group delimiters are not valid delimiters of delimited text arguments.

Speaking of group delimiters, the gratuitous begingroup and endgroup to-

kens added by vardef are usually helpful, but they can be a nuisance. For example,

suppose we want to write a zz macro such that ā˜zz1..zz2..zz3ā™ expands into

z1{dz1}..z2{dz2}..z3{dz3}

It would be trivial to do this with def :

def zz suffix $ = z${dz$} enddef;

but this makes zz a āspark.ā Letā™s suppose that we want to use vardef , so that zz

will be usable in suļ¬xes of variable names. Additional begingroup and endgroup

delimiters will mess up the syntax for paths, so we need to get rid of them. Hereā™s one

way to ļ¬nesse the problem:

vardef zz@# =

endgroup gobbled true z@#{dz@#} gobble begingroup enddef.

The gobbled and gobble functions of Appendix B will remove the vacuous expressions

ā˜begingroup endgroupā™ at the beginning and end of the replacement text.

(The initial begingroup endgroup wonā™t be gobbled if the vardef is being

read as a primary instead of as a secondary, tertiary, or expression. But in such cases

you probably donā™t mind having begingroup present.)

290 Appendix D: Dirty Tricks

2. Fortuitous loops. The ā˜maxā™ and ā˜minā™ macros in Appendix B make use of the fact max

min

that commas are like ā˜)(ā™ in argument lists. Although the deļ¬nition heading is

inorder

equally spaced

def max(expr x)(text t) text argument

ENDFOR

we can write ā˜max(a, b, c)ā™ and this makes x = a and t = ā˜b, cā™. Of course, a person isnā™t endgroup

supposed to say ā˜max(a)(b)(c)ā™. whatever

Here are two more applications of the idea: We want ā˜inorder(a, b, c)ā™ to be

true if and only if a ā¤ b ā¤ c; and we want ā˜equally spaced(x1 , x2 , x3 ) dx ā™ to produce

the equations ā˜x2 ā’ x1 = x3 ā’ x2 = dxā™.

def inorder(expr x)(text t) =

((x for u=t: <= u)

and (u endfor gobbled true true)) enddef;

def equally_spaced(expr x)(text t) expr dx =

x for u=t: - u = u endfor gobbled true

- dx enddef.

Isnā™t this fun? (Look closely.)

There is a problem, however, if we try to use these macros with loops in the

arguments. Consider the expressions

inorder(for n=1 upto 10: a[n], endfor infinity),

inorder(a[1] for n=2 upto 10: ,a[n] endfor),

inorder(a[1],a[2] for n=3 upto 10: ,a[n] endfor);

the ļ¬rst two give error messages, but the third one works! The reason is that, in the

ļ¬rst two cases, the for loop begins to be expanded before begins to read

the text argument, hence ENDFOR rears its ugly head again. We can avoid this problem

by rewriting the macros in a more complicated way that doesnā™t try to single out the

ļ¬rst argument x:

def inorder(text t) =

expandafter startinorder for u=t:

<= u endgroup and begingroup u endfor

gobbled true true endgroup) enddef;

def startinorder text t =

(begingroup true enddef;

def equally_spaced(text t) expr dx =

if pair dx: (whatever,whatever) else: whatever fi

for u=t: - u = u endfor gobbled true

- dx enddef;

Two separate tricks have been used here: (1) The ā˜endgroupā™ within ā˜inorderā™ will stop

an undelimited text argument; this gets rid of the unwanted ā˜<= uā™ at the beginning.

(2) A throwaway variable, ā˜whatever ā™, nulliļ¬es an unwanted equation at the beginning

of ā˜equally spacedā™. With the new deļ¬nitions, all three of the expressions above will

be understood, and so will things like

equally_spaced(for n=1 upto 10: x[n], endfor whatever) dx.

Furthermore the single-argument cases now work: ā˜inorder(a)ā™ will always be true, and

ā˜equally spaced(x) dx ā™ will produce no new equations.

Appendix D: Dirty Tricks 291

If we want to improve max and min in the same way, so that a person can max

min

specify loop arguments like

setu

Knuth

max(a[1] for n=2 upto 10: ,a[n] endfor) scaled

xscaled

and so that ā˜max(a) = aā™ in the case of a single argument, we have to work harder, yscaled

because max and min treat their ļ¬rst argument in quite a special way; they need to

apply the special macro setu , which deļ¬nes the type of the auxiliary variable u . The

fastest way to solve this problem is probably to use a token whose meaning changes

during the ļ¬rst time through the loop:

vardef max(text t) =

let switch_ = firstset_;

for u=t: switch_ u>u_: u_ := u ;fi endfor

u_ enddef;

vardef min(text t) =

let switch_ = firstset_;

for u=t: switch_ u<u_: u_ := u ;fi endfor

u_ enddef;

def firstset_ primary u =

setu_ u; let switch_ = if; if false: enddef.

Incidentally, the authorā™s ļ¬rst programs for max and min contained an interesting bug.

They started with ā˜save u ā™, and they tried to recognize the ļ¬rst time through the loop

by testing if u was unknown. This failed because u could be constantly unknown in

well-deļ¬ned cases like max(x, x + 1, x + 2).

3. Types. Our programs for inorder, equally_spaced, and max are careful not to make

unnecessary assumptions about the type of an expression. The ā˜roundā™ and ā˜byteā™ func-

tions in Appendix B are further examples of macros that change behavior based on the

types of their expr arguments. Letā™s look more closely at applications of type testing.

When the author was developing macros for plain , his ļ¬rst ācor-

rectā solution for max had the following form:

vardef max(text t) =

save u_; boolean u_;

for u=t: if boolean u_: setu_ u

elseif u_<u: u_ := u fi; endfor

u_ enddef.

This was interesting because it showed that there was no need to set u to true or false;

the simple fact that it was boolean was enough to indicate the ļ¬rst time through the

loop. (A slightly diļ¬erent setu macro was used at that time.)

We might want to generalize the ā˜scaledā™ operation of so that

ā˜scaled (x, y)ā™ is shorthand for ā˜xscaled x yscaled yā™. Thatā™s pretty easy:

let SCALED = scaled;

def scaled primary z =

if pair z: xscaled xpart z yscaled ypart z

else: SCALED z fi enddef;

Itā™s better to keep the primitive operation ā˜SCALED zā™ here than to replace it by the

slower variant ā˜xscaled z yscaled zā™.

292 Appendix D: Dirty Tricks

allows you to compare booleans, numerics, pairs, strings, and equality test

==

transforms for equality; but it doesnā™t allow the expression ā˜p = qā™ where p and q are

Nonlinear equations

paths or pens or pictures. Letā™s write a general equality test macro such that ā˜p == qā™ solve

will be true if and only if p and q are known and equal, whatever their type.

tertiarydef p == q =

if unknown p or unknown q: false

elseif boolean p and boolean q: p=q

elseif numeric p and numeric q: p=q

elseif pair p and pair q: p=q

elseif string p and string q: p=q

elseif transform p and transform q: p=q

elseif path p and path q:

if (cycle p = cycle q) and (length p = length q)

and (point 0 of p = point 0 of q): patheq p of q

else: false fi

elseif pen p and pen q: (makepath p == makepath q)

elseif picture p and picture q: piceq p of q

elseif vacuous p and vacuous q: true

else: false fi enddef;

vardef vacuous primary p =

not(boolean p or numeric p or pair p or path p

or pen p or picture p or string p or transform p) enddef;

vardef patheq expr p of q =

save t; boolean t; t=true;

for k=1 upto length p:

t := (postcontrol k-1 of p = postcontrol k-1 of q)

and (precontrol k of p = precontrol k of q)

and (point k of p = point k of q);

exitunless t; endfor

t enddef;

vardef piceq expr p of q =

save t; picture t;

t=p; addto t also -q;

cull t dropping origin;

(totalweight t=0) enddef;

If p and q are numeric or pair expressions, we could relax the condition that they both

be known by saying ā˜if known p ā’ q: p = q else false ļ¬ā™; transforms could be handled

similarly by testing each of their six parts. But thereā™s no way to tell if booleans, paths,

etc., have been equated when theyā™re both unknown, without the risk of irrevocably

changing the values of other variables.

4. Nonlinear equations. has a built-in solution mechanism for linear equa-

tions, but it balks at nonlinear ones. You might be able to solve a set of nonlinear

equations yourself by means of algebra or calculus, but in diļ¬cult cases it is probably

simplest to use the ā˜solve ā™ macro of plain . This makes it possible to solve

n equations in n unknowns, provided that at most one of the equations is nonlinear

when one of the unknowns is ļ¬xed.

Appendix D: Dirty Tricks 293

ńņš. 35 |