<< . .

. 2
( : 30)

. . >>

complete “round-turns,” each consisting of an entry followed by an exit.

Given a mechanical trading system that contains an entry model to generate entry
orders and an exit model to generate exit orders (including those required for
money management), how are the entries and exits evaluated to determine whether
they are good? In other words, what constitutes a good entry or exit?
Notice we used the terms entry orders and exit orders, not entry or exit sig-
nals. Why? Because “signals” are too ambiguous. Does a buy “signal” mean that
one should buy at the open of the next bar, or buy using a stop or limit order? And
if so, at what price? In response to a “signal” to exit a long position, does the exit
occur at the close, on a profit target, or perhaps on a money management stop?
Each of these orders will have different consequences in terms of the results
achieved. To determine whether an entry or exit method works, it must produce
more than mere signals; it must, at smne point, issue highly specific entry and exit
orders. A fully specified entry or exit order may easily be tested to determine its
quality or effectiveness.
In a broad sense, a good entry order is one that causes the trader to enter the
market at a point where there is relatively low risk and a fairly high degree of
potential reward. A trader™s Nirvana would be a system that generated entry orders
to buy or sell on a limit at the most extreme price of every turning point. Even if
xviii lNTR”D”Cn”N

the exits were only merely acceptable, none of the trades would have more than
one or two ticks of adverse excursion (the largest unrealized loss to occur within
a trade), and in every case, the market would be entered at the best obtainable
price. In an imperfect world, however, entries will never be that good, but they can
be such that, when accompanied by reasonable effective exits, adverse excursion
is kept to acceptable levels and satisfying risk-reward ratios are obtained.
What constitutes an elective exit? An effective exit must quickly extricate the
trader from the market when a trade has gone wrong. It is essential to preserve cap-
ital from excessive erosion by losing trades; an exit must achieve this, however,
without cutting too many potentially profitable trades short by converting them into
small losses. A superior exit should be able to hold a trade for as long as it takes to
capture a significant chunk of any large move; i.e., it should be capable of riding a
sizable move to its conclusion. However, riding a sizable move to conclusion is not
a critical issue if the exit strategy is combined with an entry formula that allows for
reentry into sustained trends and other substantial market movements.
In reality, it is almost impossible, and certainly unwise, to discuss entries
and exits independently. To back-test a trading system, both entries and exits
must be present so that complete round-turns will occur. If the market is entered,
but never exited, how can any completed trades to evaluate be obtained? An entry
method and an exit method are required before a testable system can exist.
However, it would be very useful to study a variety of entry strategies and make
some assessment regarding how each performs independent of the exits.
Likewise, it would be advantageous to examine exits, testing different tech-
niques, without having to deal with entries as well. In general, it is best to manip-
ulate a minimum number of entities at a time, and measure the effects of those
manipulations, while either ignoring or holding everything else constant. Is this
not the very essence of the scientific, experimental method that has achieved so
much in other fields? But how can such isolation and control be achieved, allow-
ing entries and exits to be separately, and scientifically, studied?

This book is intended to accomplish a systematic and detailed analysis of the
individual components that make up a complete trading system. We are propos-
ing nothing less than a scientific study of entries, exits, and other trading system
elements. The basic substance of the scientific approach as applied herein is
as f0110ws:
1. The object of study, in this case a trading system (or one or more of its
elements), must be either directly or indirectly observable, preferably
without dependence on subjective judgment, something easily achieved
with proper testing and simulation software when working with com-
plete mechanical trading systems.
2. An orderly means for assessing the behavior of the object of study must
be available, which, in the case of trading systems, is back-testing over
long periods of historical data, together with, if appropriate, the applica-
tion of various models of statistical inference, the aim of the latter being
to provide a fix or reckoning of how likely a system is to hold up in the
future and on different samples of data.
3. A method for making the investigative task tractable by holding most
parameters and system components fixed while focusing upon the effects
of manipulating only one or two critical elements at a time.
The structure of this book reflects the scientific approach in many ways.
Trading systems are dissected into entry and exit models. Standardized methods for
exploring these components independently are discussed and implemented, leading
to separate sections on entries and exits. Objective tests and simulations are run,
and statistical analyses are performed. Results are presented in a consistent manner
that permits direct comparison. This is “old hat” to any practicing scientist.
Many traders might be surprised to discover that they, like practicing scien-
tists, have a working knowledge of the scientific method, albeit in different guise!
Books for traders often discuss “paper trading” or historical back-testing, or pre-
sent results based on these techniques. However, this book is going to be more
consistent and rigorous in its application of the scientific approach to the prob-
lem of how to successfully trade the markets. For instance, few books in which
historical tests of trading systems appear offer statistical analyses to assess valid-
ity and to estimate the likelihood of future profits. In contrast, this book includes
a detailed tutorial on the application of inferential statistics to the evaluation
of trading system performance.
Similarly, few pundits test their entries and exits independently of one
another. There are some neat tricks that allow specific system components to be
tested in isolation. One such trick is to have a set of standard entry and exit strate-
gies that remain fixed as the particular entry, exit, or other element under study is
varied. For example, when studying entry models, a standardized exit strategy will
be repeatedly employed, without change, as a variety of entry models are tested
and tweaked. Likewise, for the study of exits, a standardized entry technique will
be employed. The rather shocking entry technique involves the use of a random
number generator to generate random long and short entries into various markets!
Most traders would panic at the idea of trading a system with entries based on the
fall of the die; nevertheless, such entries are excellent in making a harsh test for
an exit strategy. An exit strategy that can pull profits out of randomly entered
trades is worth knowing about and can, amazingly, be readily achieved, at least for
the S&P 500 (Katz and McCormick, March 1998, April 1998). The tests will be
done in a way that allows meaningful comparisons to be made between different
entry and exit methods.
To summarize, the core elements of the scientific approach are:
1. The isolation of system elements
2. The use of standardized tests that allow valid comparisons
3. The statistical assessment of results

Before applying the scientific approach to the study of the markets, a number of
things must be considered. First, a universe of reliable market data on which to
perform back-testing and statistical analyses must be available. Since this book is
focused on commodities trading, the market data used as the basis for our universe
on an end-of-day time frame will be a subset of the diverse set of markets supplied
by Pinnacle Data Corporation: these include the agriculturals, metals, energy
resources, bonds, currencies, and market indices. Intraday time-frame trading is
not addressed in this book, although it is one of our primary areas of interest that
may be pursued in a subsequent volume. In addition to standard pricing data,
explorations into the effects of various exogenous factors on the markets some-
times require unusual data. For example, data on sunspot activity (solar radiation
may influence a number of markets, especially agricultural ones) was obtained
from the Royal Observatory of Belgium.
Not only is a universe of data needed, but it is necessary to simulate one or
more trading accounts to perform back-testing. Such a task requires the use of a
trading simulator, a software package that allows simulated trading accounts to be
created and manipulated on a computer. The C+ + Trading Simulator from
Scientific Consultant Services is the one used most extensively in this book
because it was designed to handle portfolio simulations and is familiar to the
authors. Other programs, like Omega Research™s TradeStation or SystemWriter
Plus, also offer basic trading simulation and system testing, as well as assorted
charting capabilities. To satisfy the broadest range of readership, we occasionally
employ these products, and even Microsoft™s Excel spreadsheet, in our analyses.
Another important consideration is the optimization of model parameters.
When running tests, it is often necessary to adjust the parameters of some compo-
nent (e.g., an entry model, an exit model, or some piece thereof) to discover the
best set of parameters and/or to see how the behavior of the model changes as its
parameters change. Several kinds of model parameter optimizations may be con-
livmOD”CTlON xxi

ducted. In manual optimization, the user of the simulator specifies a parameter that
is to be manipulated and the range through which that parameter is to be stepped;
the user may wish to simultaneously manipulate two or more parameters in this
manner, generating output in the form of a table that shows how the parameters
interact to affect the outcome. Another method is brute force optimization, which
comes in several varieties: The most common form is stepping every parameter
through every possible value. If there are many parameters, each having many pos-
sible values, running this kind of optimization may take years. Brute force opti-
mization can, however, be a workable approach if the number of parameters, and
values through which they must be stepped, is small. Other forms of brute force
optimization are not as complete, or as likely to find the global optimum, but can
be run much more quickly. Finally, for heavy-duty optimization (and, if naively
applied, truly impressive curve-fitting) there are genetic algorithms. An appropri-
ate genetic algorithm (GA) can quickly tind a good solution, if not a global opti-
mum, even when large numbers of parameters are involved, each having large
numbers of values through which it must be stepped. A genetic optimizer is an
important tool in the arsenal of any trading system developer, but it must be used
cautiously, with an ever-present eye to the danger of curve-fitting. In the inves-
tigations presented in this book, the statistical assessment techniques, out-of-
sample tests, and such other aspects of the analyses as the focus on entire portfolios
provide protection against the curve-fitting demon, regardless of the optimization
method employed.

Jeffrey Owen Katz, Ph.D., and Donna E McCormick

Tools of the Trade

T o obJectlve1y study the behavior of mechanical trading systems, various exper-
imental materials and certain tools are needed.
To study the behavior of a given entry or exit method, a simulation should be
done using that method on a portion of a given market™s past performance; that
requires data. Clean, historical data for the market on which a method is being
tested is the starting point.
Once the data is available, software is needed to simulate a trading account.
Such software should allow various kinds of trading orders to be posted and
should emulate the behavior of trading a real account over the historical period of
interest. Software of this kind is called a trading simulator.
The model (whether an entry model, an exit model, or a complete system)
may have a number of parameters that have to be adjusted to obtain the best results
from the system and its elements, or a number of features to be tamed on or off.
Here is where an optimizer plays its part, and a choice must be made among the
several types of optimizers available.
The simulations and optimizations will produce a plethora of results. The sys-
tem may have taken hundreds or thousands of trades, each with its own profiVloss,
maximum adverse excursion, and maximum favorable excursion. Also generated
will be simulated equity curves, risk-to-reward ratios, profit factors, and other infor-
mation provided by the trading simulator about the simulated trading account(s). A
way to assess the significance of these results is needed. Is the apparent profitabili-
ty of the trades a result of excessive optimization? Could the system have been prof-
itable due to chance alone, or might it really be a valid trading strategy? If the system
is valid, is it likely to hold up as well in the future, when actually being traded, as in

the past? Questions such as these require the basic machinery provided by inferen-
tial statistics.
In the next several chapters, we will cover data, simulators, optimizers, and
statistics. These items will be used throughout this book when examining entry
and exit methods and when attempting to integrate entries and exits into complete
trading systems.


A determination of what works, and what does not, cannot be made in the realm
of commodities trading without quality data for use in tests and simulations.
Several types of data may be needed by the trader interested in developing a prof-
itable commodities trading system. At the very least, the trader will require his-
torical pricing data for the commodities of interest.

Commodities pricing data is available for individual or continuous contracts.
Individual contract data consists of quotations for individual commodities con-
tracts. At any given time, there may be several contracts actively trading. Most
speculators trade thefront-month contracts, those that are most liquid and closest
to expiration, but are not yet past first notice date. As each contract nears expira-
tion, or passes first notice date, the trader “rolls over” any open position into the
next contract. Working with individual contracts, therefore, can add a great deal of
complexity to simulations and tests. Not only must trades directly generated by the
trading system be dealt with, but the system developer must also correctly handle
rollovers and the selection of appropriate contracts.
To make system testing easier and more practical, the continuous contract was
invented. A continuous contract consists of appropriate individual contracts strung
together, end to end, to form a single, continuous data series. Some data massaging
usually takes place when putting together a continuous contract; the purpose is to
close the gaps that occur at rollover, when one contract ends and another begins,
Simple back-aajustment appears to be the most reasonable and popular gap-closing

method (Schwager, 1992). Back-adjustment involves nothing more than the sub-
traction of constants, chosen to close the gaps, from all contracts in a series other
than the most recent. Since the only operation performed on a contract™s prices is the
subtraction of a constant, all linear price relationships (e.g., price changes over time,
volatility levels, and ranges) are preserved. Account simulations performed using
back-adjusted continuous contracts yield results that need correction only for
rollover costs. Once corrected for rollover, simulated trades will produce profits and
losses identical to those derived from simulations performed using individual con-
tracts. However, if trading decisions depend upon information involving absolute
levels, percentages, or ratios of prices, then additional data series (beyond back-
adjusted continuous contracts) will be required before tests can be conducted.
End-of-day pricing data, whether in the form of individual or continuous
contracts, consists of a series of daily quotations. Each quotation, “bar,” or data
point typically contains seven fields of information: date, open, high, low,
close, volume, and open interest. Volume and open interest are normally
unavailable until after the close of the following day; when testing trading
methods, use only past values of these two variables or the outcome may be a
fabulous, but essentially untradable, system! The open, high, low, and close
(sometimes referred to as the settlement price) are available each day shortly
after the market closes.
Intraday pricing data consists either of a series of fixed-interval bars or of
individual ticks. The data fields for fixed-interval bars are date, time, open, high,
low, close, and tick volume. Tick volume differs from the volume reported for end-
of-day data series: For intraday data, it is the number of ticks that occur in the peri-
od making up the bar, regardless of the number of contracts involved in the
transactions reflected in those ticks. Only date, time, and price information are
reported for individual ticks: volume is not. Intraday tick data is easily converted
into data with fixed-interval bars using readily available software. Conversion soft-
ware is frequently provided by the data vendor at no extra cost to the consumer.
In addition to commodities pricing data, other kinds of data may be of value.
For instance, long-term historical data on sunspot activity, obtained from the
Royal Observatory of Belgium, is used in the chapter on lunar and solar influ-
ences. Temperature and rainfall data have a bearing on agricultural markets.
Various economic time series that cover every aspect of the economy, from infla-
tion to housing starts, may improve the odds of trading commodities successfully.
Do not forget to examine reports and measures that reflect sentiment, such as the
Commitment of Traders (COT) releases, bullish and bearish consensus surveys,
and put-call ratios. Nonquantitative forms of sentiment data, such as news head-
lines, may also be acquired and quantified for use in systematic tests. Nothing
should be ignored. Mining unusual data often uncovers interesting and profitable
discoveries. It is often the case that the more esoteric or arcane the data, and the
more difficult it is to obtain, the greater its value!
CMR I Data 5

Data may be used in its natural time frame or may need to be processed into a dif-
ferent time frame. Depending on the time frame being traded and on the nature of
the trading system, individual ticks, 5.minute bars, 20-minute bars, or daily, week-
ly, fortnightly (bimonthly), monthly, quarterly, or even yearly data may be neces-
sary. A data source usually has a natural time frame. For example, when collecting
intraday data, the natural time frame is the tick. The tick is an elastic time frame:
Sometimes ticks come fast and furious, other times sporadically with long inter-
vals between them. The day is the natural time frame for end-of-day pricing data.
For other kinds of data, the natural time frame may be bimonthly, as is the case for
the Commitment of Traders releases; or it may be quarterly, typical of company
earnings reports.
Although going from longer to shorter time frames is impossible (resolution
that is not there cannot be created), conversions from shorter to longer can be read-
ily achieved with appropriate processing. For example, it is quite easy to create a
series consisting of l-minute bars from a series of ticks. The conversion is usual-
ly handled automatically by the simulation, testing, or charting software: by sim-
ple utility programs; or by special software provided by the data vendor. lf the data
was pulled from the Internet by way of ftp (tile transfer protocol), or using a stan-
dard web browser, it may be necessary to write a small program or script to con-
vert the downloaded data to the desired time frame, and then to save it in a format
acceptable to other software packages.
What time frame is the best? It all depends on the trader. For those attracted to
rapid feedback, plenty of action, tight stops, overnight security, and many small
profits, a short, intraday time frame is an ideal choice. On an intraday time frame,
many small trades can be taken during a typical day. The nttmeroous trades hasten the
learning process. It will not take the day trader long to discover what works, and
what does not, when trading on a short, intraday time frame. In addition, by closing
out all positions at the end of the trading day, a day trader can completely sidestep
overnight risk. Another desirable characteristic of a short time frame is that it often
permits the use of tight stops, which can keep the losses small on losing trades.
Finally, the statistically inclined will be enamored by the fact that representative data
samples containing hundreds of thousands of data points, and thousands of trades,
are readily obtained when working with a short time frame. Large data samples
lessen the dangers of curve-fitting, lead to more stable statistics, and increase the
likelihood that predictive models will perform in the future as they have in the past.
On the downside, the day trader working with a short time frame needs a real-
time data feed, historical tick data, fast hardware containing abundant memory, spe-
cialized software, and a substantial amount of time to commit to actually trading.
The need for fast hardware with plenty of memory arises for two reasons: (1)
System tests will involve incredibly large numbers of data points and trades; and
(2) the real-time software that collects the data, runs the system, and draws the

charts must keep up with a heavy flow of ticks without missing a beat. Both a data-
base of historical tick data and software able to handle sizable data sets are neces-
sary for system development and testing. A real-time feed is required for actual
trading. Although fast hardware and mammoth memory can now be purchased at
discount prices, adequate software does not come cheap. Historical tick data is like-
ly to be costly, and a real-time data feed entails a substantial and recurring expense.
In contrast, data costs and the commitment of time to trading are minimal for
those operating on an end-of-day (or longer) time frame. Free data is available on
the Internet to anyone willing to perform cleanup and formatting. Software costs
are also likely to be lower than for the day trader. The end-of-day trader needs less
time to actually trade: The system can be run after the markets close and trading
orders are communicated to the broker before the markets open in the morning:
perhaps a total of 15 minutes is spent on the whole process, leaving more time for
system development and leisure activities.
Another benefit of a longer time frame is the ability to easily diversify by simul-
taneously trading several markets. Because few markets offer the high levels of
volatility and liquidity required for day trading, and because there is a limit on how
many things a single individual can attend to at once, the day trader may only be able
to diversify across systems. The end-of-day trader, on the other hand, has a much
wider choice of markets to trade and can trade at a more relaxed pace, making diver-
sification across markets more practical than for intraday counterparts.
Diversification is a great way to reduce risk relative to reward. Longer time frame
trading has another desirable feature: the ability to capture large profits from strong,
sustained trends: these are the profits that can take a $50,000 account to over a mil-
lion in less than a year. Finally, the system developer working with longer time frames
will find more exogenous variables with potential predictive utility to explore.
A longer time frame, however, is not all bliss. The trader must accept delayed
feedback, tolerate wider stops, and be able to cope with overnight risk. Holding
overnight positions may even result in high levels of anxiety, perhaps full-blown
insomnia. Statistical issues can become significant for the system developer due to
the smaller sample sizes involved when working with daily, weekly, or monthly
data. One work-around for small sample size is to develop and test systems on
complete portfolios, rather than on individual commodities.
Which time frame is best? It all depends on you, the trader! Profitable trad-
ing can be done on many time frames. The hope is that this discussion has clar-
fied some of the issues and trade-offs involved in choosing correctly.

Data quality varies from excellent to awful. Since bad data can wreak havoc with
all forms of analysis, lead to misleading results, and waste precious time, only use
the best data that can be found when running tests and trading simulations. Some
forecasting models, including those based on neural networks, can be exceeding-
ly sensitive to a few errant data points; in such cases, the need for clean, error-free
data is extremely important. Time spent finding good data, and then giving it a
final scrubbing, is time well spent.
Data errors take many forms, some more innocuous than others. In real-time
trading, for example, ticks are occasionally received that have extremely deviant,
if not obviously impossible, prices. The S&P 500 may appear to be trading at
952.00 one moment and at 250.50 the next! Is this the ultimate market crash?
No-a few seconds later, another tick will come along, indicating the S&P 500 is
again trading at 952.00 or thereabouts. What happened? A bad tick, a “noise
spike,” occurred in the data. This kind of data error, if not detected and eliminat-
ed, can skew the results produced by almost any mechanical trading model.
Although anything but innocuous, such errors are obvious, are easy to detect (even
automatically), and are readily corrected or otherwise handled. More innocuous,
albeit less obvious and harder to find, are the common, small errors in the settling
price, and other numbers reported by the exchanges, that are frequently passed on
to the consumer by the data vendor. Better data vendors repeatedly check their
data and post corrections as such errors are detected. For example, on an almost
daily basis, Pinnacle Data posts error corrections that are handled automatically by
its software. Many of these common, small errors are not seriously damaging to
software-based trading simulations, but one never knows for sure.
Depending on the sensitivity of the trading or forecasting model being ana-
lyzed, and on such other factors as the availability of data-checking software, it
may be worthwhile to run miscellaneous statistical scans to highlight suspicious
data points. There are many ways to flag these data points, or ourlieru, as they are

<< . .

. 2
( : 30)

. . >>