<< . .

. 22
( : 30)

. . >>

performance below. The tist network was the smaller of the two, having 3 layers (18-
10-I network). The second nehvork was a network with 4 layers (18-20-6-I network).

Results of the 18-10-I Nefwork. In-sample, this network performed exception-
ally well-nothing unusual, given the degree of curve-fitting involved. Out-of-
sample, there was a return to the scenario of a heavily losing system. For all three
order types (at open, on limit, and on stop, or Tests 7, 8, and 9, respectively), the
average loss per trade was in the $2,000 range, typical of many of the losing mod-
els tested in previous chapters. The heavy per-trade losses occurred although this
model was only trading long positions, which have characteristically performed
better than shorts.
In-sample, only four markets did not perform well: the British Pound, Silver,
Live Cattle, and Corn. Silver was a market that also gave all the previously tested
networks problems. Out-of-sample, the network was profitable across all three
order types for the S&P 500, the Japanese Yen, Light Crude, Unleaded Gasoline,
Palladium, Soybeans, and Bean Oil. A number of other markets were also prof-
itable with one or two of the orders.
The equity curve showed strong steady gains in-sample and losses out-of-

These results were derived from Tests l&12
Results of&e 18-20-6-I Nehvork.
(at open, on limit, and on stop, respectively). In-sample performance for this net-
work soared to unimaginable levels. With entry at open, the return was 768%
annualized, with 83% of the 699 trades taken being profitable. The average trade
produced $18,588 profit. Surprisingly, despite the larger size of this network
(therefore, the greater opportunity for curve-fitting), the out-of-sample perfor-
mance, on a dollars-per-trade basis, was better than the smaller network, especial-
ly in the case of the stop entry, where the loss per trade was down to $518.
All markets were profitable across all orders in-sample, without exception.
Out-of-sample, the S&P 500, the British Pound, Platinum, Palladium, Soybean
Meal, Wheat, Kansas Wheat, Minnesota Wheat, and Lumber were profitable
across all three order types.

Trading Results for the Top Turning-Point Model
The two networks that were selected as most likely to hold up out-of-sample,
based on their corrected multiple correlations with the target, are analyzed for
trading performance below. The first network was the smaller of the two, having
3 layers (18-10-l network). The second network had 4 layers (18-20-6-l network).

Results of the 18-10-I Network. As usual, the in-sample performance was
excellent. Out-of-sample performance was profitable for two of the orders: entry
at open (Test 13) and on limit (Test 14). There were moderate losses for the stop
order (Test 15). This is slightly surprising, given that the short side is usually less
profitable than the long side.
The market-by-market breakdown reveals that only the Canadian Dollar,
Feeder Cattle, Bean Oil, Wheat, and Cocoa were not profitable across all three
order types, in-sample. Out-of-sample, strong profits were observed across all
three orders for the Deutschemark, the Japanese Yen, Light Crude, Heating Oil,
Feeder Cattle, Live Cattle, and Corn. The Japanese Yen, Light Crude, and, to some
extent, Corn shared profitability with the corresponding bottom (long) turning-
point model; in other words, these markets held up out-of-sample for both the bot-
tom network and the top (short) network.
The equity curve (Figure 1 l-2) for entry at open depicts rapidly rising equi-
ty until August 1993, and then more slowly rising equity throughout the remain-
der of the in-sample period and through about two-thirds of the out-of-sample
period. Equity then declined slightly.
FIGURE 1 ˜I - 2

Equity Growth for Short Turning Points, 18-l O-l Net, with Entry at Open

In-sample equity


Out-of-sample equity
Y 15oooocl

Results of the 18-20-6-I Network. As expected, this network, the larger of the
two, produced greater and more consistent in-sample profits due to a higher
amount of curve-fitting. Out-of-sample, this network performed terribly across all
order types (at open, on limit, and on stop, or Tests 16, 17, and 18, respectively).
The least bad results were obtained with the stop order.
In-sample, only Silver, Wheat, Sugar, and Orange Juice did not trade prof-
itably across all orders. Out-of-sample, only Cocoa showed profitability for all
three orders. Surprisingly, all the metals showed strong out-of-sample profitability
for the entry at open and on limit, as did Feeder Cattle, Cocoa, and Cotton.
Portfolio equity showed incredibly smooth and steep gains in-sample, with
losses out-of-sample for all order types.

Table 11-5 provides in-sample and Table 11-6 contains out-of-sample perfor-
mance statistics for all of the neural network models broken down by test and mar-
ket. The SYM column represents the market being studied. The rightmost column
(COUiV˜ contains the number of profitable tests for a given market. The numbers
in the lirst row represent test identifiers. The last row (COUruT) contains the num-
ber of markets on which a given model was profitable. The data in this table pro-
vides relatively detailed information about which markets are and are not
profitable when traded by each of the models: One dash (-) indicates a moderate
loss per trade, i.e., $2,000 to $4,000; two dashes (--) represent a large loss per
trade, i.e., $4,000 or more; one plus sign (+) means a moderate profit per trade,
i.e., $1,000 to $2,000; two pluses (+ +) indicate a large gain per trade, i.e., $2,000
or more; and a blank cell means that the loss was between $0 and $1,999 or the
profit was between $0 and $1,000 per trade. (For information about the various
markets and their symbols, see Table II-1 in the “Introduction” to Part IL)
In-sample, every order and every model yielded exceptionally strong positive
returns (see Table 1 l-7). When averaged over all models, the entry at open and on
limit performed best, while the entry on stop was the worst; however, the differ-
ences are all very small. In-sample, the best dollars-per-trade performance was
observed with the large turning-point networks for the long (bottom) and short
(top) sides. Out-of-sample, the stop order provided the best overall results. The
time-reversed Slow %K and the short (top) turning-point models performed best
when averaged across all order types.

When a neural newbie model was tested on an individual market (Katz and
McCormick, November 1996), the conclusion was that such an approach does not
work at all. The out-of-sample behavior in some of the current tests was much bet-
In-Sample Results Broken Down by Test and Market

ter than expected, based on our earlier explorations of simple neural network mod-
els. In the current tests, the more encouraging results were almost certainly due to
the large number of data points in the training set, which resulted from training the
model across an entire portfolio, rather than on a single tradable. In general, the
larger the optimization (or training) sample, the greater the likelihood of contin-
ued performance in the verification sample. Sample size could be increased by


Out-of-Sample Results Broken Down by Test and Market

going back further in history, which would be relatively easy to accomplish since
many commodities contracts go back well beyond the start of our in-sample peri-
od (1985). It could also be increased by enlarging the portfolio with additional
markets, perhaps a better way to bolster the training sample.
A maxim of optimization is that the likelihood of performance holding up
increases with reduction in the number of model parameters. Given the somewhat
positive results obtained in some of the tests, it might be worthwhile to experiment
TABLE 11-7
Performance of Neural Network Models Broken Down by Model,
Order, and Sample

with more sophisticated models. Specifically, better input preprocessing, in the
sense of something that could reduce the total number of inputs without much loss
of essential predictive information, would probably lead to a very good trading
system. With a smaller number of inputs, there would be fewer parameters (con-
nection weights) in the network to estimate. Consequently, curve-fitting, an appar-
ently significant issue judging by the results and shrinkage levels, would be less of
a problem.

Under certain conditions, even neural newbie models can work. The criti-

cal issue with neural networks is the problem of achieving an adequate
ratio of sample size to free parameters for the purpose of minimizing
harmful (as opposed to beneficial) curve-fitting.
n Curve-fitting is a problem with neural networks. Any methods by which

the total number of parameters to be estimated can be reduced, without
too much loss of predictive information, are worth exploring; e.g., more
sophisticated information-compressing input preprocessing would proba-
bly improve out-of-sample performance and reduce the effects of pemi-
cious curve-fitting.
a For similar reasons, large samples are critical to the training of successful
neural network trading models. This is why training on whole portfolios
provides better results than training on individual tradables, despite the
loss of market specificity. One suggestion is to increase the number of
markets in the portfolio and, thereby, achieve a larger in-sample training
set. Carrying this to an extreme, perhaps a neural network should be
trained on hundreds of commodities, stocks, and various other trading
instruments, in an effort to develop a “universal market forecaster.” If there
are any universal “technical” price patterns that exist in all markets and
that have predictive validity, such an effort might actually be worthwhile.
. Some markets trade poorly, even in-sample. Other markets tend to hold
up out-of-sample. This has been found with other models in earlier chap-
ters: Some markets are more amenable to trading using certain techniques
than are other markets. Selecting a subset of markets to trade, based on
continued out-of-sample performance, might be an approach to take when
developing and trading neural network systems.

Genetic Algorithms

E xtrapolating from models of biology and economics, mathematician/psycholo-
gist, John Holland, developed a genetic optimization algorithm and introduced it
to the world in his book, Adaptation in Natural and Artificial Systems (1975).
Genetic algorithms (or GAS) only became popular in computer science about 1.5
years later (Yuret and de la Maza, 1994). The trading community first took notice
around 1993, when a few articles (Burke, 1993; Katz and McCormick, 1994;
Oliver, 1994) and software products appeared. Since then, a few vendors have
added a genetic training option to their neural network development shells and a
few have “industrial strength” genetic optimization toolkits.
In the trading community, GAS never really had the kind of heyday experi-
enced by neural networks. The popularity of this technology probably never grew
due to its nature. Genetic algorithms are a bit difficult for the average person to
understand and more than a bit difficult to use properly. Regardless of their image,
from our experience, GAS can be extremely beneficial for system developers.
As with neural networks, while a brief discussion is included to provide basic
understanding, it is not within the scope of this book to present a full tutorial on
genetic algorithms. Readers interested in studying this subject further should read
Davis (1991), as well as our contributions to the book Virflral Trading (Katz and
McCormick, 1995a, 1995b) and our articles (Katz and McCormick, July/August
1994, December 1996, January 1997, February 1997).

A genetic algorithm solves a problem using a process similar to biological evolution.
It works by the recombination and mutation of gene sequences. Recombination and
mutation are genetic operators; i.e., they manipulate genes. A gene is a string of
codes (the genotype) that is decoded to yield a functional organism with specific
characteristics (the phenotype). A chromosome is a string of genes. In the case of
genetic optimization, as caxried out on such problems as those being addressed here,
the string of codes usually takes the form of a series of numbers.
During the simulated evolutionary process, a GA engages in mating and
selecting the members of the population (the chromosomes). Mating involves
crossover and mutation. In crossover, the elements that comprise the genes of dif-
ferent chromosomes (members of the population or solutions) are combined to
form new chromosomes. Mutation involves the introduction of random alterations
to these elements. This provides additional variation in the sets of chromosomes
being generated. As with the process of biological selection (where less-fit mem-
bers of the population leave fewer progenies), the less-fit solutions are weeded out
so the more-fit solutions can proliferate, yielding another generution that may con-
tain some better solutions than the previous one. The process of recombination,
random mutation, and selection has been shown to be an extremely powerful prob-
lem-solving mechanism.

What would happen if a GA were allowed to search, not merely for the best para-
meters (the more common way a GA is applied by traders), but also for the best
rules? In this chapter, the consequences of using a GA to evolve a complete entry
model, by discovering both the rules and the optimal parameters for those rules,
will be explored. Although somewhat complex, this methodology proved to be
effective in our first investigation (Katz and McCormick, February 1997).
How can a GA be used to discover great trading rules? The garden variety
GA just juggles numbers. It is necessary to find a way to map sets of numbers in
a one-to-one fashion to sets of rules. There are many ways this can be accom-
plished. A simple and effective method involves the construction of a set of rule
templates. A rule template is a partial specification for a rule, one that contains
blanks that need to be filled in. For example, if some of the rules in previous chap-
ters were regarded as rule templates, the blanks to be filled in would be the values
for the look-backs, thresholds, and other parameters. Using rule templates, as
defined in this manner, a one-to-one mapping of sets of numbers to fully specified
rules can easily be achieved. The first number (properly scaled) of any set is used
as an index into a table of rule templates. The remaining numbers of the set are
then used to fill in the blanks, with the result being a fully specified rule. The code
below contains a C+ + function (Rules) that implements this mapping strategy; it
will be discussed later. Although C+ + was used in the current study, this method
can also be implemented in TradeStation using the TS-EVOLVE software from
Scientific Consultant Services (516-696-3333).
The term genetic search applies to the use of a genetic algorithm to search
through an incredibly large set of potential solutions to find those that are best, i.e.,
that have the greatest fitness. In the current application, the intention is to use the
evolutionary process to discover sets of numbers (genotypes) that translate to rule-
based entry models (phenotypes) with the greatest degree of fitness (defined in
terms of desirable trading behavior). In short, we are going to attempt to engage
in the selective breeding of rule-based entry methods! Instead of beginning with a
particular principle on which to base a model (e.g., seasonal@, breakouts, etc.),
the starting point is an assortment of ideas that might contribute to the develop-
ment of a profitable entry. Instead of testing these ideas one by one or in combi-
nations to determine what works, something very unusual will be done: The
genetic process of evolution will be allowed to breed the best possible entry model
from the raw ideas.
The GA will search an extremely broad space of possibilities to find the best
rule-based entry model that can be achieved given the constraints imposed by the
rule templates, the data, and the limitation of restricting the models to a specified
number of rules (to prevent curve-fitting). To accomplish this, it is necessary to
find the best sets of numbers (those that map to the best sets of rule-based entry
models) from an exceedingly large universe of possibilities. The kind of massive
search for solutions would be almost impossible-certainly impractical, in any
realistic sense-to accomplish without the use of genetic algorithms. There are
alternatives to GAS, e.g., brute force searching may be used, but we do not have
thousands of years to wait for the results. Another alternative might be through the
process of rule induction, i.e., where an attempt is made to infer rules from a set
of observations; however, this approach would not necessarily allow a complex
function, such as that of the risk-to-reward ratio of a trading model, to be maxi-
mized. Genetic algorithms provide an efticient way to accomplish very large
searches, especially when there are no simple problem-solving heuristics or tech-
niques that may otherwise be used.

In this exercise, a population of three-rule entry models are evolved using
OptEvolve, a C+ + genetic optimizer (Scientific Consultant Services, 516-696.
3333). Each gene corresponds to a block of four numbers and to a rule, via the
one-to-one mapping of sets of numbers to sets of rules. Each chromosome con-
tains three genes. A chromosome, as generated by the GA, therefore consists of 12
numbers: The first four numbers correspond to the first gene (or rule), the next
four correspond to the second gene (or rule), and the last four correspond to the
third gene (or rule). The GA itself has to be informed of the gene size so it does
not break up intrinsic genes when performing crossover. Crossover should only
occur at the boundaries of genes, i.e., four number blocks. In the current example,
this will be achieved by setting the “chunk size,” a property of the genetic opti-
mizer component, to four.
As mentioned, each gene is composed of four numbers. The first number is
nothing more than an index into a table of possible rule templates. For example, if
that number is 1, a price-comparison template in which the difference between two
closing prices is compared with a threshold (see code) is selected. The remaining
three numbers in the gene then control the two lookback periods for the prices being
compared and the threshold. If the first number of the four-number block is 2, a
price-to-moving-average comparison template would be selected. In that case, two
of the remaining three numbers would control the period of the moving average and
the direction of the comparison (whether the price should be above or below the
moving average). In general, if the first number in the block of four numbers that
represents a gene is n, then the n&rule template is used, and any required parame-
ters are determined by reference to the remaining three numbers in the four-number
block. This decoding scheme makes it easy to maintain an expandable database of
rule templates. Each of the three blocks of four numbers is mapped to a corre-
sponding rule. For any 12-number chromosome, a 3-rule entry model is produced.

The Rule Templates
The first rule template (case I in function Rules) defines a comparison between
two prices and a threshold: The rule takes on a value of TRUE if the closing price
lb1 bars ago is greater than some threshold factor (thr) plus the closing price lb2
bars ago. Otherwise, the rule takes on the value of FALSE. The unknowns (Ibl,
lb2, and thr) are left as the blanks to be tilled in during instantiation. This template
has been included because the kind of rule it represents was useful in previous
The second rule template (case 2) involves simple moving averages, which
are often used to determine trend. Usually the market is thought to be trending up
if the price is above its moving average and trending down if the price is below its
moving average. There are only two unknowns in this template: The first (per)
controls the number of bars in the moving average, and the second (˜4) controls
the direction of comparison (above or below).
The third rule template (case 3) is identical to the second (cuse 2), except that
an exponential moving average is used rather than a simple moving average.
Much discussion has occurred regarding the importance of open interest.
Larry Williams (1979) mentioned that a decline in total open interest, during a
period when the market has been moving sideways, indicates potential for a strong
rally. A shrinking of open interest may be interpreted as a decline in available con-
tracts, producing a condition where demand may outweigh supply. The fourth rule
template (case 4) simply computes the percentage decline in open interest from
161 bars ago to 1 bar ago (open interest is generally not available for the current
CHAPTER 12 Ge”& Algorithms

bar) and compares it with a threshold (thr). If the percentage decline is greater than
the threshold, the rule takes on the value TRUE. Otherwise it evaluates to FALSE.
The threshold and the lookback (Ibl) are the unknowns to be tilled in at the time
of instantiation.
The fifth rule template (use 5) is similar to the fourth template, but a rise,
rather than fall, in total open interest is being sought. If the increase, as a percent-
age, is greater than a threshold, then the rule evaluates to TRUE. Otherwise it eval-
uates to FALSE. As previously, the lookback and the threshold are unknowns that
must he supplied to instantiate the rule.
The sixth rule template (case 6) can be called a “new high” condition: The
template asks whether an &l-bar new high has occurred within the last lb2 bars.
A particular instance of the rule might read: “If a new 50-day high has occurred
within the last 10 days, then TRUE, else FALSE.” This rule attempts to capture a
simple breakout condition, allowing for breakouts that may have occurred several
bars ago (perhaps followed by a pull-back to the previous resistance-turned-sup-
port that another rule has detected as a good entry point). There are two blanks to
be tilled in to instantiate this template: Zbl and lb2.
The seventh rule template (case 7) is identical to the sixth rule template,
except that new lows, rather than new highs, are being detected.
The eighth rule template (case 8) examines the average directional move-
ment index with respect to two thresholds (thrl and rhr2). This is a measure of
trendiness, as discussed in the chapter on breakouts. If the average directional
movement (ADX) is above a lower threshold and below an upper threshold, the
rule evaluates to TRUE. Otherwise, the rule evaluates to FALSE.
The ninth rule template (case 9) performs a threshold comparison on the
Stochastic oscillator that is similar to that performed in Rule 8.
The tenth rule template (case 10) evaluates the direction of the slope of the
MACD oscillator. The lengths (Zbl and lb2) of the two moving averages that com-
pose the MACD, and the direction of the slope (˜4) required for the role to evalu-
ate to TRUE, are specified as parameters.

Below are the steps involved in evolving an entry model based on the rule tem-
plates being used in this study:
1. Retrieve a 12-element chromosome from the genetic optimizer compo-
nent. This represents a potential solution, one that initially will be ran-
dom and probably not very good.
2. Instantiate the rule templates to obtain three fully defined rules (one for
each gene), and compote their TRUE/FALSE values for all bars in the
data series, based upon the decoding of the genes/chromosome.
3. Proceed bar by bar through the data. If, on a given bar, all three instanti-
ated rules evaluate to TRUE, and if there is no current long (or short)
position, then post an order to the simulator component to buy (or sell)
at tomorrow™s open.
4. If a position is being held, use the standard exit strategy to handle the
5. Evaluate the trading performance of the potential solution. For this exer-
cise, the basic “goodness” of a solution is defined as the annualized risk-
to-reward ratio, a figure that is actually a resealed t-statistic.
6. Tell the genetic optimizer how fit (in the above sense) the potential solu-

<< . .

. 22
( : 30)

. . >>