74

C h a p t e r 4

PREDICTOR TESTING METHODS

The Predictors

Parallax has been producing neural-network based financial predictors since 1990

and so it is an integral part of our business to validate these predictors using

reliable statistical methods. With each predictor, we need to answer the following

set of questions:

1. What price behavior is being predicted?

2. What is the effective duration of the predictor?

3. How statistically significant is the predictor at each time step forward?

4. Is the predictor effective on all time scales?

5. Is the predictor effective on all financial series?

6. What combinations of predictors are the most effective?

The following section is an overview of a simple and reliable statistical method

and an 8-year analysis of our present stable of predictors. In order to carry out

this analysis, it is necessary to measure the price action during the time period

immediately following each prediction event. We call this the post-predictor or

outrun period.

75

Post-Predictor Z Scores

We are interested in characterizing the post-predictor time period using a scale-

independent method that allows comparisons between financial series. The Z

Score is the most appropriate measure for this job. A Z-Score is the measure of

how many standard deviations price has moved away from its price at the

prediction event, assuming that the probability of either an up or down move is

random at 50%.

By measuring local volatility at the prediction event, a normal probability

distribution can be drawn going forward in time that acts as a roadmap for

subsequent price moves. The map is centered at the closing price of the

prediction event. Each day the map widens according to normal diffusion

velocities, which are proportional to 1/Time, representing the region where the

future price is most likely to be found. An example of such a probability map is

shown below for the stock Home Depot on Feb 3, 2005:

76

Viewed from the side at two time steps, diffusion acts to spread out the region

where we might find the stock price as time elapses:

At each time step, there is a larger standard deviation and the same mean. If we

represent the actual price achieved at each step in terms of that standard

deviation, we produce a series of Z-Scores. For example, if the price at a

prediction event is $5, and then moves to $7 on day ten with a standard deviation

of $1.60, then the Z-Score on day ten would be (7-5)/1.6 = 1.25. This means

that price moved 1.25 standard deviations above the price at the prediction event.

Since the standard deviation continues to increase, in order to maintain the same

Z, price would have to increase by the same relative amount.

Diffusion causes expected prices to spread out over time

77

If all post-predictor prices for all financial series are converted to Z-Scores, and if

the predictors do not work, and if markets are random, then at every time step, a

histogram of all the Z-Scores would be expected to result in a standardized

normal distribution, with mean of zero and a standard deviation of one N(0,1) as

shown below:

We of course hope that our predictors actually predict non-random price

behavior, so the degree of deviation from the normal curve is critical.

Are Financial Series Random?

We have assumed that price movement is best characterized by a random walk

model, but this might not be the case. There is gathering evidence for other

characteristic distributions such as a Biased Random Walk, Truncated Levy

Flights, or Cauchy distribution. Our solution is to produce a quantitative

background distribution based on randomly selected dates across our entire 3000

day test period, and for all of the 2500 stocks being considered. The figure below

78

shows this background distribution of Z-scores corresponding to random

prediction dates, and a normal distribution based on a random walk assumption

plotted together. It is clear from this figure that a strict random walk assumption

is inappropriate during our 8 year test period. Instead there appears to be a

positive return bias.

To illustrate this another way, we could ask what percentage of buy predictions

are winners (Z>0) if the timing is random, and plot this percentage each day

following the randomly selected purchase dates. Normally this would be 50%,

but since the background distributions is positively biased, the figure below

shows the percent winners for randomly selected buys climbs steadily over time.

The reverse is of course true for randomly selected sells (mirror image).

79

On a weekly scale the effect is even more pronounced, as shown in the next

figure. I guess this could be called the dartboard effect, in that even a random

selection of stocks during this period showed a 59% win rate at 30 weeks after

purchase, and shorting was decidedly unprofitable.

80

We will use these curves as benchmarks for our predictors over this test period

and stock set. In order to have a non-random buy predictor then, we need to see

a win rate in excess of 59% at 30 weeks for example, and sells would need a win

rate better than 41%.

So far we have examined the background distribution of Z-Scores and the

percentage of winners. What we still need to know is how much gain is possible

from randomly selected buys and sells, so that each predictors excess gain profile

can be evaluated. The pictures below show the background performance for

randomly selected buys and sells:

81

82

Parallax Predictors

Using the randomly selected background distribution we found in the last section,

we will now statistically evaluate our main stable of predictors using a chi-squared

test. The Chi-squared test is designed for comparing distributions and a p

83

ExtremeHurst Extension Buy & Sell Predictors

Our ExtremeHurst extension predictor is Parallaxs most predictive tool. It

measures the degree of positive feedback present in price and predicts when the

end-of-trend critical point has been reached using the principle of discrete scale

invariance. Price behavior in the post-predictor period is expected to be flat or

opposite of the preceding trend. On a daily scale, we have accumulated Z-Scores

for all 30 days following the signal. The distribution of these Z-Scores on day

eleven, shown in the figure below, shows the strong positive bias, with 60% of

the returns being positive by day eleven, with a p-value = 0.00000005. This

means that these are not random signals, and that market behavior leading up to

the prediction event conditioned what happened in the post-prediction period.

ExtremeHurst is scale invariant by definition, so this same effect should be

present on weekly, monthly, or intra-day periods as well.

84

The ExtremeHurst sell predictor is extremely fast, showing 56% winners at only

4 days from the signal, with a Chi squared test p-value = 6.7x10-124. This means

that these are not random signals, and that market behavior leading up to the

prediction event significantly conditioned what happened in the post-

prediction period. ExtremeHurst is scale invariant by definition, so this same

effect should be present on weekly, monthly, or intra-day periods as

well.

85

Looking at the percentage of winning predictions for both ExtremeHurst

predictors compared to the background rate yields the following picture:

The way to interpret this picture is that we gain a significant advantage over

random for both buy and sell predictors for at least 30 days. Extension buy

signals yield the best percentage of winning predictions at 8% more than the

random background after 22 days. This tells us that if we had a large sample of

trades that were triggered by this predictor, we could expect that at 22 days, a

total of 55%+8% = 63% of the trades would be ahead. Remember that our

random trade sample had 55% winners at 22 days, so we would beat it by 8%.

This chart also tells us that we can hold trades for a relatively long time. Despite

the win rate staying constant, average portfolio returns would climb steadily.

86

The sell signal is different and faster. We immediately achieve a 7% edge above

random at 3 days and that edge diminishes steadily over 30 days. This might

make us design a faster sell trading strategy than what we would use for the long

side.

So far we have examined the overall distribution of Z-Scores for ExtremeHurst,

and the percentage of times that a buy signal goes up or a sell goes down. The

win rate is only part of the story however. What we still need to know is how

much gain is possible from these buys and sells in excess of the background

market performance. The pictures below show a gain analysis for ExtremeHurst

buys and sells. Each plot shows the median percentage gain with error bars at the

40th and 60th percentiles and the average of all returns. Note that the average is

not as good a measure of performance because of the overemphasis on outlier

signals. The count of signals is shown in parenthesis.

87

88

On a weekly scale, the power of our ExtremeHurst extensions becomes even

more evident. Remember that 0% on the graph is equivalent to a random win

rate, so any excursion above zero stacks the deck in our favor. Weekly scale

extension buy predictions reach their maximum effect after 8 weeks, with an

overall win rate that is 15% over the random background. Again we see that the

extension sell signals are faster and give only a 2.5% edge over random.

89

90

Price Wizard Fundamental Value Predictor

Price is often out of sync with value, being bid up or down based on future

expectations and irrational trend persistence. Our model incorporates all major

stores of corporate value, normalized simultaneously for sector, industry, and

economic factors. The figure below shows the Z-Scores 30 weeks following a

stock moving to undervalued from overvalued. Note the significant (p=0)

positive bias.

..

91

92

93

94

Precision Turn Cycle Predictor

..

95

96

/ColorImageDict > /JPEG2000ColorACSImageDict > /JPEG2000ColorImageDict > /AntiAliasGrayImages false /DownsampleGrayImages true /GrayImageDownsampleType /Bicubic /GrayImageResolution 300 /GrayImageDepth -1 /GrayImageDownsampleThreshold 1.50000 /EncodeGrayImages true /GrayImageFilter /DCTEncode /AutoFilterGrayImages true /GrayImageAutoFilterStrategy /JPEG /GrayACSImageDict > /GrayImageDict > /JPEG2000GrayACSImageDict > /JPEG2000GrayImageDict > /AntiAliasMonoImages false /DownsampleMonoImages true /MonoImageDownsampleType /Bicubic /MonoImageResolution 1200 /MonoImageDepth -1 /MonoImageDownsampleThreshold 1.50000 /EncodeMonoImages true /MonoImageFilter /CCITTFaxEncode /MonoImageDict > /AllowPSXObjects false /PDFX1aCheck false /PDFX3Check false /PDFXCompliantPDFOnly false /PDFXNoTrimBoxError true /PDFXTrimBoxToMediaBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXSetBleedBoxToMediaBox true /PDFXBleedBoxToTrimBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXOutputIntentProfile () /PDFXOutputCondition () /PDFXRegistryName (http://www.color.org) /PDFXTrapped /Unknown

/Description >>> setdistillerparams> setpagedevice