Rules and backtests

Mark the bar after the cross. Then catch the rule that peeks. · 11 min

Hook · Which rule peeks

Three of these can be followed. One uses a fact you would not have had.

Tap the rule that peeks.

At the open you do not know the close. Using it is look-ahead. The other three can be applied after the relevant bar has closed. Ignore wicks is a choice of definition, the one you already use, not a peek.

Squeeze · A rule two people would mark the same way

A testable rule names the market, the timeframe, the entry, the exit, and the size. Buy strength is not a rule. A backtest applies the rule to the past and writes down the trades. It is not a forecast. The usual lies are look-ahead, overfitting, survivorship, and ignoring costs. Look-ahead uses a fact that was not known yet. The classic version is entering at today’s open because today’s close crossed a line. Overfitting adds tweaks until the past looks perfect. Survivorship tests only the stocks that still exist and drops the ones that were delisted. Costs are the spread, commissions, and slippage. A test that fills at the mid and ignores them is optimistic. A handful of trades is not evidence. You already insist that a trend break is a close, not a wick, and that the entry is something a stranger could mark. The next chart uses a made-up line, not a 200-day average, and not a HOLD/LINE track record. The practice rule: when a close crosses from below the line to above it, the entry is the next bar.

Body · Mark only the entry bars

Made-up closes. The line is drawn for the toy. Practice rule: when a close crosses from below the line to above it, the entry is the next bar. Tap those entry bars only.

Wicks · A new toy line

Different closes. Same rule. Bar numbers start at 1 on the left.

Tap one entry bar. The bar that crossed is the signal. The entry is the next one. Either entry bar counts.

Bar 3 closed above the line after bar 2 was below, so the entry is bar 4. Bar 6 closed back above after bar 5 was below, so the entry is bar 7. The signal bar is not the entry. Entering on the signal bar’s open, using its close, would be look-ahead.

A test fills at the mid, ignores the spread, and drops every delisted name. It is…

You pay the spread and slippage live, and delisted companies were part of the past. Leaving both out flatters the curve. Overfitting is the separate problem of tuning until that curve looks perfect.

A toy line is crossed by the close eleven times in two years, and you only kept the four that looked good. How many trades belong in the sample?

Eleven. Dropping the ugly ones is how a test starts lying. The four are a highlight reel, not the rule.

Challenge · Fresh cheats

A new peek, then a curve with ten exceptions.

The rule says: buy at Monday’s open if Friday’s employment report, released Friday morning, was strong. Is that look-ahead?

Look-ahead is using a fact that was not known yet. Friday’s release is known by Monday. Entering Monday’s open because of Friday’s close of the report is allowed. Entering Friday’s open because of Friday’s close is not.

You added ten filters until the past sample had no losing trades. The careful name is…

Overfitting is tuning until the past looks perfect. Survivorship is a different lie: ignoring names that no longer exist. You can commit both. This sentence is the first one.

Reveal · Two crosses, worked

Toy closes 98, 99, 101, 102, 103, 99, 97, 101, 100.5, 104. Line at 100. The close goes from 99 to 101: that bar is the signal, known only once it has closed. The entry is the next bar, the one that opens after you know. Later the close goes from 97 to 101: again the entry is the following bar. A test that buys the signal bar’s open, using that bar’s close, peeked. A test that then keeps only the names still listed today, and fills at the middle of the spread, flatters the result a second and third time. None of this is a HOLD/LINE track record. The scanner’s listings are history too.

Entering at today’s open because today’s close crossed the line is…

A rule you could have followed waits until the signal bar has closed, then uses a later price, such as the next bar’s open.

Case · Case file: would the test have cheated

The bars after the last one are hidden. Decide, then the path is shown.

Check · Fresh tests

New rules. Same four lies.

The rule says enter the day after a close crosses above a line. The test fills you at that signal day's open. The lie is…

Answer

Using a price from before the signal existed. The open was already gone when the close printed. The test used a price you could not have traded.

You try twelve versions of a rule on the same chart and keep the one that looks best. The sample you are admiring is…

Answer

One result picked after seeing the others. The winner of a search. It has not been a single rule stated in advance.

A rule two people would not mark on the same bar cannot be tested. The missing piece is…

Answer

An exit a stranger could mark the same way. A definition that does not depend on the tester's mood. Otherwise the backtest is a story.

The test ignores the spread and fills every stop at the exact price. On a gap, that test will…

Answer

Overstate the result relative to a live fill. Look kinder than the live book. A stop-market can fill through the price. The spread was a real cost.

Key terms

Look-ahead bias
Using a fact that was not known yet. The classic version is entering at today’s open because today’s close crossed a line. At the open, the close does not exist.
Overfitting
Adding tweaks until the past looks perfect. A curve that needed ten exceptions rarely describes the next sample.
Survivorship bias
Testing only the stocks that still exist. The ones that were delisted or collapsed were part of the history. Leaving them out flatters the result.
Costs
Spread, commissions, and slippage. A test that fills at the mid and ignores them is optimistic. A handful of trades is not evidence.