Rules and backtests
Mark the bar after the cross. Then catch the rule that peeks. · 11 min
Hook · Which rule peeks
Three of these can be followed. One uses a fact you would not have had.
Tap the rule that peeks.
At the open you do not know the close. Using it is look-ahead. The other three can be applied after the relevant bar has closed. Ignore wicks is a choice of definition, the one you already use, not a peek.
- Enter on the next bar’s open after a close back above the line.
- Exit if a later close finishes back under the line.
- Enter at today’s open if today’s close ends above the line.
- Ignore a wick that does not close through the line.
Squeeze · A rule two people would mark the same way
A testable rule names the market, the timeframe, the entry, the exit, and the size. Buy strength is not a rule. A backtest applies the rule to the past and writes down the trades. It is not a forecast. The usual lies are look-ahead, overfitting, survivorship, and ignoring costs. Look-ahead uses a fact that was not known yet. The classic version is entering at today’s open because today’s close crossed a line. Overfitting adds tweaks until the past looks perfect. Survivorship tests only the stocks that still exist and drops the ones that were delisted. Costs are the spread, commissions, and slippage. A test that fills at the mid and ignores them is optimistic. A handful of trades is not evidence. You already insist that a trend break is a close, not a wick, and that the entry is something a stranger could mark. The next chart uses a made-up line, not a 200-day average, and not a HOLD/LINE track record. The practice rule: when a close crosses from below the line to above it, the entry is the next bar.
Body · Mark only the entry bars
Made-up closes. The line is drawn for the toy. Practice rule: when a close crosses from below the line to above it, the entry is the next bar. Tap those entry bars only.
Wicks · A new toy line
Different closes. Same rule. Bar numbers start at 1 on the left.
Tap one entry bar. The bar that crossed is the signal. The entry is the next one. Either entry bar counts.
Bar 3 closed above the line after bar 2 was below, so the entry is bar 4. Bar 6 closed back above after bar 5 was below, so the entry is bar 7. The signal bar is not the entry. Entering on the signal bar’s open, using its close, would be look-ahead.
A test fills at the mid, ignores the spread, and drops every delisted name. It is…
You pay the spread and slippage live, and delisted companies were part of the past. Leaving both out flatters the curve. Overfitting is the separate problem of tuning until that curve looks perfect.
- Optimistic relative to a real account
- The only fair test
- A cure for overfitting
A toy line is crossed by the close eleven times in two years, and you only kept the four that looked good. How many trades belong in the sample?
Eleven. Dropping the ugly ones is how a test starts lying. The four are a highlight reel, not the rule.
Challenge · Fresh cheats
A new peek, then a curve with ten exceptions.
The rule says: buy at Monday’s open if Friday’s employment report, released Friday morning, was strong. Is that look-ahead?
Look-ahead is using a fact that was not known yet. Friday’s release is known by Monday. Entering Monday’s open because of Friday’s close of the report is allowed. Entering Friday’s open because of Friday’s close is not.
- Yes. Monday cannot know Friday
- No. Friday’s report exists before Monday’s open
- Yes, because all backtests cheat
You added ten filters until the past sample had no losing trades. The careful name is…
Overfitting is tuning until the past looks perfect. Survivorship is a different lie: ignoring names that no longer exist. You can commit both. This sentence is the first one.
- A robust rule
- Overfitting. A curve that needed ten exceptions rarely describes the next sample
- Survivorship, by definition
Reveal · Two crosses, worked
Toy closes 98, 99, 101, 102, 103, 99, 97, 101, 100.5, 104. Line at 100. The close goes from 99 to 101: that bar is the signal, known only once it has closed. The entry is the next bar, the one that opens after you know. Later the close goes from 97 to 101: again the entry is the following bar. A test that buys the signal bar’s open, using that bar’s close, peeked. A test that then keeps only the names still listed today, and fills at the middle of the spread, flatters the result a second and third time. None of this is a HOLD/LINE track record. The scanner’s listings are history too.
Entering at today’s open because today’s close crossed the line is…
A rule you could have followed waits until the signal bar has closed, then uses a later price, such as the next bar’s open.
- Look-ahead. The close did not exist at the open
- A conservative fill
- Required for a valid sample
Case · Case file: would the test have cheated
The bars after the last one are hidden. Decide, then the path is shown.
Check · Fresh tests
New rules. Same four lies.
The rule says enter the day after a close crosses above a line. The test fills you at that signal day's open. The lie is…
- Using a price from before the signal existed
- Slippage, because the open is busy
- A sample that is simply too short
Answer
Using a price from before the signal existed. The open was already gone when the close printed. The test used a price you could not have traded.
You try twelve versions of a rule on the same chart and keep the one that looks best. The sample you are admiring is…
- More reliable, because you checked twelve
- The average of the twelve versions
- One result picked after seeing the others
Answer
One result picked after seeing the others. The winner of a search. It has not been a single rule stated in advance.
A rule two people would not mark on the same bar cannot be tested. The missing piece is…
- More years of data
- An exit a stranger could mark the same way
- A higher win rate
Answer
An exit a stranger could mark the same way. A definition that does not depend on the tester's mood. Otherwise the backtest is a story.
The test ignores the spread and fills every stop at the exact price. On a gap, that test will…
- Overstate the result relative to a live fill
- Match the live book, because stops are exact
- Understate the result, because gaps help stops
Answer
Overstate the result relative to a live fill. Look kinder than the live book. A stop-market can fill through the price. The spread was a real cost.
Key terms
- Look-ahead bias
- Using a fact that was not known yet. The classic version is entering at today’s open because today’s close crossed a line. At the open, the close does not exist.
- Overfitting
- Adding tweaks until the past looks perfect. A curve that needed ten exceptions rarely describes the next sample.
- Survivorship bias
- Testing only the stocks that still exist. The ones that were delisted or collapsed were part of the history. Leaving them out flatters the result.
- Costs
- Spread, commissions, and slippage. A test that fills at the mid and ignores them is optimistic. A handful of trades is not evidence.