Top 3 mistakes of trading system development

March 31, 2014 07:00 PM

For most traders, strategy development is merely a means to an end—a necessary process to be endured on the path to making profits. Just as spending gold is more enjoyable than panning for it, so it is with trading system development: Actual trading is usually more fun than developing a trading approach.

Unfortunately, this leads traders to take all sorts of shortcuts, and make all kinds of mistakes, as they develop their trading methodology. While this makes the trader’s life easier in the short run, mistakes and shortcuts inevitably come back to roost in real-time trading. 

The good news is that most traders make the same mistakes in development, and once identified, they can be overcome and corrected. Eliminating these issues won’t make strategy development any easier, regretfully. Strategy development, performed the right way, becomes much more difficult when the shortcuts are eliminated. 

So, what are some of the most common mistakes and shortcuts? Here are three of the more common ones. We explain why they are harmful, how they can be recognized and how they can be corrected. Avoiding these common mistakes will make any trader a better builder of trading systems.

No. 1: Complications & obfuscations

If you have been involved with trading for any length of time, you’ve undoubtedly come across complicated trading approaches. For a discretionary trader, this might look like the chart shown in “Analysis paralysis” (below). Many indicators, many lines, and many support and resistance areas adorn each and every chart. For an algorithmic trader, a complicated approach might be thousands of lines of code, with dozens or even hundreds of variables to optimize and tweak.

Both of these approaches have one thing in common: They are extremely complicated and involved. Many inexperienced traders think this is the way to develop a system. Their thinking is that the more indicators that line up, and the better the fit of their algorithm to past data, the better off the strategy is. Nothing could be further from the truth.

The fallacy in thinking a complicated approach is better is that a superior result with historical data does not equate to real-time success. In fact, constantly tinkering with an approach, adding an indicator, or inserting another rule in the algorithm to make a trading approach more complicated, usually only gives the trader a false sense of confidence. Improving and adding rules to a strategy does not mean it will work better in the future.

As hard as it is for most to believe, the simple approaches are usually the best. For a discretionary trader, a clean chart with only one or two indicators, coupled with a thorough understanding of price action and market dynamics, is always better than a chart clouded with lines and indicators. For algorithmic traders, one simple entry rule is typically better than five or 10 conditions that must be satisfied for a trade to be taken.

A simple example of this is shown in “Clear direction” (below). While far from a perfect system, the strategy is very simple. Such a strategy is unlikely to be overfit to historical data, and therefore is more likely to perform well in the future.

No. 2: Friction-free trading

Take a look at most trading systems for sale and you’ll usually notice a small disclaimer at the bottom: commissions and slippage not included. Likewise, many do-it-yourself developers also ignore commissions and slippage during development. If they do include these costs, they usually underestimate the amount.

You’ll hear various reasons for not including commissions and slippage. “Different brokers charge different commissions” is a common excuse. “My system uses only limit orders” is another. The real reason, though, is to make a system appear better than it really is. Profitable systems become a lot more difficult to discover when real-world trading costs are added in.

For example, take a simple scalping system for the E-mini S&P (CME:ESM14) futures. Without commissions and slippage, such a system might take 20 trades per day, and average $15 profit per trade. The trader might see the result of $300 per day and think that is a nice payday. But add in $5 round-turn commissions, and one tick round-turn slippage (which is probably optimistic), and the $300 per day actually turns into -$50 per day.

One hidden effect of evaluating a strategy without commission and slippage costs is that it leads the trader to take more trades than necessary. Here’s an example. Suppose System A is the scalping system mentioned above. It earns $300 per day before commissions and slippage. Compare that to System B, which only trades one time per day, and averages $50 per trade before trading costs. Anyone evaluating both systems side by side, would surely pick System A over System B. Yet, when commissions and slippage are added in, the exact opposite is true; System B is the only worthwhile approach. It trades less, and commissions and slippage are a much smaller percentage of the gross profits.

It is critical, then, to include adequate amounts for commissions and slippage at the start of development. For the E-mini S&P 500 futures, $5 per round turn for commissions, and one to two ticks round turn for slippage, is a reasonable place to begin. 

No. 3: All data used

A third mistake many traders make during development is using all the available historical data for their testing. Most inexperienced traders will run optimizations and analysis up until the present day. The thinking usually is that the trader wants to be sure the strategy is “tuned” to the most recent data.

Of course, if the first test is a failure, the trader will go back, add a rule or a filter (thereby getting closer and closer to committing the complication mistake discussed earlier), and run through all the data again. 

Eventually, the trader finds something that works, and then moves to live trading. This approach, though, is almost always a recipe for real-time disaster.

A much preferred, and much more difficult, path is to use an out-of-sample test to verify a trading system. For instance, the trader might develop a strategy over the past 10 years of data, but leave the most recent year of data untouched and untested. Once development is complete, the strategy is run on the unseen (out-of-sample) data. If the system performs acceptably on the out-of-sample data, then the system can be considered for live trading.

Taken one step further, walk-forward testing can be used, which combines multiple out-of-sample periods. Such an approach is more likely to lead to  success, because any resulting equity curve is entirely composed of out-of-sample results.

The pitfall to walk-forward or out-of-sample testing is that once the test is run once, any further tests will not truly be “out-of-sample.” So, it is easy to inadvertently make an out-of-sample test an in-sample test, if you perform it multiple times. But performing out-of-sample tests is much better than optimizing on all data. 

No easy way

Creating a viable trading strategy is extremely difficult. In fact, many traders never accomplish this, and many times will instead take shortcuts or make simplifying mistakes while developing a system. Certainly, adding rule after rule, filter after filter and condition after condition is easier to do than finding one simple rule that works reasonably well.

Similarly, uncovering workable strategies is fairly easy, if you do not account for the grinding friction of commissions and slippage. 

Finally, running an optimization on all available data is also much simpler, and seemingly much more fruitful, than running laborious walk-forward or out-of-sample tests. 

The point being, if you optimize with all your data, you create a system that would have made money if you applied it in the period you were testing. This, of course, would require the creation of a time machine, which is even more complicated than building a profitable trading system. 

The simple lesson in these development mistakes is this: If the approach makes finding systems easier, or always creates better backtests, it is a warning sign that something may be wrong. Proper system development is tough. 

Yet, in the long run, developing the correct way is always preferable to losing money in the market due to a development mistake.

Prior to starting a full-time trading career in 2008, Kevin J. Davey was a quality assurance and engineering executive for an aerospace company. Kevin is the author of the upcoming Wiley Finance book “Building Winning Algorithmic Trading Systems,” scheduled for release in July 2014. Reach him via www.kjtradingsystems.com.

About the Author

Kevin J. Davey has been trading for more than 25 years. Kevin is the author of “Building Winning Algorithmic Trading Systems.”