Walk forward optimization

Walk forward optimization is a method used in finance to determine the optimal parameters for a trading strategy. The trading strategy is optimized with in-sample data for a time window in a data series. The remaining data is reserved for out of sample testing. A small portion of the reserved data following the in-sample data is tested and the results are recorded. The in-sample time window is shifted forward by the period covered by the out of sample test, and the process repeated. Lastly, all of the recorded results are used to assess the trading strategy.

After the most suitable parameters are found, the system is run using another segment of data. The two segments of data do not overlap each other. It is the culmination of the following methods and aids in creation of robust systems.

Past data is used for Backtesting of a trading system. It refers to applying a trading system to historical data to verify how a system would have performed during the specified time period and is useful if a system was not profitable in the past.

Forward testing (also known as Walk forward testing) is the simulation of the real markets' data on paper only. One moves along the markets live and is not using real money, but virtually trading in the markets to understand their movements better. Hence, it is also called Paper Trading. Forward performance testing is a simulation of actual trading and involves following the system's logic in a live market.

Overview

One of the biggest issues with system development is that many systems do not hold up into the future. There are several reasons for this. The first is that the system is not based on a valid premise. Another is that the testing is not sound for reasons such as:

• Lack of robustness in a system due to improper parameters. A system is considered robust if it runs well in any market conditions.
• Inconsistent rules and improper testing of the system using 'out-of-sample' and 'in-sample' data.

Walk Forward Analysis does optimization on a training set; test on a period after the set and then rolls it all forward and repeats the process. We have multiple out-of-sample periods and look at these results combined. Walk forward analysis was originally discussed by Robert E. Pardo. Walking forward can keep a trading model a step ahead. Walk forward is so called, as we have multiple walk training and testing periods is less likely to suffer from overfitting.

Walk forward testing allows us to develop a trading system while maintaining a reasonable 'degree of freedom'. Walk-forward testing carries the idea of 'out-of-sample' testing to the next level. It is a specific application of a technique known as Cross-validation. It means to take a segment of your data to optimize a system, and another segment of data to validate. Hence, here you optimize a window of data say past 1000 bars, and then test it on next 200 bars. Then roll the whole thing forward 200 bars and repeat the process. This gives you a large out of sample period and allows you to see how stable the system is over time.

Suppose you consider a strategy around a moving average. You take the first 3 months of data, and find that for that period a 20-minute moving average was optimal (using tick data). You then validate this rule by assessing its performance for the 4th month (i.e. profit, reward/risk or any other statistic of interest). Next, you repeat the optimization using data from month 2–4, and validate using month 5, and keep repeating this until you've reached the end of the data. The performance you get for the validation months (4-13) are your out-of-sample performance.

The basics behind the data used

Before doing the backtesting or optimization, one needs to set up the data required which is the historical data of a specific time period. This historical data segment is divided into the following two types:

• In-Sample Data: It is a past segment of market data (historical data) reserved for testing purposes. This data is used for the initial testing and any optimization and is the original parameters of a system under test.
• Out-of-Sample Data: It is the reserved data set (historical data) which is not a part of the in-sample data. It is important as this ensures that the system is tested on another period of historical data not earlier thus removing any bias or influences in the checking of the system's performance.

The process is to first develop a trading system using in-sample data and then apply the out-of-sample data to the system. The results of both cases can then be compared and tested.

Conclusion

For a better understanding, please see the example here.

In order to evaluate any system, one should check out its performance when using the "Out-of-Sample Data" (test data) and not the "In-Sample Data" (data used for optimization of the system). Thus, walk forward test determines the optimized system performance as follows:

• Was it realistic? It is considered realistic if it could fit to the entire test data (or at least to a larger segment of the test data) used. It implies that the system has the characteristics of the real time markets and is robust.
• Is it overfitting? If the system does not perform well using the test data and seems to fit only chance characteristics (not necessarily part of the test data), the system is considered to be overfitting. It is neither a robust nor reliable one and ought not to be used for trading.

Hence, the out-of-sample data plays a crucial role in determining the validity and reliability of the system and is a realistic estimate of how a system should work in real markets.

See also

• Backtesting
• Mathematical optimization
• Overfitting
• Trading strategy

Literature

• Katz, Jeffrey Owen, and McCormick, Donna L. "The Encyclopedia of Trading Strategies." McGraw-Hill, 2000.
• Essential technical analysis: tools and techniques to spot market trends By Leigh Stevens
• The encyclopedia of technical market indicators By Robert W. Colby

External links

• Out-of-sample and walk-forward testing