Building an algorithmic trading system using a Monte-Carlo simulation

When you want to calculate the risk or consistency of your trading system, you simply need to test your system. The Monte-Carlo simulation is a good tool for this. The Monte-Carlo method is based on a simulation where all possibilities are evaluated by a generation of random numbers and all possible scenarios are simulated.

What we want is to generate as many random numbers as possible, in order to simulate as many of our trading scenarios as possible. It is the same as seeing what would happen if certain variables like the spread, entry and exit or the price itself were changed. You will see how each option evolves over time, if it is viable in the long and short term, and what other variables can influence its price.

Further below, we'll discuss how this test can help you in your trading, give some practical examples, and we'll also see how to use it in a concrete way.

How does one use the Monte-Carlo method?

The Monte-Carlo simulation relates to the economy, both in terms of companies and investments, the latter being where it is most often seen in action: to create, evaluate and analyse investment portfolios. It is also used to evaluate complex financial products, such as financial options or risk management models.

As ROIs (return on investment) are unpredictable, this type of method is used to evaluate different types of scenarios. A simple example is in trading. As you know, price movements cannot be predicted. They can be estimated, but not with precision. This is where Monte-Carlo simulation comes in, where we try to imitate the behaviour of a trading system or a set of systems to assess how they work over time. Once the simulation has been carried out, a very large number of possible scenarios are extracted from it.

Origin of the Monte-Carlo name

Monte-Carlo is the name of a famous casino located in Monaco (south of France). It is known as the "gambling capital" because roulette is a simple random number generator. The system was devised in 1947 when mathematician Stanislas Ulam came up with an effective method to improve his solo playing.

It was on this occasion that he realised that it would be easier to approach the overall result of a solo match by doing multiple tests with the cards and counting the proportions of the results than calculating all the possible combinations one by one. He presented this idea to another mathematician, John Neumann, in its most rudimentary form. The latter was so impressed with the system that he tried to perfect the formula.

Advances in technology, along with Alan Turing's computer and theories, facilitated the advancement of the study of this particular financial simulation. A letter from Neumann to the Los Alamos laboratory helped share the formula with everyone. The use of the Monte-Carlo model as a research tool stems from work carried out as part of the development of the atomic bomb during WWII at the National Laboratory of Los Alamos (USA).

Their work focused on the simulation of probabilistic hydrodynamic problems linked to the scattering of neutrons in fission material. This diffusion is eminently random in how it acts. Today, it is a fundamental element of ray tracing algorithms for the generation of 3D images. In principle, it was idealised by Neumann for evaluating multiple integrals. Today it is used in labour market for all kinds of statistics, and oddly enough, for high-risk administrative decisions, where it would be difficult to verify the validity of a variant.

Understanding Monte-Carlo

The most essential thing to keep in mind when doing your calculations is that you need to generate a fair amount of random numbers. How can you generate random numbers? In the Monte-Carlo casino this is used with a roulette wheel, so it would take you longer than expected. The best solution is to simply use software.

Suppose we want to generate 10,000 random numbers. How long would it take to calculate each probability? Computer programmes are used to generate these numbers. They are not considered purely random numbers because they are created by the program using a formula. However, they are very similar to the random variables in reality. They are called pseudo-random numbers.

A practical example:

Software programmes can quickly determine volumes in highly complex models. These programmes can't usually determine a volume (for example, for a prism, the base area multiplied by the height). One of the things that it can do, though, is to divide the model into a set of small sub-models with which a volume can be determined. However, this consumes a lot of resources for calculating the volume of each of the elements.

For this, they use Monte-Carlo simulations, which are more robust and efficient. The software knows the analytical expression of the model geometry (position of nodes, edges and surfaces) and can approximate a point that is inside or outside the model at a much lower cost.

First, the software places the model in a known volume (for example, a 1 m3 cube). It then generates a random point inside the known volume and records whether the point has "fallen" inside or outside the model. This process is repeated several times (thousands or millions), which makes it possible to obtain a very large record of the number of times it falls both inside and outside.

The probability that a point falls inside is proportional to the volume of the model, so the proportion of points that fall inside, in relation to the total points generated, is the same proportion of volume that the model occupies in the 1 m3 cube. If 50% of the points fall into it, the model occupies 50% of the total volume, i.e. 0.5 m3. Obviously, the more points the software generates, the smaller the error in the volume estimation.

Despite what you are told, trading is difficult. And developing a trading strategy (algorithm) is difficult. To build an algo properly, there are a number of things you need to do correctly. For example, you need to know how to not over-optimise your strategy. Easier said than done!

However, in the rest of this article I will focus on what happens AFTER you have tested and optimised your strategy.

Testing and developing algorithmic trading systems - what should I do after I have tested and optimised it?

Most traders know how difficult it is to design trading systems that have a lasting edge. Many people can look at a chart and establish trading rules that work well for that particular chart. However, when they test their rules against a large amount of historical data, they find that their system is worthless.

Other traders will develop a system that omits factors such as commissions and slippage. The trading system looks great, and the trader is very enthusiastic, until these real fractional costs add up, and a winning system turns into a break-even system or a losing one.

Improving algorithmic trading

If you have a hard time developing an algo trading strategy, you really need to understand HOW to properly build an algo. With proper testing and development methodology, it is common for traders to test dozens, hundreds, and even thousands of trading ideas before finding one that works.

Here's what the strategy development process looks like:

Many traders assume that testing - optimisation, Walk-Forward tests, etc. - constitute the major part of the development process. Yet once a trading system has passed the initial development and testing phase, the job is far from over. Other steps are required before you risk real money in a trading system. Following these post-development rules can save you thousands of dollars and prevent unwelcome stress.

One of the mistakes traders make is to base their trading decisions on the shape of a historical backtest equity curve. Many traders look at the equity curve and say to themselves "since the historical maximum drawdown is $X, I will probably never have a larger drawdown than that". Or they will say "this system has made money every year, so it's safe".

Unfortunately, both of these assumptions are incorrect. This is because the historical stock curve is just one of many possible paths the trading system could have taken over time. This is shown in Figure 1. If we tweak a few trades here and there, the maximum drawdown could have been much worse, and profitable periods could easily have turned into losing periods. On the other hand, the performance could also have been better than it was.

Monte-Carlo simulation

Is there a way to analyse this phenomenon - to see the different paths a trading system could have taken? The answer is yes - using a method called Monte Carlo analysis. Click here to download a free Monte-Carlo simulator.

Monte-Carlo analysis is based on the idea that trades made in the past will re-occur in the future, but in a different and unknown order. By scrambling the trades in a different order, different equity curves will be generated.

Even though there are times where Monte Carlo analysis isn't appropriate, for most trading systems the analysis is valid and can provide great insight into the characteristics of a trading system. With Monte Carlo analysis, you just write each trade result on a piece of paper, drop them in a hat, and then choose the trades one by one to create an equity curve. Computers can obviously do this sampling process much faster, and create thousands of simulated equity curves in just a few seconds.

When you analyse thousands of Monte Carlo simulations, you get statistics on the biggest possible drawdown, possible annual return, and the risk of bankruptcy. If you have performance targets for simulation results, then you can easily see if your system meets your targets. It's impossible to get such information just by looking at the historical curve of stocks.

The trading system's incubation period

When most traders finish developing a strategy, they are usually so enthusiastic that they rush into real money trading. This can be a big mistake, especially for newbies. This is largely due to trading software. Most modern trading software makes it very easy to create systems, change rules, and perform multiple optimisations. In fact, most software is designed to make optimisation easy, thereby encouraging it.

So, because the software encourages many of these bad habits, the historical stock curve will invariably look better with each new test. But, most of the time, a better looking backtest has no correlation (or even a negative correlation) with future performance. Why is this? Simply put, adding rules, filters, and performing more optimisations has the effect of tailoring the trading strategy to historical data. In reality, a trader wants the exact opposite: a trading strategy that works well on previous data, but is not 100% suited for it. These types of systems work better in the future.

Often, the observation of a strategy after its development provides important clues. This process is called incubation. In its simplest form, you let the strategy work in a live setting, but without trading, for a period of 3-6 months. Once a month, you review the results and add these new results to your historical backtest equity curve. An example of this is shown in the image above, where the results of the incubation are added to the historical results of the walk-forward (advanced optimisation). Since in this case the results of the incubation are similar to the historical results, this gives the trader the confidence that the development of the strategy has been carried out correctly.

After a few months of incubation, you will likely see one of two things: The performance of the system "looks" like historical tests, or it doesn't. One trick is to print the equity curve (historical performance and incubated performance) as large as possible, and stick it on a wall about a few meters away. If you can see where the historical test ends and where the incubation begins, you have a problem. Ideally, there should be no change in performance between the historical results and those of the incubation period.

There are more statistically oriented tests that you can run on the strategy's results to determine if performances have changed. A histogram of daily results is one such method. Again, this is to see if the performances of the system change or not after you develop the system. What could cause a change in performance? It is certain that if the market behaves differently from what it has been in the history of testing, performances can change dramatically. Assuming you've tested years of data, this probably isn't too likely. It is more likely that an error was made during development. For example, the strategy may have been over-optimised, with too many variables. Or, the developer may have looked at current data before selecting their rules, which is a form of hindsight bias.

Analysing correlations

Another area that many traders overlook after developing a system is correlation with other systems. Obviously, if you are only trading one system, you don't have to worry. But if you are adding a system to your trading portfolio, you need to make sure that your new system complements your other systems. You don't want all of your systems to buy and sell at the same time. In other words, you want your systems to correlate as little as possible.

One way to get correlation is to trade different markets during different time periods. Often, this provides sufficient correlation. If you like mathematical proof of correlation, you can perform linear regression of trading system results. The best way to do this is to compare the results of two strategies, and see what the correlation coefficient is. If the results are between -0.5 and +0.5, you have a pair of systems that are not correlated with each other. The image above shows what correlated systems and uncorrelated systems can look like. Traders should always look for uncorrelated systems and avoid correlated systems.

Trading system correlation of is important because a key factor in trading is proper risk control. When two or more systems are highly correlated, your level of risk increases dramatically. As an alternative, you really want systems that aren't correlated. This creates a smoother equity curve and is key to proper diversification.

Limited tests using real money

After performing a Monte Carlo simulation of your trading systems, "incubating" your system for a few weeks, and verifying that your new system doesn't correlate with the other systems you use, it might be time to actually test it with a few small trades.

Why just a few trades, rather than a large number? There are several reasons, first of all, it is always possible that systems that have passed all the previous steps will fail. Second, trading just a few times is a great way to check any assumptions you have made regarding slippage. You may find that the slippage is much more, or much less, than you expected, and therefore your position sizing approach when you scale upwards might change. Finally, you may decide that you don't like the system, even though it is profitable. Many people who like to follow trends, for example, have difficulty using reverting strategies. It is best to find out that you don't like trading a system while you are trading on a small scale.

Conclusion

Once you've developed a system, it's clear that trading with real money is not the next step. Instead, the best approach is to proceed with caution and continue with the analysis. From Monte Carlo simulations and incubation, correlation analysis and limited real money testing, these extra steps can make things less stressful for you and your trading account. In the end, you will have a more robust strategy and a more realistic view of what to expect when using this strategy with real money.