Multi-algorithm trading strategy testing- Portfolio and Methodology

"Two heads are better than one", the saying goes. But can averaging improve trading systems? Can a multi-algorithm technique improve profitability and lower risk? After compiling the "Summary of Strategy Performance", I was very curious to quantify if and for how long strategies such as those published here were profitable for. To do that, I used out-of-sample testing which is very easy to do with SignalSolver.

In out-of-sample testing, you first find a strategy which worked for a specific time period. You then see how that strategy performed on data outside that specific time period. You can either go forward or backwards in time, but in these tests I go forward because that's pretty much how you would do it in reality. By multi-algorithm I mean running several algorithms simultaneously on the same stock. We look at running from 1 to 8 algorithms on each stock in a portfolio. Every stock has a different set of algorithms, found by optimization.

To get a handle on average success rates, I needed to analyze a reasonable number of stocks. Accordingly, I will report on 30 stocks with 128 out-of-sample results on each, a total of 960 algorithms each run for 4 time periods.

The A-Z Portfolio

I tried to create a portfolio which didn't look like I had cherry-picked stocks to put a rosy face on things. Accordingly, I used stock symbols A though Z which comprises 24 stocks (because symbols J and U are not assigned at this time). This I will call the "A-Z" portfolio. Other stocks were tested: AMZN, DUST, FAS, FB, NUGT, SPY and UWTI. I will report on these later, but the results are also in the spreadsheets if you are interested ahead of time.

Methodology

I have been careful to ensure that anyone with a recent copy of SignalSolver can reproduce the results I report. You just need to make sure the Settings are as per the Settings tab on the results spreadsheets. Also, you will need to adjust the Web Query end date to get the exact OHLC data I used, (or increase the Offset to move the data back in time).

First, load the daily OHLC data for the stock into SignalSolver by typing the symbol and making sure data interval was set at "Daily". Set the "Days To Analyze" to 250 days which is almost a year of data and the Offset set to 100 days leading to an exclusion of the most recent 100 days. For example, for stock symbol "A" the included data was 6/15/15 to 6/9/16, the excluded data 6/10/16 to 10/31/16. Other exclusion days tested were 200 and 300 days. The buy and sell scan ranges were set to -20% to +20%, and the investment style set to "Long and Short", starting in the "Out" state. I'll talk about optimizations later.

Clear all the algorithms on the Algorithm tab (select the rows and press delete) so SignalSolver won't scan through any pre-configured strategies. Set the Seek/Scan level to "Seek level 2". and press Solve.  This will search 1000 algorithms using about 400,000 backtests. The seek doesn't search the same 1000 algorithms each time, but follows logic as to what may constitute a good strategy. It would take about 5 minutes to finish on my machine after which the top 100 algorithms will appear on the Algorithms tab, with the top 8 being transferred to the Strategy and Report Tabs.

For this study, I looked only at the top 8 strategies found. Here is an example Seek result:

Example of a seek result

Notice that the strategies are ordered with the highest Figure of Merit first, as this was a FOM optimization (see below).

Next step is to see how each of these 8 strategies played out moving forward in time. I started out looking at the next 25, 50 days and 99 days, but it quickly became clear that I'd have to also look at a shorter time interval, so I opted for 10 days.

Start State of out-of-sample testing

Start State is the start state of the backtest and is especially important for the short time results as these may not trade much, if at all. I set the Start State of the out-of-sample test to Out, unless a clear Start State was indicated by the traffic lights on the 250/100 seek result. For example, in the above illustration, 5 algorithms are green (bullish) and 3 are red (bearish). Only if the vote was 6/2, 7/1 or 8/0 would I set the start state to that indicated by the traffic lights, Long for green and Short for red, otherwise use Out.

After setting the Start State, I would read off the out-of-sample results by setting the "Days to Analyze" and the "Offset". Here is an example, same stock, same algorithms:

Out-of-sample result

Notice the data analyzed starts from 6/16/16. I had to avoid using 6/15/16 data because it is used in the traffic light calculation, that's why I set the offset to 89 and not 90 days. Its vital that none of the backtested data is used in the out-of-sample analysis. After 10/89 I would set the Days To Analyze/Offset to 25/74, then 50/49 then 99/0. Each time I would cut and paste the top 8 Strategies and the Annual Returns into the results spreadsheet.

Optimizations and bands

I followed this procedure for two different Solve optimizations, one using a simple Return optimization on the Percentage Band (PB) (like most of the algorithms on this site), the other a Figure of Merit optimization using a 50 period (H+L)/2 Exponential Moving Average band. Why use an EMA band? It seemed to give better results than the PB band, but I haven't confirmed this properly yet.

Bands are explained here. The FOM optimization used these weights-

With these FOM settings we are essentially looking for algorithms with good 250 day return, low standard deviation of returns for each quartus (62 days in this case), a good value for the minimum of all four quartus returns, and good returns for the most recent quartus. By setting up this kind of optimization we hope to avoid algorithms which performed very well for only short period. For more information on FOM, see here.

There is a secondary effect to running multiple algorithms on the same stock; a change in efficiency. All the tests were done using investment style "Long and Short", where you are essentially tying up the capital 100% of the time. If you run 2 algorithms  on the same stock, however, there may be times when algorithm A puts you long while algorithm B puts you short, effectively taking you (mostly) out of the market. Since, as I hope to show, profits on each algorithm average out to be similar, you have essentially increased the effective annual return by reducing the time your capital is in the market. We don't take this effect into account in the Annualized Returns reported here, we assume all capital is in the market for 100% of the time. But I believe this effect is real--please correct me if I am wrong about this.

In the next post, we shall look at the results.

This is a trading system for Keryx Biopharmaceuticals Inc. (KERX). It requires daily intervention, trading at the open of business.

KERX Trading System: Description and Performance

A trading system which worked for FEYE (FireEye). This is based on daily data, so traded at most once per day.

FEYE Trading System: Equity curve, signals and positions

As always, future performance is not guaranteed.

Here is a Walt Disney Company (DIS) trading strategy with daily maintenance which had quite nice characteristics; 59% annualized return over the last 2 years.

DIS trading strategy, equity curves, signals and positions.

The performance was better than long buy-hold with lower drawdown and about three times the reward-risk. Signal reinforcement was good, and not many dual signal days.

DIS trading strategy (daily), and performance characteristics over the last 2 years.

Below I show the return for the DIS trading strategy for each 6 month period going back 2 years. Comparing with long buy-hold you can see better consistency (higher minimum and lower StdDev).

Please note that this is simply a measurement, not an opinion or financial advice.

This post was corrected 1/8/2016 to correct a miscalculation in the short-side returns.

Update Oct 21st 2016

Strategy peaked 11/23/15, shortly after the stock price peaked.

DIS Trading Strategy (Daily) Update Oct 21st 2016

GLD is the much traded SPDR Gold Trust ETF. I find these two GLD trading strategies interesting because they gave reasonable results (32.6% and 48% annualized return) for each of the four 6 month periods of the analysis. The strategies require daily intervention.

Strategy 1: BCS AHC

This is a buy on fall, sell on rise strategy using the close price as the buy reference and the high price as the sell reference. As you can see from the life chart and the longevity analysis, the lowest return for the last four 6 month periods was close to 30% annualized. You can easily find algorithms with over 50% annualized return for GLD, but they are not as consistent, with lowest quartus returns of around 14%.

I would prefer to see more reinforcement on the signals, but there it is. As of Sat Sept 5th, this strategy is Short with no transactions pending.

Strategy 2: AOO AHCI

In many ways this strategy shows better results than the BCS AHC strategy, for example there was lower drawdown, higher return, better signal reinforcement and good consistency (minimum 6 month quartus return of 38.43%). On the other hand, signals were cluttered, with 67 dual signal days, 50 buy signal days and 75 sell signal days. Also, its not an trading strategy that makes intuitive sense; buy on rise, sell on rise. Maybe it is one of those serendipitous occurrences, we shall see. Notice the sell strategy is almost the same as for BCS AHC but the sell signal percentage is quite different.

As of Sat Sept 5th, the strategy is Short with no transactions pending.  For a list of trades in Excel format: GLD.D2 Trades. For a more detailed explanation of the above charts, please go here.

Algorithms were discovered by SignalSolver.

Please note, the above analysis was corrected on 12/28/2015 to reflect a bug fix in SignalSolver. Original returns were \$9530 and \$12753 respectively.

Update Oct 21st 2016

Both algorithms peaked 12/30/2015.

GLD Trading Strategy AOO AHCI Update Oct 21st 2016

GLD Trading Strategy BCS AHC Update Oct 21st 2016

This TSLA trading strategy would have given a 1062% return over 2.1 years vs. a buy-hold return of 86% for the same period. The strategy is based on buying and selling when the stock price rises above specific thresholds. The buy side keyed off the day’s open price; the buy and cover signal appeared when the price rose 4.27% above the open price of the day, and the buy is at the signal price, so you would have to set up stop orders for the buy and the cover.

The sell and short signals came along when the price rose 2.82% above the previous day’s high, and the sell actions occured at the subsequent open using market orders. Every day you would have to recalculate the buy or sell point to find the new buy or sell price. SignalSolver will re-calculate the prices each time you update the prices from the web or enter the latest prices manually. Here is the list of trades.

The equity curve shows the return of \$10,000 over time for the algorithm (yell0w) and buy-hold (gray):

This algorithm spent 64% of its time short. Looking at the signals at the bottom of the chart, you may notice that they are fairly thin, and there are 21 dual signal days, 28 buy signal days and 36 sell signal days. There is occasional reinforcement of signals, OK but not great.

From the performance table you can see that the long side of the algorithm worked much harder than the short side (the leftmost two columns), but the combination (always being long or short) gave annualized return of 223%.

Lets look at sensitivity to the buy and sell parameters.

The dotted line is buy-hold annualized return at 34.45%. The colored lines are the return for different values of buy and sell percentage in different time periods. The blue line is the overall performance for the whole 2.1 year period (528 daily data points), the other green, red, yellow and white lines each represent one quarter of the data (which I call a quartus). As you move the buy or sell point out of the region, you can see that some quartus’s would have been lossy. The worst performing quartus for the chosen buy and sell points was the most recent one, 02/19/15 to 08/26/15, and the annualized return was 161%. The algorithm was found by instructing SignalSolver to find strategies with the best minimum quartus return.

If you map the return for a large range of buy and sell points you notice that the overall surface is a little peaky.

While there is a fair amount of space under the peaks, there are also steep cliffs in the vicinity, so if the buy and sell points were to move around over time you would be in trouble. For that reason I would not expect such high gains in the future.

I will be paper trading this strategy for a while and will post the results from time to time.

Andrew

The above analysis has been corrected 12/29/15 for a bug in the short side return.

Update Oct 21st 2016

TSLA Trading Strategy (Daily) Update Oct 21st 2016

TSLA Trading Strategy (Daily) Update Oct 21st 2016

Frequent reversals characterize this strategy for Omeros Corporation

This OMER trading strategy is signal rich; there were 174 dual signal days out of the 528 days in the analysis. Added to that 151 buy signal only days and 39 sell signal only days and you get 364 signal days, of which 250 were actionable signals leading to trades, of which only 146 were good. Still, all that activity led to a theoretical \$311,341 profit from \$10K invested over the 2 year period from 7/12/13 to 8/14/15.

The algorithm itself is a bit of an odd one with buys triggering off price changes from the open price and sells triggering off price changes from the previous day’s open price. Is it just a fluke that it has worked so consistently? In its worst quartus (132 trading days in this case) this algorithm returned 250% annualized or 93% actual return.

You can see from the list of trades and the equity chart that there were several periods of daily reversals from long to short and back again. You might think that ignoring dual signals would work better, but it doesn’t–it leads to an 80% reduction in returns.

This strategy was found by optimizing for minimum quartus returns and then doing some minor tweaking by hand, which is quite easy to do since the Scan charts in Signalsolver are interactive. I just moved the buy and sell points to areas away where there were lower returns. I’m just going to post the charts, if you need help interpreting them I would refer you to yesterday’s AAPL post where I discuss the methodology in detail.

By the way, OMER took off today gapping up 70% or so. I was working on it before that so this change doesn’t show on the data, but the proceeds would have jumped to \$929,212. It would be cheating to track this strategy knowing it had already added 70%, so I’m not planning on doing so.

Andrew

Equity curve for the OMER trading strategy showing \$545,057 in returns over a 2 year period.

OMER Daily algorithm–current and pending trades, if you are looking for synchronization info.

This algorithm peaked and then failed dramatically immediately after the 70% gap up on Aug 18th, very shortly after publication. Here is the 528 day equity curve:

Here are the stats:

Buy-hold would have been a much better option. For a profit, one solution was a buy point of 2.0% and a sell point of 4.4%, which would have given a return of \$13,964 over the 94 days. Unfortunately, these parameter changes don’t appear to be predictable.

Andrew

This UWTI trading strategy would have returned \$676,147 for \$10K outlay over a 2 year period. It was a very straightforward strategy with simple maintainence, once a day you would have put in either market orders to cover and buy or stop orders to sell and short before the open. This is another result discovered by optimizing minimum quartus returns, a SignalSolver backest optimization technique. A quartus is one-fourth of the data and we look for algorithms which give the best minimum return of all four quartus results. In this case it was 206% annualized for the period June 2013 to Jan 2014. It happened to be the fourth best result found for this particular scan of 500 algorithms but I chose it because the drawdown was significantly better than for the other 3 (30.7% vs 55% for short-hold).

Results for this strategy were not consistent, most of the gains were made in the period June 2014 to Feb 2015.

Update 12-30-2015:

The above analysis has been corrected for a miscalculation of the short-side return of this algorithm.

Update 10-20-2016

This algorithm turned a complete loss:

In contrast to yesterday’s TNA trading strategy optimized for low drawdown, this one is optimized for minimum quartus annual return, a new feature of SignalSolver. A quartus is one quarter of the data, 132 days in this instance, and the minimum return was 99% annualized for the most recent quartus Feb 3 to Aug 11th 2015.

You can see how quartus annualized returns fluctuate with buy/sell parameters from the lifetime graphs. Note that this algorithm took both long and short positions–when you were not long, you were short. I have no idea how or why this kind of algorithm works–you would think the buy and sell signals are so similar that it would give more random returns, but you can see from the surface plot that the results are positive for most of the parameter space.

The “user defined price” was found by averaging the previous day’s high, the previous day’s low and the current day’s open price. Buying and selling was done at the daily close or next day’s open.

Update 8/26/2015

Trades since the original backtest endpoint on 8/11/2015:

Update 12/30/2015

The above analyses have been corrected for an error in the short-side returns. Overall gain of the original erroneous post was \$104,971

Here are the updates for 12/30/15. The algorithm peaked around Oct 13th and now appears to be failing:

Update 10/20/2016

A bit of an improvement:

TNA Trading Strategy Update Oct 2016 Equity