Preliminary results of multi-algorithm testing

The portfolio and methodology for this multi algorithm study are described in my previous post. Here are links to the results spreadsheets:

EMA/Figure of merit optimization periods: Sept 2014-15  Jan 2014-15 June 2015-16

PB/Return optimization period:  Jun 2015-16

On these spreadsheets you can use the autofilter to include and exclude stocks of interest. For any stock you can click on the + sign on the left to see detailed results for each of the 8 algorithms used in the analysis. If you change the "Algs Included" cell, it will change the number of algorithms included in the averaging and summarized below the portfolio. For example, if you set it to 5 the top 5 algorithms for each stock in the portfolio (that is in the autofiltered list of stocks) will be averaged. If you set it to one, you will only include the top (best performing) algorithm. These calculations are also done automatically in the data tables below, and displayed on the graphs.

A-Z Portfolio June 2015-16 Optimizations

The top graph in each set shows the average return on $10K on running from one to 8 algorithms for a 10day, 25 day, 50 day or 90 day period on the out-of-sample data immediately following the 250 days of data the optimization were run on. So, for example, for the stock A (Agilent), the algorithms were found by using the data from June 15 2015 to June 9th 2016. The 10 day result was on the data from June 13th 2016 to June 24th 2016. The 25 day result was on data June 13th through July 18th 2016, and so on. These are trading days, not calendar days.

The buy-hold performance of this portfolio is shown below:

So if you had correctly guessed to go short for the 10 day period and long after that, you would have made more profit. On the other hand, if you had guessed wrongly, you would have been better off following the algorithms.

 

Percentage Band/Return optimization vs. EMA Band/Figure of Merit optimization

A three algorithm average for the PB/Return optimization for June 2015-16 gave the best Annualized Return (AR) results overall, much better than just running one algorithm. But there seems to be little point in running them beyond 10 days--profits barely increased for the next 15 days and by 50 days had soured into a loss.

The EMA/FOM algorithms had better longevity and, on average, were not lossy until after 50 days. Peak profits were at 25 days and slightly better than those for PB/Return, but best AR was still at 10 days, the same as the PB/Return result but in most cases lower.

The Figure of Merit optimizations gave more consistent results than simply optimizing for return, especially when averaged over several algorithms.

A-Z Portfolio Jan 2015-16 Optimizations

Again we see a breakdown in returns for the 99 day runs, but the shorter time-frame runs are all positive. Buy-hold performance for the same periods are shown below:

Running a single algorithm on each stock in the portfolio realized most of the potential profit for the 10 and 25 day periods, but running for 50 or 99 days gave disappointing results. Averaging across many algorithms did not lead to improvements.

 

A-Z Portfolio Sept 2014-15 Optimizations

Clearly, the best results for this timeframe were achieved when more algorithms were averaged, however the results were tepid at best and did not exploit the potential of the data, which is hinted at in the buy-hold performance for the portfolio in same time-frame, shown below:

 

 

 

Multi-algorithm trading strategy testing- Portfolio and Methodology

"Two heads are better than one", the saying goes. But can averaging improve trading systems? Can a multi-algorithm technique improve profitability and lower risk? After compiling the "Summary of Strategy Performance", I was very curious to quantify if and for how long strategies such as those published here were profitable for. To do that, I used out-of-sample testing which is very easy to do with SignalSolver.

In out-of-sample testing, you first find a strategy which worked for a specific time period. You then see how that strategy performed on data outside that specific time period. You can either go forward or backwards in time, but in these tests I go forward because that's pretty much how you would do it in reality. By multi-algorithm I mean running several algorithms simultaneously on the same stock. We look at running from 1 to 8 algorithms on each stock in a portfolio. Every stock has a different set of algorithms, found by optimization.

To get a handle on average success rates, I needed to analyze a reasonable number of stocks. Accordingly, I will report on 30 stocks with 128 out-of-sample results on each, a total of 960 algorithms each run for 4 time periods.

The A-Z Portfolio

I tried to create a portfolio which didn't look like I had cherry-picked stocks to put a rosy face on things. Accordingly, I used stock symbols A though Z which comprises 24 stocks (because symbols J and U are not assigned at this time). This I will call the "A-Z" portfolio. Other stocks were tested: AMZN, DUST, FAS, FB, NUGT, SPY and UWTI. I will report on these later, but the results are also in the spreadsheets if you are interested ahead of time. 

Methodology

I have been careful to ensure that anyone with a recent copy of SignalSolver can reproduce the results I report. You just need to make sure the Settings are as per the Settings tab on the results spreadsheets. Also, you will need to adjust the Web Query end date to get the exact OHLC data I used, (or increase the Offset to move the data back in time).

First, load the daily OHLC data for the stock into SignalSolver by typing the symbol and making sure data interval was set at "Daily". Set the "Days To Analyze" to 250 days which is almost a year of data and the Offset set to 100 days leading to an exclusion of the most recent 100 days. For example, for stock symbol "A" the included data was 6/15/15 to 6/9/16, the excluded data 6/10/16 to 10/31/16. Other exclusion days tested were 200 and 300 days. The buy and sell scan ranges were set to -20% to +20%, and the investment style set to "Long and Short", starting in the "Out" state. I'll talk about optimizations later.

Clear all the algorithms on the Algorithm tab (select the rows and press delete) so SignalSolver won't scan through any pre-configured strategies. Set the Seek/Scan level to "Seek level 2". and press Solve.  This will search 1000 algorithms using about 400,000 backtests. The seek doesn't search the same 1000 algorithms each time, but follows logic as to what may constitute a good strategy. It would take about 5 minutes to finish on my machine after which the top 100 algorithms will appear on the Algorithms tab, with the top 8 being transferred to the Strategy and Report Tabs.

For this study, I looked only at the top 8 strategies found. Here is an example Seek result:

Example of a seek result

Example of a seek result

Notice that the strategies are ordered with the highest Figure of Merit first, as this was a FOM optimization (see below).

Next step is to see how each of these 8 strategies played out moving forward in time. I started out looking at the next 25, 50 days and 99 days, but it quickly became clear that I'd have to also look at a shorter time interval, so I opted for 10 days.

Start State of out-of-sample testing

Start State is the start state of the backtest and is especially important for the short time results as these may not trade much, if at all. I set the Start State of the out-of-sample test to Out, unless a clear Start State was indicated by the traffic lights on the 250/100 seek result. For example, in the above illustration, 5 algorithms are green (bullish) and 3 are red (bearish). Only if the vote was 6/2, 7/1 or 8/0 would I set the start state to that indicated by the traffic lights, Long for green and Short for red, otherwise use Out.

Reading of the out-of-sample result

After setting the Start State, I would read off the out-of-sample results by setting the "Days to Analyze" and the "Offset". Here is an example, same stock, same algorithms:

Out-of-sample result

Out-of-sample result

Notice the data analyzed starts from 6/16/16. I had to avoid using 6/15/16 data because it is used in the traffic light calculation, that's why I set the offset to 89 and not 90 days. Its vital that none of the backtested data is used in the out-of-sample analysis. After 10/89 I would set the Days To Analyze/Offset to 25/74, then 50/49 then 99/0. Each time I would cut and paste the top 8 Strategies and the Annual Returns into the results spreadsheet.

Optimizations and bands

I followed this procedure for two different Solve optimizations, one using a simple Return optimization on the Percentage Band (PB) (like most of the algorithms on this site), the other a Figure of Merit optimization using a 50 period (H+L)/2 Exponential Moving Average band. Why use an EMA band? It seemed to give better results than the PB band, but I haven't confirmed this properly yet.

Bands are explained here. The FOM optimization used these weights-

FOM Weights

With these FOM settings we are essentially looking for algorithms with good 250 day return, low standard deviation of returns for each quartus (62 days in this case), a good value for the minimum of all four quartus returns, and good returns for the most recent quartus. By setting up this kind of optimization we hope to avoid algorithms which performed very well for only short period. For more information on FOM, see here.

There is a secondary effect to running multiple algorithms on the same stock; a change in efficiency. All the tests were done using investment style "Long and Short", where you are essentially tying up the capital 100% of the time. If you run 2 algorithms  on the same stock, however, there may be times when algorithm A puts you long while algorithm B puts you short, effectively taking you (mostly) out of the market. Since, as I hope to show, profits on each algorithm average out to be similar, you have essentially increased the effective annual return by reducing the time your capital is in the market. We don't take this effect into account in the Annualized Returns reported here, we assume all capital is in the market for 100% of the time. But I believe this effect is real--please correct me if I am wrong about this.

In the next post, we shall look at the results.

 

IBB Trading Strategy Details and Performance

IBB Trading System (Weekly)

This weekly trading strategy for IBB had 3 times better annualized returns than IBB and more than 10 times better return over the 10.1 years. For each 2.5 year period within the 10 years,  the annualized return was between 30 and 50%.

IBB Trading Strategy Details and Performance

IBB Trading Strategy Details and Performance

IBB Weekly Trading Strategy: Equity Curve

IBB Weekly Trading Strategy: Equity Curve

FEYE Trading System: Performance

FEYE Trading System

A trading system which worked for FEYE (FireEye). This is based on daily data, so traded at most once per day.

FEYE Trading System: Performance

FEYE Trading System: Performance

FEYE Trading System: Equity curve, signals and positions

FEYE Trading System: Equity curve, signals and positions

FEYE Trading System: Parameter surface

FEYE Trading System: Parameter surface

FEYE Trading System: Parameter scan

FEYE Trading System: Parameter scan

FEYE Trading System: Trades List

As always, future performance is not guaranteed.

AMZN 20,000,000% total return

Over 500 times better return than buy-hold

In the time frame May 16th 1997 to Aug 1st 2016, this strategy gave a return of 88.9% compounded which amounts to over $219,000 for every dollar invested, that's 518 times better than the performance of buy-hold which only gave $423.

This screenshot shows the strategy description and performance along with messages warning if something looks odd about the data. The data was checked and all the price jumps turn out to be real with the prices agreeing with prices from other sources.

amzn-m-table

The strategy used Long and Short investment style, so was always in the market long or short. The short side trades didn't do nearly as well as the long side trades, but well enough to boost the overall performance by a factor of almost 10. Also, the short trades had an efficiency of 69%, so for the short time that the strategy was short it did quite well.

Trading was at most once per month, trading a total of 67 round trips over the lifetime. 43 of these trades were good

This is a straightforward type of Percentage Band algorithm. Buy signals occurred in months when the stock price failed to rise, at any time, 5.6%  above the high of the previous month. This happened 143 times out of the 243 months of data. The buy was at the following month's open, but it only happened 34 times because the position was already long in most cases.

Sell signals occurred whenever the stock price fell 5.56% below the previous month's low at any time in the month. Usually, there was a buy signal the same month as a sell signal; there were 92 buy signal only months, 3 sell signal only months and 53 dual signal months. For this type of algorithm, if you are long you have to sell if a sell signal occurs, and you have to buy if a buy signal occurs when you are short. This is all explained in this video.

Here is the equity curve for the strategy, along with the signals and positions mapped out.

amzn-m-equity-curve

It looks as though the equity curve is still rising rapidly, however a closer look at the returns over different time periods (below) shows that most of the strategy gains occurred in the 1997 to 2002 time period. In fact, in the most recent quartus  (one fourth of the data, 2011 -16) the strategy did not quite keep up with buy/hold (31% vs 35.6% annualized).

amzn-m-longevity-scan

amzn-m-longevity

Looking at how changes in the parameters affect return, we see a broad area over which the buy parameters worked with a slightly narrower range for the sell parameter.

amzn-m-return-scan

The parameter surface shows a nice large structure, showing that sensitivity to parameter changes was fairly low, but remember most of the gains were focused in the 97-02 timeframe.

amzn-m-return-surface-scan

Drawdown for the strategy was much better than for buy-hold (46% vs 93%) and not overly sensitive to parameter change:

amzn-m-drawdown-scan

You can take a look at the buy-sell points on the trades list. As always, please be aware that there is no guarantee that this strategy will work in the future.

Recent performance

For the most recent 2.9 years, here is the performance table.

amzn-m-recent-table

There are not enough trades to make any firm conclusions about where the optimum parameters are now, but there is evidence to suggest that the buy point has pulled in somewhat. I suggest you download SignalSolver and optimize for different timeframes to get a feel for where the parameters might have moved to.

Berkshire Hathaway trading strategy

This is a Berkshire Hathaway trading strategy which would have given almost ten times the return performance of buy/hold over the last 10 years with half the drawdown. The strategy is detailed in the table below, it was straightforward, with 123 trades over the 10 year backtest period. All trading was done at the weekly close of business. This was a strategy where there was usually a buy and a sell signal every week (398 out of 528 weeks), but selling was initiated by the presence of a sell signal and an absence of a buy signal. There was a strong buy bias, appropriate for the underlying positive trend of the stock.

Equity curve, signals and positions are shown in the growth chart below. Notice the preponderance of white signals which indicate dual signal days.

If the sell point was set at zero percent, the algorithm gave positive results for all buy points greater than 0.25%. Below we show how return varies with buy and sell parameters.

The returns for every combination of buy and sell parameter are shown in the surface plot below.

Looking at the minimum annualized returns for each of the four 132 week periods, you can see that the algorithm was much better behaved than the underlying stock, worst case was 19.26% which happened in the most recent quartus. The long side of the algorithm showed a min return of 15% with a stddev of only 3.37%, which is quite tight.

As always, this is not a recommendation to trade using this algorithm, just an interesting backtest result. For a list of trades, see here: BRKA.W Trades.

Please note, the above result was corrected 12/28/2015 to address a bug fix in the short side calculations.


Update Oct 21st 2016

 

BRK-A Trading Strategy (Weekly) Update 10/21/16

BRK-A Trading Strategy (Weekly) Update 10/21/16

GOOGL Trading Strategy (Weekly)

Here is a Google Inc (GOOGL) trading strategy with once a week intervention which would have performed significantly better than buy-hold over the last 10 years. Annualized return was 31.5% vs. 16.5% (returning $149K for 10K outlay vs $36.7K, compounded), drawdown was 40.2% vs. 62.4%, so reward/risk was better.

The buy and cover signal (see table below) was present every week where the price dropped 0.03% below the last price the stock was sold at which happened 174 times over the course of the 528 weeks of the analysis, and usually happened the week following a sell/short.

The sell and short signal happened every week the stock price rose 8.13% or more above the open price of the current week. This happened only 29 times, so there is a strong buy-side bias to this strategy. All trading would have been done at the open of the week following the signal.

The equity curve for this GOOGL trading strategy shows that the stock was held short (the red bands in the background) for small periods, typically a week.

The scan below shows how the annualized return changed, had the buy and sell parameters changed. At 2.35% buy point, the algorithm gets stuck, resulting in a loss. This is characteristic of trading strategies which reference buy or sell prices. At 1.5% sell point and below, the algorithm made a loss, but returns for all buy points above that were positive, for the 0.03% buy point.

One nice characteristic this algorithm had was consistent returns for each of the 132 week  periods in the backtest. You can see from the table below that the annualized return was between 28% and 34.8% for every period. Compare that with buy-hold which ranged from -4.6% to 26%

You can also see this characteristic on the scans for each quartus:

You can download the list of trades in .xlsx format: GOOGL.W Trades.

As of Sept 16th 2015, the algorithm is long, awaiting a sell signal if the price hits 708.933. Last sell price was 654.34.

Please note, while this is an interesting backtest result, it is not a suggestion to trade this way. As always I don’t know how this strategy will fare in the future, but will track it from time to time.

Andrew

This post was edited 12/28/15 to correct an error in the short-side returns.


Update Oct 21st 2016

This strategy has pretty much followed buy-hold long:

GOOGL Trading Strategy Update Oct 21 2016

GOOGL Trading Strategy Update Oct 21 2016

DIS Trading Strategy

Here is a Walt Disney Company (DIS) trading strategy with daily maintenance which had quite nice characteristics; 59% annualized return over the last 2 years.

DIS trading strategy, equity curves, signals and positions.

The performance was better than long buy-hold with lower drawdown and about three times the reward-risk. Signal reinforcement was good, and not many dual signal days.

DIS trading strategy (daily), and performance characteristics over the last 2 years.

Below I show the return for the DIS trading strategy for each 6 month period going back 2 years. Comparing with long buy-hold you can see better consistency (higher minimum and lower StdDev).

DIS.D Longevity Corr1

Please note that this is simply a measurement, not an opinion or financial advice.

This post was corrected 1/8/2016 to correct a miscalculation in the short-side returns.


Update Oct 21st 2016

Strategy peaked 11/23/15, shortly after the stock price peaked.

DIS Trading Strategy (Daily) Update Oct 21st 2016

DIS Trading Strategy (Daily) Update Oct 21st 2016