NeighborTrader back-tested the CiG trade before he ever talked to me about it last fall, and he's been combing through data ever since to find more trades he can run. I've been meaning to do the same with CS|MACO for quite some time, and I finally did this weekend. I learned some interesting things, and I found a few changes I want to make.
I grabbed daily historical prices for SPY from January 1993 through March 2011. I also grabbed the AAII sentiment data for that same period of time. I wrote myself a little Python script to collate the data together, and then plugged all of that into a spreadsheet that created signals just like my present-day trading spreadsheet. To this, I added some calculations to figure out the results of the trades, and compare them to simply buying SPY and holding it.
As designed and outlined in this post, CS|MACO underperformed SPY over the 18-year period from 1993-2011. Then I abstracted away all of the parameters so I could change them easily, and started playing around. Next I evaluated various time periods based on the sort of market they covered: I looked for bullish and bearish periods, triangular moves up and down, and sideways choppiness. I compared CS|MACO against SPY in bottom-to-bottom and top-to-top time periods, as well as a simple 5-year rolling time period throughout the 90s. Once I had a feel for how CS|MACO behaved in various market scenarios, I started changing the parameters, and learned some things.
The first thing I learned is that the 25/200 moving average crossover component of MACO is far too responsive, and tends to trade into choppy sideways markets, losing money on every reversal. To catch the really big trends, much bigger moving average periods, and more similarly sized periods, are far better: 200/300 seemed to be a good mix.
The next thing I learned is that the arbitrary 10% collar I have on the CS component is about right, but only for the buy signal. This outcome was fascinating, and I think it gives insight into individual investor psychology. If I'm right, it means that the CS buy signal (which is based on below-average levels of bullishness in the survey) is a leading indicator while the CS sell signal (which is based on above-average levels of bullishness in the survey) is a lagging indicator.
Bear in mind, this all just my viewpoint: I think we as humans tend to invest our emotions as well as our money, and we are very slow to accept that we are in a losing position and get out of it. On the other hand, we are much quicker to jump into a new position if we think there is opportunity there. The vast majority of us do not short-sell anything (my father thinks it's un-American and somehow Satanic), and so statistically, investors tend to become bullish faster, and become bearish much slower.
To handle this lopsided behavior, I changed things so that I could control the bullish/bearish thresholds independently. Then I tried turning one and then the other off by setting them so wide that the indicator could never reach them (+/- 100% certainly works). I discovered that turning CS off entirely made things worse: MACO, by itself, is not a winning strategy. Actually, let me be clear: it does have positive returns, but it does not beat SPY itself. Turning on only the buy (bearish investors) signal had the most positive effect.
So, how about the results? In rolling 5-year periods, CS|MACO was profitable in just about all of them - can't say that for SPY, not by a long shot. When it beat SPY, it beat it badly; when SPY beat it, it wasn't nearly as big a difference. The best part is that CS|MACO tended to diverge up from SPY in down markets, and pace it fairly well in up markets. It really only lost ground in prolonged sideways chop markets. And by prolonged I mean like longer than a year of nothing but sideways chop - that's pretty rare.
A big danger of back-testing is sample bias, also known as curve-fitting or false optimization. This is where you optimize your strategy against all the data you have, and assume that tomorrow will just like your data sample. In a perfect world, we would like to use a sample of, say, 1995-2000 to train our strategy, and then make sure it still works from 2000-2011 before committing real money to it. This is called split-sample testing. However, I feel that the behavior of the markets and the attitudes and psychology of the individual investors have changed somewhat over the last 18 years. For me to find a strategy that works well in the 90s, and expect it to continue working in 2012 and beyond, is naive. So I have to flirt with that sample bias problem, but I try to watch for it and be aware that it is always there without falling into it.
Below is a graph that compares SPY to CS|MACO for the whole 1993-2011 period. SPY is the red line, and CS|MACO is the blue line. Notice how when SPY suffers, CS|MACO profits. This makes it a very viable strategy for running alongside a standard retirement account holding index funds. And for me, that's just perfect.
(click for the original size)
No comments:
Post a Comment