About three or four months ago I was pleased to see that Morningstar Inc. was following me on Twitter. About two weeks ago I wrote a blog about the value of their rating system for mutual funds. That blog post pointed out that, in my opinion and based on studies that have been done, Morningstar’s rating system does not work. I said that they had taken on an impossible task and that nobody can reliably identify which mutual funds are likely to outperform the market, let alone match it, and that Index Funds are the best way to go.
This prompted Morningstar to send me an email pointing out what they felt were errors in my analysis and asked that I set the record straight. Morningstar is a first-class organization, their email was very professional, and their request to set the record straight is fair.
So, let’s set the record straight.
Below is Morningstar’s email to me in italics, followed by my responses in bold interweaved with each of their points. You will see is that I did make one error in my research but that my general conclusion about the effectiveness of their rating system still stands. Also, while looking into this subject a bit more, I found a new study that further reinforces my conclusions.
Dear Ted. We just read your latest post, “The Failure of the Morningstar Rating System” and request that you correct the inaccuracies in the following section:
“Even after stating that their star system was “overrated” Morningstar did nothing to try to solve or change their rating system for another 12 years. Finally, about 6 weeks ago, Morningstar announced they were replacing their existing star rating system with a “new and better one”.
Let me state the obvious here. They got it wrong for a long time, 20 years, (12 years by their own admission) and by now it should be very clear that almost nobody can reliably beat the market.”
First, we’re not replacing our star rating system, we’ve added qualitative analyst ratings as a complement to the quantitative star ratings. Below is a link with more information.
http://corporate.morningstar.com/us/asp/subject.aspx?xmlfile=174.xml&filter=PR4681
The article this links to says that they didn’t replace their system but added an additional, new rating system. They say, “The new Morningstar Analyst RatingTM for funds is a qualitative, forward-looking measure based on analyst research, whereas the well-known Morningstar RatingTM for funds, or the “star rating,” is a purely quantitative, backward-looking measure that rates historical performance…” So, while they did add a new system, it does not replace the old one but “supplements” it. My opinion is that adding a new “forward looking measure” to the old system really amounts to a new system. The purpose of the star-rating system has always been to predict future performance, and adding an entirely new “forward looking” element (the part we are interested in) constitutes a new system in my eyes.
Second, it is untrue that we made no changes to our star rating system in 20 years. The system was revamped in 2002 to better rank funds against their category peers and put more emphasis on downside risk. Below is a link with more information. http://corporate.morningstar.com/US/asp/subject.aspx?xmlfile=174.xml&filter=PR3990
Morningstar did indeed go on the record as having made a significant change to their rating system methodology in 2002 and I apologize for neglecting to dig this up and accurately report it.
Third, the quote attributed to us was taken out of context. We view the star ratings as one valuable measure among many, including analyst ratings and research reports, style box, and comprehensive data on fees, management, etc., that investors should use to better compare and select actively managed funds, index funds, and ETFs.
I re-posted the quote in question verbatim from a relevant website that I felt was an acceptable source: http://www.financial-choices-matter.com/compare-mutual-funds.htm. In an email response I let Morningstar know that I believed the quote has been out there for over a year and that I felt it reasonable to ask that they address the matter with the source of this information. I also asked Morningstar to provide me with the full context that originally surrounded that quote, so that I may read it for myself.
They have not provided that information.
While revisiting this subject, I came across an article on a new study that examined the performance of Morningstar’s five-star rated funds between 1999 and 2009. This article says, “Of the 248 stock funds with five-star ratings at the start of the period, just four still kept that rank after 10 years. And the 218 domestic stock funds with the rating typically lagged their category averages over the period — not just the benchmarks, but other mutual funds. The exceptions were 30 foreign large-cap funds, which had a 10-year annualized return of 1.44% compared to their category average of 1.32%… In other words, it’s not just that five-star funds don’t, on average, continue to lead their peers — they actually do worse.”
This study’s conclusions, like the others I mentioned in my original article, “might convince investors to kick their star-rating habit.” Simply put, I still believe that the task of predicting fund success is an impossible one and that most investors should stick to broad based no load index funds and nothing more.