Buffett was recently on CNBC discussing investing for retirement. He said that most people should simply buy an index fund that tracks the S&P 500 “through thick and thin, and especially through thin.”  That’s it. Consistently buying shares in a S&P 500 index fund is pretty simple advice. Given that Social Security will likely be scaled back sometime in the future, it’s even more important to make sure you’re managing your private portfolio correctly.

My goal is to help people maximize their income in retirement by writing about different options that are available. It’s not just about investing either. You will have up to four different sources of income in retirement: 1. Social Security, 2. Pensions & Annuities, 3. Portfolio Income, 4. Employment Income. You’ll have to handle each type of income differently based on how it’s taxed and your unique situation. For example, most people pay very little tax on Social Security due to the nature of how this type of income is taxed.

retirement income sources

So back to Warren Buffett’s retirement advice. When he’s talking about people buying shares in a low-cost index fund, he’s referring to your private investments (outside of pension and Social Security). This would include both tax-deferred and regular taxable accounts. Private investment in assets leads to portfolio income, my favorite type of income. What Buffett’s really saying is that putting your money into a S&P 500 index fund in regular intervals is the best way to build up your retirement nest egg. He’s talking about the accumulation phase of investing. It’s not exactly holistic retirement advice but it makes a good sound clip and there’s a lot of truth to what Buffett is saying.

The Buffett 401(k)

Setting up your 401(k) in Warren Buffett manner would then involve directing 100% of your contributions to a low-cost index fund. For example, if your company’s 401(k) is through Vanguard, you’ll likely have the option to invest in VFIAX which is Vanguard’s S&P 500 Index Fund (Admiral shares) and has a ridiculously low cost. You would then direct all of your 401(k) contributions into this fund in good times and bad. And according to Buffett, you want to make sure you do it through the bad times.

“Sometimes it rains gold.”

This is another Buffett quote. He’s referring to the market tanking and investors selling off shares in company so low that it doesn’t make sense. This happens from time to time. It happened during the Great Recession, the Flash Crash, the government shutdown over the debt limit. There’s all sorts of scenarios where stocks will sell off huge. You want to make sure you’re buying during these times and not selling. Buffett wants you to be an investor with conviction, not bending to the whims of Mr. Market.

What Happens When I Need My Money?

Ok Mr. Buffett, you’ve said to invest in the S&P 500 through a low-cost index fund through thick and thin, but what happens when I want to take my money out? Do I cash out everything? Do I convert to bonds? Do I do nothing, sit on my sofa and watch Netflix until I can no longer think?

There are many key issues that come up when people withdraw their funds to use for retirement.

1) Sequence-of-returns risk – This is the fact that you’ll experience different rates of returns in different years of retirement. Those who retired in 2008 and had their money in equities know all too well what this type of risk can do. If the market takes a dive, would you still want to withdraw X percent or would you keep the money invested? Most advisors who are worth a damn will have a good plan if the market decreases substantially. That means that you may need to get income from other sources until the market comes back. This way you’re not selling at a low.

2) Tax Burden – The tax burden you face in retirement is impossible to calculate in your head. Our country has made the tax system this way*. They (big brother) uses the tax system to incentivise people into doing what they want. You’ll want to make sure you’re not paying more than you have to pay. Sometimes people pay more than they have to because they made a bad move. For example, maybe they took money from their traditional IRA which then bumped them over a threshold so that their Social Security was taxed at a higher rate. This happens all the time and people don’t even know it. It’s important to plan out your income so you don’t have to deal with these tax surprises.

*Here’s a solution to all this mess. Reform the system so that you pay a flat tax rate on the amount you earn minus what it cost to earn that income. 

Tax Burden = (Income – Cost of Earning Income) x 10%

This would get rid of all the silly things we do in calculating our tax liability. No more deductions for health accounts, mortgage interest, etc. Who thought it was a good idea for such a complex system? Even a fifth grader would be able to calculate their tax burden if you did it like this. And of course this will never happen but I thought I’d give it a go.

3) Low-interest rate risk – Knowing how many shares to sell of your S&P 500 index fund per year is important.You’re living in a low-interest rate world and it doesn’t seem to be normalizing. You may not be able to withdraw 4% on your money depending on your situation. Or you may be able to safely withdraw 6% a year. You may be able to sell shares every year to cover your living expenses while seeing your account balance increase. Isn’t investing great?!

Conclusion

Warren Buffet’s retirement advice is a good starting point but there are many things that will come up that you’ll need to prepare for. I’m not saying that Buffett is wrong in this; quite the opposite. Investing in low-cost S&P 500 index funds is a great idea. I am saying retirement is a bit more nuanced than what Buffett makes it out to be. Buffett knows that too. He’s just on TV and has to give an answer that acts as a folksy blanket of wisdom.

The best way to make sure you’re investing in a low cost way to is to run this fee analyzer to see if you’re over paying. I did it and since I’m investing through Vanguard in my 401(k), my fees are below average. Also, I’m investing in DFEOX through my 401(k) so I am following Buffett’s advice and didn’t even realize it. However, I do invest in individual stocks, bonds, and alternative investments outside of my 401(k).