$4 Million Early Retirement? Not Easy, But Here’s What You’ll Enjoy

When do you plan to retire (or, as I prefer, reach work-optional status)?

Are you expecting to retire early?

It isn’t easy and despite the volume of articles talking about people planning to retire in their 50s, 40s, or even 30s, when you survey people about their retirement thoughts, a different picture emerges.

According to a Natixis survey, here’s when different generations of Americans who participate in employer retirement plans said they expect to retire, and how much they think they’ll need.

• Millennials (born between 1981 and 1996 so currently between 27 and 42 years old) plan to retire at 60 with $891k.
• Generation X workers (ages 43–58) plan to retire at 65, hoping to amass $1.2 million.
• Baby Boomers (ages 59-75) plan to work, on average, until age 70 and think they’ll need $1.1 million.

Notice a pattern?

As people get older, the age they expect to retire gets pushed out further, about 5 years per generation.

Some of this is likely due to sampling bias – those who already retired probably aren’t included in the survey – but part of it is how hard it is for Americans to invest enough to retire while dealing with the ever-increasing cost of living.

Shining Examples of Super-Early Retirement

One of the most famous examples of early retirement, Billy and Akaisha Kaderli, retired at age 38 with $500k. However, their example bears scrutiny.

• Their $500k as of January 14, 1991, would equal over $1.14 million In today’s dollars. 
• Their timing was incredibly fortunate. From the time they retired, the market soared with no significant interruption for nearly 10 years, peaking at over 5.7 times its January 1991 value (in nominal dollars). Further, despite a sickening 44.7 percent drop over 25 months when the dot-com bubble burst, the market never dropped below triple its value when they retired (again, in nominal dollars). Even with the advantage of hindsight, you’d be hard-pressed to find a better time to call it a career.
• Plus, though officially retired, the Kaderlis continue blogging, writing, etc. to this day, supplementing their investment income. 
• They also reduced their cost of living dramatically to ensure their portfolio lasted decades.

Then, there’s Peter Adeney, a.k.a. Mr. Money Mustache, who retired with his wife in 2005 at age 30, having amassed a $600k portfolio.

Here too, the story isn’t so simple.

• In today’s dollars, their portfolio was worth just under $970k.
• They also owned their $200k home ($323k in today’s dollars) free and clear, so their net worth was over $1.2 million in today’s dollars.
• Mr. Adeney wasn’t quite as fortunate as the Kaderlis in terms of the market. It went up just under 35 percent before peaking and, over 16 months, dropping to 34 percent below its value when he retired. However, once he weathered that crash, the market took under nine months to recover to its 2005 baseline and proceeded to grow almost uninterrupted until peaking at 5.5 times its 2005 value in December 2021. It’s currently a bit lower, at 5-fold its 2005 value (all values in nominal dollars).
• Adeney has been blogging since around 2011. He also started a small house-building company (later transitioning to smaller projects). His wife didn’t retire at the same time, so in the first year of his retirement, between her income and his business profits, they brought in about $110k ($171k in today’s dollars). When Adeney says he retired, it really means they were work-optional and kept earning.
• They also kept their living expenses (no mortgage, remember?) to $25k a year ($39k in today’s dollars).

How About Retiring with $4 Million?

Both the Kaderlis and Adeneys earned high salaries and lived uber-frugally for a decade or more to reach over $1.1 million in today’s dollars before quitting their jobs while in their 30s.

Kudos, but that’s not enough for a luxurious retirement. For that, you’d need much more.

How about $4 million?

That would obviously take much more than $1 million. More time, more luck, and or high(er) investing rates.

Here’s what it would take, with certain assumptions, to reach $4 million in today’s dollars in 33 years.

That’s, e.g., from age 22 to age 55.

• Assuming you invest 10 percent of your gross income annually, you’d need to earn about $590k, putting you solidly in the top percentile (people making $407.5k or more).
• Increase your investing rate to 20 percent, and you could make it on $285k a year, in the 97^th percentile.
• How about a 30 percent investing rate? You’d get by on $190k, or 95^th percentile.
• Can you hit 40 percent investing? An income of $140k would suffice, or 90^th percentile.
• Can you even imagine investing half your gross income? With $110k, or 84^th percentile, that would get you to the target.

Don’t want to work for 33 years?

Here’s how much you’d need to invest annually, to reach $4 million (inflation-adjusted) assuming a 7 percent annual return after inflation.

The annual investment needed grows as you reduce the number of years you’re willing to work for it. 

• At 30 years, it’s $39.6k invested annually (and don’t forget, you’d need to increase the amount each year to counter inflation).

• For 25 years, you’d need to set aside $59.1k.
• Twenty years? $91.2k!
• Fifteen years requires $148.8k (inflation-adjusted) annually.
• A single decade? That requires $270.6k! Annually! Increasing each year to counter inflation!

But as John Stoj, founder of flat-fee RIA Verbatim Financial, points out, “If you want to retire, ‘early,’ that’s great, but no matter when you want to retire, you give yourself the most options by saving as much as possible while you can. I say, ‘when you can,’ because there will almost always be times in our lives when high savings rates, and maybe any saving at all, simply won’t be possible.”

What Makes Retiring Early on $4 Million Hard

To reach this point, you’d have needed to be hyper-focused, disciplined, and frugal for years, if not decades, sacrificing much of your day-to-day enjoyment of experiences and goodies to help your future self reach a well-deserved cushy retirement.

That’s the first problem, and don’t underestimate it.

After so much time denying yourself so much, don’t expect to be able to start spending freely and enjoying a rich life with the flip of a mental switch.

On the financial side, you’ll have to wait many years before Medicare kicks in, so you’ll need to buy private health insurance. And since you’re living large, don’t expect to fund it on the cheap. Think $1k a month for an ACA Bronze high-deductible plan.

Social Security won’t be available for a while either, even if you claim early, at age 62, taking a 30 percent “haircut” for life, while your spouse’s benefits are cut by as much as 35 percent. On top of this, if you contributed to Social Security for fewer than 35 years, your benefit takes a hit because the Social Security Administration uses your 35 highest years of earnings, filling any missing years with zeros. 

In most cases, withdrawing funds from (non-Roth) IRAs and 401(k) plans before age 59.5 results in a 10 percent penalty on top of getting taxed. Plus, withdrawals at any age get taxed at the highest rates – those for regular income.

One useful way to avoid this is to draw essentially equal amounts each year until you reach age 59.5 (or five years, whichever is shorter).

Next, since your nest egg has to provide income for many more years, your safe draw percentage is lower. According to Morningstar, if you want a 90 percent likelihood of not running out of money over a 30-year retirement, your highest safe initial draw is 3.8 percent of a 50/50 stock/bond portfolio. Extend that to 35 years and you’re down to 3.4 percent. Want a 40-year retirement, don’t draw more than 3.2 percent.

The Advantages of Retiring Early on $4 Million

It’s not too difficult to imagine these.

Retiring early means you have many more years to enjoy whatever activities light you up, without having to keep running on the hamster wheel with everyone else.

Plus, the years you added to your leisure are the healthiest ones you’ll have from this point forward, so you can do so much more than you’ll be willing and able to do in your 70s and 80s (let alone your 90s).

Retiring at a younger age means you have more time for activities that keep you healthier. You’ll have far less stress in your life, so you’ll likely stay healthier longer. 

This also reduces your healthcare costs.

Amar Shah, CFA, CFP®, Founder & CIO, Client First Capital, shares, “We often see successful business owners and corporate executives having a hard time stepping away from their roles. But there’s nothing like a health scare to realize how finite life can be. When a friend or family member experiences health-related issues, that often causes people to reassess their priorities in life and consider retiring early.”

This Is What Retiring Early on $4 Million Looks Like

According to DQYDJ, a net worth of $4 million puts you squarely in the 92^nd percentile of wealth (assuming you include your home equity, otherwise, it’s the 93^rd percentile).

At that level, the average fraction of net worth tied in home equity is 21.2 percent.

That means you’d have about $3.15 million invested.

Since we’re talking early retirement, say at age 55, your life expectancy is at least 24 years. However, that means you have at least a 50 percent chance of living longer (even more so with the high standard of living your wealth affords you). To be on the safe side, we’ll assume a 40-year retirement.

Using a variant of the 4-percent rule, modified for 40 years, you draw 3.2 percent of your portfolio in Year 1 of your retirement and increase the dollar amounts annually to counter inflation.

That’s about $100.8k a year. And since this is retirement income, we’ll assume it’s more or less equivalent to earning 25 percent more as a worker, or $126k. That puts you in the 88^th percentile of income.

The next question is, how will this income be taxed?

• Withdrawals from traditional retirement accounts are taxed as regular income.
• Withdrawals from your taxable portfolio get taxed using lower rates for capital gains and dividends.
• Withdrawals from Roth accounts are tax-free.

For the following, let’s assume your money is divided 50 percent in traditional IRAs/401(k) plans, 15 percent in Roth accounts, and 35 percent in taxable accounts.

There are various ways to optimize the order in which you draw down different account types. Those usually tend to draw first from taxable accounts, then tax-deferred traditional accounts, and then tax-free Roth accounts.

For simplicity here, we’ll assume you draw pro-rated, i.e., 50 percent from tax-deferred, 15 percent from Roth, and 35 percent from taxable.

For your $100.8k draw, you’d thus have to pay about 20 percent regular income taxes on $50.4k and 15 percent capital gains tax on $35.3k. Adding, say, 8 percent state tax, you’d have $78.6k left to live on.

Once you reach age 62, Social Security would chip in up to $46k more annually if you claim early. Claim at age 67 and that maximum increases to $65k. Delay to age 70 and that goes to $82k.

However, you must account for having contributed to Social Security for fewer than 35 years.

If you contributed for 33 years, your maximum benefit would be about $43k, $61k, and $77k, respectively for the three claiming ages.

If you contributed for fewer years, your benefits would shrink more dramatically.

Can’t I Draw More Than 3.2 Percent?

You can draw as high a percentage of your portfolio as you want. However, if you draw more than 3.2 percent and adjust each year for inflation, your risk of running out increases to more than 10 percent.

To avoid this, you can use a dynamic withdrawal strategy such as the Guardrails Approach if you’re willing to trim spending when the market drops. 

For a 30-year retirement using a 50/50 allocation, this increases your safe initial draw from 3.8 percent to 4.8 percent. If you’re willing to take on more risk by increasing your stock allocation to, say, 70 percent, your initial draw can grow to 5.3 percent.

Since Morningstar didn’t report how these draw percentages decrease for a 35- or 40-year retirement, let’s simply scale using the impact on static draws. This gives us a 4.0 percent initial draw with a 50/50 allocation and 4.5 percent using 70/30.

You can also allocate more of your stock position to small-cap stocks and to international equities, which will, on average, increase your return (and your risk), to have a higher likely average return that can afford a higher initial draw.

At 4.5 percent, you could draw as much as $142k in Year 1, equivalent to about $177k working income (94^th percentile of income). 

After taxes, this would leave you about $109k.

The Bottom Line

Reaching $4 million is no mean feat. It takes dedication, discipline, planning, and sacrifice over years and decades.

Doing so in time to retire early is that much harder.

However, if you manage to accomplish this feat, you’re looking at a long life of (relatively) stress-free, luxurious, high net worth lifestyle than few ever achieve.

Disclaimer: This article is intended for informational purposes only, and should not be considered financial advice. You should consult a financial professional before making any major financial decisions.

Find a Financial Advisor

Do you have questions about your financial future? Find a financial advisor who can help you enjoy life with less money stress by visiting Wealthtender’s free advisor directory.

Whether you’re looking for a specialist advisor or prefer to find a financial advisor near you, you deserve to work with a professional who understands your unique circumstances.

Have a question to ask a financial advisor? Submit your question and it may be answered by a Wealthtender community financial advisor in an upcoming article.

This article originally appeared on Wealthtender. To make Wealthtender free for our readers, we earn money from advertisers, including financial professionals and firms that pay to be featured. This creates a natural conflict of interest when we favor their promotion over others. Wealthtender is not a client of these financial services providers.