Retiring Early Is Looking Easier. Here’s How to Do It With No Regrets.

Grimley is part of an exodus of older Americans from the labor force during the pandemic that has contributed to an overarching labor shortage. Baby boomers—the 70 million-plus cohort born from 1946 to 1964—have been in the midst of a retirement boom for more than a decade, but the pandemic has accelerated it, contributing to the labor shortage and complicating the inflation picture for the Federal Reserve, as well as requiring a redo of the retirement calculus for even those individuals who have hit their savings goals.

There were slightly more than three million excess retirements during the pandemic as of August beyond the demographic trend already under way, estimates St. Louis Fed economist Miguel Faria-e-Castro. Fewer people plan on working longer, as well, with only 32% of Americans expecting to stay on the job beyond 67 and just half beyond 62, according to a July survey by the New York Fed.

There was an 8% decline in the labor-force participation rate for those 65 and older from February 2020 to September 2021, when the rate sat at 19.2%. That contrasts with longer-term trends: The participation rate for older adults steadily rose over the past two decades to 20.8% in February 2020 from 12.9% in 2000 as they worked longer to shore up retirement security.

One reason for the wave of retirements: The economic downturn was brief and came amid booming stock and real estate markets. But only a small swath of the population is fortunate to be able to retire on its own terms.

Some people who lost their jobs or left them because of Covid-related issues, including caregiving, would rather still be working. And not everyone has a retirement savings portfolio or home equity to fall back on. More than half of households headed by someone 60 to 65 still hold substantial mortgages; a quarter of those households don’t own their homes, and roughly 31% of them have no savings in 401(k) or 403(b) plans, individual retirement accounts, or pensions, according to Teresa Ghilarducci, a professor of economics and policy analysis at the New School who also leads the Retirement Equity Lab.

Economic Reverberations

For the economy, the rush of older workers opting to retire earlier than planned could mean the labor market is healthier than it appears. Of the estimated five million workers on the sidelines, half of those may not come back because they have retired. Already, younger workers have been drawn back into the labor market as companies ramp up hiring, while participation rates among older workers haven’t changed much.

That has big implications for economic and monetary policy from the Fed, which remains committed to low interest rates as it aims to bring down unemployment, particularly among minorities. The takeaway: With a tighter labor market, inflationary pressures could emerge faster and not be as temporary as central bank officials believe.

Some of the shortages in the labor market are a result not just of Covid but also longer-term trends related to demographics. Relatively low tax rates on income, dividends, and capital gains have allowed older workers to accumulate assets faster and leave the workforce sooner, says TS Lombard chief U.S. economist Steven Blitz. And the cohort entering the workforce—Generation Z—is smaller than the millennials before them, creating a shortage of workers for entry-level jobs.

The trend of older workers staying in their jobs longer to bolster retirement savings had provided an offset to these other trends, helping the prepandemic labor market. But as more of these older workers retire, staffing shortages could continue, impeding the economic recovery in the near term and crimping productivity over the long run. “A growing labor force sparks economic growth and productivity gains,” says Richard Johnson, director of the Urban Institute’s program on retirement policy. “If the labor force stagnates, the economy stagnates.”

Retirement Realism

Advisors are worried that the bull market and reduced spending during the pandemic among those who retired early by choice may have skewed their perspective on retirement readiness. They recommend a dose of realism.

“There’s an exaggerated financial optimism happening right now,” says Sharon Oberlander, managing director and wealth management advisor at Merrill Lynch. “Bills that were $15,000 a month may be $6,000 to $8,000 during the pandemic, but people don’t think about all the things they will want to do when they can. It’s a bit of a fantasy [that] we don’t spend any money.”

While older Americans may not be going on long trips through Italy and spending weekends at the theater as they had anticipated, many advisors say their clients are instead incurring big-ticket expenses like buying a recreation vehicle, building a pool, or adding on to their homes to accommodate children or grandchildren.

For those who are retiring before qualifying for Medicare, there are also healthcare costs to consider. In modeling what could be needed, Mariner Wealth Advisors’ Jon Bresnen recommends adding $20,000 in expenses, assuming the highest-deductible options, every year until qualifying for Medicare.

David Blanchett, head of retirement research at QMA, worries that people may be basing their early-retirement decisions on unrealistic expectations for long-term market performance in light of the S&P 500 index’s 40% run-up since the beginning of 2020.

“Congrats, if you are near retirement and your portfolio is way up, but there’s also a growing likelihood the market is due for a correction in the near future,” Blanchett says. “Let’s say you retire early and you enjoy yourself by tapping the portfolio, and then have a severe market correction at 70. You are likely not readily employable and could still have another 25 to 30 years in retirement. It just creates more fragility.”

One shorthand version to see if a portfolio may be hardy enough for earlier retirement is to see how much has to be withdrawn to fund retirement needs. If someone needs only needs 2% to 4% a year from his or her portfolio, Blanchett says that’s probably OK, but more could mean that retirees will have to make adjustments to their spending over time or risk running out of money.

For Glenn Neugebauer, the market’s run gave him some comfort to retire about a year earlier than anticipated after the pandemic changed his work life dramatically. From traveling once a week for six months of the year, dining out, and visiting retailers like Macy’s and Nordstrom in New York as a distributor for a Canadian shoe company, he went to days of virtual meetings and sales calls on Skype.

It was a bad fit, and he and his wife, a preschool administrator, decided they had enough.

But Neugebauer still feels the sting from the 30% hit he took during the global financial crisis. “My portfolio keeps growing, but I’m a realist,” he says. “We have been lucky for the past 10 to 12 years. I’m moderately aggressive, but we will be rebalancing” the portfolio to become more moderate.

Too much risk after the market’s run-up is a common problem for many near-retirees, and Robert Gilliland, senior wealth advisor at Concenture Wealth Management, advises clients to think more about guarantees of income than return. “It doesn’t mean you give up on growth, but you have to make sure you get through tough times,” he says.

As a result, Gilliland segments portfolios into buckets, with the first focused on guaranteed income like Social Security, pensions, certificates of deposit, and short-term bonds to cover about five years of expenses. Funds for the next six to 10 years can be invested in a mix of 65% stocks and 35% bonds, diversified by size and style—large-cap growth and value, for example, as well as mid-cap and small-caps, emerging and developed markets, and bond ladders. The rest of the portfolio is invested more aggressively among diversified assets, including alternatives.

For those in the top tax brackets, investing in municipal bonds in a taxable account may make sense, especially if the bond allocation is designed to create cash flow, since the income will be tax-free, Gilliland says.

Tax Strategies

Thinking about taxes in retirement has always been important when analyzing which accounts to pull from to fund spending, but it could become even more crucial as Congress considers raising taxes on wealthy individuals, especially those making more than $1 million a year.

Part of that plan includes raising the top rate on capital gains to 25% from 20%, a reason that advisors are nudging clients who retired early to take the tax hit sooner rather than later, especially on long-term equity holdings that have surged and represent big gains. That could serve the dual purpose of trimming stock exposure that may have gotten too high during the bull market and building up cash for expenses in the first years of retirement. It can also help mute a tax hit down the road.

Another source of cash: exercising stock options that have vested to avoid a bigger tax bill if rates do rise, says Brandt Kuhn, a financial planner at Beacon Pointe Advisors.

When it comes to the popular Roth IRA conversion, views are mixed. Some advisors recommend early retirees take advantage of the opportunity to convert some of their 401(k) assets and pay taxes today, potentially at lower rates than in the future, especially if quitting their jobs puts them in one of the lower tax brackets.

It’s also used as a strategy for Medicare planning, with advisors like Kuhn noting that clients can do a Roth conversion about two years before qualifying for Medicare. They take the tax hit, which lowers their income in the year Medicare uses to determine Part B premiums.

But the conversion probably isn’t for those who have the bulk of their assets in taxable accounts already. For those who are tapping their portfolio several years earlier than anticipated, the required minimum distribution at age 72 may be enough to fund their lifestyle—rather than leaving them with excess if they hadn’t tapped their savings early—another reason to not bother with a conversion. With the tax backdrop changing and early retirees feeling out what life without a paycheck feels like, advisors recommend reassessing their plan frequently in the next year or two—and leaving enough flexibility to accommodate a change of mind.

While the pandemic was a tipping point for Grimley, the dentist in the Chicago suburbs, he hasn’t made any dramatic moves in his first months of retirement. No big purchases or trips. Instead, he is decluttering his home, spending more time on the golf course, and taking longer trips with his wife, who also retired, to see their grandchildren in Georgia. “Change isn’t easy. We have done some exploring, but right now we have to just let the dust settle,” Grimley says.

enjoy this complimentary download:

By submitting your contact information, you consent to be contacted regarding retirement income strategies that utilize investments and insurance products.