For a growing number of Americans, “retirement” simply means changing where or how you work.

More than half of Americans age 40 and up expect to continue doing paid work after retirement, according to TD Ameritrade. This includes 86% of people in their 50s, while a remarkable 92% of people in their 40s expect to keep working after dialing back their main career.

Some people embrace working in retirement as a positive choice, since they enjoy the mental challenge or socialization a job provides. Others even undertake a later-life “encore career” and launch a new professional chapter entirely.

Of course, money is a big consideration as well. For some people, working in retirement is the only way to make ends meet. A new report from the Transamerica Center for Retirement Studies found that 84% of women who plan to work in retirement say their motivation is financial necessity. And if you have your heart set on realizing a lifelong dream to, say, open a distillery, you need to take a clear-eyed look at your expectations for turning a profit.

If you plan to work into your retirement years, financial advisors say there are a few factors to keep in mind.

Although we tend to assume that more money should always be the goal, a lucrative retirement gig could mean higher taxes and bigger medical insurance bills — and a potential haircut to your Social Security benefits. “People don’t look at how it impacts their other sources of income,” says Matt Nadeau, a wealth advisor at Piershale Financial Group near Chicago.

Hold off Drawing Social Security

Drawing Social Security earlier than full retirement age (that’s 67 years old for people born in or after 1960) reduces your monthly payment, so conventional wisdom says you should wait until full retirement age to claim if possible.

“With longer lifespans, it becomes more important to consider delaying Social Security,” says Christine Russell, senior manager of retirement and annuities for TD Ameritrade.

If you plan both to earn income and draw Social Security between the ages of 62 and when you reach full retirement age, take note: For that period of time, your Social Security benefits will drop by $1 for every $2 earned above $17,640, and $1 for every $3 earned above $46,920. These deductions stop after you reach full retirement age.