46-year-old early retiree who had $380,000 a year in passive income heads back to work—here's why

For the first time since 2012, Sam Dogen is getting a day job.

That's the year that Dogen quit his job as an investment banker, having spent 13 years working, saving, investing and generally burning himself out.

At age 34, his portfolio and real estate investments were generating about $80,000 a year — enough for he and his wife to live on in perpetuity. So, he took his severance and left. His wife did the same in 2015.

"We've been a dual, no income household for a while," Dogen, the founder of Financial Samurai and the author of "Buy This, Not That," tells CNBC Make It.

Well, not W-2 income anyway. As the couple's plans changed, the $80,000 a year they thought would last them for life needed to be bumped up. In 2017, the pair welcomed their first child, followed by another in 2019.

Over the years, Dogen built his passive income streams to about $380,000 annually — $288,000 net of taxes. That was enough to cover the family budget while living in San Francisco — until now.

In a recent post on his website, Dogen, now 46, detailed his choice to sell a portion of his stock and bond holdings to buy a multimillion-dollar house in cash.

By swapping income-producing assets for a house, "I basically have a lot more dead money now," Dogen says. That means his passive income streams no longer cover his family budget — so back to work it is.

Giving up financial independence was always part of the plan

The headline on Dogen's post reads, "Blew Up My Passive Income, No Longer Financially Independent." That's true, although like all good headlines, it makes things sound sudden and exciting and new.

After 12 years of financial independence, Dogen knew giving it up for a while was a possibility. In fact, he sort of planned on it.

After his younger child was born, "I made a promise to be a stay-at-home dad for five years. Then they go to school full-time, and I would like to do something else, like consult or work," Dogen says. "My daughter is going to school full-time this September. So I said OK, I believe the best time to own the nicest home you can afford is when your kids are at home."

By buying the home in cash, Dogen sacrificed his financial independence — a state in which your investment and passive income covers your living expenses — for the time being.

Between his four rental properties, distributions from his portfolio and other forms of passive income, such as book royalties, Dogen estimates he'll bring in about $230,000 in nonworking income in 2024. That puts him about $113,000 short of his estimated expenses for the year at what he calls a "realistic and comfortable" lifestyle.

Dogen's plan going forward

In the short term, Dogen hopes to pick up a consulting job, which would see him work about 20 hours a week for a salary of about $145,000. That would cover this year's shortfall while still allowing him to spend ample time with his children before and after school.

Over the intermediate term, he hopes to build his passive income streams back to where he's once again financially independent. One avenue, he says, is eventually selling the house he and his family are vacating, but he doesn't currently view San Francisco as a seller's market.

"We're past the bottom of the San Francisco real estate downturn," he says. "We're gonna pick up over the next several years because of artificial intelligence and technology and everything. So I want to rent [the house] out for now."

He thinks he could collect a little more than $100,000 in annual rent, for a net profit of about $40,000 — money he can use to help build his investments back up. The same goes for an eventual sale.

In the meantime, Dogen says he's excited to put his efforts into something else now that he and his wife are scaling back their hours, as it were, being full-time parents.

"The more you invest in something, like being a stay-at-home parent, the more you have to fill this void of emptiness once they go to school full time," he says.

DON'T MISS: Want to be smarter and more successful with your money, work & life? Sign up for our new newsletter!

Want to land your dream job in 2024? Take CNBC's new online course How to Ace Your Job Interview to learn what hiring managers are really looking for, body language techniques, what to say and not to say and the best way to talk about pay. Get started today and save 50% with discount code EARLYBIRD. 

CHECK OUT: ‘Loud budgeting’ is mentally and financially ‘healthy,’ says expert—and it’s good etiquette too