For more than a decade, Paul Tudor Jones’s home in the Belle Haven district of Greenwich, Connecticut, hosted one of the most elaborate Christmas light shows on the East Coast. Each December, cars snaked through the secluded area, occupants craning necks to get a peek at light-sculpture angels, wise men, and stars as they burst onto the lawn of his Monticello-inspired mansion. Christmas music was even piped into the local FM radio station, including a rendition of “Oh Come All Ye Faithful” by Jones’s daughter Caroline, an aspiring country singer-songwriter.
Given the extravagant holiday tradition, it might be hard to argue that Greenwich is not where Jones’s heart lies — often a key element in determining whether one is domiciled in a state, and therefore must pay its income taxes. So, much to the chagrin of his Christmas display spectators, after Jones bought a $71 million home in Palm Beach, Florida, in 2015 and was seeking nonresidency tax status in Connecticut, he canceled the light show.
To be sure, Jones — whose net worth is estimated by Forbes at $4.5 billion — still owns his Belle Haven manse, and Tudor Investment Corp., the hedge fund firm he runs, is still headquartered in Connecticut. But Jones, who was one of the wealthiest taxpayers in Connecticut for years, no longer pays any income taxes to that state.
Whether or not Jones is required to pay Connecticut income taxes is not, as many people assume, just a question of being in the state for fewer than 183 days. To prove that requires a taxpayer to keep copious records that serve to meet a statutory residency requirement. But nonresidents can also be required to prove that their so-called domicile has changed as well — and that’s where it gets messy.
Registering to vote, opening local bank accounts, and getting a driver’s license in the new state are important, but not the end-all people think they are, say tax lawyers. Possibly even more important is proving the new residence is your true home by showing an emotional attachment and moving all your valuables and family mementos there.
Being the CEO of a business in the state one has left, as is the case for Jones, is likely to raise eyebrows, if not an audit. “Having a business connection to your prior sate is a significant impediment to proving a change of domicile,” says Joseph Lipari, a tax attorney with Roberts & Holland.
Jones is hardly alone in claiming a change in domicile — or at least wanting to do so. Changes in the tax code last year, which limit the amount of state income and property taxes individuals can deduct from their federal income taxes, have made moving to lower-taxed locales a topic of consideration for many of the metropolitan area’s wealthy.
Florida — which has no state income tax and no inheritance tax — has become a lure for several billionaire financiers who in recent years have managed to keep luxurious homes or significant business operations (or both) in the high-tax states of Connecticut, New York, and New Jersey while avoiding paying these states’ income taxes.
In addition to Jones, these men include Edward Lampert, Barry Sternlicht, and Thomas Peterffy — all current or former residents of Greenwich — and Leon Cooperman and David Tepper of New Jersey. All have changed their domicile to Florida.
Perhaps the most famous case is that of hedge fund manager Julian Robertson, a New York state resident who also has a Manhattan apartment. In 2010, Robertson won a years-long battle with the city of New York over $26,702,341 plus interest the city claimed he owed, based on the statutory requirement that led to extraordinary efforts he made to limit his days spent in the city to meet the 183-day test.
“It is a growing problem,” says Kevin Sullivan, speaking on the final day of his job as Connecticut’s commissioner of revenue services last month. “If one of the very top should go . . . that’s a noticeable blip on the radar.”
If Connecticut is feeling the loss, it should come as no surprise. Hedge funds, led by Tudor, moved to Greenwich in droves in the early 1990s to escape New York City’s high taxes. But in recent years Connecticut’s marginal tax rate on the highest earners jumped to 6.99 percent, up from 4.5 percent a few years ago. Now it’s close to New York state’s top income tax rate of 8.82 percent — though New York City residents pay an additional 3.88 percent. New Jersey’s top marginal income tax rate has also been rising and is now 8.97 percent.
The top 1 percent pay an estimated third or more of total state income taxes in these states — or at least they used to. New Jersey acknowledged it faced a budget crunch when Tepper, its wealthiest resident, who is worth $11 billion, quit paying state income taxes in 2016, reportedly costing the state hundreds of millions of dollars in income taxes. This year Tepper ranked second on Institutional Investor’s Rich List of the top hedge fund earners, earning $1.5 billion in 2017, on top of $700 million in 2016. That’s $2.2 billion that won’t get taxed by New Jersey, despite the fact that Appaloosa Management, his hedge fund firm, maintains its office in Short Hills, New Jersey. (It also opened one in Miami Beach, Florida, where Tepper lives most of the year. He declined to comment.)
As for Connecticut, Jones, Sternlicht, and Peterffy all moved their domiciles to Florida in recent years. By 2016, when they filed Connecticut nonresident tax returns, their departures were creating a problem for the state’s coffers. Alluding to their exits, Sullivan says, “In December of 2016 we woke up to a $450 million revenue shortfall because of people in that group making certain decisions.”
“New York state often will not hesitate to challenge a tax return reporting a high-net-worth individual’s change of domicile where certain factors are present — for example, where the individual continues to maintain a residence in New York or continues to spend a significant amount of time in New York,” says Craig Fields, co-chair of Morrison & Foerster’s tax department.
That said, last year Fields successfully argued that a former CFO of Colgate-Palmolive Co. was no longer a New York tax resident despite the fact that he had been in the city more than half of the year in question. The main reason he avoided paying? The executive, Stephen Patrick, married his high school sweetheart, gave her a solid-gold, heart-shaped necklace (a replica of a gold-plated one he had given her when they were teenagers), retired, and moved to Paris, all the while keeping a multimillion-dollar penthouse apartment in New York City.
But domicile is where the heart is, and “the love story behind it . . . made this case different,” says Fields. The case was decided in favor of Patrick, saving him $2.2 million in New York state taxes.
New York nonresidents paid about $6.2 billion in state taxes in 2014, the latest year for which there are statistics, according to the New York State Department of Taxation and Finance. Some 941,361 nonresidents filed with the state, and the vast majority — 761,117 — ended up owing taxes.
Nonresidents are supposed to check a box on a New York state tax form claiming their nonresidency status when they don’t pay the state taxes, which can lead to an audit. But there’s no assurance that all of them even go to the effort of filing a state return. “It’s a little bit of a mystery as to how they identify someone’s presence in New York who wasn’t filing returns,” says one tax attorney. That doesn’t stop the state from going after them, he says. To find potential scofflaws, he says, “I believe they take a look at property records.”
Fred Feingold, the attorney who successfully fought back New York City’s efforts to tax Robertson, says going after nonresident returns has become a “big profit center for New York.” And the money in individual cases can be huge. “We have handled domicile cases where the amount involved was over $100 million for one year,” he says.
In 2012, The New Yorker reported that New York audited about 4,000 returns a year, bringing in about $200 million annually in delinquencies from those “wrongly or fraudulently claiming nonresident status.” Feingold says the chance of getting audited is “somewhat of a lottery. But if you’re a very prominent person, you have a target on your back.” He tells the story of an entertainment celebrity client of his who was audited simply because the auditor wanted the person’s autograph.
The Department of Taxation and Finance declined to say how many returns it is now auditing or how much it is bringing in as a result, but the overall nonresident take has jumped about $1.5 billion, or 30 percent, since 2012.
“It’s always been an issue for people who have a choice — but many people don’t have a choice. They live in New York and have to live fairly close to where they work,” says Feingold. Typically, those who do claim nonresident nonpaying status have several homes. “New York, Paris, south of France . . . . If you stay in enough places, you can practice the rhythm method of taxation and not be in any one place long enough to be taxed,” he jokes.
Now it seems more people want to get in on the act. “There’s a new incentive for high-end earners that aren’t just jet-setting around: the tax law,” explains Feingold. “Now that people realize they’re not going to get a deduction for state income taxes on their federal income taxes, all of a sudden New York has become a bigger issue for them.”
Paying $10 million to $20 million for a house elsewhere may cost more than the taxes, he says, “but it’s become a principle.” He adds, “A lot of those people are going to end up meeting me, or someone like me, when New York says, ‘You really haven’t changed your domicile.’”
Feingold admits that some people will try to get one over on the state, even by hiding the fact that they have apartments in New York. “They are just plain liars. They think they’ll never get caught,” he says.
But lawyers warn against that, saying New York’s tax auditors are true detectives. “They are very diligent,” says Robert Goldstein, a tax attorney with Schroder & Strom. “They aren’t flying by the seat of their pants. They tend to be very meticulous, which means that it’s uncomfortable for the taxpayer.”
New York’s extensive guidelines lay out the process in excruciating detail. In most cases, the state says, the first communication a taxpayer receives from the Department of Taxation and Finance is a cover letter with the residency questionnaire requesting information on domicile and the number of days spent in New York.
It’s then up to the taxpayer to prove they don’t owe the state. To meet the statutory test of being out of the state more than 183 days, past cases have shown that “contemporaneously maintained diaries or calendars supported by credible testimony” are required, according to the state guidelines. Any time spent in New York during the day counts as a New York day. As one New York administrative law judge puts it, “There is unfortunately no shopping or dining exception in the statute, regulation, or case law.”
Because the statutory guidelines are so strict, Feingold says he tells his clients to put “ten days in the bank” in case something happens at year-end to force them to be in New York longer than planned.
The state’s auditors don’t stop with checking out diaries.
“The auditor should make every attempt to visit the New York place of abode. The location of the neighborhood, the facility itself, and the relationship to the lifestyle of the taxpayer are important to the establishment of residency in New York,” the guidelines state. “This personal observation should include checking the names on the mailbox, checking the license numbers of any vehicles on the premises, interviewing the doorman, building superintendent, and mailman, if necessary. The auditor should make notes of his observations and, if possible, take pictures of the residence.” The guidelines also suggest that the auditor visit the taxpayer’s residences outside of New York as well for purposes of comparison.
An auditor could look at credit card statements, E-ZPass statements, bank statements, leases, mortgages, and the size of the properties in and outside of New York, says Goldstein. “They might want to look at the floor plan. They can’t barge into your house and start taking pictures, but there have been instances where I advised my client to invite the auditor in. When they see everything that’s there, it’s easier to understand that it’s your primary residence.”
While the rules do allow it, Goldstein says, “I’ve never had them fly all the way to Florida. I’ll either ask somebody to have a professional videography of the residence or lots of photos with time stamps.”
If at the conclusion of an audit the state rules against the individual, that person is also subject to a penalty of 25 percent of the underpaid tax. Most cases are settled without going to court, but if the taxpayer wants to fight it, says Goldstein, “the state will go on the offensive.”
Given New York’s tough standards, J.P. Morgan has sent out advisories to its clients on the matter. “We cannot emphasize enough how critical it is that you pay attention to both tests for determining New York tax residency. Even if you are no longer domiciled in New York, you still may be deemed to be a statutory resident and owe New York taxes,” it warns. In its note the bank discusses how the domicile test hinges on the determination of where one’s “home” is, including the “involvement in an active trade or business in New York, the time spent in New York, the location of items important to the taxpayer, and family connections.”
“Nothing speaks to ‘home’ more clearly than where you host family gatherings, spend the holidays, and keep items important to you, such as family heirlooms and photos,” J.P. Morgan says. “If these primary factors alone fail to make a taxpayer’s domicile clear, authorities may consider others, such as where the taxpayer receives important mail, maintains a safe deposit box, and registers to vote.”
Most people are surprised it’s a lot more complicated to prove you’ve moved than just being in the new state more than 183 days, says Jordan Sprechman, head of U.S. wealth advisory at J.P. Morgan Private Bank. “Things like where your dog is matters, where your artwork is matters. Those kinds of things are part of the texture.”
As he explains, “It’s actually easy to move if you actually move. The hangup comes where people want to establish tax residency in another state but keep the lives they have in their existing state.”
“New York has litigated many cases where people have said they’ve moved to Florida,” says Feingold. “But if you continue to have your hedge fund run from New York, and that’s the center of your business, you’re going to end up with a contest there. One of the significant factors is where you work.”
Adds William Sinclair, a managing director at J.P. Morgan Private Bank: “Continuing to operate a business you’ve operated for years in a high-tax jurisdiction is simply a bad fact, but it’s not that you can’t overcome it.”
Somehow several billionaires have managed to do just that. Cooperman, now 75, still runs his hedge fund firm, Omega Advisors, which is located in New York City. But Cooperman, who is also a Short Hills, New Jersey, resident, says he now often works from his home in Boca Raton, Florida.
“For 45 years the alarm clock went off at 5:15, and I spent an hour and ten minutes commuting from Short Hills to New York City and the same time coming home each night. In Florida I work out of an approved home office,” he says, explaining that he has been audited and “passed.”
“I save two hours and 18 minutes a day,” he says. “It’s a life-saver.”
Cooperman says he lives in Florida seven months out of the year but has kept his Short Hills house because “my wife won’t let me sell it.”
He told Institutional Investor that his move, which he made seven years ago, “has nothing to do with taxes,” though the tax savings “is nice.”
“My house in Florida is substantially nicer and more expensive. Every test one would take, I’d pass it,” he says, explaining that his doctors and dentist are also in Florida. The size or value of the home isn’t really determinant, according to lawyers. But there’s a more important fact in Cooperman’s favor: his burial plot is in Florida.
The situation is different for 63-year-old Jones, though he seems in the clear too.
In July 2016, several months after Jones changed his domicile, his $10 billion hedge fund shed 15 percent of its employees following poor performance of its funds. By last December, Jones had liquidated his five-year-old Tudor Discretionary Macro funds and told investors he would play a bigger role in managing money at the firm.
It’s uncertain if being more involved in Tudor means being in the firm’s new Stamford, Connecticut, offices more often. When he moved to Florida, Tudor opened an office in Palm Beach, which houses one investment professional — himself.
Jones declined to comment for this story. But there is another reason he probably doesn’t have to worry about paying state taxes in Connecticut — the many high-paid professionals at Tudor who still work and live there.
“Being associated with a business in a state, but not conducting any personal business in that state, is not the end-all and be-all” of the determination of a person’s tax status in Connecticut, former commissioner Sullivan says. “I could conceive of a circumstance where a person could be the owner of such an entity and still have a role and not be taxed here.”
When asked about Jones, he says, “We’ve been concerned. But as much as we hate losing the individual, what we really don’t want, what we really hate, is losing the business.”
So far, the state has mostly managed to avoid that when the billionaires leave — largely because their employees don’t want to uproot their families and lives. Peterffy’s Interactive Brokers Group remains headquartered in Greenwich, and while Sternlicht’s Starwood Capital Group is now headquartered in Miami, where Sternlicht lives, it still has an office in Greenwich.
In 2012, Lampert moved his ESL Investments hedge fund to Bay Harbor, Florida, but nearly all of the firm’s former employees stayed behind. They went to work for Adrian Maizey, Lampert’s former chief financial officer, whose company provided the same function as when they were part of ESL. They even worked out of its former Greenwich offices.
Lampert, who is also the CEO of troubled Sears Holding Corp., has maintained his Greenwich estate, and was interviewed there for a recent Vanity Fair profile that described in great detail the 10,000-square-foot house jutting into Long Island Sound. Lampert told the magazine that he and his family spend most of their time in Florida, where the children go to school.
Moving to Florida has helped Lampert preserve what’s left of his net worth, which has almost halved since he made the move — to about $1.7 billion — given the gargantuan losses at Sears. ESL is also a shadow of its former self, with $1.3 billion in regulatory assets under management, according to its latest filings with the Securities and Exchange Commission, compared with an estimated $5 billion in 2012.
Connecticut was fine with the arrangement, given that most of ESL’s high-paid employees remained in the state.
“We were very pleased to see the result they came to,” says Sullivan. “We would have loved to have him [Lampert] stay too. But it’s possible for others to do exactly what they did.” As the former commissioner explains, “If the company has one principal and 50 investment managers, if we lose both the principal and the 50 others, that’s really bad.”
One problem for Connecticut, which is facing a $192 million budget hole this year alone, is that being too tough could lead to even more of the state’s wealthy fleeing.
On the political front, Sullivan says, “There is a group that believes it’s highly inequitable that the very wealthy are able to evade so much taxation. They want us to go to their mansions and shake them down.”
But the ease with which the richest citizens can move elsewhere in today’s world of mobile connectivity makes that impractical.
“You can tax yourself to such progressivity that you have no tax base left,” he says.
