The demolition was slated to begin Saturday. The mansion, known as Land's End, will be replaced by five houses priced at $10 million each. The 13-acre property faces the Long Island Sound.
Preservationist Alexandra Wolfe told Newsday that regardless of F. Scott Fitzgerald's true connection to the mansion, his relationship to the house has become local lore.
Newsday said Winston Churchill, the Marx Brothers and Ethel Barrymore attended parties there in the 1920s and 1930s.
The house featured marble, parquet floors, Palladian windows and hand-painted wallpaper.
Historians say hundreds of mansions on Long Island's Gold Coast have been lost in the past 50 years because of rising taxes and maintenance costs.
The Associated Press contributed to this report.
