They complain that some of the visitors renting homes and rooms in their residential neighborhoods create noise, drive up home and rental prices and destroy the sense of community.
"When you live in a place like Hawaii that is a resort 24-7, you need areas where people can feel at home," said Katherine Bryant-Hunter, chairwoman of a neighborhood board on the island of Oahu. "Our neighbors change everyday. They don't coach volleyball. They don't go to church with us. They are not part of the community fabric."
Their concerns have led to a flurry of proposals by lawmakers on Oahu, Maui and Kauai. The legislators are trying to address what they say is a proliferation of short-term rentals that are operating without required county approval.
But owners of short-term rental units call them an asset to Hawaii's tourism industry. They say they set rules ensuring their guests aren't disruptive.
"In my area, I never had noise that I know of, and there were always house rules," said Angie Larson, who runs a bed and breakfast out of her home in an upscale Oahu neighborhood near the beach. She is part of a group fighting to lift a ban on new short-term rentals on Oahu.
Larson and other short-term rental owners argue that their guests help them pay the mortgage and meet other expenses, not get rich. But disputes between owners and their neighbors have sparked angry confrontations and lawsuits.
"There's a lot of ugly things occurring in neighborhoods," Bryant-Hunter said. "People are really frustrated because they don't want to fight with their neighbors, and nobody wants to throw out the visitors."
In one of the most contentious cases, Oahu resident Susan Cummings sued her neighbor Marlene Roth, accusing her of operating an illegal bed and breakfast. She has installed a video camera on the side of her beach cottage to catch renters going up and down the driveway she shares with Roth.
Cummings, who will be 75 this year, said a stream of guests coming in and out of Roth's house at all hours have ruined her dream of a quiet retirement.
"This isn't the way I intended these years of my life to be," she said recently.
She won a lawsuit against Roth two years ago, but said she still has little relief.
Roth is appealing the case and has accused Cummings in court papers of spying and harassment. Her attorney said she was unavailable for comment.
Vacation rentals and bed and breakfasts are a small part of Hawaii's tourism industry. Hotel rooms account for most of the roughly 73,000 units of visitor accommodations.
Studies financed by real estate and vacation rental groups in 2005 found more than 4,000 short-term rental units on Oahu, Maui and Kauai - most of them illegal. The state Department of Business, Economic Development and Tourism counted more than 1,100 units on the state's remaining islands.
Bed and breakfast rooms in Hawaii are available for as little as $80 a night, while the average hotel room is nearly $200, according to Hospitality Advisors LLC.
Short-term vacation rentals tend to be more expensive, going for a few hundred dollars a night.
Maui is considering a proposal to eliminate special permits allowing short-term vacation rentals in residential areas, said Jeff Hunt, Maui County planning director. The proposal would simplify the process of getting a bed and breakfast permit and expand the areas where short-term vacation rentals can operate without a permit.
"We're trying to find some middle ground," Hunt said.
Vacation rental homes tend to incite the most outrage among neighbors, because unlike bed and breakfasts, they have no owner or manager on site, Hunt said. The county has about 1,100 illegal bed and breakfasts and vacation rentals, he estimated.
On Kauai, the county council recently passed a law that prohibits single-family homes from serving as vacation rentals outside certain resort areas. The previous law had mentioned only multifamily units, leading some people to argue it didn't apply to single-family homes, according to Councilwoman JoAnn Yukimura, who proposed the new law.
Oahu banned new bed and breakfasts and short-term vacation rentals in the late 1980s. About 1,000 property owners - grandfathered in at the time of the ban - have city approval to rent rooms for less than 30 days. But hundreds of others operate illegally in oceanside communities, critics say.
City officials are considering a measure that would nearly quadruple the property tax on homes used as vacation rentals and bed-and-breakfasts.
"If they are going to be making money out of residential districts operating as basically a hotel, I think it's only fair they pay something that is equivalent to a hotel rate," said Councilman Gary Okino, who introduced the proposal.