One of them, a 45-year-old woman from Munich, who would give her name only as Andrea S., said she felt duped and betrayed by Fritzl.
"If I would see him now, I would ask ... 'How can you do such a thing to your children?"' she told AP Television News in an interview Saturday.
That question was also on the minds of classmates of three of the children whom Fritzl fathered with Elisabeth - then allegedly smuggled out of the basement and dropped on the doorstep with notes police say he forced his daughter to write saying she couldn't raise them.
"How could the father do such a thing?" asked Jelena Krsic, 12, who goes to school with Elisabeth's 12-year-old son, one of the three being raised by his grandparents.
Jelena, in an interview alongisde her parents, described the boy as a top student who always helped the others when they struggled in mathematics, English or German. She said he once cried in front of the other children when he failed in a jumping exercise, but was otherwise among the best athletes.
When the children returned to school after the scandal broke, "the teachers cried," Jelena said.
"He has a lot of feelings for the others and whenever someone cried, he helped them," Jelena said. "Without him recess is really boring."
Investigators have said that Fritzl was an "absolute ruler" in his household whose tyranny caused most of the seven children he had with his wife to flee the home as soon as they were old enough.
"He forbade anyone to ask even where he was or what he was doing," chief investigator Col. Franz Polzer told The AP.
Fritzl's children have given investigators consistent reports of daily life in the family with his wife, Rosemarie, and Elisabeth has described the difficulties she experienced with her father even before her imprisonment, Polzer said.
He declined to confirm an Austrian media report that she ran away from home as a teenager, but said she might have been sexually abused by her father as young as 12 or 13.
Questions have also surfaced about whether Rosemarie was aware of the abuse - a notion rejected by police and by friends.
"If she had known, she definitely would have done something to get her daughter out," said Andrea S., who visited the family several times in their home. "She loves her children. You can also see that in how she raises her grandchildren."
She asked that her full name not be used because of the international attention the saga has attracted.
The woman - who with her boyfriend visited the Fritzl home in Amstetten - said Josef and Rosemarie interacted normally in front of them. But on a vacation in Thailand without his wife, she learned that Josef had a "girlfriend" for whom he purchased sexy underwear.
Andrea S. said she is shocked by what she has now learned - that the "girlfriend" may have been Fritzl's daughter - but at the time was "satisfied" with Fritzl's explanation: "We never asked about it afterward."
Authorities first began to unravel the complex story on April 19, when Elisabeth's eldest daughter was admitted to a hospital suffering from an unidentified infection.
Doctors, unable to find any medical records for the girl, appealed on TV for her mother to come forward. Fritzl then accompanied Elisabeth to the hospital April 26 and opened up to police.
Fritzl faces up to 15 years in prison if convicted on rape charges, the most grave of his alleged offenses, unless prosecutors can charge him with "murder through failure to act" in connection with the death of an infant, whose body he has told police he burned after the child died shortly following birth. That is punishable by up to 20 years in prison.
Everyone from average Austrians to politicians have pondered whether that is enough.
"I have to say, I absolutely no sympathy for him," Andrea S. said. "Neither does my boyfriend. He was really angry at him" when he heard the news. "And he still is."