The incident left both the family and the children traumatized, they say, as their mother was described as homeless and a drug user who had lost legal custody of her children.
Ten-year-old Liam Jasso was emotional as he reflected on that day he was taken and kept in an unfamiliar location away from his family for nearly a dozen hours.
"It makes me scared," said Liam, who attends the afterschool program with his 8-year-old sister. "Sad, kind of angry and like I can't believe that happened."
They say the incident happened May 14 at Lanai Road Elementary School in Encino, when the two children were attending the Beyond the Bell afterschool program.
Lucia Jasso is the grandmother of the two children and adopted them when she says her own daughter became addicted to drugs and lost legal rights to her children.
She says she showed up at the school that day to pick up Liam and his sister - as she did every day - and was stunned to hear someone had already taken them.
"I asked for the kids and I was told they were not there, that somebody had already picked them up," she recalled.
Lucia was frantic when she says she learned the administrators of the afterschool program did nothing to stop the children's mother from walking up to the school gate, calling the children over and taking them.
"Then after that I started crying. I was desperate. They had already called the police."
"When my mom told me the kids were missing, I didn't know what to think," said Jaylene Rios, the aunt of the children.
"When we finally found out that it was Isabel who had taken them we got even more worried because she doesn't have a home. We were scared that the kids were gonna be sleeping somewhere cold."
Jaylene says it took about nine hours to track the children down in Compton at the home of Isabel's boyfriend's mother.
Jaylene says upon arrival they found the children but not their mother. Jaylene says luckily the children were not harmed.
"They were scared because they were in a house," Jaylene said. "They don't know these people."
The family has retained an attorney and is suing the school district.
"It is just infuriating," said Christa Ramey with Acts Law. "LAUSD has a program, Beyond the Bell, on its campuses, that is administrated through LAUSD. These are employees that failed."
LAUSD issued a statement: "Los Angeles Unified takes the safety and security of all students very seriously. The District is looking into these claims. However, Los Angeles Unified does not comment on pending or ongoing litigation."