Smith's mother, Shawna Smith, along with her partner and a third woman appeared at the family home in Menifee briefly Thursday and quickly went inside.
About 30 minutes later, the mother got into a blue Ford SUV and drove away. She didn't say anything, but her partner, Dennise Bugna, asked for privacy.
"We're just asking for a little time to grieve, everybody has known Terry, if they're lucky, for five minutes, we've known him for 11 years, and we have 11 years of grieving to do, they have three days of searching," she said. "We appreciate all the pain that they have, but they have to remember ours, it's a lot deeper."
Not only is the family grieving the loss of the 11-year-old boy, but presumably, it's the 16-year-old half brother that is under arrest for the boy's murder.
In an interview with Eyewitness News, Smith's dad said the 16-year-old was violent, and his mother knew it.
"(The mother) knew that he did get violent with his brother and sister. I begged her to get him the right kind of help," said Terry Smith Sr., who lives in West Virginia. "In fact, I begged her to get the right kind of help for herself, and it just never happened."
The teen was taken into custody on Wednesday. News of the arrest came hours after authorities revealed they found a body matching the boy's description in a shallow grave behind the family's home.
The identity of the remains has not been confirmed, but authorities said the remains are of a male juvenile that fits Smith's description.
Hundreds of people had volunteered to search for the boy, who was reportedly autistic. But Terry Smith Sr. said his son did not have autism.
"He was rambunctious. He was not autistic. I'll fight that until the day I die," he said. "He was a typical little boy. He loved life. And Thomas the Train set."
The school district also disagreed with the family about whether the boy was ever diagnosed with autism.
"I know it's going to be a bit of a shock to people to hear this, but no, he was not autistic," said Jenny Smith, who said she taught the boy.
What happened between the two boys remains unknown. But Smith's father said after a phone call with the mother earlier this week, he grew worried.
"She told me (the half brother) had been sneaking out of the house, and Terry Jr. went to follow him, and him and (the half brother) got into it. She didn't go any further than that, but as soon as she said him and (the half brother) got into it, I knew they wouldn't find Terry Jr. alive," he said.
It is also unknown why it took detectives four day to find a body that was less than 100 feet from the home. According to Pam Ragland, the woman who says she found the body, the size and rural characteristics of the property could have been a factor. She also thinks people weren't expecting to find a body.
"I just believe that when everybody was looking initially, they probably, I'm sure, looked for signs of foul play, decided that they were looking for a child, so when they looked up in that area, they were looking for a child. I really think that's why they missed him."
Ragland, of South Orange County, says it was a vision she had that led her to the house two nights ago with her children.
It was dark, and she says it was actually one of her kids who found the body.
"My kids found this one spot, and they said, 'Mommy, we found a dead animal,'" she said. "And I went over and looked at it and said, 'Oh, that's not an animal.' It was a head."
Ragland admits that she is no forensics expert, but she said in her opinion, it appeared the body had been there for some time. The sheriff's department said deputies and volunteers had searched the inside and the outside of the home several times.
According to Terry Smith Sr., the boy was born in West Virginia and moved to California with his mother about three years ago. But the boy wanted to return to West Virginia, he said.
"Terry Jr.'s going to get his last wish. And that's to be back home here," he said. "He was born back here, this is where his life started, and this is where he's going to wind up at."
Smith's mother and her partner spent several hours Wednesday at the sheriff's station in Perris for questioning. Authorities have not said what role they had in this case, but they are not facing criminal charges.
"This has been a devastating week for our family and for everybody involved in the search," the family said in a statement. "This community has been amazing and we appreciate all the support."
Anyone with information is asked to contact the Riverside County Sheriff's Department at (951) 776-1099.