The remains were found Monday during a search on the outskirts of Amalia, a tiny, remote town near the Colorado border marked by scattered homes, sagebrush and open plains.
Authorities were awaiting positive identification of the remains by medical examiners.
Taos County Sheriff Jerry Hogrefe indicated the body appeared to be that of a boy similar in age to Abdul-ghani, who suffers from seizures and was reported missing in December after his father said he was taking him to a park in Jonesboro, Georgia, south of Atlanta.
The father, Siraj Ibn Wahhaj, had told his wife he wanted to perform an exorcism on the child, authorities said.
"We discovered the remains yesterday on Abdul's fourth birthday," Hogrefe said, appearing to fight back tears.
The search for Abdul-ghani led authorities Friday to the squalid compound shielded by old tires, wooden pallets and an earthen wall studded with broken glass. Investigators said they found the heavily armed Wahhaj along with four other adults and 11 hungry children living in filthy conditions .
Wahhaj was taken into custody without incident.
All the adults were arrested on suspicion of child abuse. Wahhaj is also is being detained on a Georgia warrant that seeks his extradition to face a charge of abducting his son. He was expected to appear in court Wednesday.
Authorities returned to search the compound after interviews on Friday and Saturday led them to believe the boy might still be on the property.
"We had a good idea of a target location to look for the child," Hogrefe said.
The Georgia arrest warrant said the boy suffers from severe medical issues including a defect caused by lack of oxygen and blood flow around the time of birth. His mother said he can't walk and requires constant attention.
At a news conference in Taos, Hogrefe described FBI surveillance efforts in recent months that included photographs of the compound and interviews. He said the images were shared with the mother of Abdul-ghani but she did not spot her son.
"I had no probable cause to get a search warrant to go onto this property," the sheriff said.
He said FBI officials were invited to the news conference but declined to attend. An FBI spokesman didn't immediately return a call seeking comment.
Hogrefe said the "breaking point" in seeking a search warrant came when Georgia authorities received a message that may have originated within the compound that children were starving inside.
It wasn't clear who sent the message or how it was communicated. Georgia detectives forwarded it to the Taos County Sheriff's Office.
Children ages 1 to 15 were rescued from the compound that had been under investigation for months. The sheriff said it appeared the children hadn't eaten for days.
Property owner Jason Badger said he and his wife had pressed authorities to remove the group that he said had built the compound on his acreage instead of a neighboring tract owned by Lucas Morton, one of the men arrested during the raid.
"I started to try and kick them off about three months ago and everything I tried to do kept getting knocked down," Badger said.
Court records show a judge dismissed an eviction notice filed by Badger against Morton in June. The records didn't provide further details on the judge's decision.
Tyler Anderson, who lives near the compound, believes the group had moved to the area to live off the grid, just as he had done.
Anderson said he had helped the newcomers install solar panels after they arrived in December. But he eventually stopped visiting the compound. He said the children at first played at neighboring properties but stopped in recent months.
The women, believed to be mothers of some of the children, have been identified as 35-year-old Jany Leveille, 38-year-old Hujrah Wahhaj, and 35-year-old Subhannah Wahhaj.
Jail booking photos show them wearing traditional Muslim veils or hijabs.
Aleks Kostich, managing attorney in the Taos County public defender's office, said the office was gathering information and assigning attorneys to the defendants. He declined to comment on their behalf, citing the early stage of the case.
However, he questioned the "legal sufficiency" of the criminal complaints filed against the men and women, saying they were vague.
"I'm not sure how much investigating has been done," he said. "I'm not sure how much law enforcement knows and how long they've known it for."