Gabriel Fernandez died two years ago in May. Prosecutors said he was the victim of repeated abuse.
"Social workers play a vital role in society. We entrust them to protect our children from harm," District Attorney Jackie Lacey said in a statement. "When their negligence is so great as to become criminal, young lives are put at risk."
A complaint for an arrest warrant filed on March 28 charges social workers Stefanie Rodriguez, 30, and Patricia Clement, 65, and their respective supervisors, Kevin Bom, 36, and Gregory Merritt, 60, each with one felony count of child abuse and one felony count of falsifying public records, according to the district attorney's office.
The young boy suffered a fractured skull, several broken ribs and had been shot with a BB gun. He was declared dead on May 24, 2013.
MORE: Death penalty will be sought against ex-couple in Palmdale boy's death
The boy's mother, 31-year-old Pearl Sinthia Fernandez, and her former boyfriend, 35-year-old Isauro Aguirre, have both been charged with capital murder in Gabriel's death.
The family of Gabriel Fernandez held a news conference after the charges were announced.
"I have a message for the four social workers. You've brought this upon yourself. Your conviction will be our greatest victory," said Emily Carranza, the boy's cousin.
The Department of Children and Family Services (DCFS) opened a case on the boy in October 2012. Following an investigation into Gabriel's death, the four social workers were fired.
One of the four successfully appealed his termination to the county's Civil Service Commission and has been reinstated, prompting Children's Services to appeal that ruling in court.
The district attorney's office said that Rodriguez and Clement are accused of falsifying reports that should have documented signs of Gabriel's escalating physical abuse, and Bom and Merritt knew or should have known that they were approving false reports that conflicted with the evidence of Gabriel's deteriorating physical well-being.
MORE: Palmdale mom charged in beating death of son confronts ex in court
"We believe these social workers were criminally negligent and performed their legal duties with willful disregard for Gabriel's well-being," Lacey said. "They should be held responsible for their actions."
DCFS Director Philip Browning said in a statement, "I want to make it unambiguously clear that the defendants do not represent the daily work, standards or commitment of our dedicated social workers, who, like me, will not tolerate conduct that jeopardizes the well-being of children. For the vast majority of those who choose this demanding career, it is nothing short of a calling."
The four were scheduled to be arraigned on Thursday, but the arraignment was postponed until April 21. If convicted, each defendant faces up to 10 years in state prison.
The Associated Press contributed to this report.