Joshua Robey, 24, is facing attempted murder, torture and child abuse charges.
Anaheim police say Robey was supposed to be looking after his 2-month-old son.
The child's mother had a hidden camera set up because she thought Robey was cheating on her.
Instead, the video apparently shows Robey swinging the child with a blanket around the baby's neck. Investigators say he also punched, slapped and shook the baby.
The child is hospitalized with unknown injuries.
Robey was held on $1 million bail and could face 15 years to life in prison if convicted.