The incident occurred near the 3100 block of East Highland Avenue around 11:41 a.m.
Authorities said the men were both serving time in living quarters at the hospital when the attack occurred.
The 65-year-old victim was rendered unconscious and taken to Loma Linda University Medical Center for treatment of severe head injuries. He died of his injuries about six hours later.
The suspect, identified as George Carver, 40, of San Bernardino was arrested and charged with murder at West Valley Detention Center.
The victim has not been identified pending notification of kin.
Anyone with information regarding this case was urged to contact Detective Marco Granado at (909) 384-5628 or Sergeant Robertson at (909) 841-5361.