Officers responded to a home in the 200 block of South Golden Avenue just before 8 p.m. Sunday after receiving a call from a man who claimed his life was being threatened.
After forcing their way into a bathroom, officers said they found the suspect, 43-year-old Richard Matus, holding a six-inch knife to the victim's throat.
Police said an officer pulled the victim away and shot Matus, who died at a hospital.
The victim, 73-year-old Peter Zabala, was not hurt.
Police did not reveal the nature of the relationship between the two men except to say that they were acquaintances.
Investigators said Matus had arrived to the victim's house Sunday night to avoid the rain and that the victim reluctantly let him.
Police said the person who called 911 saying his life was threatened was apparently not the victim.
"We understand it was probably the suspect who called 911," said Lt. Jarrod Burguan of the San Bernardino Police Department. "There's something going on where he was disturbed."
Matus had a long criminal history ranging from drug possession to resisting arrest.
San Bernardino authorities did not identify the officer who shot Matus but did say he is a 10-year veteran of the force.
Officials said the case will be deferred to the district attorney's office per department protocol.