"There's always denial, we don't want to get old," Phillips said. "When you have a good quality of life, you want to preserve it."
A talk with his doctor revealed the real problem was andropause, commonly known as male menopause.
Pennsylvania State University's College of Medicine says it affects half of all men over age 50, or about 25 million men.
"The men's symptoms, a lot of the times with andropause, it's more of an insidious onset," said hormone expert Dr. Benita Swartout of BodyLogicMD.
In fact, male testosterone levels fade slowly, 1 percent per year after age 40.
That drop in testosterone can mean a maddening lack of sleep and energy, and can translate into a decrease in sex drive and muscle mass.
"There is a little bit of a reluctance if, for no other reason than just embarrassment or shyness, to speak of it," Swartout said.
Swartout said hormone replacement is one solution to boost testosterone.
But doctors at Johns Hopkins University said that can also boost red blood cell production in some men.
And there are other side effects.
Any man with a history of prostate cancer can't get the treatment because the added testosterone could actually trigger cancer growth.
Fertility can also be a problem, because a man's body may stop producing testosterone when he's getting it from an outside source, and that change can be permanent.
But for some men going through "man-o-pause," testosterone therapy can make a big impact.
Joel said hormone replacement worked for him.
"There's a total difference," Phillips said. "I feel more like the man I used to be, when I was 25 to 40 (years old)."
Now, the only grind in his morning is that cup of coffee, just the way he likes it.