It's not just rail thin runway models, but those sedentary yet strict calorie counters that has Brown concerned.
"You can do all of your weight loss by just dieting and that's fine, but that's not going to change your percentage of body fat to your lean body tissue," said Brown.
So common is this trend that health experts have coined the term "skinny fat" or "TOFI," thin outside - fat inside. It might not seem like an issue to those fighting their obvious girth, but research out of England suggests otherwise.
Some doctors are concerned that internal fat that might appear to be invisible on thin people, sits around their vital organs like the heart, liver and pancreas, putting this group at risk for things like heart disease and diabetes.
Out of the 800 slim people that received an MRI to scan for visceral fat, 45 percent had excessive levels of internal fat with some as high as 60 percent, yet exteriorly they appeared slight.
The good news is exercise can help considerably, as working out helps boost the amount of lean body mass, helping to utilize stored fat, increasing circulation and more.
Even sumo wrestlers are suggested to have a better metabolic profile as fat in active people tends to be healthier than those that are skinny an inactive.
"When you have more lean body mass, specifically muscle, then you're going to have a higher metabolism so it's easier for you to keep your body weight off if you're exercising," said Brown.
Since the scale isn't the best indicator, having someone administer a body fat test using a skin calipher provides a barometer, or you can check a body mass index chart found on many Web sites. A healthy woman should have between 14 and 24 percent body fat, a healthy man should have between 5 and 17 percent.