The animal seemed to have followed a mother and her child into the store, took a look around and then headed toward the restroom.
"There was this little goat, and he was just adorable," said Lorrie Bridges of the Piano Gallery.
The goat was picked up by animal control officers, who named the little guy Beethoven.
If his owner doesn't show up to pick him up, he'll be adopted. Someone has already reserved him if that happens.