The new law states that animals must be obtained from an animal shelter and the store must post the name of the agency where it got the animal; no breeders will be allowed.
The change comes from the Pet Rescue and Adoption Act, which was signed into law by Gov. Jerry Brown in October 2017.
The law requires the stores to keep public records that show where each dog, cat or rabbit came from. A violation would mean a $500 fine.
Thirty-six cities in California, including Sacramento, Los Angeles and San Francisco already had similar bans in place, but no statewide bans existed.
"Californians spend more than $250 million a year to house and euthanize animals in our shelters," O'Donnell said in a statement to the Associated Press. "Protecting the pets that make our house a home is an effort that makes us all proud."
Those looking to purchase a new furry friend still have the option to buy from a private breeder.