But now, Rowling has revealed Edinburgh is actually not where the writing of the boy who lived began.
"I was renting a room in a flat over what was then a sports shop," Rowling said on Twitter, with an image of where she resided at the time of writing the first book. "The first bricks of Hogwarts were laid in a flat in Clapham Junction."
And fans were -- to borrow from the books -- were essentially petrified.
The news came after a fan posted a picture on Twitter of the Elephant House, a coffee shop in Edinburgh which on its website describes itself as the place "made famous as the place of inspiration to writers such as J.K. Rowling, who sat writing much of her early novels in the back room overlooking Edinburgh Castle."
The fan asked Rowling to explain "the truth about this 'birthplace' of Harry Potter."
Rowling, who is known to drop various bombshells and unknown tidbits about the franchise on Twitter, explained that the real "pen to paper" birth of Harry Potter himself, happened in her flat.
However, she said, the moment where she had the idea for the series itself was while she was riding on a train.
This is the true birthplace of Harry Potter, if you define 'birthplace' as the spot where I put pen to paper for the first time.* I was renting a room in a flat over what was then a sports shop. The first bricks of Hogwarts were laid in a flat in Clapham Junction. pic.twitter.com/HVORnPVboK— J.K. Rowling (@jk_rowling) May 21, 2020
"If you define the birthplace of Harry Potter as the moment when I had the initial idea, then it was a Manchester-London train," Rowling tweeted.
"But I'm perennially amused by the idea that Hogwarts was directly inspired by beautiful places I saw or visited, because it's so far from the truth."
As for the high flying, bludger smashing game of Quidditch, Rowling said it was birthed in 1991 after a one-night stay at the the Bourneville Hotel in Manchester.
Rowling went on to dispel some of the other rumors around her inspirations for key icons from the Harry Potter world.
For example, a fan tweeted a picture from the bookstore Livraria Lello, in Porto, Portugal, that supposedly inspired the Hogwarts library. The only trouble is, Rowling said she has never been there.
She did acknowledge that she wrote in a cafe in Rua Santa Catarina, Portugal.
"If it cheers up the people who're disappointed about the bookshop in Oporto, I wrote in here sometimes," she wrote. "This was probably the most beautiful café I ever wrote in, actually."
Rowling also said that rumors around her inspiration for Hogwarts itself miss the mark as well.
"I sometimes hear Hogwarts was based on one or other of Edinburgh's schools, but that's 100% false, too," Rowling said.
"Hogwarts was created long before I clapped eyes on any of them! I did finish Hallows in the Balmoral, though, and I can't lie, I'd rate it a smidge higher than the Bournville."
Out of all the "utter nonsense about Potter landmarks," Rowling said her favorite is people going to a parking meter she supposedly used in Edinburgh while writing the "Deathly Hallows."
"I can't drive," she wrote.
Unfortunately, when it comes to any spoilers about upcoming work, Dumbledore voiced disappointment best: "Alas, earwax."