The surprise win left rap artist Eminem shut out in all of the top categories.
Arcade Fire beat out Lady Antebellum's "Need You Now," Katy Perry's "Teenage Dream," "The Fame Monster" by Lady Gaga and Eminem's "Recovery" - 2010's best selling album.
This is now the third time Eminem has lost the best album trophy. He led all nominees with 10 and won two.
Lady Antebellum's "Need You Now" won song of the year and record of the year.
Another upset came as Esperanza Spalding - a jazz bassist and singer who sold a fraction of Justin Bieber's music and is perhaps best identified by her voluminous Afro - beat the perfectly coifed 16-year-old pop phenomenon, and also Florence & the Machine, Mumford & Sons and Drake for best new artist.
She is the first jazz artist to ever win the category.
The Associated Press contributed to this story.
