"Today's been a long time coming, almost 50 years," Eli Broad said.
The $140 million gift to Los Angeles was courted by other cities, a fact L.A. Mayor Eric Garcetti acknowledged.
"Los Angeles is the right home for The Broad," he said.
PHOTOS: The Broad museum features 2,000 works of art
The contemporary art museum will house 2,000 works of art by 200 artists, and more than 250 of the pieces are in the inaugural display.
Visitors will enter the ground-floor lobby to ride the 105-foot escalator.
Jeff Koons' steel sculpture "Tulips" is on display in The Broad's third-floor galleries. Koons is a friend of the collectors, who for the first time are seeing their museum with people in it.
"It's exciting. The art we collect, it has messages, social messages about what's going on in the world," Eli Broad said.
Curator and founding director Joanne Heyler said the collection includes a "deep concentration" of pop art from the 1950s and 1960s, providing "a truly unique opportunity to experience these rare master works free."
From a new Robert Longo charcoal drawing of police in riot gear in Ferguson, Missouri to Andy Warhol's take on pop culture to a piece that includes a living model, the collection is meant to evoke emotion, but art is not the Broads' only world.
"This is also just one part of our life, we're interested in science and medical research," Edythe Broad said.
Timelapse video shows construction of The Broad in DTLA
The Frank Gehry-designed Walt Disney Concert Hall is located next door to The Broad. Museum planners initially wondered how they would "design a building that doesn't' clash with Frank Gehry's masterpiece but also is not anonymous."
Their answer came in the form of a design by the firm Diller Scofidio+Renfro.
"While Disney Hall is shiny, exuberant, reflective, ours would be absorbent, more modest," architect Elizabeth Diller said.
Heyler said at least 85,000 tickets have already been reserved prior to the museum's grand opening on Sunday.
"There's tremendous interest in this museum, and we can't wait to open on Sunday and welcome the first of those," she said.
Once open to the public, The Broad, at 221 S. Grand Ave., will be open 11 a.m. to 5 p.m. on Tuesdays and Wednesdays, and 11 a.m. to 8 p.m. on Thursdays and Fridays. The museum is open at 10 a.m. on weekends, closing at 8 p.m. on Saturdays, and 6 p.m. on Sundays. The museum is closed on Mondays, Thanksgiving Day and Christmas Day.
Tune in to Eyewitness Newsmakers on ABC7 Sunday at 11 a.m. for a personal tour of the museum and to see Adrienne Alpert's special interview with Eli Broad.
City News Service contributed to this report.