An accompanying banner says the message was issued to mark the 41st anniversary of the 1967 Arab-Israeli war, during which Egypt lost the Gaza Strip and Sinai Peninsula, Syria lost the Golan Heights and Jordan lost the West Bank and east Jerusalem.
Trying to minimize the shock of the defeat, Arabs have long called the war the "naksa" - "setback" in Arabic - but it remains a deep wound.
In al-Zawahri's recording, titled "In Memory of the Naksa ... Break the Siege of Gaza," Osama bin Laden's deputy blames Arab regimes for the 1967 defeat. He says Arab governments were "impotent and unable to protect the Muslim nation, its sanctuaries and its wealth."
"The sons of the nation should break the shackles of the treacherous regimes and move to wage jihad, which has become a duty," al-Zawahri says.
The tape follows a audio message from Bin Laden on May 18 in which he criticized Arab states for not waging war against Israel.
Al-Zawahri's message seemed especially directed at Hamas, the Islamic militant group that seized control of the Gaza Strip last June in fighting with supporters of Palestinian President Mahmoud Abbas. Abbas' government is now based in the West Bank, leaving the Palestinian territories split.
Hamas' seizure of Gaza and its near daily firing of rockets at towns in southern Israel has prompted Israel to impose a blockade on the strip and mount airstrikes and occasional ground operations in Gaza.
Al-Zawahri lashes out at Egyptian authorities in his new message, declaring Egyptian President Hosni Mubarak and his troops "criminal traitors" for perpetuating Israel's blockade by sealing its own boundary with Gaza.
"You have the right to enter Egypt whenever you like and destroy the treacherous siege," al-Zawahri tells Gazans. "Those who confront you should not blame anyone but themselves."
Just hours before al-Zawahri's message was released, Abbas called for renewed dialogue with Hamas.
Al-Zawahri often issues audio and video recordings, speaking on a wide range of topics. He has frequently discussed Palestinian issues.