The 10-story banner shows James and bears the phrase "We Are All Witnesses" in downtown Cleveland.
Some Clevelanders said they hoped it would remain as a tribute to James' years with the Cavs.
RELATED: LA fans, businesses prepare for impact of new Lakers star LeBron James
Fans, businesses prepare for LeBron James' arrival in LA
James is set to join the Los Angeles Lakers, leaving Cleveland for the second time in his career.
It will take crews several days to remove the banner.