According to the local broadcaster, members of Ukraine's National Corps party and Right Sector extremist organisation used a rope attached to the bust to pull it off the pedestal.
In place of the bust, the protesters planted a Ukrainian flag.
Following the incident, Kharkov Mayor Hennadiy Kernes said that Zhukov's bust would be restored.
In 2015, the Ukrainian parliament approved a set of laws as part of decommunisation policy. It banned the use of Soviet symbols and provided for the demolition of monuments commemorating communist figures and renaming streets and inhabited areas named after them.