The attack comes one day after a suicide bombing killed 34 in a Shiite area in the city.
The firefight erupted near a training center overseen by the National Security Directorate - Afghanistan's intelligence agency - with the gunmen holed up in a construction site near residential buildings, an official at the scene said. "It was horrific and numerous students were torn to pieces".
Mohammad Arif Shahjahan, an MP from Ghazni, told CNN Monday Taliban fighters had taken control of key buildings, including the police headquarters and some government offices. In the past two years, there were at least 13 attacks on the Shiite community in Kabul alone, he said.
Afghan forces appeared to have finally pushed Taliban fighters from the strategic provincial capital, as the United Nations warned that reports suggested up to 150 civilians might have been killed in the fighting.
Hossain recounted to The Associated Press how when he entered Fareba's classroom, he saw parts of human bodies all over student desks and benches.
Afghan President Ashraf Ghani condemned the "terrorist" attack that "martyred and wounded the innocent" - students attending class - and ordered an investigation into the attack.
With parliamentary elections due on October 20, authorities had been bracing for more attacks in Kabul and other cities, but even so, the scale of the violence has come as a shock to a government facing bitter criticism over its handling of the war.
On Tuesday, the AP says, the Taliban killed 17 soldiers in northern Faryab province after besieging an army base for several days.
The United States and North Atlantic Treaty Organisation launched airstrikes and sent military advisers to aid Afghan forces as they fight for the city, just 120 kilometers (75 miles) from the Afghan capital with a population of some 270,000 people.
"It took Afghan forces - assisted by USA advisors - five days to repel the insurgents", NPR's Diaa Hadid reports.
Elsewhere in Afghanistan, the Taliban attacked a police checkpoint in the southern Zabul province early Wednesday, killing four policemen, according to the provincial police chief, Mustafa Mayar. "They are killing our educated people and everyday they are killing us".
The brief respite sparked hopes the truce could clear the way for talks to end the almost 17-year-old conflict.