Police said the suspected cell had links to a group associated with Osama Bin Laden's al-Qaeda network.
They seized the imam and two other men, all Moroccans, and have a warrant for a fourth man believed to be abroad.
|We have discovered and neutralised a real 'terror school'|
Carlo De Stefano
anti-terrorism police head
The three detainees refused to reply to questions when they were brought before a local magistrate.
Twenty foreign students were also arrested in a related dawn raid and police said that those without residence permits would be deported.
Perugia, a popular tourist destination because of its medieval and Renaissance palaces, is home to Italy's University for Foreigners, where hundreds of students from the Middle East are enrolled in university courses in Italian and other subjects.
The discovery of the alleged terrorist training centre is a matter for serious concern to the Italian authorities, the BBC's David Willey reports from Rome.
Italian Interior Minister Giuliano Amato said it was now necessary to pay close attention to mosques being used for activities unrelated to religion.
The suspects were running an "in-depth operation of instruction and training in the use of weapons and combat techniques suitable for terrorist acts", police said.
Chemicals - including acids and cyanide - were found in the mosque's cellar and equipment for remote detonation of explosives was also discovered, they added.
"The investigation has shown that... there was a continued training for terrorist activity," anti-terrorism police head Carlo De Stefano said.
"We have discovered and neutralised a real 'terror school,' which was part of a widespread terrorism system made up of small cells that act on their own."
According to the police, the Perugia cell had contacts with two members of the Moroccan Islamic Combat Group arrested around two years ago in Belgium.
The group is believed to have ties to al-Qaeda and has been linked to the 2004 Madrid bombings and 2003 attacks in Casablanca.
The detained men were named as imam Korchi el Moustapha, 41, Mohamed el Jari, 47, and Driss Safika, 46.
Officers are reported to have spent two years investigating activities at the mosque.
Between daily prayers, the small mosque at Ponte Felcino doubled as a training camp, a police statement said.
The imam allegedly held courses, showed propaganda messages and made fiery sermons inciting a small group of disciples, some of them children, to join a Holy War.
The director of the Cultural Institute of the Italian Islamic Community, Sheikh Abdul Adid Palazzi, told BBC News that he was not surprised to hear of the arrests.
"It is the top of the iceberg in our country - like in the rest of Western Europe. Most mosques are controlled by extremist pro-terror organisations - 90% of mosques," he said.
"And I think the percentage is more or less the same in Italy, Britain, France and Germany."
However the imam at the central mosque in Perugia, which has a 10,000-strong Muslim community, said the Ponte Felcino group had not appeared dangerous.
"Generally it's a quiet community," Abdel Qader told Italian news agency Ansa.
"A few made some noise over the international situation but those were just words. We trust justice... and if any [of the suspects] has made a mistake, he will have to pay."