Hundreds of Libyans gathered in the capital, Tripoli, Wednesday to protest against former rebel fighters who took part in the ouster of former leader Moammar Gadhafi months ago but are still camping out in the capital.

The protesters waved Libyan flags and chanted “We want safety, not weapons.” They said they want the militias, who came to Tripoli mostly from the cities of Misrata and Zintan, to go home. They complained of feeling unsafe on the streets and of militiamen causing problems.

On Tuesday, the interim government and the city council gave the former rebels until December 20 to leave.

Wednesday's protest followed a smaller one earlier in the day by judges and lawyers outside Tripoli's courthouse, who demanded protection. They said that security must now be in hands of the national army and police and insisted that the militias leave.

Late last month, the United Nations said Libya's former rebel fighters still hold about 7,000 people in prisons and makeshift detention centers.

A report by U.N. Secretary-General Ban Ki-moon said that many detainees had no access to due process and some of them have reportedly been subjected to torture and ill treatment.

The report noted that sub-Saharan Africans, suspected of being mercenaries hired by Gadhafi, make up the majority of those detained.