Adrone is helping forces intercept boats carrying in the Mediterranean, who are then "hauled back to arbitrary detention and abuse", Human Rights Watch said Monday.
The drone, operated out of EU-member Malta, plays a "crucial role" detecting boats leaving Libya, information that the EU's border agency Frontex then hands to the Libyan coast guard in the North African country, it said.
"Frontex claims the surveillance is to aid rescue, but the information facilitates interceptions and returns to Libya ... (despite) overwhelming evidence of torture and exploitation of migrants and refugees in Libya," HRW said in a statement.
War-torn Libya is a major launchpad for people fleeing violence further south in Africa to seek refuge in Europe, and Frontex has backed Libyan efforts to intercept the boats.
HRW said Frontex's drone appeared to have helped Libyan forces intercept a boat in international waters, without informing nearby rescuers.
It said it was investigating how "the shift from sea to air surveillance contributes to the cycle of extreme abuse in Libya".
The New York-based rights group said some 32,450 people were intercepted by Libyan forces last year and "hauled back to arbitrary detention and abuse" in the country.
United Nations experts have said acts of "murder, enslavement, torture, imprisonment (and) rape" against detained migrants in Libya may amount to crimes against humanity.
The HRW report comes days after media reports that Frontex routinely covered up illegal "pushbacks" of migrants by the Greek coastguard.
The agency's head at the time, Fabrice Leggeri, resigned in April amid an investigation by the European anti-fraud office OLAF, reportedly into alleged mismanagement.