Ocarina of Time
As the name suggests, the Gerudo Guards are the Gerudos in Gerudo's Fortress that secure the area of intruders. When Link is an adult, the Carpenters have joined the Gerudos, but were put in jail. Link must stay vigilant to avoid being caught by them, or he can simply stun them with certain items, such as the Fairy Bow. If they see Link, they will throw him into jail. When Link has saved all of the Carpenters, the Gerudo Warrior will come and give him the Gerudo Token, which makes the Gerudos become friendly toward Link. With the token, Link can enter new areas, such as the Gerudo Training Ground, the Horseback Archery Range and the Haunted Wasteland. There are more skilled Gerudo Guards, called Gerudo Thieves, who attempts to prevent Link from saving the Carpenters.
In Termina, the Gerudo are pirates, led by Aveil, and Gerudo Guards patrol their Pirate's Fortress much like their Hyrulean counterparts patrol the Gerudo's Fortress there. If they catch Link, rather than throwing him in jail, they throw him out of the current area of the fortress.