Boss difficulty. The dungeons themselves were difficult enough, but I didn't find any of the bosses to be overly challenging. The methods of defeating them were easy to figure out, and I never really found myself getting it. One of the few things that disappointed me about the game.
Also, losing rented items when you died was pretty annoying. I didn't have enough rupees to buy the items until near the end of the game, so I had to rent them. I didn't die a lot, so it wasn't too bad, but having to spend the money to rent the items again when I did was pretty annoying.