Hmm ... that's a very tough one.
Kakariko Village as a Child in Ocarina of Time is a great place. The slow, laid back guitar music sets the mood as a small town. The people there are lovable, and the activities in town are most memorable as a child. The Cucco Lady's chicken game, the cow inside the house with the heart piece, the kid in the graveyard, etc. It seems when all the commerce comes to town, the town loses that flair of being small and doing their own thing.
Kokiri Forest is a cool location, though for the scenery and rather not the people and stuff.
I love the Fishing Pond. Even after I've recieved anything worthwhile, I'll just head in to waste time. The Kakariko Village music really sets the calming feel that a fishing hole should have. The game is simple and can be considered boring to some people, but I love it. I had no idea about the Golden Scale or the Sinking Lure. I accidentally'd both of those by spending so much time there.
Now, in Majora's Mask, North Clock Town is the best of both worlds. You still get the feel that you're in a busy town, but it's a small garden area for peace and stuff.
Ikana Castle is an interesting place with an even more interesting history. That place is great, and theorists seem to love Ikana in general. I'm no theorist, but I find the old Kingdom of Ikana interesting, myself. I won't delve into every detail of the castle to find out what happened to them or what they were exactly, but just the interesting history behind them makes that part of the game that much more interesting.