Nothing can beat the GameBoy Advance SP. First off, you have the GameBoy Advance system as it first began... a portable 32-bit console. At least for me, the notion was mindblowing, and I've consistently, including to this day, seriously dug the visuals on GBA games. The SP was the same thing, but with the sleekest, tightest, most perfect design for any handheld console ever. I really can't get over the design. Sleek, stylish, small, and with everything that was the GBA packed onto it. Very nice. Only thing it was missing was the speaker jack, as I see it anyway. It was also the first handheld console that I was aware of to have a backlit screen, which is now a necessity.
My favorite Nintendo console is now the original DS, for a variety of other reasons, but in terms of Nintendo handhelds, the GBA SP is right behind it, and it's definitely the best of the actual GameBoy systems.