[Updated]For really retro chiptune music, you’ll want an original Game Boy. But the new Nintendo Game Boy could be the first since the original to really work well for portable music making:
Now, the big question is, what does it sound like? Previous modern GBAs, strangely enough, actually sound worse than the original: audio output is noisy and thin. The new Game Boy Micro could be perfect for Game Boy music carts like Nanoloop when released in the fall. In the meantime, I’ll stick with my DS (for Nanoloop) and original Game Boy. Stay tuned.
Updated: Major downside is that, like the GBA SP, carts load from the bottom, so if you’re using original-style Game Boy cartridges, the select/start buttons just above the cartridge slot will be hard to reach. (Pointed out by several people on the LSDj list.) This won’t be an issue if you’re using Nanoloop 2.0 or other GBA cartridges.
For more info on the Micro, see Nintendo’s excellent Japanese site (worth it even if you can’t read the text)