Just talk to the guy at the counter of a stable. 20 Rupees to name it and get a saddle + be able to board it. You can take it out at any stable, abd if you left your horse in the wild somewhere you can still take it out of the stable and they get it for you.