The first time I got Epona that was the way I got out. Everything happened the exact same way.
Except the cutscene won't load to happen there if you weren't in a race.
So I was using UoT and took a look around the ranch in it. I could not find the actor cube of the brown horse, because all the actors are from child Link's time only. I could not remove the collision of the fence. So, what to do? Should I replace one of the actors for the horse? And what's the brown horse's file name? I guess I will have to look it all myself.