Kinda going through the same process myself with my D4 build. Just one random thing I noticed with both your builds: it seems at some point Fly's speed was increased, but the flight speed cap was not, as such, it's extremely easy to hit the flight speed cap at 50. By extremely easy I mean, if your level 50, and turn on an unslotted fly, you'll hit the cap.
I'm throwing run speed in swift and a single -End in fly.