When Bill Joy created the vi text editor he used the ADM-3A terminal, which had the arrows on hjkl keys. Naturally he reused the same keys and the rest is history!
source
Read more posts by Adarsh, and follow Adarsh on Twitter.