from_disk: Do not recurse in ignored directories

Using os.walk() does not make much sense when we want to control what directories to recurse into.

Additionally, this uses os.scandir directly, which allows us to directly sort symlinks and files apart from directories (while os.walk groups symlinks with directories) without two extra system calls.

Includes commits from !359 (merged)