Using os.walk() does not make much sense when we want to control what
directories to recurse into.

Additionally, this uses os.scandir directly, which allows us to directly
sort symlinks and files apart from directories (while os.walk groups
symlinks with directories) without two extra system calls.