to_disk: Use a BFS to recursively list a directory instead of a DFS

It enables to push new files to download asynchronously while fetching
sub-directories and thus slightly improve the overall cooking performance.

It should also reduce the memory consumption of the cooking process.