swh.indexer.metadata should not use directory_ls

The swh.indexer.metadata module uses self.storage.directory_ls() in order to get all entries in a directory.

However, there may be many entries, causing the server to crash because of timeouts (which is why this method is deprecated). Even if it worked, it could take a significant amount of memory.

Instead, swh.indexer.metadata should use self.storage.directory_get_entries(), which supports pagination.