loader: Fix empty pack file handling since dulwich 0.21

dulwich >= 0.21 raises an AssertionError when the pack is empty
rather than setting the PackData version and length to None.

So do not try to access the pack file when its size is 0.

Fixes #4747