Skip to content

package loaders: Make test failures more helpful

vlorentz requested to merge generated-differential-D6607-source into generated-differential-D6607-target

Some tests did the following:

  1. build a snapshot
  2. get the snapshot from the storage
  3. compare it with the expected snapshot
  4. get the origin visit from the storage and check it

If the loader built a wrong snapshot, the test fails at step 2, and the only information displayed is that the expected snapshot id does not exist, which is very unhelpful.

Instead, I reordered them as: 1, 4, 2, 3. This way, if a wrong snapshot is build by the loader, it is detected when comparing the visit, and pytest shows the two hashes. Then, the test can be modified to use the hash that is actually generated to show the actual snapshot.

This is consistent with what was already done in the pypi loader.

Additionally, I made the following changes:

  1. always check stats last (because a difference in numbers is hardly actionable without testing other objects)
  2. add a few more snapshot id checks in visits
  3. deduplicated a hardcoded snapshot id.

Migrated from D6607 (view on Phabricator)

Merge request reports