crates: Bump csv field size limit

A size limit of 1000000 was not enough to properly process all CSV crates data so bump to a higher value.

crates: Bump csv field size limit
b2ece7ca · Antoine Lambert · f7abfaff · b2ece7ca
Commit b2ece7ca authored 7 months ago by Antoine Lambert
--- a/swh/lister/crates/lister.py
+++ b/swh/lister/crates/lister.py
@@ -131,7 +131,7 @@ class CratesLister(Lister[CratesListerState, CratesListerPage]):
                        members_to_extract.append(member)
                tf.extractall(members=members_to_extract, path=db_dump_path)

-            csv.field_size_limit(1000000)
+            csv.field_size_limit(10000000)

            (crates_csv_path,) = list(db_dump_path.glob("*/data/crates.csv"))
            (versions_csv_path,) = list(db_dump_path.glob("*/data/versions.csv"))