Skip to content

utils: Fix some author parsing errors

Antoine Lambert requested to merge generated-differential-D1494-source into generated-differential-D1494-target

Digging into the production logs of the npm loader, the following errors are reported:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/swh/loader/core/loader.py", line 895, in load
    more_data_to_fetch = self.fetch_data()
  File "/usr/lib/python3/dist-packages/swh/loader/npm/loader.py", line 203, in fetch_data
    data = next(self.new_versions)
  File "/usr/lib/python3/dist-packages/swh/loader/npm/client.py", line 149, in prepare_package_versions
    version_data)
  File "/usr/lib/python3/dist-packages/swh/loader/npm/client.py", line 210, in _prepare_package_version
    author = extract_npm_package_author(package_json)
  File "/usr/lib/python3/dist-packages/swh/loader/npm/utils.py", line 95, in extract_npm_package_author
    author_data = parse_npm_package_author(author_str)
  File "/usr/lib/python3/dist-packages/swh/loader/npm/utils.py", line 52, in parse_npm_package_author
    author_str.replace('<>', '').replace('()', ''),
AttributeError: 'list' object has no attribute 'replace'
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/swh/loader/core/loader.py", line 895, in load
    more_data_to_fetch = self.fetch_data()
  File "/usr/lib/python3/dist-packages/swh/loader/npm/loader.py", line 203, in fetch_data
    data = next(self.new_versions)
  File "/usr/lib/python3/dist-packages/swh/loader/npm/client.py", line 149, in prepare_package_versions
    version_data)
  File "/usr/lib/python3/dist-packages/swh/loader/npm/client.py", line 210, in _prepare_package_version
    author = extract_npm_package_author(package_json)
  File "/usr/lib/python3/dist-packages/swh/loader/npm/utils.py", line 97, in extract_npm_package_author
    author_str = _author_str(package_json['authors'][0])
KeyError: 0

The first error happens when the author field in a package.json file corresponds to a list instead of a string, while the second one happens when the author field corresponds to a dict instead of a string,

Migrated from D1494 (view on Phabricator)

Merge request reports