Gitorious import: Overflow error in revision time

Error with the gitorious task import: {'args': {'directory': '/srv/storage/space/mirrors/gitorious.org/mnt/repositories/nginx-catap/mainline.git', 'date': 'Wed, 30 Mar 2016 09:40:04 +0200', 'origin_url': 'https://gitorious.org/nginx-catap/mainline.git'}, 'exception': "OverflowError('timestamp out of range for platform time_t',)"}

Trying to reproduce, i have not exactly that error though.

Repository: /srv/storage/space/mirrors/gitorious.org/mnt/repositories/nginx-catap/mainline.git

repo = 'mainline.git'
origin_url = 'http://nginx-catap/mainline.git'

import logging
logging.basicConfig(level=logging.DEBUG)

from swh.loader.git.tasks import LoadDiskGitRepository

t = LoadDiskGitRepository()
t.run(origin_url=origin_url, directory=repo, date='2016-05-03T15:16:32+00:00')

Output:

python3
Python 3.5.3 (default, Jan 19 2017, 14:11:04)
[GCC 6.3.0 20170118] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> repo = 'mainline.git'
>>> origin_url = 'http://nginx-catap/mainline.git'
>>>
>>> import logging
>>> logging.basicConfig(level=logging.DEBUG)
>>>
>>> from swh.loader.git.tasks import LoadDiskGitRepository

>>>
>>> t = LoadDiskGitRepository()
>>> t.run(origin_url=origin_url, directory=repo, date='2016-05-03T15:16:32+00:00')
DEBUG:swh.scheduler.task.LoadDiskGitRepository:Creating git origin for http://nginx-catap/mainline.git
DEBUG:swh.scheduler.task.LoadDiskGitRepository:Done creating git origin for http://nginx-catap/mainline.git
DEBUG:swh.scheduler.task.LoadDiskGitRepository:Sending 8230 contents
DEBUG:swh.scheduler.task.LoadDiskGitRepository:Done sending 8230 contents
DEBUG:swh.scheduler.task.LoadDiskGitRepository:Sending 4751 directories
DEBUG:swh.scheduler.task.LoadDiskGitRepository:Done sending 4751 directories
DEBUG:swh.scheduler.task.LoadDiskGitRepository:Sending 847 revisions
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/tony/work/inria/repo/swh/swh-environment/swh-scheduler/swh/scheduler/task.py", line 35, in run
    raise e from None
  File "/home/tony/work/inria/repo/swh/swh-environment/swh-scheduler/swh/scheduler/task.py", line 32, in run
    result = self.run_task(*args, **kwargs)
  File "/home/tony/work/inria/repo/swh/swh-environment/swh-loader-git/swh/loader/git/tasks.py", line 39, in run_task
    return loader.load(origin_url, directory, dateutil.parser.parse(date))
  File "/home/tony/work/inria/repo/swh/swh-environment/swh-loader-git/swh/loader/git/base.py", line 434, in load
    self.send_all_revisions(self.get_revisions())
  File "/home/tony/work/inria/repo/swh/swh-environment/swh-loader-git/swh/loader/git/base.py", line 370, in send_all_revisions
    send_in_packets(revisions, self.send_revisions, packet_size)
  File "/home/tony/work/inria/repo/swh/swh-environment/swh-loader-git/swh/loader/git/base.py", line 40, in send_in_packets
    sender(formatted_objects)
  File "/usr/lib/python3/dist-packages/retrying.py", line 49, in wrapped_f
    return Retrying(*dargs, **dkw).call(f, *args, **kw)
  File "/usr/lib/python3/dist-packages/retrying.py", line 206, in call
    return attempt.get(self._wrap_exception)
  File "/usr/lib/python3/dist-packages/retrying.py", line 247, in get
    six.reraise(self.value[0], self.value[1], self.value[2])
  File "/usr/lib/python3/dist-packages/six.py", line 686, in reraise
    raise value
  File "/usr/lib/python3/dist-packages/retrying.py", line 200, in call
    attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
  File "/home/tony/work/inria/repo/swh/swh-environment/swh-loader-git/swh/loader/git/base.py", line 279, in send_revisions
    self.storage.revision_add(revision_list)
  File "/home/tony/work/inria/repo/swh/swh-environment/swh-storage/swh/storage/storage.py", line 640, in revision_add
    lambda rev: parents_filtered.extend(rev['parents']))
  File "/home/tony/work/inria/repo/swh/swh-environment/swh-storage/swh/storage/db.py", line 178, in copy_to
    for d in items:
  File "/home/tony/work/inria/repo/swh/swh-environment/swh-storage/swh/storage/storage.py", line 633, in <genexpr>
    if revision['id'] in revisions_missing)
  File "/home/tony/work/inria/repo/swh/swh-environment/swh-storage/swh/storage/converters.py", line 166, in revision_to_db
    date = date_to_db(revision['date'])
  File "/home/tony/work/inria/repo/swh/swh-environment/swh-storage/swh/storage/converters.py", line 150, in date_to_db
    timestamp = datetime.datetime.fromtimestamp(seconds, datetime.timezone.utc)
ValueError: year is out of range

---

Migrated from T823 (view on Phabricator)