Gitorious import: Overflow error in revision time
Error with the gitorious task import: {'args': {'directory': '/srv/storage/space/mirrors/gitorious.org/mnt/repositories/nginx-catap/mainline.git', 'date': 'Wed, 30 Mar 2016 09:40:04 +0200', 'origin_url': 'https://gitorious.org/nginx-catap/mainline.git'}, 'exception': "OverflowError('timestamp out of range for platform time_t',)"}
Trying to reproduce, i have not exactly that error though.
Repository: /srv/storage/space/mirrors/gitorious.org/mnt/repositories/nginx-catap/mainline.git
repo = 'mainline.git'
origin_url = 'http://nginx-catap/mainline.git'
import logging
logging.basicConfig(level=logging.DEBUG)
from swh.loader.git.tasks import LoadDiskGitRepository
t = LoadDiskGitRepository()
t.run(origin_url=origin_url, directory=repo, date='2016-05-03T15:16:32+00:00')Output:
python3
Python 3.5.3 (default, Jan 19 2017, 14:11:04)
[GCC 6.3.0 20170118] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> repo = 'mainline.git'
>>> origin_url = 'http://nginx-catap/mainline.git'
>>>
>>> import logging
>>> logging.basicConfig(level=logging.DEBUG)
>>>
>>> from swh.loader.git.tasks import LoadDiskGitRepository
>>>
>>> t = LoadDiskGitRepository()
>>> t.run(origin_url=origin_url, directory=repo, date='2016-05-03T15:16:32+00:00')
DEBUG:swh.scheduler.task.LoadDiskGitRepository:Creating git origin for http://nginx-catap/mainline.git
DEBUG:swh.scheduler.task.LoadDiskGitRepository:Done creating git origin for http://nginx-catap/mainline.git
DEBUG:swh.scheduler.task.LoadDiskGitRepository:Sending 8230 contents
DEBUG:swh.scheduler.task.LoadDiskGitRepository:Done sending 8230 contents
DEBUG:swh.scheduler.task.LoadDiskGitRepository:Sending 4751 directories
DEBUG:swh.scheduler.task.LoadDiskGitRepository:Done sending 4751 directories
DEBUG:swh.scheduler.task.LoadDiskGitRepository:Sending 847 revisions
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/tony/work/inria/repo/swh/swh-environment/swh-scheduler/swh/scheduler/task.py", line 35, in run
raise e from None
File "/home/tony/work/inria/repo/swh/swh-environment/swh-scheduler/swh/scheduler/task.py", line 32, in run
result = self.run_task(*args, **kwargs)
File "/home/tony/work/inria/repo/swh/swh-environment/swh-loader-git/swh/loader/git/tasks.py", line 39, in run_task
return loader.load(origin_url, directory, dateutil.parser.parse(date))
File "/home/tony/work/inria/repo/swh/swh-environment/swh-loader-git/swh/loader/git/base.py", line 434, in load
self.send_all_revisions(self.get_revisions())
File "/home/tony/work/inria/repo/swh/swh-environment/swh-loader-git/swh/loader/git/base.py", line 370, in send_all_revisions
send_in_packets(revisions, self.send_revisions, packet_size)
File "/home/tony/work/inria/repo/swh/swh-environment/swh-loader-git/swh/loader/git/base.py", line 40, in send_in_packets
sender(formatted_objects)
File "/usr/lib/python3/dist-packages/retrying.py", line 49, in wrapped_f
return Retrying(*dargs, **dkw).call(f, *args, **kw)
File "/usr/lib/python3/dist-packages/retrying.py", line 206, in call
return attempt.get(self._wrap_exception)
File "/usr/lib/python3/dist-packages/retrying.py", line 247, in get
six.reraise(self.value[0], self.value[1], self.value[2])
File "/usr/lib/python3/dist-packages/six.py", line 686, in reraise
raise value
File "/usr/lib/python3/dist-packages/retrying.py", line 200, in call
attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
File "/home/tony/work/inria/repo/swh/swh-environment/swh-loader-git/swh/loader/git/base.py", line 279, in send_revisions
self.storage.revision_add(revision_list)
File "/home/tony/work/inria/repo/swh/swh-environment/swh-storage/swh/storage/storage.py", line 640, in revision_add
lambda rev: parents_filtered.extend(rev['parents']))
File "/home/tony/work/inria/repo/swh/swh-environment/swh-storage/swh/storage/db.py", line 178, in copy_to
for d in items:
File "/home/tony/work/inria/repo/swh/swh-environment/swh-storage/swh/storage/storage.py", line 633, in <genexpr>
if revision['id'] in revisions_missing)
File "/home/tony/work/inria/repo/swh/swh-environment/swh-storage/swh/storage/converters.py", line 166, in revision_to_db
date = date_to_db(revision['date'])
File "/home/tony/work/inria/repo/swh/swh-environment/swh-storage/swh/storage/converters.py", line 150, in date_to_db
timestamp = datetime.datetime.fromtimestamp(seconds, datetime.timezone.utc)
ValueError: year is out of rangeMigrated from T823 (view on Phabricator)