Smoke test ingestion of bitbucket repositories with latest loader mercurial
Smoke test it with remote and local (archived from BitBucket) repositories. loader mercurial version v2.1
Migrated from T3337 (view on Phabricator)
Designs
- Show closed items
Activity
-
Newest first Oldest first
-
Show all activity Show comments only Show history only
- Phabricator Migration user marked this issue as related to #3336 (closed)
marked this issue as related to #3336 (closed)
added Mercurial loader System administration priority:High labels
- Phabricator Migration user marked this issue as related to #3338 (closed)
marked this issue as related to #3338 (closed)
- Antoine R. Dumont changed title from Deploy swh.loader.mercurial 1.0 in production to Smoke test ingestion of bitbucket repositories with latest loader mercurial
changed title from Deploy swh.loader.mercurial 1.0 in production to Smoke test ingestion of bitbucket repositories with latest loader mercurial
- Antoine R. Dumont changed the description
changed the description
- Owner
Smoke test it with remote repositories.
Well, our check end to end loading the hgview [1] repository is now green again and no longer flip-floping so that takes care of it.
Some other remote repositories were tested as well without any issues [2]
-
[2] migrated/migration$1100
Smoke test it with local bitbucket repositories
that's next.
- Antoine R. Dumont added state:wip label
added state:wip label
- Owner
Smoke test it with local bitbucket repositories
that's next.
It's working as expected [1].
This is ready to be deployed in production.
Here is their visit status result and analysis next to visit status that matters.
12:46:22 softwareheritage@belvedere:5432=> select o.url, ovs.date, ovs.status, ovs.snapshot from origin o inner join origin_visit_status ovs on o.id=ovs.origin where o.url in ('https://bitbucket.org/sunithacharan/pocketbook1', 'https://bit bucket.org/alienonland/ownershipandriskweb', 'https://bitbucket.org/alienonland/ownershipandriskweb/wiki', 'https://bitbucket.org/rarchk/jsmn', 'https://bitbucket.org/rarchk/jsmn/wiki', 'https://bitbucket.org/k4nagatsuki/cw_wsn1example-hg' , 'https://bitbucket.org/csimpson/pythononrails', 'https://bitbucket.org/tindelli/polytope') and status='full' order by origin desc,visit desc; +------------------------------------------------------------+-------------------------------+--------+--------------------------------------------+ | url | date | status | snapshot | +------------------------------------------------------------+-------------------------------+--------+--------------------------------------------+ | https://bitbucket.org/k4nagatsuki/cw_wsn1example-hg | 2021-07-30 10:30:36.532165+00 | full | (null) | 2nd run uneventful | https://bitbucket.org/k4nagatsuki/cw_wsn1example-hg | 2021-07-30 10:29:41.131538+00 | full | \x4e85afea09148d757966425d12026f99c40cd181 | 1st run evenfful | https://bitbucket.org/rarchk/jsmn/wiki | 2021-07-30 10:30:30.356632+00 | full | (null) | 2nd run uneventful | https://bitbucket.org/rarchk/jsmn/wiki | 2021-07-30 10:29:32.817241+00 | full | \x9731b94489737bc6a459a591dedcbeb4e2adde1d | 1st run evenfful | https://bitbucket.org/alienonland/ownershipandriskweb/wiki | 2021-07-30 10:30:26.201182+00 | full | (null) | 2nd run uneventful | https://bitbucket.org/alienonland/ownershipandriskweb/wiki | 2021-07-30 10:29:27.681315+00 | full | \x564df4553f129c77c7f683931929907266f9f072 | 2nd run uneventful | https://bitbucket.org/tindelli/polytope | 2021-07-30 10:43:12.058411+00 | full | (null) | 3rd run uneventful (i had another run independently from the paste, overlapping range) | https://bitbucket.org/tindelli/polytope | 2021-07-30 10:30:41.082185+00 | full | (null) | 2nd run uneventful | https://bitbucket.org/tindelli/polytope | 2021-07-30 10:30:07.399909+00 | full | \x4cfe3bcc2a05672715ca91b2634eaf7e923a5e71 | 1st run uneventful | https://bitbucket.org/tindelli/polytope | 2019-09-23 16:33:01.485429+00 | full | \x5d36d39afc5ff7685c8e3ad69229eb3a4add9e27 | previous snapshot prior to the new release so snapshot was different because of the "default" extid version 0 | https://bitbucket.org/tindelli/polytope | 2019-09-19 13:29:25.313042+00 | full | \x5d36d39afc5ff7685c8e3ad69229eb3a4add9e27 | ... | https://bitbucket.org/tindelli/polytope | 2019-09-11 13:28:52.070991+00 | full | \x5d36d39afc5ff7685c8e3ad69229eb3a4add9e27 | | https://bitbucket.org/tindelli/polytope | 2019-08-25 11:10:14.362887+00 | full | \x5d36d39afc5ff7685c8e3ad69229eb3a4add9e27 | | https://bitbucket.org/tindelli/polytope | 2019-07-06 03:26:54.404194+00 | full | \x5d36d39afc5ff7685c8e3ad69229eb3a4add9e27 | | https://bitbucket.org/rarchk/jsmn | 2021-07-30 10:30:28.378383+00 | full | (null) | | https://bitbucket.org/rarchk/jsmn | 2021-07-30 10:29:30.613391+00 | full | \xc07b7f6b2c84a59b2ed38f867831cec466db02cb | | https://bitbucket.org/rarchk/jsmn | 2019-12-10 11:18:00.14157+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-12-08 19:09:42.092199+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-12-07 05:46:00.746191+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-12-05 10:56:01.93435+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-12-02 20:46:13.740019+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-12-01 08:25:17.093066+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-11-29 18:12:02.422935+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-11-28 00:27:55.639441+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-11-26 06:37:10.417357+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-11-24 18:23:50.054938+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-11-23 08:01:28.024955+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-11-21 15:57:13.062139+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-11-19 21:53:58.539205+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-11-18 03:55:22.896125+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-11-16 16:12:55.214658+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-11-15 00:06:22.283524+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-11-13 04:10:10.904844+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-11-11 11:37:13.496159+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-11-09 22:42:06.663271+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-11-08 06:31:49.302642+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-11-06 13:37:37.053864+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-11-04 21:04:16.742105+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-11-01 21:49:58.214468+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-10-29 16:40:36.82471+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-10-17 07:24:10.446816+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-10-15 00:00:33.547458+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-10-12 07:08:42.233412+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-10-09 22:56:17.834132+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-10-07 21:31:22.227515+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-10-05 14:08:13.437975+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-10-03 06:17:19.175246+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-09-30 09:57:31.611388+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-09-26 00:01:59.286519+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-09-08 22:29:07.337007+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-08-23 12:34:55.080055+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-07-01 12:53:13.721234+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/sunithacharan/pocketbook1 | 2021-07-30 10:30:21.770151+00 | full | (null) | empty repository [2] | https://bitbucket.org/sunithacharan/pocketbook1 | 2021-07-30 10:29:22.310871+00 | full | (null) | | https://bitbucket.org/csimpson/pythononrails | 2021-07-30 10:30:38.998849+00 | full | (null) | same pattern as before... | https://bitbucket.org/csimpson/pythononrails | 2021-07-30 10:29:44.559665+00 | full | \x042f695a1768bff19b8dfdd497ad4f6504e0fc6f | | https://bitbucket.org/csimpson/pythononrails | 2019-12-10 17:42:53.270323+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-12-08 23:35:46.74614+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-12-07 10:55:28.856216+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-12-05 17:08:36.001726+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-12-03 02:07:46.069676+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-12-01 12:46:48.708118+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-11-29 23:23:17.158027+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-11-28 05:46:14.40683+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-11-26 12:20:18.439113+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-11-24 22:17:57.685404+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-11-23 12:39:16.899187+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-11-21 21:10:34.829363+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-11-20 02:42:45.542426+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-11-18 08:05:14.350113+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-11-16 20:41:21.411309+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-11-15 04:40:16.795394+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-11-13 10:13:04.192558+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-11-11 16:21:47.134495+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-11-10 02:49:53.025858+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-11-08 11:29:23.697376+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-11-06 18:48:27.53713+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-11-05 01:43:06.324477+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-11-02 06:24:22.09214+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-10-30 01:47:24.573198+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-10-17 17:33:33.105511+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-10-15 06:26:47.392683+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-10-12 13:37:10.587124+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-10-10 04:59:49.034164+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-10-08 03:51:15.326945+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-10-05 22:06:30.09897+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-10-03 23:20:14.156245+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-10-01 16:11:35.54927+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-09-29 01:41:35.957731+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-09-24 23:01:59.883073+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-09-07 09:03:17.9129+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-08-13 02:03:07.679798+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-06-30 15:38:58.06108+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/alienonland/ownershipandriskweb | 2021-07-30 10:30:24.049421+00 | full | (null) | | https://bitbucket.org/alienonland/ownershipandriskweb | 2021-07-30 10:29:25.438277+00 | full | \x4f429edb601ea9aa1ab1d48aab4e961c3eab159e | | https://bitbucket.org/alienonland/ownershipandriskweb | 2019-10-12 20:16:01.14027+00 | full | \x661d74b97dfd5519d8840e3f1fdd42978db10a98 | | https://bitbucket.org/alienonland/ownershipandriskweb | 2019-10-10 10:40:15.296953+00 | full | \x661d74b97dfd5519d8840e3f1fdd42978db10a98 | | https://bitbucket.org/alienonland/ownershipandriskweb | 2019-10-08 08:12:09.836132+00 | full | \x661d74b97dfd5519d8840e3f1fdd42978db10a98 | | https://bitbucket.org/alienonland/ownershipandriskweb | 2019-10-06 03:57:51.121655+00 | full | \x661d74b97dfd5519d8840e3f1fdd42978db10a98 | | https://bitbucket.org/alienonland/ownershipandriskweb | 2019-10-04 04:05:41.913737+00 | full | \x661d74b97dfd5519d8840e3f1fdd42978db10a98 | | https://bitbucket.org/alienonland/ownershipandriskweb | 2019-10-01 19:54:23.079177+00 | full | \x661d74b97dfd5519d8840e3f1fdd42978db10a98 | | https://bitbucket.org/alienonland/ownershipandriskweb | 2019-09-30 04:12:31.91622+00 | full | \x661d74b97dfd5519d8840e3f1fdd42978db10a98 | | https://bitbucket.org/alienonland/ownershipandriskweb | 2019-09-28 17:51:05.256742+00 | full | \x661d74b97dfd5519d8840e3f1fdd42978db10a98 | | https://bitbucket.org/alienonland/ownershipandriskweb | 2019-09-27 15:44:13.862242+00 | full | \x661d74b97dfd5519d8840e3f1fdd42978db10a98 | | https://bitbucket.org/alienonland/ownershipandriskweb | 2019-09-26 03:56:42.959891+00 | full | \x661d74b97dfd5519d8840e3f1fdd42978db10a98 | | https://bitbucket.org/alienonland/ownershipandriskweb | 2019-09-24 00:32:53.056976+00 | full | \x661d74b97dfd5519d8840e3f1fdd42978db10a98 | | https://bitbucket.org/alienonland/ownershipandriskweb | 2019-09-22 20:49:03.454986+00 | full | \x661d74b97dfd5519d8840e3f1fdd42978db10a98 | | https://bitbucket.org/alienonland/ownershipandriskweb | 2019-09-20 19:37:44.966056+00 | full | \x661d74b97dfd5519d8840e3f1fdd42978db10a98 | | https://bitbucket.org/alienonland/ownershipandriskweb | 2019-09-08 15:52:01.564134+00 | full | \x661d74b97dfd5519d8840e3f1fdd42978db10a98 | | https://bitbucket.org/alienonland/ownershipandriskweb | 2019-08-31 01:27:53.877789+00 | full | \x661d74b97dfd5519d8840e3f1fdd42978db10a98 | | https://bitbucket.org/alienonland/ownershipandriskweb | 2019-07-30 01:52:57.936951+00 | full | \x661d74b97dfd5519d8840e3f1fdd42978db10a98 | | https://bitbucket.org/alienonland/ownershipandriskweb | 2019-06-28 01:51:50.188403+00 | full | \x661d74b97dfd5519d8840e3f1fdd42978db10a98 | +------------------------------------------------------------+-------------------------------+--------+--------------------------------------------+ (112 rows)
-
[1] migrated/migration$1101
-
[2] Emtpy repository
ardumont@worker16:/srv/storage/space/mirrors/boatbucket$ base_dir=/srv/storage/space/mirrors/boatbucket ardumont@worker16:/srv/storage/space/mirrors/boatbucket$ grep sunithacharan/pocketbook1 mapping-to-repos.txt refugees/data/f6/f6f2cb2b-7bb5-482a-9974-c86fc28a1ef8/main-repo.mercurial https://bitbucket.org/sunithacharan/pocketbook1 ardumont@worker16:/srv/storage/space/mirrors/boatbucket$ repo=$(grep sunithacharan/pocketbook1 mapping-to-repos.txt | awk '{print $1}') ardumont@worker16:/tmp% hg clone $base_dir/$repo not trusting file /srv/storage/space/mirrors/boatbucket/refugees/data/f6/f6f2cb2b-7bb5-482a-9974-c86fc28a1ef8/main-repo.mercurial/.hg/hgrc from untrusted user 1002, group aeviso destination directory: main-repo.mercurial no changes found updating to branch default 0 files updated, 0 files merged, 0 files removed, 0 files unresolved swhworker@worker0:/tmp$ cd main-repo.mercurial/ swhworker@worker0:/tmp/main-repo.mercurial$ hg log -> nothing to show here
-
- Antoine R. Dumont removed state:wip label
removed state:wip label
- Antoine R. Dumont closed
closed