swh/scanner : Strip root path from json output

Fixes #2731 (closed)

---

Migrated from D5213 (view on Phabricator)

Build has FAILED

Patch application report for D5213 (id=18670)

Rebasing onto 433f5563...

Current branch diff-target is up to date.

Changes applied before test

commit c5b36a731047234588fcd2336ff8058129d51948
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Sat Mar 6 20:14:38 2021 +0530

    Strip root path from json output

Link to build: https://jenkins.softwareheritage.org/job/DTSCN/job/tests-on-diff/104/ See console output for more information: https://jenkins.softwareheritage.org/job/DTSCN/job/tests-on-diff/104/console

Updating !48 (closed): swh/scanner : Strip root path from json output

Build has FAILED

Patch application report for D5213 (id=18671)

Rebasing onto 433f5563...

Current branch diff-target is up to date.

Changes applied before test

commit 2ce1b5d929d631fd5f64cd3d54f759802b6eb97c
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Sat Mar 6 20:14:38 2021 +0530

    Strip root path from json output

Link to build: https://jenkins.softwareheritage.org/job/DTSCN/job/tests-on-diff/105/ See console output for more information: https://jenkins.softwareheritage.org/job/DTSCN/job/tests-on-diff/105/console

All the tests are passing when !47 (closed) is applied. It is expected that the build errors will get fixed once !47 (closed) is merged.

Tests are still failing on this, so some changes are needed for sure.

In addition, I'm suggesting above to change the interface of the model so that Path is still used, instead of dumbing down path to strings there.

Build has FAILED

Patch application report for D5213 (id=18671)

Rebasing onto 6c191b6e...

First, rewinding head to replay your work on top of it...
Applying: Strip root path from json output

Changes applied before test

commit 451baabd2f21aedfb1fbbd45e93e092ca49f63a0
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Sat Mar 6 20:14:38 2021 +0530

    Strip root path from json output

Link to build: https://jenkins.softwareheritage.org/job/DTSCN/job/tests-on-diff/106/ See console output for more information: https://jenkins.softwareheritage.org/job/DTSCN/job/tests-on-diff/106/console

Tests are still failing, but that's caused by an aiohttp released published last saturday with an invalid version number

Updating !48 (closed): Use Path.relative_to instead of str.replace

Build is green

Patch application report for D5213 (id=18711)

Rebasing onto 41c57740...

Current branch diff-target is up to date.

Changes applied before test

commit ff4d4acfb5c54825ed8d506eaed36be503a7a344
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Sat Mar 6 20:14:38 2021 +0530

    Strip root path from json output

See https://jenkins.softwareheritage.org/job/DTSCN/job/tests-on-diff/109/ for more details.

Thanks, this fix looks good.

However, it introduces an inconsistency between the json (which now strips the root path) and the text output (which does not strip it).

Can you make the textual output behave the same? (and retitle this diff accordingly)

Merge request was returned for changes

Hi @zack In my opinion, the textual output doesn't print anything other than the directory structure so instead of removing the whole root path we can just put the directory name (it looks better).

website-folder
│   js/
│   │   main.js
│   │   table.js
│   .gitignore
│   table.html
│   index.html
│   login.html

VS

│   js/
│   │   main.js
│   │   table.js
│   .gitignore
│   table.html
│   index.html
│   login.html

But the ndjson output must be stripped.

Updating !48 (closed): Fix ndjson and textual output

Build is green

Patch application report for D5213 (id=18724)

Rebasing onto 41c57740...

Current branch diff-target is up to date.

Changes applied before test

commit 9154ec2b8fd5347dec09131e190f5fb82b74c3d6
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Sat Mar 6 20:14:38 2021 +0530

    Clean path in scan output
    
    Clean scan outputs for ndjson,json and text format

See https://jenkins.softwareheritage.org/job/DTSCN/job/tests-on-diff/110/ for more details.

! In !48 (closed), @KShivendu wrote: In my opinion, the textual output doesn't print anything other than the directory structure so instead of removing the whole root path we can just put the directory name (it looks better). [...] But the ndjson output must be stripped.

OK, I'm sold, thanks!

But please also add yourself to CONTRIBUTORS as part of this diff. Then I'll be happy to accept this one.

Also, minor point, we usually use "Closes XXX" in commit messages rather than "Fixes XXX". (They probably both work, but I haven't checked, and consistency is better anyway :-))

Merge request was returned for changes

Updating !48 (closed): swh/scanner : Add Kumar Shivedu to CONTRIBUTORS

Build is green

Patch application report for D5213 (id=18736)

Rebasing onto 41c57740...

Current branch diff-target is up to date.

Changes applied before test

commit 9c9b39892bb36f9c2c03313ffbcc7f41cc2e0a31
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Sat Mar 6 20:14:38 2021 +0530

    CLI : Clean path in scan output
    
    Clean scan outputs for ndjson, json, and text formats

See https://jenkins.softwareheritage.org/job/DTSCN/job/tests-on-diff/111/ for more details.

Updating !48 (closed): swh/scanner : Add newline in CONTRIBUTORS file

Build is green

Patch application report for D5213 (id=18737)

Rebasing onto 41c57740...

Current branch diff-target is up to date.

Changes applied before test

commit eb04424434021f39b8869b32a17c11c18872472f
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Sat Mar 6 20:14:38 2021 +0530

    CLI : Clean path in scan output
    
    Clean scan outputs for ndjson, json, and text formats

See https://jenkins.softwareheritage.org/job/DTSCN/job/tests-on-diff/112/ for more details.