npm: Add workaround for mangled package descriptions
Null bytes in JSON produced by indexers cause the indexer-storage to crash (#4277 (closed)), and this case seems to be the only current source of such crashes; so this should fix the issue for now.
A future commit will sanitize all JSON documents before storage.
Migrated from D7992 (view on Phabricator)
Merge request reports
Activity
Build is green
Patch application report for D7992 (id=28796)
Rebasing onto 71046713...
Current branch diff-target is up to date.Changes applied before test
commit 026e9fb86c138df9f62473a0a03cf1cf9fd1617b Author: Valentin Lorentz <vlorentz@softwareheritage.org> Date: Wed Jun 15 18:29:09 2022 +0200 npm: Add workaround for mangled package descriptions Null bytes in JSON produced by indexers cause the indexer-storage to crash, and this case seems to be the only current source of such crashes; so this should fix the issue for now. A future commit will sanitize all JSON documents before storage.See https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/255/ for more details.
-
Build has FAILED
Patch application report for D7992 (id=28802)
Rebasing onto 71046713...
Current branch diff-target is up to date.Changes applied before test
commit 92fc18eae4c239e0c5811060d706986ea54063f7 Author: Valentin Lorentz <vlorentz@softwareheritage.org> Date: Wed Jun 15 18:29:09 2022 +0200 npm: Add workaround for mangled package descriptions Null bytes in JSON produced by indexers cause the indexer-storage to crash, and this case seems to be the only current source of such crashes; so this should fix the issue for now. A future commit will sanitize all JSON documents before storage.Link to build: https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/256/ See console output for more information: https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/256/console
-
Build has FAILED
Patch application report for D7992 (id=28803)
Rebasing onto 71046713...
Current branch diff-target is up to date.Changes applied before test
commit 4c1a829a6b3331c89a0af1c1234b58e271c25b52 Author: Valentin Lorentz <vlorentz@softwareheritage.org> Date: Wed Jun 15 18:29:09 2022 +0200 npm: Add workaround for mangled package descriptions Null bytes in JSON produced by indexers cause the indexer-storage to crash, and this case seems to be the only current source of such crashes; so this should fix the issue for now. A future commit will sanitize all JSON documents before storage.Link to build: https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/257/ See console output for more information: https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/257/console
-
Build is green
Patch application report for D7992 (id=28804)
Rebasing onto 71046713...
Current branch diff-target is up to date.Changes applied before test
commit 38bd1f45cda2599730821b07d446c9c37f0d8af1 Author: Valentin Lorentz <vlorentz@softwareheritage.org> Date: Wed Jun 15 18:29:09 2022 +0200 npm: Add workaround for mangled package descriptions Null bytes in JSON produced by indexers cause the indexer-storage to crash, and this case seems to be the only current source of such crashes; so this should fix the issue for now. A future commit will sanitize all JSON documents before storage.See https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/258/ for more details.
169 # is a common mistake, which indicates a README.md was saved as UTF-16, 170 # and some NPM tool opened it as UTF-8 and used the first line as 171 # description. 172 173 description_bytes = description.encode() 174 175 # Strip the the two unicode replacement characters 176 assert description_bytes.startswith(b"\xef\xbf\xbd\xef\xbf\xbd") 177 description_bytes = description_bytes[6:] 178 179 # If the following attempts fail to recover the description, discard it 180 # entirely because the current indexer storage backend (postgresql) cannot 181 # store zero bytes in JSON columns. 182 description = None 183 184 if not description_bytes.startswith(b"\x00"): mentioned in issue #4277 (closed)
-
Build is green
Patch application report for D7992 (id=28808)
Rebasing onto 71046713...
Current branch diff-target is up to date.Changes applied before test
commit 62db0cb2086be7a4c6c48f6488fb425485e56093 Author: Valentin Lorentz <vlorentz@softwareheritage.org> Date: Wed Jun 15 18:29:09 2022 +0200 npm: Add workaround for mangled package descriptions Null bytes in JSON produced by indexers cause the indexer-storage to crash, and this case seems to be the only current source of such crashes; so this should fix the issue for now. A future commit will sanitize all JSON documents before storage.See https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/259/ for more details.
