Skip to content

indexer.metadata: Warn and skip incomplete entries from the journal

Antoine R. Dumont requested to merge generated-differential-D8165-source into generated-differential-D8165-target

Detected through swh/infra/sysadm-environment#4406 (closed)

According to sentry, stuff like the 2nd item [1] can appear and it's not currently dealt with. This diff is an attempt to unstuck the extrinsic metadata journal client.

Related to #4412 (closed)

  • [1]
{
raw_extrinsic_metadata: [
{
authority: [Filtered], 
discovery_date: datetime.datetime(2021, 10, 18, 12, 43, 22, tzinfo=datetime.timezone.utc), 
fetcher: {"metadata":"{}","name":"'swh.loader.package.npm.loader.NpmLoader'","version":"'0.25.0'"}, 
format: 'replicate-npm-package-json', 
id: b'\x10u\xb6\xb8\xd7\xfe\xb1\x92^\xa7\xa30\xa5^\xf4\xa6\x8dpF\r', 
metadata: b'{"name": "@arkecosystem/platform-sdk-lsk", "description": "Cross-Platform Utilities for ARK Applications", "version": "0.9.362", "contributors": [], "license": "MIT", "main": "dist/index", "types": "dist/index", "scripts": {"build": "yarn clean && tsc", "build:watch": "yarn build -w", "build:docs": "typedoc --out docs src", "clean": "rimraf .coverage dist tmp", "test": "jest", "test:watch": "jest --watchAll", "coverage:report": "codecov", "publish": "yarn build && yarn npm publish --access public --tol..., 
origin: 'https://www.npmjs.com/package/@arkecosystem/platform-sdk-lsk', 
revision: 'swh:1:rev:77436d8960cc5c3997535a03711e34ca63cfd97b', 
target: 'swh:1:dir:18fd0df245b17e8552e32cb950db1d1837063fc2'
}, 
{
authority: [Filtered], 
discovery_date: datetime.datetime(2021, 10, 18, 12, 43, 22, tzinfo=datetime.timezone.utc)
}
]
}

Migrated from D8165 (view on Phabricator)

Merge request reports