UniqueViolation: duplicate key value violates unique constraint ... (May 2023 edition)

Sentry Issue: (directory_pkey) SWH-STORAGE-2WXZ

UniqueViolation: duplicate key value violates unique constraint "directory_pkey"
DETAIL:  Key (id)=(\x6c0a3adce8ee0bd09d6a29f82e7eb85a50e0e46e) already exists.
CONTEXT:  SQL statement "insert into directory (id, dir_entries, file_entries, rev_entries, raw_manifest)
    select id, dir_entries, file_entries, rev_entries, raw_manifest from tmp_directory t
    where not exists (
        select 1 from directory d
	where d.id = t.id)"
PL/pgSQL function swh_directory_add() line 7 at SQL statement

(21 additional frame(s) were not displayed)
...
  File "swh", line 8, in <module>
    sys.exit(main())

There are very consistently 10-15 such events every hour, for the past three days

changed title from UniqueViolation: duplicate key value violates unique constraint "directory_pkey" to UniqueViolation: duplicate key value violates unique constraint ... (May 2023 edition)

changed the description

mentioned in issue #4685 (closed)

Sentry issue (skipped_content_add): SWH-STORAGE-2WXX

Sentry issue (release_pkey): SWH-STORAGE-1EX

Sentry issue (directory_entry_file_target_name_perms_id): SWH-LOADER-GIT-1M1

Sentry issue (revision_pkey): SWH-LOADER-BZR-77

mentioned in commit swh/infra/ci-cd/swh-charts@b1cdb499

It should be fixed with the addition of the buffer filter on the replayer configurations

/cc @guillaume

oh, so it was caused by the lack of buffer proxy?

do you think it's because of the fewer transactions, or because the buffer proxy also deduplicates?

closed