cassandra `min_index_interval` value too small for a couple of tables
It seems a limit is reached on cassandra11 regarding the value of min_index_interval for the content_by_blake2s256 and content_by_sha256 tables.
Those logs are traced during the server restart. It indicates cassandra can't read the index from disk and need to recompute en whole sstables. It takes 15mn for one and 10 minutes for the other delaying the server restart.
cassandra11 ~ % grep -A1 -B1 min_index_interval /var/log/cassandra/instance1/system.log
INFO [SSTableBatchOpen:8] 2025-02-06 18:06:40,760 SortedTableReaderLoadingBuilder.java:55 - Opening /srv/cassandra/instance1/data/swh/content_by_blake2s256-bb6de3802d2e11ed970a612d80206516/nb-50732-big (15.696GiB)
WARN [SSTableBatchOpen:6] 2025-02-06 18:06:44,343 IndexSummaryBuilder.java:115 - min_index_interval of 128 is too low for 5373707200 expected keys of avg size 64; using interval of 181 instead
INFO [SSTableBatchOpen:6] 2025-02-06 18:21:08,577 SortedTableReaderLoadingBuilder.java:55 - Opening /srv/cassandra/instance1/data/swh/content_by_blake2s256-bb6de3802d2e11ed970a612d80206516/nb-23703-big (247.645GiB)
--
INFO [SSTableBatchOpen:13] 2025-02-06 18:21:15,638 SortedTableReaderLoadingBuilder.java:55 - Opening /srv/cassandra/instance1/data/swh/content_by_sha256-bab011c02d2e11edb23fb91705299a22/nb-38306-big (27.396GiB)
WARN [SSTableBatchOpen:22] 2025-02-06 18:21:19,064 IndexSummaryBuilder.java:115 - min_index_interval of 128 is too low for 4391992028 expected keys of avg size 64; using interval of 148 instead
INFO [SSTableBatchOpen:22] 2025-02-06 18:32:52,312 SortedTableReaderLoadingBuilder.java:55 - Opening /srv/cassandra/instance1/data/swh/content_by_sha256-bab011c02d2e11edb23fb91705299a22/nb-38303-big (189.048GiB)Some resources regarding this parameter:
- The configuration is per table via an alter: https://cassandra.apache.org/doc/5.0/cassandra/reference/cql-commands/create-table.html
- lower value: faster reads, less memory, higher value: faster writes, more memory
-
https://stackoverflow.com/questions/73544362/cassandra-service-startup-delays-with-warn-message-min-index-interval-of-128-is
- Changing the compaction type could help (?)
- Changing the value of
min_index_intervalimpacts only the new sstables
- To check: The size of the partitions of these tables
