Rename the project as swh-shard, migrate to pybind11 and add cli tools

It does not make much sense to call it perfecthash, since the aim of this package is creating, reading and manipulating shard files (which do use cmph to speed extracting content objects from the shard file, but this is an implementation detail, really).

Use pybind11 to wrap the cmph and shard manipulation code instead of cffi, it makes is a bit easier to add (C/C++) features in the extension.

Add a cli tool to manipulate shard files. Currently, it allows to:

• read the header of the shard file
• list entries in the shard file (as a list of {key: length})
• get an object from a shard file
• create a shard file from a list of files
• delete one (or more) entry from a shard

The extension source files have been moved to src/_shard, and the python source files for the swh.shard package have been moved to src/swh/shard, moving away from all other swh package structure. This is required to prevent side effects of having the local 'swh' directory in the working directory of the developer, thus in the sys.path (by default), breaking the dark magic involved in the loading of the package when it is installed in editable mode (i.e. not to break pytest when executed directly from the source with the package being installed in editable mode).

extra-tox-environments: pypy310

docs/format.rst

@@ -8,3 +8,58 @@ The Read Shard has the following structure:
 * bytes \[``objects_position``, ``index_position``\[: ``objects_count`` times the size of the object (``u_int64_t``) followed by the content of the object
 * bytes \[``index_position``, ``hash_position``\[: An array of index entries. The size of the array is provided by ``cmph_size`` after building the hash function. An index entry is made of the key (of ``SHARD_KEY_LEN`` bytes) and the object position (``u_int64_t``) in the range \[``objects_position``, ``index_position``\[. If the object position is ``UINT64_MAX``, this means the object has been deleted.
 * bytes \[``hash_position``, ...\[: The hash function, as written by ``cmph_dump``
+
+In more details:
+
++--------------------------+------+----------------------------+
+| Section                  | pos  | description (length)       |
++==========================+======+============================+
+| **SHARD_MAGIC**          | 0    | SHARD_OFFSET_MAGIC (32)    |
++--------------------------+------+----------------------------+
+| **header**               | 32   | Header (56)                |
++--------------------------+------+----------------------------+
+| ``version``              |      | uint64_t (8)               |
++--------------------------+------+----------------------------+
+| ``objects_count``        |      | uint64_t (8)               |
++--------------------------+------+----------------------------+
+| ``objects_position`` <op>|      | uint64_t (8)               |
++--------------------------+------+----------------------------+
+| ``objects_size``         |      | uint64_t (8)               |
++--------------------------+------+----------------------------+
+| ``index_position`` <ip>  |      | uint64_t (8)               |
++--------------------------+------+----------------------------+
+| ``index_size``           |      | uint64_t (8)               |
++--------------------------+------+----------------------------+
+| ``hash_position`` <hp>   |      | uint64_t (8)               |
++--------------------------+------+----------------------------+
+| **Objects**              | <op> |                            |
++--------------------------+------+----------------------------+
+| ``object0 size``         |      | uint64_t (8)               |
++--------------------------+------+----------------------------+
+| ``object0 data``         |      | bytes (<object0 size>)     |
++--------------------------+------+----------------------------+
+| ``object1 size``         |      | uint64_t (8)               |
++--------------------------+------+----------------------------+
+| ``object1 data``         |      | bytes (<object1 size>      |
++--------------------------+------+----------------------------+
+|   ...                    |      |                            |
++--------------------------+------+----------------------------+
+| **Index**                | <ip> |                            |
++--------------------------+------+----------------------------+
+| ``object0 key``          |      | SHARD_KEY_LEN (32)         |
++--------------------------+------+----------------------------+
+| ``object0 offset``       |      | uint64_t (8)               |
++--------------------------+------+----------------------------+
+|   ...                    |      |                            |
++--------------------------+------+----------------------------+
+| **Hash map**             | <hp> |                            |
++--------------------------+------+----------------------------+
+| ``hash function``        |      | <as written by cmph_dump>  |
++--------------------------+------+----------------------------+
+
+
+``SHARD_MAGIC`` is the constant ``SWHShard`` (with ``\x00`` padding to 32
+characters).
+
+Index entries for deleted content are using the special value
+``{key=\x00...\x00, offset=2**64-1}``.