Tags give the ability to mark specific points in history as being important
-
v8.0.7
317bfdc3 · ·v8.0.7 Bug fixes (Python): * Rename to a more explicit name the tool that maps author IDs to full names * Rename end-to-end tests to end-to-end checks * Rework end-to-end checks so that they can work on different graphs * Add checks for the mapping between author IDs and full names * Fix luigi pipeline when compressing a graph with sensitive data Improvements (Python): * CLI: Improve the rpc-serve and grpc-serve help
-
v8.0.5
56a11f30 · ·v8.0.5 Bug fixes (Python): * Add missing propagation of local_sensitive_graph_path * Add missing docstring that makes Sphinx error * Change default LLP gammas to match production Soundness fixes (Rust): * Fix unaligned array in in-memory Node2Type * Fix unaligned array reads in GOV and GOV * Add workarounds to support Miri
-
v8.0.0
c8034bed · ·v8.0.0 Breaking changes (Python): * Remove SENSITIVE_DIR from example dataset Breaking changes (Rust): * Split NodeSet trait into ReadNodeSet and NodeSet * Move swh_graph::collections to swh_graph_stdlib New features (Python): * Add SubdatasetOriginsFromFile * Add CLI to symlink a graph (with option to copy some files) * Add default values to config checking inner function * Generalize SWHID::from_origin_url to support borrowed strings * http_client: Enable requests retry on transient errors New features (Rust): * Make id2fullname an integral part of swh-graph * stdlib: Introduce tree diff utility * Add SmallNodeSet * Add ContiguousSubgraph * Add swh-subgrapher script to build the list of SWHIDs for a subdataset * Inline DataFilesAvailability methods * Make iter_nodes support arbitrary IntoIterators instead of just slice. * Relax unnecessary SwhLabeledForwardGraph constraint to SwhForwardGraph Fixes (Python): * download: Execute zstdmt in quiet mode to avoid polluting stdout * Don't ignore configured batch size * Fix mismatch of object_types names between swh-graph and swh-export Fixes (Rust): * Make 'arrow' an optional dependency of swh-graph-topology Internal: * Refactor properties backends to provide indexing themselves * Use `shell.py` to run the compression pipeline * stdlib: Add missing crate metadata
-
v7.0.0
3ff4c945 · ·v7.0.0 Breaking changes (gRPC server): * Fix inconsistent extension for stats of .json graphs Breaking changes (Rust): * Refactor SwhidMphf to embed a permutation * Move stdlib to a separate `swh-graph-stdlib` crate Breaking changes (Python): * shell.py: Add support for capturing stderr New features (gRPC server): * Add num_nodes_by_type and num_arcs_by_type to StatsResponse New features (Rust): * Add accessors num_nodes_by_type/actual_num_nodes/num_arcs_by_type * Rename FilenameId to LabelNameId (there is an alias for backward-compatibility) * Add num_label_names() accessor, and iterators on LabelNameId * Switch AdaptiveNodeSet from SipHash to RapidHash * Extract full names and provide mapping from their corresponding author ID Fixes (gRPC server): * grpc-server: Fix deadlock when processing as many requests as there are CPU threads in Traverse Fixes (Python): * Fail luigi task when end-to-end tests fail * Don't expect -labelled.* files to be produced for graphs with no label * Pass missing `test_flavor` argument * Pass YAML config parameters to luigi * Turn end to end tests into a luigi task * Fix handling of local_sensitive_export_path with luigi Internal (Rust): * with NodeConstraint to parse CLI args * Fixes for Clippy 1.88.0 * Update tonic * pyproject.toml: Use extend-exclude flake8 option instead of exclude * Apply swh-py-template v0.3.4 with copier * cli: Remove no longer needed type ignore * requirements-test: Remove use of deprecated types-click stubs * Fix MPH tests * Improve debug profile detection * Improve typing * Rename 'target' to 'profile' in places it had been forgotten Internal (Python): * Remove redundant OFFSETS and TRANSPOSE_OFFSETS steps
-
v6.7.4
5331845a · ·v6.7.4 Fixes (Rust): * Switch to released webgraph version Fixes (Python): * rust/tests/compress_label_names: Prevent possible infinite loop * Unmark ExtractNodes and ExtractLabels as using all CPU threads * Remove incorrect dependency of NODE_PROPERTIES on MAPS Internal (Rust): * Update dsi-progress-logger to 0.8.0
-
v6.7.3
7cf372de · ·example_dataset: Update some edge labels and regenerate dataset e2e_tests: Fix comparison of graph elements with successors example_dataset: Fix generate_dataset script Stop publishing graph.{labels,persons}.csv.zst pytest_plugin: Use safer multiprocessing start method for graph server pytest_plugin: Ensure to kill swh-graph-grpc-serve process on teardown Fix compiler errors and Clippy warnings on Rust 1.87.0 Update webgraph to fix 'swh-graph-index labels-ef' With webgraph >=0.2.0, labels-ef now expects num_nodes instead of num_nodes+1 Add missing dependency of swh-graph-grpc-server on swh-graph's serde feature Update Webgraph to fix incorrect LLP output Add exit code to the log file Document how to generate custom graphs to run tests on grpc-server: Add support for --graph-format json grpc-server: Add support for using empty lists of labels/arcs if not available Add WebgraphAdapter -
v6.7.2
45537435 · ·v6.7.2 Evolutions: - Add support for making properties optional at runtime - Add methods SwhGraph::{iter_nodes, par_iter_nodes} - Set 'profile' in config.py instead of cli.py - Rename 'target' to 'profile' in e2e tests Fixes: - Fix label width computation - Fix SUCCESSOR mask in node_builder_bitmasks to cover SUCCESSOR_LABEL - Fix missing renames - Remove Java-specific memory estimation - Simplify type of deserialized VecGraphs - stdlib/visit: Relax trait constraints on the graph - ordered_swhids: Fix unsound concurrent writes to a shared array Dependency updates: - Regenerate Cargo.lock - Update to webgraph 0.2.x - Adapt to new version of swh-export - pyo3: update to 0.24.0 - Switch to released version of orc-rust - Remove unused dependency on swh-storage Documentation: - Add compression_steps.dot to .gitignore - Automatically generate compression tasks diagram - Update hardware and recommended system config for graph compression - Use anonymous references to avoid duplicate ids - Fix markdown syntax - README: add reference to the WWW 24 paper about swh-graph-rs - Document all features on docs.rs -
v6.7.0
9596534c · ·v6.7.0 Fixes (Rust): * Fix SwhGraphProperties::label_name_id to correctly bisect on 2023-09-06 and newer graphs Positive results of label_name_id ("the name exists and has this id"), fs_resolve_name, and fs_resolve_path were correct, negative results may be incorrect. Positive results of find_head_rev may be incorrect (in the case of repositories with both `refs/heads/main` and `refs/heads/master`) as well as its negative results. The gRPC server is not affected. Improvements (Rust): * Replace jemalloc with mimalloc as swh-graph-extract's allocator Fixes (Python): * Make swh-export an optional dependency, with a new 'export' extra * Fix compression for example dataset generation * Migrate from deprecated pkg_resources package to importlib.metadata New features (Python): * Expose a base Luigi Task class to upload to S3 without creating an Athena table Internal: * Fix flakiness of statsd tests on Jenkins * Fix Clippy warnings on Rust 1.85. * new tool: counters. Counts nodes by type, and total/max content length -
v6.6.0
f3002276 · ·v6.6.0 New features (Python): * Make swh.graph.shell.Rust usable by other packages * Add end-to-end tests for newly compressed graphs New features (Rust): * Relax constraint that UnderlyingGraph labels must impl Copy to simplify code Internal: * luigi/compressed_graph: Fix flake8 warnings * webgraph: Fix black formatting * Apply swh-py-template v0.3.3 with copier * requirements-test: Remove no longer needed pytest-asyncio dependency * Move dataset generation to other repositories Documentation: * Document that 'cargo install' is needed to reindex
-
v6.5.0
1c916584 · ·v6.5.0 Changes: * Rename swh-dataset dependency to swh-export New features: * Test new compressed graphs Fixes: * Add missing dependency of CompressGraph on Stats step * Fix pytest detection * docker: Update dockerfile to use swh-graph rust binaries * doc: Update CLI commands to compress a graph
-
v6.4.0
d69b49a3 · ·v6.4.0 The Rust crate is not released in this version of swh-graph, as it depends on an un-released version of orc-rust, which orc-rust would reject Python minor breaking changes: * Drop support for python 3.7 Python improvements: * Add some defaults values to simplify automation * Add RunNewGraph meta-task Python bug fixes: * Add support for blobs.csv.zst referencing unknown or skipped contents * Add a warning notice about the output of 'swh graph list-datasets' * docs/compression: Add missing --no-sign-request option to aws command * shell: Fix hangs in pytest * Create provenance directory when it does not exist * luigi: Fix crash in derived datasets declaration when compressed graph is not available * Add version restriction to avoid a regression in pyarrow * Add grpcio-tools version requirement * Update protobuf to 5.29 * docs/quickstart: Add pv in required debian package to install * aggregate_datasets: Use precomputed revisions-in-origins table from provenance Rust bug fixes: * Update dataset-writer to avoid exceeding Parquet's max row-group count * Update orc-rust to v0.5.0+three patches Internal: * grpc_server: Fix clippy::needless_lifetimes warnings Rust new features: * provenance: Add revisions-in-origins index generation