Skip to content
Snippets Groups Projects

Compare revisions

Changes are shown as if the source revision was being merged into the target revision. Learn more about comparing revisions.

Source

Select target project
No results found

Target

Select target project
  • lunar/swh-fuse
  • anlambert/swh-fuse
  • swh/devel/swh-fuse
  • douardda/swh-fuse
  • martin/swh-fuse
5 results
Show changes
Commits on Source (20)
Hide whitespace changes
Inline Side-by-side
Showing
with 252 additions and 229 deletions
.copier-answers.yml 0 → 100644
View file @ ed2a4502
# Changes here will be overwritten by Copier
_commit: v0.3.3
_src_path: https://gitlab.softwareheritage.org/swh/devel/swh-py-template.git
description: Software Heritage virtual file system
distribution_name: swh-fuse
have_cli: true
have_workers: false
package_root: swh/fuse
project_name: swh.fuse
python_minimal_version: '3.7'
readme_format: rst
.git-blame-ignore-revs
View file @ ed2a4502
# python: Reformat code with black 22.3.0
# python: Reformat code with black
82ad1d6e501637fce226bc2812aecdd12f20cbdc
02ae6b6dac247f9d4b7100422ffd9b11c13ebc2a
.gitignore
View file @ ed2a4502
*.egg-info/
*.pyc
*.sw?
*~
.coverage
.eggs/
.hypothesis
.mypy_cache
.tox
__pycache__
build/
dist/
version.txt
.mypy_cache/
# these are symlinks created by a hook in swh-docs' main sphinx conf.py
docs/README.rst
docs/README.md
# this should be a symlink for people who want to build the sphinx doc
# without using tox, generally created by the swh-env/bin/update script
docs/Makefile.sphinx
.vscode/
\ No newline at end of file
.pre-commit-config.yaml
View file @ ed2a4502
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.3.0
rev: v5.0.0
hooks:
- id: trailing-whitespace
- id: check-json
- id: check-yaml
- repo: https://github.com/python/black
rev: 25.1.0
hooks:
- id: black
- repo: https://github.com/PyCQA/isort
rev: 6.0.0
hooks:
- id: isort
- repo: https://github.com/pycqa/flake8
rev: 5.0.4
rev: 7.1.1
hooks:
- id: flake8
additional_dependencies: [flake8-bugbear==22.9.23]
additional_dependencies: [flake8-bugbear==24.12.12, flake8-pyproject]
- repo: https://github.com/codespell-project/codespell
rev: v2.2.2
rev: v2.4.1
hooks:
- id: codespell
name: Check source code spelling
args: [-L nin, -L nIn, -L crate, -L grammer]
stages: [commit]
stages: [pre-commit]
exclude:
swh/fuse/tests/data/api_data.py
- id: codespell
name: Check commit message spelling
stages: [commit-msg]
- repo: local
hooks:
- id: mypy
name: mypy
entry: env DJANGO_SETTINGS_MODULE=swh.deposit.settings.testing mypy
entry: mypy
args: [swh]
pass_filenames: false
language: system
types: [python]
- repo: https://github.com/PyCQA/isort
rev: 5.11.5
hooks:
- id: isort
- repo: https://github.com/python/black
rev: 22.10.0
hooks:
- id: black
- id: twine-check
name: twine check
description: call twine check when pushing an annotated release tag
entry: bash -c "ref=$(git describe) &&
[[ $ref =~ ^v[0-9]+\.[0-9]+\.[0-9]+$ ]] &&
(python3 -m build --sdist && twine check $(ls -t dist/* | head -1)) || true"
pass_filenames: false
stages: [pre-push]
language: python
additional_dependencies: [twine, build]
CODE_OF_CONDUCT.md
View file @ ed2a4502
......@@ -6,7 +6,7 @@ In the interest of fostering an open and welcoming environment, we as Software
Heritage contributors and maintainers pledge to making participation in our
project and our community a harassment-free experience for everyone, regardless
of age, body size, disability, ethnicity, sex characteristics, gender identity
and expression, level of experience, education, socio-economic status,
and expression, level of experience, education, socioeconomic status,
nationality, personal appearance, race, religion, or sexual identity and
orientation.
......
MANIFEST.in deleted 100644 → 0
View file @ b17957eb
include Makefile
include requirements*.txt
include version.txt
include README.md
recursive-include swh py.typed
README.rst deleted 120000 → 0
View file @ b17957eb
docs/README.rst
\ No newline at end of file
README.rst 0 → 100644
View file @ ed2a4502
Software Heritage Filesystem (SwhFS)
====================================
User-space POSIX filesystem to browse the `Software Heritage
<https://www.softwareheritage.org/>`_ `archive
<https://archive.softwareheritage.org/>`_, as if it were locally available.
Built using the `FUSE <https://github.com/libfuse/libfuse>`_ framework.
Demo
----
A live demo of SwhFS in action is available as a `screencast
<https://www.youtube.com/watch?v=2L4ANVlICaE>`_.
Bibliography
------------
In addition to accompanying technical documentation, SwhFS is also described in
the following scientific paper. If you use SwhFS for your scientific results,
please acknowledge it by citing the paper as follows:
.. note::
Thibault Allançon, Antoine Pietri, Stefano Zacchiroli. `The Software Heritage
Filesystem (SwhFS): Integrating Source Code Archival with Development
<https://arxiv.org/pdf/2102.06390.pdf>`_. In proceedings of `ICSE 2021
<https://conf.researchr.org/home/icse-2021>`_: The 43rd International
Conference on Software Engineering, May 2021, Madrid, Spain. IEEE 2021.
Links: `preprint <https://arxiv.org/pdf/2102.06390.pdf>`_, `bibtex
<https://upsilon.cc/~zack/research/publications/saner-2020-swh-graph.bib>`_.
docs/Makefile
View file @ ed2a4502
include ../../swh-docs/Makefile.sphinx
-include Makefile.local
include Makefile.sphinx
include Makefile.local
docs/README.rst deleted 100644 → 0
View file @ b17957eb
Software Heritage Filesystem (SwhFS)
====================================
User-space POSIX filesystem to browse the `Software Heritage
<https://www.softwareheritage.org/>`_ `archive
<https://archive.softwareheritage.org/>`_, as if it were locally available.
Built using the `FUSE <https://github.com/libfuse/libfuse>`_ framework.
Demo
----
A live demo of SwhFS in action is available as a `screencast
<https://www.youtube.com/watch?v=2L4ANVlICaE>`_.
Bibliography
------------
In addition to accompanying technical documentation, SwhFS is also described in
the following scientific paper. If you use SwhFS for your scientific results,
please acknowledge it by citing the paper as follows:
.. note::
Thibault Allançon, Antoine Pietri, Stefano Zacchiroli. `The Software Heritage
Filesystem (SwhFS): Integrating Source Code Archival with Development
<https://arxiv.org/pdf/2102.06390.pdf>`_. In proceedings of `ICSE 2021
<https://conf.researchr.org/home/icse-2021>`_: The 43rd International
Conference on Software Engineering, May 2021, Madrid, Spain. IEEE 2021.
Links: `preprint <https://arxiv.org/pdf/2102.06390.pdf>`_, `bibtex
<https://upsilon.cc/~zack/research/publications/saner-2020-swh-graph.bib>`_.
docs/README.rst 0 → 120000
View file @ ed2a4502
../README.rst
\ No newline at end of file
docs/configuration.rst
View file @ ed2a4502
......@@ -14,8 +14,24 @@ The configuration file location is subject to the `XDG Base Directory
well as explicitly overridden on the :ref:`command line <swh-fuse-cli>` via the
``-C/--config-file`` flag.
The following sub-sections and fields can be used within the ``swh > fuse``
stanza:
You can choose how ``swh-fuse`` will fetch content from the archive.
The simplest (and default) way is to query the SWH public API.
This method can be configured with the following block:
- ``web-api``:
- ``url``: archive API URL
- ``auth-token``: (optional) authentication token used with the API URL
You can use a :ref:`compressed graph <swh-graph>` close to your server, via its gRPC API,
to traverse the folder hierarchy much faster.
This can be configured with the following block:
- ``graph``:
- ``grpc-url``: URL to the graph's :ref:`gRPC server <swh-graph-grpc-api>`.
``swh-fuse`` will also search for the following options:
- ``cache``:
......@@ -28,11 +44,6 @@ stanza:
specified using a ``maxram`` entry (either as a percentage of available RAM,
or with disk storage unit suffixes: ``B``, ``KB``, ``MB``, ``GB``).
- ``web-api``:
- ``url``: archive API URL
- ``auth-token``: authentication token used with the API URL
- ``json-indent``: number of spaces used to print JSON metadata files (setting
it to ``null`` disables indentation).
......@@ -45,12 +56,10 @@ If no configuration is given, default values are:
- ``json-indent``: 2 spaces.
Example
-------
Examples
--------
Here is a full ``~/.config/swh/global.yml`` example, showcasing different cache
storage strategies (in-memory for metadata, on-disk for blob, 20% RAM for
direntry), using the default Web API service:
Here is a full ``~/.config/swh/global.yml`` equivalent to the default configuration:
.. code:: yaml
......@@ -58,15 +67,32 @@ direntry), using the default Web API service:
fuse:
cache:
metadata:
in-memory: true
path: "/path/to/cache/blob.sqlite"
blob:
path: "/path/to/cache/blob.sqlite"
direntry:
maxram: 20%
maxram: 10%
web-api:
url: "https://archive.softwareheritage.org/api/1/"
auth-token: eyJhbGciOiJIUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJhMTMxYTQ1My1hM2IyLTQwMTUtO...
json-indent: 2
This example uses a local compressed graph, an in-memory cache for metadata,
and authenticates against the SWH public API to benefit from higher rate limits:
.. code:: yaml
swh:
fuse:
cache:
metadata:
in-memory: true
blob:
path: "/path/to/cache/blob.sqlite"
graph:
grpc-url: localhost:50091
web-api:
url: "https://archive.softwareheritage.org/api/1/"
auth-token: eyJhbGciOiJIUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJhMTMxYTQ1My1hM2IyLTQwMTUtO...
Logging
-------
......
mypy.ini deleted 100644 → 0
View file @ b17957eb
[mypy]
namespace_packages = True
warn_unused_ignores = True
# 3rd party libraries without stubs (yet)
[mypy-aiosqlite.*]
ignore_missing_imports = True
[mypy-daemon.*]
ignore_missing_imports = True
[mypy-pkg_resources.*]
ignore_missing_imports = True
[mypy-psutil.*]
ignore_missing_imports = True
[mypy-pytest.*]
ignore_missing_imports = True
[mypy-pyfuse3.*]
ignore_missing_imports = True
[mypy-pyfuse3_asyncio.*]
ignore_missing_imports = True
pyproject.toml
View file @ ed2a4502
[project]
name = "swh.fuse"
authors = [
{name="Software Heritage developers", email="swh-devel@inria.fr"},
]
description = "Software Heritage virtual file system"
readme = {file = "README.rst", content-type = "text/x-rst"}
requires-python = ">=3.7"
classifiers = [
"Programming Language :: Python :: 3",
"Intended Audience :: Developers",
"License :: OSI Approved :: GNU General Public License v3 (GPLv3)",
"Operating System :: OS Independent",
"Development Status :: 3 - Alpha",
]
dynamic = ["version", "dependencies", "optional-dependencies"]
[tool.setuptools.packages.find]
include = ["swh.*"]
[tool.setuptools.dynamic]
dependencies = {file = ["requirements.txt", "requirements-swh.txt"]}
[tool.setuptools.dynamic.optional-dependencies]
testing = {file = ["requirements.txt", "requirements-swh.txt", "requirements-test.txt"]}
[project.entry-points."swh.cli.subcommands"]
"swh.fuse" = "swh.fuse.cli"
[project.urls]
"Homepage" = "https://gitlab.softwareheritage.org/swh/devel/swh-fuse"
"Bug Reports" = "https://gitlab.softwareheritage.org/swh/devel/swh-fuse/-/issues"
"Funding" = "https://www.softwareheritage.org/donate"
"Documentation" = "https://docs.softwareheritage.org/devel/swh-fuse/"
"Source" = "https://gitlab.softwareheritage.org/swh/devel/swh-fuse.git"
[build-system]
requires = ["setuptools", "setuptools-scm"]
build-backend = "setuptools.build_meta"
[tool.setuptools_scm]
fallback_version = "0.0.1"
[tool.black]
target-version = ['py37']
target-version = ['py39', 'py310', 'py311', 'py312']
[tool.isort]
multi_line_output = 3
......@@ -9,3 +53,37 @@ use_parentheses = true
ensure_newline_before_comments = true
line_length = 88
force_sort_within_sections = true
known_first_party = ['swh']
[tool.mypy]
namespace_packages = true
warn_unused_ignores = true
explicit_package_bases = true
# ^ Needed for mypy to detect py.typed from swh packages installed
# in editable mode
plugins = []
# 3rd party libraries without stubs (yet)
[[tool.mypy.overrides]]
module = [
"aiosqlite.*",
"daemon.*",
]
ignore_missing_imports = true
[tool.flake8]
select = ["C", "E", "F", "W", "B950"]
ignore = [
"E203", # whitespaces before ':' <https://github.com/psf/black/issues/315>
"E231", # missing whitespace after ','
"E501", # line too long, use B950 warning from flake8-bugbear instead
"W503" # line break before binary operator <https://github.com/psf/black/issues/52>
]
max-line-length = 88
[tool.pytest.ini_options]
norecursedirs = "build docs .*"
asyncio_mode = "strict"
consider_namespace_packages = true
asyncio_default_fixture_loop_scope = "function"
\ No newline at end of file
pytest.ini deleted 100644 → 0
View file @ b17957eb
[pytest]
norecursedirs = build docs .*
asyncio_mode = strict
requirements-swh.txt
View file @ ed2a4502
swh.core >= 0.10.0
swh.model >= 3.0.0
swh.web.client >= 0.2.5
swh.graph >= 6.7.1
requirements-test.txt
View file @ ed2a4502
pytest
requests # workaround for https://forge.softwareheritage.org/T2634
pytest >= 8.1
requests-mock
swh.core[testing]
types-python-dateutil
types-psutil
types-pyyaml
types-requests
requirements.txt
View file @ ed2a4502
pyyaml
aiosqlite
psutil
pyfuse3
pyfuse3 >= 3.4.0
python-daemon
requests
protobuf >=5.29.3
grpc-stubs
\ No newline at end of file
setup.cfg deleted 100644 → 0
View file @ b17957eb
[flake8]
# E203: whitespaces before ':' <https://github.com/psf/black/issues/315>
# E231: missing whitespace after ','
# E501: line too long, use B950 warning from flake8-bugbear instead
# W503: line break before binary operator <https://github.com/psf/black/issues/52>
select = C,E,F,W,B950
ignore = E203,E231,E501,W503
max-line-length = 88
setup.py deleted 100755 → 0
View file @ b17957eb
#!/usr/bin/env python3
# Copyright (C) 2020 The Software Heritage developers
# See the AUTHORS file at the top-level directory of this distribution
# License: GNU General Public License version 3, or any later version
# See top-level LICENSE file for more information
from io import open
from os import path
from setuptools import find_packages, setup
here = path.abspath(path.dirname(__file__))
# Get the long description from the README file
with open(path.join(here, "README.rst"), encoding="utf-8") as f:
long_description = f.read()
def parse_requirements(name=None):
if name:
reqf = "requirements-%s.txt" % name
else:
reqf = "requirements.txt"
requirements = []
if not path.exists(reqf):
return requirements
with open(reqf) as f:
for line in f.readlines():
line = line.strip()
if not line or line.startswith("#"):
continue
requirements.append(line)
return requirements
setup(
name="swh.fuse",
description="Software Heritage virtual file system",
long_description=long_description,
long_description_content_type="text/x-rst",
python_requires=">=3.7",
author="Software Heritage developers",
author_email="swh-devel@inria.fr",
url="https://forge.softwareheritage.org/source/swh-fuse",
packages=find_packages(), # packages's modules
install_requires=parse_requirements() + parse_requirements("swh"),
tests_require=parse_requirements("test"),
setup_requires=["setuptools-scm"],
use_scm_version=True,
extras_require={"testing": parse_requirements("test")},
include_package_data=True,
entry_points="""
[swh.cli.subcommands]
fuse=swh.fuse.cli
""",
classifiers=[
"Programming Language :: Python :: 3",
"Intended Audience :: Developers",
"License :: OSI Approved :: GNU General Public License v3 (GPLv3)",
"Operating System :: OS Independent",
"Development Status :: 3 - Alpha",
],
project_urls={
"Bug Reports": "https://forge.softwareheritage.org/maniphest",
"Funding": "https://www.softwareheritage.org/donate",
"Source": "https://forge.softwareheritage.org/source/swh-fuse",
"Documentation": "https://docs.softwareheritage.org/devel/swh-fuse/",
},
)