Skip to content
GitLab
Explore
Sign in
Register
Primary navigation
Search or go to…
Project
S
swh-fuse
Manage
Activity
Members
Labels
Plan
Issues
Issue boards
Milestones
Wiki
Code
Merge requests
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Build
Pipelines
Jobs
Pipeline schedules
Artifacts
Deploy
Releases
Package registry
Model registry
Operate
Environments
Terraform modules
Monitor
Incidents
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
Antoine Lambert
swh-fuse
Commits
1ad41c84
Commit
1ad41c84
authored
4 years ago
by
Stefano Zacchiroli
Browse files
Options
Downloads
Patches
Plain Diff
doc: uniform naming around SwhFS
parent
bb963b44
No related branches found
Branches containing commit
No related tags found
Tags containing commit
No related merge requests found
Changes
3
Hide whitespace changes
Inline
Side-by-side
Showing
3 changed files
docs/README.rst
+6
-6
6 additions, 6 deletions
docs/README.rst
docs/design.md
+23
-22
23 additions, 22 deletions
docs/design.md
swh/fuse/cache.py
+8
-7
8 additions, 7 deletions
swh/fuse/cache.py
with
37 additions
and
35 deletions
docs/README.rst
+
6
−
6
View file @
1ad41c84
Software Heritage
- V
irtual file
system
=======================================
Software Heritage
v
irtual filesystem
(SwhFS)
=======================================
=====
Virtual file
system to browse the
`Software Heritage
<https://www.softwareheritage.org/>`_
`archive
<https://archive.softwareheritage.org/>`_,
using the `FUSE
<https://github.com/libfuse/libfuse>`_ framework.
Virtual filesystem to browse the
`Software Heritage
<https://www.softwareheritage.org/>`_
`archive
<https://archive.softwareheritage.org/>`_,
using the `FUSE
<https://github.com/libfuse/libfuse>`_ framework.
This diff is collapsed.
Click to expand it.
docs/design.md
+
23
−
22
View file @
1ad41c84
# S
WH FUSE —
Design notes
# S
oftware Heritage virtual filesystem (SwhFS) ---
Design notes
```
{warning}
this document describes design notes for SWH FUSE, which is still under active
development and hence **not yet available** for general use.
this document describes design notes for the Software Heritage virtual
filesystem (SwhFS), which is still under active development and hence **not yet
available** for general use.
```
The
[
Software Heritage
](
https://www.softwareheritage.org/
)
{ref}
`data model <data-model>`
is a
[
Direct Acyclic
Graph
](
https://en.wikipedia.org/wiki/Directed_acyclic_graph
)
(
DAG
)
with nodes of
different types that correspond to source code artifacts
such as directories,
commits, etc. Using
this
[
FUSE
](
https://en.wikipedia.org/wiki/Filesystem_in_Userspace
)
module
(
*SWH FUSE*
for short) you can locally mount, and then navigate as a (virtual)
file system,
parts of the archive identified by
{ref}
`Software Heritage
identifiers <persistent-identifiers>`
(SWHIDs).
The
[
Software Heritage
](
https://www.softwareheritage.org/
)
{ref}
`data model
<data-model>`
is
a
[
Direct Acyclic
Graph
](
https://en.wikipedia.org/wiki/Directed_acyclic_graph
)
(DAG) with nodes of
different types that correspond to source code artifacts
such as directories,
commits, etc. Using
this
[
FUSE
](
https://en.wikipedia.org/wiki/Filesystem_in_Userspace
)
module
(
*SwhFS*
for short) you can locally mount, and then navigate as a (virtual)
file system,
parts of the archive identified by
{ref}
`Software Heritage
identifiers <persistent-identifiers>`
(SWHIDs).
To retrieve information about the source code artifacts
the FUSE module
interacts over
the network with the Software Heritage archive via its
{ref}
`Web API
<swh-web-api-urls>`
.
To retrieve information about the source code artifacts
, SwhFS interacts over
the network with the Software Heritage archive via its
{ref}
`Web API
<swh-web-api-urls>`
.
## Command-line interface
$ swh fuse mount <DIR> [SWHID]...
will mount the Software Heritage archive at the local
`<DIR>`
, the
*
S
WH FUSE
mount
point
*
. From there, the user will be able to lazily load and navigate the
will mount the Software Heritage archive at the local
`<DIR>`
, the
*
S
whFS mount
point
*
. From there, the user will be able to lazily load and navigate the
archive using SWHID at entry points.
If one or more SWHIDs are also specified, the corresponding objects will be pre-
...
...
@@ -38,7 +39,7 @@ For more details see the {ref}`CLI documentation <swh-graph-cli>`.
## Mount point
The S
WH FUSE
mount point contain:
The S
whFS
mount point contain:
-
`archive/`
: initially empty, this directory is lazily populated with one entry
per accessed SWHID, having actual SWHIDs as names.
...
...
@@ -83,7 +84,7 @@ Directory nodes are represented as directories on the file-system, containing
one entry for each entry of the archived directory. Entry names and other
metadata, including permissions, will correspond to the archived entry metadata.
Note that
the FUSE
mount
is
read-only, no matter what the permissions say. So it
Note that
SwhFS is
mount
ed
read-only, no matter what the permissions say. So it
is possible that, in the context of a directory, a file is presented as
writable, whereas actually writing to it will fail with
`EPERM`
.
...
...
@@ -137,7 +138,7 @@ Each entry is a symlink pointing into `archive/` to the branch target SWHID.
## Caching
S
WH FUSE
retrieves both metadata and file contents from the Software Heritage
S
whFS
retrieves both metadata and file contents from the Software Heritage
archive via the network. In order to obtain reasonable performances several
caches are used to minimize network transfer.
...
...
@@ -151,8 +152,8 @@ rather than in SQLite, e.g., under `$XDG_CACHE_HOME/swh/fuse/objects/`
```
All caches are persistent (i.e., they survive the restart of the S
WH FUSE
process)
and global (i.e., they are shared by concurrent S
WH FUSE
processes).
All caches are persistent (i.e., they survive the restart of the S
whFS process)
and global (i.e., they are shared by concurrent S
whFS
processes).
We assume that no cache
*invalidation*
is necessary, due to intrinsic properties
of the Software Heritage archive, such as integrity verification and append-only
...
...
This diff is collapsed.
Click to expand it.
swh/fuse/cache.py
+
8
−
7
View file @
1ad41c84
...
...
@@ -15,21 +15,22 @@ from swh.web.client.client import typify_json
class
FuseCache
:
"""
SWH FUSE
retrieves both metadata and file contents from the Software
Heritage archive
via the network. In order to obtain reasonable performances
several caches are used
to minimize network transfer.
"""
SwhFS
retrieves both metadata and file contents from the Software
Heritage archive
via the network. In order to obtain reasonable performances
several caches are used
to minimize network transfer.
Caches are stored on disk in SQLite databases located at
`$XDG_CACHE_HOME/swh/fuse/`.
All caches are persistent (i.e., they survive the restart of the SWH FUSE
process) and global (i.e., they are shared by concurrent SWH FUSE
processes).
All caches are persistent (i.e., they survive the restart of the SwhFS process) and
global (i.e., they are shared by concurrent SwhFS processes).
We assume that no cache *invalidation* is necessary, due to intrinsic
properties of the Software Heritage archive, such as integrity verification
and append-only archive changes. To clean the caches one can just remove the
corresponding files from disk.
"""
corresponding files from disk.
"""
def
__init__
(
self
,
cache_conf
:
Dict
[
str
,
Any
]):
self
.
cache_conf
=
cache_conf
...
...
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment