Related to swh/meta#5075.

Create the file to write directly in the method implementation using
a context manager instead of providing it as parameter.

It enables to avoid creating too many temporary files by keeping file
bytes in memory when the file size is lower than a cutoff value.

When using pserver protocol, the previous implementation could fail to
checkout some files while it is working fine when using the cvsimport
command from git.

So align the pserver commands sent to checkout a file with the ones used
in the git cvsimport implementation as it effectively fixes the observed
file checkout issues.

Otherwise it will find `swh/loader/cvs/rcsparse/setup.py` which
will annoy Sphinx as `swh.loader.cvs.rcsparse` is not a package.

Refs. swh/infra/sysadm-environment#4906

Previously an assertion was raised when encountering a CVS module with
no changesets. As such module is empty, we should rather produce an empty
snapshot and marks the loading as uneventful (as in the git loader).

Fixes #4648.

When attempting to fetch CVS repository data using rsync protocol but
that process exits with return code 23, it means the repository is
no longer available so NotFound must be raised.

Fixes #4572

When using the rsync protocol the loader parses the CVSROOT/config file
to look for a custom keyword definition.

However, it exists CVS repositories where such file is missing and thus
loadings were failing as file existence was not checked prior opening it.

GitLab will display the content of the README file when browsing the
repository. But in case the file is a symlink, it will display the
path pointed by the symlink. There is a 6 year old issue about this:
https://gitlab.com/gitlab-org/gitlab/-/issues/15093

We can workaround the issue by having the content at the root of the
repository and a symlink to this file in the `docs/` directory.

Tested in swh-py-template!27

Related to swh/meta#4960

Related to swh/meta#4959

This fixes python 3.7 support due to poetry, a dependency of isort, that
removed support for that Python version in a recent release.

In order to remove warnings about /apidoc/*.rst files being included
multiple times in toc when building full swh documentation, prefer to
include module indices only when building standalone package documentation.

Related to T4496

Some CVS servers have restriction regarding the expansion of the $Log$
keyword in file to checkout and might skip that operation.

Nevertheless, official CVS client will still checkout the file but
without expanding the $Log$ keyword so we should have the same behavior
in the loader.

Closes T4646

- pre-commit from 4.1.0 to 4.3.0,
- codespell from 2.2.1 to 2.2.2,
- black from 22.3.0 to 22.10.0 and
- flake8 from 4.0.1 to 5.0.4. Also freeze flake8 dependencies.

Also change flake8's repo config to github (the gitlab mirror
being outdated).

Use helper fixture loading_task_creation_for_listed_origin_test from
swh-loader-core and remove redundant test.

This fixes debian package builds.

Previously, after each revision replay all files and directories of the
CVS repository being loaded were collected and sent to the storage.
This is a real bottleneck in terms of loading performances as it delegates
the filtering of new objects to archive to the storage filtering proxy.

As we known exactly the set of paths that have been modified in a CVS
revision, prefer to do that filtering on the loader side and only
send modified objects to storage instead of the whole set of contents
and directories from the reconstructed filesystem.

This should greatly improve loading performance for large repositories
but also reduce loader memory consumption.

Instead of creating a from_disk.Directory instance after each replayed
CVS revision by recursively scanning all directories of the repository,
prefer to have a single one as class member kept synchronized with the
recontructed filesystem after each revision replay.

This should improve loader in terms of performance, especially when
delaing with large repositories.

CVS rlog for a given module sent by server is a concatenation of
rlog entries. Each entry has a header containing the path to a
RCS file plus other info.

It exist cases where a rlog entry header is empty which makes the
rlog parsing fail.

So instead of stopping rlog parsing by raising an exception, prefer
to skip that entry and process the next one.

Closes T4629

Instead of using the readlines method on file objects that retrieve all
lines of a file and store them in memory, prefer to read files line
by line by using the lazy generator of lines from file objects.

This significantly reduce loader memory consumption when processing
a large rlog output stored in a file.

That case was handled when using rsync protocol but not when using pserver
or ssh protocol.

Closes T4631

When attempting to fetch the rlog for a path that does not exist in
the repository, the CVS server will respond with the following lines:

E cvs rlog: could not read RCS file for <path>
ok

That error case was not handled in fetch_rlog so ensure it returns None
when encountering it.

The issue was spotted when the loader attempts to fetch more rlog data from
Attic directories. The paths of these Attic directories are computed from
those of the files in the repositories but it exist cases where those
directories do not exist.

Since Python 3.10, support for PyArg_ParseTuple() # formats
requires PY_SSIZE_T_CLEAN macro to be defined.

It exists CVS respositories where revision numbers greather than 1.x
are used to version files.

Previous loader implementation was raising an error when encountering
such kind of revision so ensure it will be processed as the other ones.

Also fix tag names extraction from rlog output.

Related to T4043

It exists cases where rsync output will not be ascii decodable so prefer
to use utf-8 instead.

It makes the loading process fail otherwise.

Some CVS repositories have paths which are non valid UTF-8 (typically
ISO-8859-1 ones) but the loader implementation assumed all paths can
be safely encoded to UTF-8 and was raising UnicodeEncodeError when
attempting to encode non UTF-8 paths.

That commit modifies the way CVS paths are handled by the loader by
using their raw bytes representation instead of their UTF-8 decoded
string representation.

Also rcsparse.rcsfile constructor has been modified to take bytes path
as argument instead of an unicode one in order to be able to successfully
open non UTF-8 paths.

Such CVS repositories can now be successfully loaded, either using rsync
or pserver protocol.

Related to T3980

Connection to an existing pserver might sometimes fail (on SourceForge
for instance), retrying the operation usally fixes the issue.

Some CVS servers (SourceForge and OSDN for instance) return an error if
the path sent with the "Directory" pserver request is not absolute.

So fix that issue to ensure loading of such CVS repositories.

The CVS client was raising an error when trying to connect to such pserver
URL: pserver://anonymous@cvs.example.org/cvsroot/project/module

But numerous CVS pserver URLs that can be found in the wild (notably on
SourceForge and OSDN) are in that form.

So add support for such URL form in the CVS client.

Also remove use of external dependency urllib3.util.parse_url and prefer
to use urllib.parse.urlparse from standard Python library.

A change in the base loader will allow to increase it punctually if needed in debugging
mode.