Skip to content
Snippets Groups Projects

tests: Reduce the number of warnings raised

Merged Renaud Boyer requested to merge rboyer/swh-web:fix-warnings into master

TODO

  • pytest:
    • Unknown pytest.mark.inbound_message
    • unused pytest asyncio_mode config option
  • Converter 'swhid' is already registered
  • Vault & Indexer storage DeprecationWarning while init using args
  • docutils:
    • frontend.OptionParser is deprecated
    • use Node.findall instead of Node.traverse
  • UrlField's default scheme -> transitional setting + silence warning
  • Naive datetime value used for an aware field
  • Re-implemented GitLoaderFromDisk to remove the last dependency to the deprecated swh.loader.git.from_disk
  • Identifying contents by sha1 instead of hash dicts {"sha1": b"..."} is deprecated
  • test_message method is not a test

Will be fixed elsewhere

Wontfix here

Results

Down from 41630 warnings in master to 1185, mostly (1136) due to a deprecation warning fixed in swh-loader-git!189 (merged) 🎉

The only change impacting swh-web behavior is setting https the default scheme for UrlFields (hence the change in e2e tests).

Edited by Renaud Boyer

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • Renaud Boyer added 2 commits

    added 2 commits

    • 62cf7e93 - 1 commit from branch swh/devel:master
    • 1d8d1f1f - Fix PytestUnknownMarkWarning

    Compare with previous version

  • Jenkins job DWAPPS/gitlab-builds #1100 succeeded .
    See Console Output, Blue Ocean and Coverage Report for more details.

  • Renaud Boyer changed the description

    changed the description

  • Renaud Boyer added 2 commits

    added 2 commits

    Compare with previous version

  • Jenkins job DWAPPS/gitlab-builds #1102 failed .
    See Console Output, Blue Ocean and Coverage Report for more details.

  • Renaud Boyer added 1 commit

    added 1 commit

    • e9acb00e - Fix register SWHIDConverter only once

    Compare with previous version

  • Jenkins job DWAPPS/gitlab-builds #1103 failed .
    See Console Output, Blue Ocean and Coverage Report for more details.

  • Renaud Boyer changed title from Reduce the number of warnings raised by test execution to tests: Reduce the number of warnings raised

    changed title from Reduce the number of warnings raised by test execution to tests: Reduce the number of warnings raised

  • Renaud Boyer added 1 commit

    added 1 commit

    • 423c81ab - Fix register SWHIDConverter only once

    Compare with previous version

  • Renaud Boyer marked the checklist item Fix PytestUnknownMarkWarning: Unknown pytest.mark.inbound_message as completed

    marked the checklist item Fix PytestUnknownMarkWarning: Unknown pytest.mark.inbound_message as completed

  • Renaud Boyer marked the checklist item Fix RemovedInDjango60Warning: Converter 'swhid' is already registered. Support for overriding registered converters is deprecated and will be removed in Django 6.0 as completed

    marked the checklist item Fix RemovedInDjango60Warning: Converter 'swhid' is already registered. Support for overriding registered converters is deprecated and will be removed in Django 6.0 as completed

  • Jenkins job DWAPPS/gitlab-builds #1104 succeeded .
    See Console Output, Blue Ocean and Coverage Report for more details.

  • Renaud Boyer added 1 commit

    added 1 commit

    • abd53aaa - Fix vault DeprecationWarning using args

    Compare with previous version

    • Resolved by Renaud Boyer

      Regarding the registration of URL converters, I had to explicitely register them in each django application they are used as otherwise running tests for a specific django application can fail, see below after I have applied your patches:

      $ pytest -sv swh/web/vault/
      /home/anlambert/.virtualenvs/swh/lib/python3.11/site-packages/pytest_asyncio/plugin.py:208: PytestDeprecationWarning: The configuration option "asyncio_default_fixture_loop_scope" is unset.
      The event loop scope for asynchronous fixtures will default to the fixture caching scope. Future versions of pytest-asyncio will default the loop scope for asynchronous fixtures to function scope. Set the default fixture loop scope explicitly in order to avoid unexpected behavior in the future. Valid fixture loop scopes are: "function", "class", "module", "package", "session"
      
        warnings.warn(PytestDeprecationWarning(_DEFAULT_FIXTURE_LOOP_SCOPE_UNSET))
      ================================================================================================================================== test session starts ==================================================================================================================================
      platform linux -- Python 3.11.2, pytest-8.3.2, pluggy-1.5.0 -- /home/anlambert/.virtualenvs/swh/bin/python
      cachedir: .pytest_cache
      django: version: 5.1.1, settings: swh.web.settings.tests (from ini)
      hypothesis profile 'swh-web-fast' -> deadline=None, max_examples=5, suppress_health_check=[HealthCheck.too_slow, HealthCheck.filter_too_much, HealthCheck.function_scoped_fixture], database=DirectoryBasedExampleDatabase(PosixPath('/home/anlambert/swh/swh-environment/swh-web/.hypothesis/examples'))
      rootdir: /home/anlambert/swh/swh-environment/swh-web
      configfile: pytest.ini
      plugins: django-4.9.0, swh.journal-1.5.3.dev4+g0705dcc, pytest_httpserver-1.1.0, docker-3.1.1, dotenv-0.5.2, django-test-migrations-1.4.0, dash-2.17.1, redis-3.1.2, asyncio-0.24.0, click-1.1.0, hypothesis-6.111.2, swh.core-3.4.1.dev5+gea02022, testinfra-10.1.1, cov-5.0.0, postgresql-6.1.0, flaky-3.8.1, mock-3.14.0, requests-mock-1.12.1, xdist-3.6.1, anyio-4.4.0, timeout-2.3.1
      asyncio: mode=Mode.STRICT, default_loop_scope=None
      collected 19 items / 1 error                                                                                                                                                                                                                                                            
      
      ======================================================================================================================================== ERRORS =========================================================================================================================================
      ___________________________________________________________________________________________________________________ ERROR collecting swh/web/vault/tests/test_app.py ____________________________________________________________________________________________________________________
      ../../../.virtualenvs/swh/lib/python3.11/site-packages/django/urls/resolvers.py:275: in _route_to_regex
          converter = all_converters[raw_converter]
      E   KeyError: 'swhid'
      
      The above exception was the direct cause of the following exception:
      swh/web/vault/tests/test_app.py:13: in <module>
          from swh.web.vault.urls import urlpatterns
      build/__editable__.swh.web-0.7.1.dev5+gba351e0d4.d20240906-py3-none-any/swh/web/vault/urls.py:11: in <module>
          from swh.web.vault.api_views import vault_api_urls
      build/__editable__.swh.web-0.7.1.dev5+gba351e0d4.d20240906-py3-none-any/swh/web/vault/api_views.py:102: in <module>
          @api_route(
      build/__editable__.swh.web-0.7.1.dev5+gba351e0d4.d20240906-py3-none-any/swh/web/api/apiurls.py:124: in decorator
          api_urls.add_url_pattern(url_pattern, api_view_f, view_name)
      build/__editable__.swh.web-0.7.1.dev5+gba351e0d4.d20240906-py3-none-any/swh/web/utils/urlsindex.py:43: in add_url_pattern
          self.urlpatterns.append(url(pattern, view, name=view_name))
      ../../../.virtualenvs/swh/lib/python3.11/site-packages/django/urls/conf.py:81: in _path
          pattern = Pattern(route, name=name, is_endpoint=True)
      ../../../.virtualenvs/swh/lib/python3.11/site-packages/django/urls/resolvers.py:319: in __init__
          self._regex, self.converters = _route_to_regex(str(route), is_endpoint)
      ../../../.virtualenvs/swh/lib/python3.11/site-packages/django/urls/resolvers.py:277: in _route_to_regex
          raise ImproperlyConfigured(
      E   django.core.exceptions.ImproperlyConfigured: URL route 'api/1/vault/flat/<swhid:swhid>/' uses invalid converter 'swhid'.

      I think the simplest solution is to rework the register_url_path_converters function the following way:

      from django.urls.converters import REGISTERED_CONVERTERS, register_converter
      
      def register_url_path_converters():
          if "swhid" not in REGISTERED_CONVERTERS:
              register_converter(SWHIDConverter, "swhid")
  • Jenkins job DWAPPS/gitlab-builds #1105 succeeded .
    See Console Output, Blue Ocean and Coverage Report for more details.

  • Renaud Boyer added 2 commits

    added 2 commits

    • a5fad3e3 - Revert "Fix register SWHIDConverter only once"
    • 51a88cf9 - Register SWHIDConverter only when needed

    Compare with previous version

  • Renaud Boyer changed the description

    changed the description

  • Jenkins job DWAPPS/gitlab-builds #1106 succeeded .
    See Console Output, Blue Ocean and Coverage Report for more details.

  • Renaud Boyer changed the description

    changed the description

  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
Please register or sign in to reply
Loading