Skip to content
Snippets Groups Projects

misc/coverage: Use django cache instead of LRU ones

Closed misc/coverage: Use django cache instead of LRU ones
generated-differential-D7224-source into generated-differential-D7224-target
Closed Antoine Lambert requested to merge generated-differential-D7224-source into generated-differential-D7224-target

It enables to update origin counters more frequently in the coverage widget of the production webapp.

Also disable cache use when running swh-web in development or staging environment to get accurate counters on each homepage refresh.

Depends on !735 (closed)

Migrated from D7224 (view on Phabricator)

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • Software Heritage Continuous Integration for public repositories
    Software Heritage Continuous Integration for public repositories @swh-public-ci ·

    Build is green

    Patch application report for D7224 (id=26182)

    Could not rebase; Attempt merge onto c14cfd1b...

    Updating c14cfd1b..cb5f4ca9
Fast-forward
 swh/web/common/utils.py             | 40 +++++++++++++++++++++++++++----------
 swh/web/config.py                   |  5 +++--
 swh/web/misc/coverage.py            | 21 +++++++++++++++----
 swh/web/tests/common/test_utils.py  | 19 +++++++++++++++++-
 swh/web/tests/misc/test_coverage.py | 17 ++++++++++++----
 swh/web/tests/test_templates.py     | 11 +++++++---
 6 files changed, 89 insertions(+), 24 deletions(-)
    Changes applied before test
    commit cb5f4ca9386386b70c529048a739b1f4e7a67962
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Feb 22 17:10:51 2022 +0100

    misc/coverage: Bypass LRU caches when not in production
    
    It enables to get accurate origin counters when executing swh-web
    in a development environment or in staging.

commit cc5736263984312914af673831932af01f6779eb
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Feb 22 16:26:14 2022 +0100

    common/utils: Add utility functions to check swh-web run environment
    
    Add functions to check if we are running a development, staging or
    production version of swh-web.

    See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1399/ for more details.

  • Antoine Lambert
    Antoine Lambert @anlambert ·
    Author Maintainer

    Fix typo in comment

  • vlorentz
    vlorentz @vlorentz ·
    Maintainer

    This code feels rather weird. I think you should get rid of the lru_cache decorator on _get_listers_metrics, and implement caching directly in _get_listers_metrics, with a boolean argument to toggle it.

  • vlorentz
    vlorentz @vlorentz ·
    Maintainer

    Merge request was returned for changes

  • Software Heritage Continuous Integration for public repositories
    Software Heritage Continuous Integration for public repositories @swh-public-ci ·

    Build is green

    Patch application report for D7224 (id=26184)

    Could not rebase; Attempt merge onto c14cfd1b...

    Updating c14cfd1b..880e4e1f
Fast-forward
 swh/web/common/utils.py             | 40 +++++++++++++++++++++++++++----------
 swh/web/config.py                   |  5 +++--
 swh/web/misc/coverage.py            | 21 +++++++++++++++----
 swh/web/tests/common/test_utils.py  | 19 +++++++++++++++++-
 swh/web/tests/misc/test_coverage.py | 17 ++++++++++++----
 swh/web/tests/test_templates.py     | 11 +++++++---
 6 files changed, 89 insertions(+), 24 deletions(-)
    Changes applied before test
    commit 880e4e1f60fd269b4f443859ebb2d6f366609212
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Feb 22 17:10:51 2022 +0100

    misc/coverage: Bypass LRU caches when not in production
    
    It enables to get accurate origin counters when executing swh-web
    in a development environment or in staging.

commit cc5736263984312914af673831932af01f6779eb
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Feb 22 16:26:14 2022 +0100

    common/utils: Add utility functions to check swh-web run environment
    
    Add functions to check if we are running a development, staging or
    production version of swh-web.

    See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1400/ for more details.

  • Antoine Lambert
    Antoine Lambert @anlambert ·
    Author Maintainer

    ! In !736 (closed), @vlorentz wrote: This code feels rather weird. I think you should get rid of the lru_cache decorator on _get_listers_metrics, and implement caching directly in _get_listers_metrics, with a boolean argument to toggle it.

    I could use django cache instead, this will also have the advantage of resetting the cache on a regular basis and update counters more frequently.

  • Antoine Lambert
    Antoine Lambert @anlambert ·
    Author Maintainer

    Use django cache instead of LRU caches.

  • Antoine Lambert
    Antoine Lambert @anlambert ·
    Author Maintainer

    Update comment in test

  • Software Heritage Continuous Integration for public repositories
    Software Heritage Continuous Integration for public repositories @swh-public-ci ·

    Build has FAILED

    Patch application report for D7224 (id=26187)

    Could not rebase; Attempt merge onto c14cfd1b...

    Updating c14cfd1b..77856583
Fast-forward
 swh/web/common/utils.py             |  40 +++++++++---
 swh/web/config.py                   |   5 +-
 swh/web/misc/coverage.py            | 117 ++++++++++++++++++++++--------------
 swh/web/tests/common/test_utils.py  |  19 +++++-
 swh/web/tests/misc/test_coverage.py |  33 +++++-----
 swh/web/tests/test_templates.py     |  11 +++-
 6 files changed, 144 insertions(+), 81 deletions(-)
    Changes applied before test
    commit 77856583981ce1080c19bd11c28d6bdce8c6ce86
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Feb 22 17:10:51 2022 +0100

    misc/coverage: Use django cache instead of LRU ones
    
    It enables to update origin counters more frequently in the coverage
    widget of the production webapp.
    
    Also disable cache use when running swh-web in development or staging
    environment to get accurate counters on each homepage refresh.

commit cc5736263984312914af673831932af01f6779eb
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Feb 22 16:26:14 2022 +0100

    common/utils: Add utility functions to check swh-web run environment
    
    Add functions to check if we are running a development, staging or
    production version of swh-web.

    Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1401/ See console output for more information: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1401/console

  • Software Heritage Continuous Integration for public repositories
    Software Heritage Continuous Integration for public repositories @swh-public-ci ·

    Build has FAILED

    Patch application report for D7224 (id=26188)

    Could not rebase; Attempt merge onto c14cfd1b...

    Updating c14cfd1b..4d14f4b3
Fast-forward
 swh/web/common/utils.py             |  40 +++++++++---
 swh/web/config.py                   |   5 +-
 swh/web/misc/coverage.py            | 117 ++++++++++++++++++++++--------------
 swh/web/tests/common/test_utils.py  |  19 +++++-
 swh/web/tests/misc/test_coverage.py |  33 +++++-----
 swh/web/tests/test_templates.py     |  11 +++-
 6 files changed, 144 insertions(+), 81 deletions(-)
    Changes applied before test
    commit 4d14f4b335948954bce0e8054bb6c6d35e48bda4
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Feb 22 17:10:51 2022 +0100

    misc/coverage: Use django cache instead of LRU ones
    
    It enables to update origin counters more frequently in the coverage
    widget of the production webapp.
    
    Also disable cache use when running swh-web in development or staging
    environment to get accurate counters on each homepage refresh.

commit cc5736263984312914af673831932af01f6779eb
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Feb 22 16:26:14 2022 +0100

    common/utils: Add utility functions to check swh-web run environment
    
    Add functions to check if we are running a development, staging or
    production version of swh-web.

    Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1402/ See console output for more information: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1402/console

  • vlorentz
    vlorentz @vlorentz ·
    Maintainer

    great, thanks!

  • vlorentz
    vlorentz @vlorentz ·
    Maintainer

    Merge request was accepted

  • vlorentz approved this merge request

    approved this merge request

  • Antoine Lambert
    Antoine Lambert @anlambert ·
    Author Maintainer

    Return adequate default value when value is not in cache.

  • Software Heritage Continuous Integration for public repositories
    Software Heritage Continuous Integration for public repositories @swh-public-ci ·

    Build has FAILED

    Patch application report for D7224 (id=26199)

    Could not rebase; Attempt merge onto c14cfd1b...

    Updating c14cfd1b..7b917f18
Fast-forward
 swh/web/common/utils.py             |  40 +++++++++---
 swh/web/config.py                   |   5 +-
 swh/web/misc/coverage.py            | 117 ++++++++++++++++++++++--------------
 swh/web/tests/common/test_utils.py  |  19 +++++-
 swh/web/tests/misc/test_coverage.py |  33 +++++-----
 swh/web/tests/test_templates.py     |  11 +++-
 6 files changed, 144 insertions(+), 81 deletions(-)
    Changes applied before test
    commit 7b917f18fd155e8d76f2df6edb0c5b72aabe8530
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Feb 22 17:10:51 2022 +0100

    misc/coverage: Use django cache instead of LRU ones
    
    It enables to update origin counters more frequently in the coverage
    widget of the production webapp.
    
    Also disable cache use when running swh-web in development or staging
    environment to get accurate counters on each homepage refresh.

commit cc5736263984312914af673831932af01f6779eb
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Feb 22 16:26:14 2022 +0100

    common/utils: Add utility functions to check swh-web run environment
    
    Add functions to check if we are running a development, staging or
    production version of swh-web.

    Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1403/ See console output for more information: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1403/console

  • Software Heritage Continuous Integration for public repositories
    Software Heritage Continuous Integration for public repositories @swh-public-ci ·

    Build has FAILED

    Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1404/ See console output for more information: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1404/console

  • Software Heritage Continuous Integration for public repositories
    Software Heritage Continuous Integration for public repositories @swh-public-ci ·

    Build is green

    Patch application report for D7224 (id=26199)

    Could not rebase; Attempt merge onto c14cfd1b...

    Updating c14cfd1b..7b917f18
Fast-forward
 swh/web/common/utils.py             |  40 +++++++++---
 swh/web/config.py                   |   5 +-
 swh/web/misc/coverage.py            | 117 ++++++++++++++++++++++--------------
 swh/web/tests/common/test_utils.py  |  19 +++++-
 swh/web/tests/misc/test_coverage.py |  33 +++++-----
 swh/web/tests/test_templates.py     |  11 +++-
 6 files changed, 144 insertions(+), 81 deletions(-)
    Changes applied before test
    commit 7b917f18fd155e8d76f2df6edb0c5b72aabe8530
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Feb 22 17:10:51 2022 +0100

    misc/coverage: Use django cache instead of LRU ones
    
    It enables to update origin counters more frequently in the coverage
    widget of the production webapp.
    
    Also disable cache use when running swh-web in development or staging
    environment to get accurate counters on each homepage refresh.

commit cc5736263984312914af673831932af01f6779eb
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Feb 22 16:26:14 2022 +0100

    common/utils: Add utility functions to check swh-web run environment
    
    Add functions to check if we are running a development, staging or
    production version of swh-web.

    See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1405/ for more details.

  • Antoine Lambert
    Antoine Lambert @anlambert ·
    Author Maintainer

    Merge request was merged

  • Antoine Lambert closed

    closed

Please register or sign in to reply