Hook up processing of inbound emails for add_forge_now
Closed
requested to merge generated-differential-D7326-source into generated-differential-D7326-target
Depends on !811 (closed) Related to T3999
Test Plan
Added an integration test in cypress that checks email processing and that the message source link shows up properly.
Migrated from D7326 (view on Phabricator)
Merge request reports
Activity
Build is green
Patch application report for D7326 (id=26499)
Could not rebase; Attempt merge onto 51aa6fbc...
Merge made by the 'recursive' strategy. swh/web/add_forge_now/__init__.py | 0 swh/web/add_forge_now/apps.py | 10 ++ swh/web/add_forge_now/migrations/0001_initial.py | 109 +++++++++++++++++++ swh/web/add_forge_now/migrations/__init__.py | 0 swh/web/add_forge_now/models.py | 118 +++++++++++++++++++++ swh/web/add_forge_now/signal_receivers.py | 76 +++++++++++++ swh/web/add_forge_now/tests/test_migration.py | 62 +++++++++++ swh/web/api/urls.py | 1 + swh/web/api/views/add_forge_now.py | 108 +++++++++++++++++++ swh/web/config.py | 1 + swh/web/inbound_email/__init__.py | 0 swh/web/inbound_email/apps.py | 11 ++ .../management/commands/process_inbound_email.py | 73 +++++++++++++ swh/web/inbound_email/signals.py | 36 +++++++ swh/web/inbound_email/tests.py | 3 + swh/web/inbound_email/utils.py | 66 ++++++++++++ swh/web/settings/common.py | 2 + swh/web/tests/api/views/test_add_forge_now.py | 117 ++++++++++++++++++++ swh/web/tests/inbound_email/test_utils.py | 113 ++++++++++++++++++++ swh/web/tests/utils.py | 4 +- 20 files changed, 909 insertions(+), 1 deletion(-) create mode 100644 swh/web/add_forge_now/__init__.py create mode 100644 swh/web/add_forge_now/apps.py create mode 100644 swh/web/add_forge_now/migrations/0001_initial.py create mode 100644 swh/web/add_forge_now/migrations/__init__.py create mode 100644 swh/web/add_forge_now/models.py create mode 100644 swh/web/add_forge_now/signal_receivers.py create mode 100644 swh/web/add_forge_now/tests/test_migration.py create mode 100644 swh/web/api/views/add_forge_now.py create mode 100644 swh/web/inbound_email/__init__.py create mode 100644 swh/web/inbound_email/apps.py create mode 100644 swh/web/inbound_email/management/commands/process_inbound_email.py create mode 100644 swh/web/inbound_email/signals.py create mode 100644 swh/web/inbound_email/tests.py create mode 100644 swh/web/inbound_email/utils.py create mode 100644 swh/web/tests/api/views/test_add_forge_now.py create mode 100644 swh/web/tests/inbound_email/test_utils.pyChanges applied before test
commit 977ffa190ef93d2bcf8e2828fe72d863920705fc Merge: 51aa6fbc 3804c203 Author: Jenkins user <jenkins@localhost> Date: Wed Mar 9 16:58:56 2022 +0000 Merge branch 'diff-target' into HEAD commit 3804c20354bd0cfdd259ce1d99df07a7ca3b9df0 Author: Nicolas Dandrimont <nicolas@dandrimont.eu> Date: Wed Mar 9 17:43:36 2022 +0100 wip: hook up processing of inbound emails for add_forge_now commit e49b4cee5906f89629d6eb89bbceacdd7ba91cc7 Author: Nicolas Dandrimont <nicolas@dandrimont.eu> Date: Wed Mar 9 17:48:45 2022 +0100 inbound_email: add some logging statements for unhandled exceptions commit 889a84771fce10d5483de6cd5991d4feee9d1234 Author: Nicolas Dandrimont <nicolas@dandrimont.eu> Date: Wed Mar 9 16:37:54 2022 +0100 inbound_email: add utilities to extract and filter email recipients commit 1c00cc8a08add62f30468a26b1d8784d51b3a2ec Author: Nicolas Dandrimont <nicolas@dandrimont.eu> Date: Wed Mar 9 14:29:37 2022 +0100 Bootstrap a common app for management of inbound emails commit 130e9faa9bcab18bc3c76dc898546edb89f3f9b8 Author: Antoine Lambert <anlambert@softwareheritage.org> Date: Tue Mar 8 15:23:50 2022 +0100 api: Add endpoint to create an add-forge request Related to [T3990](https://forge.softwareheritage.org/T3990 'view original for T3990 on Phabricator') commit 03101208803501e5178f35d18d171e740ee4ca76 Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org> Date: Tue Mar 8 11:34:44 2022 +0100 add_forge_now: Bootstrap app and modelSee https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1434/ for more details.
-
Build is green
Patch application report for D7326 (id=26549)
Could not rebase; Attempt merge onto cdcc1efd...
Merge made by the 'recursive' strategy. swh/web/add_forge_now/__init__.py | 0 swh/web/add_forge_now/apps.py | 10 + swh/web/add_forge_now/migrations/0001_initial.py | 109 +++++ swh/web/add_forge_now/migrations/__init__.py | 0 swh/web/add_forge_now/models.py | 146 ++++++ swh/web/add_forge_now/signal_receivers.py | 76 ++++ swh/web/add_forge_now/tests/test_migration.py | 62 +++ swh/web/api/urls.py | 1 + swh/web/api/views/add_forge_now.py | 352 +++++++++++++++ swh/web/config.py | 1 + swh/web/inbound_email/__init__.py | 0 swh/web/inbound_email/apps.py | 11 + .../management/commands/process_inbound_email.py | 73 +++ swh/web/inbound_email/signals.py | 36 ++ swh/web/inbound_email/utils.py | 63 +++ swh/web/settings/common.py | 2 + swh/web/tests/add_forge_now/test_models.py | 26 ++ swh/web/tests/api/views/test_add_forge_now.py | 489 +++++++++++++++++++++ .../tests/inbound_email/test_management_command.py | 161 +++++++ swh/web/tests/inbound_email/test_utils.py | 113 +++++ swh/web/tests/utils.py | 10 +- 21 files changed, 1738 insertions(+), 3 deletions(-) create mode 100644 swh/web/add_forge_now/__init__.py create mode 100644 swh/web/add_forge_now/apps.py create mode 100644 swh/web/add_forge_now/migrations/0001_initial.py create mode 100644 swh/web/add_forge_now/migrations/__init__.py create mode 100644 swh/web/add_forge_now/models.py create mode 100644 swh/web/add_forge_now/signal_receivers.py create mode 100644 swh/web/add_forge_now/tests/test_migration.py create mode 100644 swh/web/api/views/add_forge_now.py create mode 100644 swh/web/inbound_email/__init__.py create mode 100644 swh/web/inbound_email/apps.py create mode 100644 swh/web/inbound_email/management/commands/process_inbound_email.py create mode 100644 swh/web/inbound_email/signals.py create mode 100644 swh/web/inbound_email/utils.py create mode 100644 swh/web/tests/add_forge_now/test_models.py create mode 100644 swh/web/tests/api/views/test_add_forge_now.py create mode 100644 swh/web/tests/inbound_email/test_management_command.py create mode 100644 swh/web/tests/inbound_email/test_utils.pyChanges applied before test
commit 176c43d04ee5ed9f1cf43214f510c5ddb6268d0b Merge: cdcc1efd c60fc7c6 Author: Jenkins user <jenkins@localhost> Date: Mon Mar 14 12:37:32 2022 +0000 Merge branch 'diff-target' into HEAD commit c60fc7c6c8fe8d7e392f94533142b447ad70f435 Author: Nicolas Dandrimont <nicolas@dandrimont.eu> Date: Wed Mar 9 17:43:36 2022 +0100 wip: hook up processing of inbound emails for add_forge_now commit 531a8bd04ad6bc938a66565f0d146da08b172eeb Author: Nicolas Dandrimont <nicolas@dandrimont.eu> Date: Wed Mar 9 16:37:54 2022 +0100 inbound_email: add utilities to extract and filter email recipients commit 99413db8665d649e7a598233e972f374d6856e93 Author: Nicolas Dandrimont <nicolas@dandrimont.eu> Date: Wed Mar 9 14:29:37 2022 +0100 Bootstrap a common app for management of inbound emails commit 26748e56ecce5877fce3a215eba06a19ebb8342f Author: Antoine Lambert <anlambert@softwareheritage.org> Date: Thu Mar 10 14:09:36 2022 +0100 api: Add endpoint to get details about an add-forge request Related to [T4030](https://forge.softwareheritage.org/T4030 'view original for T4030 on Phabricator') commit 294a95c711bc40622c6d718afa2535f377b90dfc Author: Antoine Lambert <anlambert@softwareheritage.org> Date: Wed Mar 9 16:30:06 2022 +0100 api: Add endpoint to list add-forge requests Related to [T4027](https://forge.softwareheritage.org/T4027 'view original for T4027 on Phabricator') commit 1bf17d6a75fd7265b1da9c9e87220efa62869c93 Author: Antoine Lambert <anlambert@softwareheritage.org> Date: Wed Mar 9 16:28:03 2022 +0100 api: Add endpoint to update an add-forge request Related to [T4026](https://forge.softwareheritage.org/T4026 'view original for T4026 on Phabricator') commit 130e9faa9bcab18bc3c76dc898546edb89f3f9b8 Author: Antoine Lambert <anlambert@softwareheritage.org> Date: Tue Mar 8 15:23:50 2022 +0100 api: Add endpoint to create an add-forge request Related to [T3990](https://forge.softwareheritage.org/T3990 'view original for T3990 on Phabricator') commit 03101208803501e5178f35d18d171e740ee4ca76 Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org> Date: Tue Mar 8 11:34:44 2022 +0100 add_forge_now: Bootstrap app and modelSee https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1453/ for more details.
- Restrict pytest-postgresql to < 4.0.0
- inbound_email: split recipient matching logic out
- inbound_email: add support for signed email addresses
- add_forge_now: add inbound email address to model
- inbound_email: add function to extract the plaintext from a mail
- Add forge now: prepare fields for inbound email processing
- Add forge now: Add Cc and Reply-To to forge administrator email
- Add forge now: process inbound emails
-
Build was aborted
Patch application report for D7326 (id=27157)
Rebasing onto ad5add7d...
Current branch diff-target is up to date.Changes applied before test
commit 276076996b038406f51694768e41ec73165534d9 Author: Nicolas Dandrimont <nicolas@dandrimont.eu> Date: Fri Apr 1 17:30:19 2022 +0200 Add forge now: process inbound emails Add a signal receiver to process inbound emails to the address set up in config. The text part of the emails is parsed if available, and the raw message is saved as a last resort. commit cdd0fb00bf4d0c67407caa46ac950314eff530ff Author: Nicolas Dandrimont <nicolas@dandrimont.eu> Date: Fri Apr 1 17:16:36 2022 +0200 Add forge now: Add Cc and Reply-To to forge administrator email This sets Cc: and Reply-To to the inbound email for the add-forge-now feature. commit 6c8b7fdb3970c5d804a905931d2d9286b0d7a57b Author: Nicolas Dandrimont <nicolas@dandrimont.eu> Date: Fri Apr 1 17:08:47 2022 +0200 Add forge now: prepare fields for inbound email processing - add raw message field to request history table - add email as choice for the request history actor roles commit 7d45ac4711455e84720f504f07d4f57285171bf4 Author: Nicolas Dandrimont <nicolas@dandrimont.eu> Date: Fri Apr 1 17:04:30 2022 +0200 inbound_email: add function to extract the plaintext from a mail This function decodes the html part if no text part is available. commit a677e9d164e156b60f0281ab7ff20c84a237d102 Author: Nicolas Dandrimont <nicolas@dandrimont.eu> Date: Fri Apr 1 10:38:38 2022 +0200 add_forge_now: add inbound email address to model commit e88291be19b8f52167c3c38804cabc0410578a8f Author: Nicolas Dandrimont <nicolas@dandrimont.eu> Date: Thu Mar 31 16:01:56 2022 +0200 inbound_email: add support for signed email addresses These utilities allow us to generate addresses of the form `<localpart>+<integer>.<signature>@<domain>`, where the integer is the primary key of a given object for which we want to track email exchanges. The signature prevents the addresses from being forged, that is, the addresses have to be explicitly generated and displayed by the web app to be discovered. The counterpart function retrieves all relevant email addresses from the list of recipients of an email message, and validates the signatures to recover the integer values that won't have been tampered with. These new utilities are expected to be used in the views and signal handlers pertaining to processing of inbound email messages. commit f8fb7a04639729d6530f126ce1c7dd87ce82364c Author: Nicolas Dandrimont <nicolas@dandrimont.eu> Date: Fri Apr 1 15:16:11 2022 +0200 inbound_email: split recipient matching logic out This allows calling the function on a single recipient rather than on a whole message, when one isn't available. commit 1864b158225d2ae23bc81ac73cb3966caa2c65a1 Author: Nicolas Dandrimont <nicolas@dandrimont.eu> Date: Fri Apr 1 15:15:05 2022 +0200 Restrict pytest-postgresql to < 4.0.0 Other modules still need psycopg2 and pytest-postgresql 4 introduced a hard dependency on psycopg3.Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1648/ See console output for more information: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1648/console
-
Build is green
Patch application report for D7326 (id=27157)
Rebasing onto ad5add7d...
Current branch diff-target is up to date.Changes applied before test
commit 276076996b038406f51694768e41ec73165534d9 Author: Nicolas Dandrimont <nicolas@dandrimont.eu> Date: Fri Apr 1 17:30:19 2022 +0200 Add forge now: process inbound emails Add a signal receiver to process inbound emails to the address set up in config. The text part of the emails is parsed if available, and the raw message is saved as a last resort. commit cdd0fb00bf4d0c67407caa46ac950314eff530ff Author: Nicolas Dandrimont <nicolas@dandrimont.eu> Date: Fri Apr 1 17:16:36 2022 +0200 Add forge now: Add Cc and Reply-To to forge administrator email This sets Cc: and Reply-To to the inbound email for the add-forge-now feature. commit 6c8b7fdb3970c5d804a905931d2d9286b0d7a57b Author: Nicolas Dandrimont <nicolas@dandrimont.eu> Date: Fri Apr 1 17:08:47 2022 +0200 Add forge now: prepare fields for inbound email processing - add raw message field to request history table - add email as choice for the request history actor roles commit 7d45ac4711455e84720f504f07d4f57285171bf4 Author: Nicolas Dandrimont <nicolas@dandrimont.eu> Date: Fri Apr 1 17:04:30 2022 +0200 inbound_email: add function to extract the plaintext from a mail This function decodes the html part if no text part is available. commit a677e9d164e156b60f0281ab7ff20c84a237d102 Author: Nicolas Dandrimont <nicolas@dandrimont.eu> Date: Fri Apr 1 10:38:38 2022 +0200 add_forge_now: add inbound email address to model commit e88291be19b8f52167c3c38804cabc0410578a8f Author: Nicolas Dandrimont <nicolas@dandrimont.eu> Date: Thu Mar 31 16:01:56 2022 +0200 inbound_email: add support for signed email addresses These utilities allow us to generate addresses of the form `<localpart>+<integer>.<signature>@<domain>`, where the integer is the primary key of a given object for which we want to track email exchanges. The signature prevents the addresses from being forged, that is, the addresses have to be explicitly generated and displayed by the web app to be discovered. The counterpart function retrieves all relevant email addresses from the list of recipients of an email message, and validates the signatures to recover the integer values that won't have been tampered with. These new utilities are expected to be used in the views and signal handlers pertaining to processing of inbound email messages. commit f8fb7a04639729d6530f126ce1c7dd87ce82364c Author: Nicolas Dandrimont <nicolas@dandrimont.eu> Date: Fri Apr 1 15:16:11 2022 +0200 inbound_email: split recipient matching logic out This allows calling the function on a single recipient rather than on a whole message, when one isn't available. commit 1864b158225d2ae23bc81ac73cb3966caa2c65a1 Author: Nicolas Dandrimont <nicolas@dandrimont.eu> Date: Fri Apr 1 15:15:05 2022 +0200 Restrict pytest-postgresql to < 4.0.0 Other modules still need psycopg2 and pytest-postgresql 4 introduced a hard dependency on psycopg3.See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1649/ for more details.
-
rebase on top of !1065 (closed)
-
Build has FAILED
Patch application report for D7326 (id=27238)
Could not rebase; Attempt merge onto 8126ea65...
Merge made by the 'recursive' strategy. .pre-commit-config.yaml | 1 + MANIFEST.in | 1 + assets/src/bundles/add_forge/request-dashboard.js | 4 +- swh/web/add_forge_now/apps.py | 6 + .../migrations/0005_prepare_inbound_email.py | 30 +++++ swh/web/add_forge_now/models.py | 9 ++ swh/web/add_forge_now/signal_receivers.py | 70 +++++++++++ swh/web/api/views/add_forge_now.py | 4 +- swh/web/config.py | 1 + swh/web/inbound_email/utils.py | 126 ++++++++++++++++++- .../templates/add_forge_now/request-dashboard.html | 2 +- swh/web/tests/api/views/test_add_forge_now.py | 16 +++ swh/web/tests/inbound_email/__init__.py | 0 swh/web/tests/inbound_email/resources/__init__.py | 0 .../tests/inbound_email/resources/multipart.eml | 24 ++++ .../tests/inbound_email/resources/plaintext.eml | 15 +++ swh/web/tests/inbound_email/test_utils.py | 133 +++++++++++++++++++++ 17 files changed, 438 insertions(+), 4 deletions(-) create mode 100644 swh/web/add_forge_now/migrations/0005_prepare_inbound_email.py create mode 100644 swh/web/add_forge_now/signal_receivers.py create mode 100644 swh/web/tests/inbound_email/__init__.py create mode 100644 swh/web/tests/inbound_email/resources/__init__.py create mode 100644 swh/web/tests/inbound_email/resources/multipart.eml create mode 100644 swh/web/tests/inbound_email/resources/plaintext.emlChanges applied before test
commit 69154f9376857c9bd8f46d442696a6a9d0294a8f Merge: 8126ea65 18c55d79 Author: Jenkins user <jenkins@localhost> Date: Tue Apr 5 15:51:28 2022 +0000 Merge branch 'diff-target' into HEAD commit 18c55d79c98f206b525a243b86a97c7d73df107e Author: Nicolas Dandrimont <nicolas@dandrimont.eu> Date: Fri Apr 1 17:30:19 2022 +0200 Add forge now: process inbound emails Add a signal receiver to process inbound emails to the address set up in config. The text part of the emails is parsed if available, and the raw message is saved as a last resort. commit ea50915c8caf95c45906f94d1b918247a7ca7998 Author: Nicolas Dandrimont <nicolas@dandrimont.eu> Date: Fri Apr 1 17:16:36 2022 +0200 Add forge now: Add Cc and Reply-To to forge administrator email This sets Cc: and Reply-To to the inbound email for the add-forge-now feature. commit 65ef2fac94f634529f807b23c06246ea0f49d11c Author: Nicolas Dandrimont <nicolas@dandrimont.eu> Date: Fri Apr 1 17:08:47 2022 +0200 Add forge now: prepare fields for inbound email processing - add raw message field to request history table - add email as choice for the request history actor roles commit 7802e3b9d01cd177b08a615fe675531eec6d2238 Author: Nicolas Dandrimont <nicolas@dandrimont.eu> Date: Fri Apr 1 10:38:38 2022 +0200 add_forge_now: add inbound email address to model commit b690064679fde54f912c8b77b2ea4ac064db19ac Author: Nicolas Dandrimont <nicolas@dandrimont.eu> Date: Fri Apr 1 17:04:30 2022 +0200 inbound_email: add function to extract the plaintext from a mail This function uses the html part if no text part is available. commit 620d14c60e2c3152bacf9ccaa1d23edbc4a5f98b Author: Nicolas Dandrimont <nicolas@dandrimont.eu> Date: Thu Mar 31 16:01:56 2022 +0200 inbound_email: add support for signed email addresses These utilities allow us to generate addresses of the form `<localpart>+<integer>.<signature>@<domain>`, where the integer is the primary key of a given object for which we want to track email exchanges. The signature prevents the addresses from being forged, that is, the addresses have to be explicitly generated and displayed by the web app to be discovered. The counterpart function retrieves all relevant email addresses from the list of recipients of an email message, and validates the signatures to recover the integer values that won't have been tampered with. These new utilities are expected to be used in the views and signal handlers pertaining to processing of inbound email messages.Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1684/ See console output for more information: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1684/console
Below is a diff to allow end to end testing with cypress with inbound email processing. I only handle the mail sent by us to the forge administrator but simulating his response should be quite similar.
diff --git a/assets/src/bundles/add_forge/add-request-history-item.ejs b/assets/src/bundles/add_forge/add-request-history-item.ejs index 57b43972..869e1786 100644 --- a/assets/src/bundles/add_forge/add-request-history-item.ejs +++ b/assets/src/bundles/add_forge/add-request-history-item.ejs @@ -19,7 +19,9 @@ </div> <div id="collapse<%= index %>" class="collapse" aria-labelledby="historyItem<%= index %>" data-parent="#requestHistory"> <div class="card-body"> - <p><%= event.text %></p> + <%if (event.text) { %> + <pre><%= event.text %></pre> + <% } %> <%if (event.new_status !== null) { %> <p> <span>Status changed to:</span> <strong><%= swh.add_forge.formatRequestStatusName(event.new_status) %></strong> diff --git a/cypress/integration/add-forge-now-request-dashboard.spec.js b/cypress/integration/add-forge-now-request-dashboard.spec.js index 70bdaeca..2e8c88bb 100644 --- a/cypress/integration/add-forge-now-request-dashboard.spec.js +++ b/cypress/integration/add-forge-now-request-dashboard.spec.js @@ -6,29 +6,33 @@ */ let requestId; +let requestInboundEmailAddress; + +const requestData = { + forge_type: 'bitbucket', + forge_url: 'test.example.com', + forge_contact_email: 'test@example.com', + forge_contact_name: 'test user', + submitter_forward_username: true, + forge_contact_comment: 'test comment' +}; function createDummyRequest(urls) { cy.task('db:add_forge_now:delete'); cy.userLogin(); - cy.getCookie('csrftoken').its('value').then((token) => { + return cy.getCookie('csrftoken').its('value').then((token) => { cy.request({ method: 'POST', url: urls.api_1_add_forge_request_create(), - body: { - forge_type: 'bitbucket', - forge_url: 'test.example.com', - forge_contact_email: 'test@example.com', - forge_contact_name: 'test user', - submitter_forward_username: true, - forge_contact_comment: 'test comment' - }, + body: requestData, headers: { 'X-CSRFToken': token } }).then((response) => { // setting requestId from response requestId = response.body.id; + requestInboundEmailAddress = response.body.inbound_email_address; // logout the user cy.visit(urls.swh_web_homepage()); cy.contains('a', 'logout').click(); @@ -36,6 +40,45 @@ function createDummyRequest(urls) { }); } +function genEmailSrc() { + return `Message-ID: <d5c43e75-2a11-250a-43e3-37034ae3904b@inria.fr> +Date: Tue, 19 Apr 2022 14:00:56 +0200 +MIME-Version: 1.0 +User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 + Thunderbird/91.8.0 +To: ${requestData.forge_contact_email} +Cc: ${requestInboundEmailAddress} +Subject: [swh-add_forge_now] Request 1 +Content-Language: en-US +From: Antoine Lambert <anlambert@softwareheritage.org> +Content-Type: text/plain; charset=UTF-8; format=flowed +Content-Transfer-Encoding: 7bit + +Dear forge administrator, + +The mission of Software Heritage is to collect, preserve and share all the +publicly available source code (see https://www.softwareheritage.org for more +information). + +We just received a request to add the forge hosted at https://test.cgit.org to the +list of software origins that are archived, and it is our understanding that you +are the contact person for this forge. + +In order to archive the forge contents, we will have to periodically pull the +public repositories it contains and clone them into the +Software Heritage archive. + +Would you be so kind as to reply to this message to acknowledge the reception +of this email and let us know if there are any special steps we should take in +order to properly archive the public repositories hosted on your infrastructure? + +Thank you in advance for your help. + +Kind regards, +The Software Heritage team +`; +} + describe('Test add forge now request dashboard load', function() { before(function() { @@ -148,15 +191,16 @@ function populateAndSubmitForm() { describe('Test forge now request update', function() { beforeEach(function() { - createDummyRequest(this.Urls); - - const url = this.Urls.add_forge_now_request_dashboard(requestId); - cy.adminLogin(); - // intercept GET API on page load - cy.intercept(`${this.Urls.api_1_add_forge_request_get(requestId)}**`).as('forgeRequestGet'); - // intercept update POST API - cy.intercept('POST', `${this.Urls.api_1_add_forge_request_update(requestId)}**`).as('forgeRequestUpdate'); - cy.visit(url); + createDummyRequest(this.Urls).then(() => { + + this.url = this.Urls.add_forge_now_request_dashboard(requestId); + cy.adminLogin(); + // intercept GET API on page load + cy.intercept(`${this.Urls.api_1_add_forge_request_get(requestId)}**`).as('forgeRequestGet'); + // intercept update POST API + cy.intercept('POST', `${this.Urls.api_1_add_forge_request_update(requestId)}**`).as('forgeRequestUpdate'); + cy.visit(this.url); + }); }); it('should submit correct details', function() { @@ -212,6 +256,20 @@ describe('Test forge now request update', function() { .should('have.length', 2); }); + it('should update the dashboard after sending or receiving email', function() { + cy.wait('@forgeRequestGet'); + cy.task('processAddForgeNowInboundEmail', genEmailSrc()); + cy.visit(this.url); + + cy.get('#requestHistory') + .children() + .should('have.length', 2); + + cy.get('#requestHistory') + .children() + .should('contain', 'New status: Waiting for feedback'); + }); + it('should show an error on API failure', function() { cy.intercept('POST', `${this.Urls.api_1_add_forge_request_update(requestId)}**`, diff --git a/cypress/plugins/index.js b/cypress/plugins/index.js index fac4b035..a7f7146b 100644 --- a/cypress/plugins/index.js +++ b/cypress/plugins/index.js @@ -6,6 +6,7 @@ */ const axios = require('axios'); +const {execFileSync} = require('child_process'); const fs = require('fs'); const sqlite3 = require('sqlite3').verbose(); @@ -159,6 +160,16 @@ module.exports = (on, config) => { }); db.close(); return true; + }, + processAddForgeNowInboundEmail(emailSrc) { + try { + execFileSync('django-admin', + ['process_inbound_email', '--settings=swh.web.settings.tests'], + {input: emailSrc}); + return true; + } catch (_) { + return false; + } } }); return config;-
Rebase on top of !811 (closed).
- Use and tweak @anlambert's suggestions to properly test email processing in cypress.
- Add tests that the message source endpoint works as expected
-
Build was aborted
Patch application report for D7326 (id=28195)
Rebasing onto 5d37baf2...
Current branch diff-target is up to date.Changes applied before test
commit 7650e3e36a61f98ea6eb7281c762f9dcb54d99e7 Author: Nicolas Dandrimont <nicolas@dandrimont.eu> Date: Tue May 10 15:43:22 2022 +0200 add forge now: only catch expected exceptions in request dashboard The unqualified catch would hide all exceptions instead of only setting an error message when the request itself fails. commit 4179f3d5510755604a0b8634018efb8c1a1a2011 Author: Nicolas Dandrimont <nicolas@dandrimont.eu> Date: Fri Apr 1 17:30:19 2022 +0200 Add forge now: process inbound emails Add a signal receiver to process inbound emails to the address set up in config. The text part of the emails is parsed if available, and the raw message is saved as a last resort. commit ecf5d88828036f208a8fb63f22f58fb095100cf1 Author: Nicolas Dandrimont <nicolas@dandrimont.eu> Date: Fri Apr 1 17:16:36 2022 +0200 Add forge now: set Cc and Reply-To for admin emails This sets the Cc and Reply-To headers to the email address for inbound message processing, when sending an email to the forge administrator.Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1822/ See console output for more information: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1822/console
-
Build is green
Patch application report for D7326 (id=28196)
Rebasing onto 5d37baf2...
Current branch diff-target is up to date.Changes applied before test
commit 080c988fe92067d4c82a56884e37eb79f719d5e4 Author: Nicolas Dandrimont <nicolas@dandrimont.eu> Date: Tue May 10 16:21:10 2022 +0200 Add forge now: only catch expected exceptions in request dashboard The unqualified catch would hide all exceptions instead of only setting an error message when the request itself fails. commit dc5d570ea30acbcf6e8b1b2a6a8dae9b80a19a9f Author: Nicolas Dandrimont <nicolas@dandrimont.eu> Date: Tue May 10 16:20:57 2022 +0200 Add forge now: process inbound emails Add a signal receiver to process inbound emails to the address set up in config. The text part of the emails is parsed if available, and the raw message is saved as a last resort. commit f80541830b617ff48df8360dda5046306c6bd165 Author: Nicolas Dandrimont <nicolas@dandrimont.eu> Date: Tue May 10 16:20:46 2022 +0200 Add forge now: set Cc and Reply-To for admin emails This sets the Cc and Reply-To headers to the email address for inbound message processing, when sending an email to the forge administrator.See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1823/ for more details.
Please register or sign in to reply