Actions Engine Possibilities
The actions engine has some amazing possibilities that I wanted to highlight, and hopefully see from the Currents team if any of these would be feasible given its existing implementation. My proposition: Could the Actions Engine accept some sort of custom script that will do whatever we might want? This would allow any user to hook into the results of their tests, powered by, and easilly visible and modifiable from within Currents. Hereβs a use case for us that would be compelling: Our team has been building out a comprehensive flake reporting system for our Cypress E2E test suite. We have a bad flakiness problem, and have written a custom reporting plugin that will: 1. detect a flake 2. create a jira ticket for the flake for a given unique test and error message 3. if the flake occurs again, it will find this jira ticket and update it with more information Some feedback within our team on this would be: 1. can we then attach grafana logs if an error contains a request ID? 2. can we only assign someone if the flake has occurred more than once so we can prioritise? the possibilities are endless, so having the Actions Engine attempt to build a one-size-fits-all solution for all the things people might want to do with test results might be an endless job. Putting the power into the hands of the users to script their own actions would be sick! What do others think? Is this feasible within the current engine?
Parker Rowe 18 days ago
Integration
Actions Engine Possibilities
The actions engine has some amazing possibilities that I wanted to highlight, and hopefully see from the Currents team if any of these would be feasible given its existing implementation. My proposition: Could the Actions Engine accept some sort of custom script that will do whatever we might want? This would allow any user to hook into the results of their tests, powered by, and easilly visible and modifiable from within Currents. Hereβs a use case for us that would be compelling: Our team has been building out a comprehensive flake reporting system for our Cypress E2E test suite. We have a bad flakiness problem, and have written a custom reporting plugin that will: 1. detect a flake 2. create a jira ticket for the flake for a given unique test and error message 3. if the flake occurs again, it will find this jira ticket and update it with more information Some feedback within our team on this would be: 1. can we then attach grafana logs if an error contains a request ID? 2. can we only assign someone if the flake has occurred more than once so we can prioritise? the possibilities are endless, so having the Actions Engine attempt to build a one-size-fits-all solution for all the things people might want to do with test results might be an endless job. Putting the power into the hands of the users to script their own actions would be sick! What do others think? Is this feasible within the current engine?
Parker Rowe 18 days ago
Integration
Have failure reason in weekly emails
See main failure reasons can help reducing time checking why tests failed mainly and knowing where to focus efforts in bug fixing.
miguel@currents.dev 2 months ago
Have failure reason in weekly emails
See main failure reasons can help reducing time checking why tests failed mainly and knowing where to focus efforts in bug fixing.
miguel@currents.dev 2 months ago
Planned
Send an alert if a test exceeds a flakiness rate threshold
Whenever a test exceed a certain threshold of flakiness (or other metric), send an alert / notification to an email address or to an integration. Example scenario If a test exceeds 20% flakiness rate based on 14-day recordings, from a particular branch, then we send an alert to test owner or a designated email.
Andrew Goldis 2 months ago
Observability
Planned
Send an alert if a test exceeds a flakiness rate threshold
Whenever a test exceed a certain threshold of flakiness (or other metric), send an alert / notification to an email address or to an integration. Example scenario If a test exceeds 20% flakiness rate based on 14-day recordings, from a particular branch, then we send an alert to test owner or a designated email.
Andrew Goldis 2 months ago
Observability
Accessibility reports / dashboard
Playwright has a built-in support for accessibility testing https://playwright.dev/docs/accessibility-testing. Currents can process accessibility violation reports, and emphasize those in the dashboard: highlight violations in a more user-friendly and visible manner (today only accessible if accessibility report attached to a test) collect and track accessibility metrics (possibly via lighthouse reports)
Andrew Goldis 2 months ago
Accessibility
Accessibility reports / dashboard
Playwright has a built-in support for accessibility testing https://playwright.dev/docs/accessibility-testing. Currents can process accessibility violation reports, and emphasize those in the dashboard: highlight violations in a more user-friendly and visible manner (today only accessible if accessibility report attached to a test) collect and track accessibility metrics (possibly via lighthouse reports)
Andrew Goldis 2 months ago
Accessibility
v8 coverage reports for Playwright
Currently we only support collecting coverage reports from instrumented code (via istanbuljs), but Playwright can collect v8 coverage reports directly from supporting browsers.
Andrew Goldis 3 months ago
Coverage
v8 coverage reports for Playwright
Currently we only support collecting coverage reports from instrumented code (via istanbuljs), but Playwright can collect v8 coverage reports directly from supporting browsers.
Andrew Goldis 3 months ago
Coverage
Enable integration with Jira
Enable integration with Jira to open tickets related to test failures. P.S. Can be implemented today with webhooks on a Run level.
Andrew Goldis 3 months ago
Integration
Enable integration with Jira
Enable integration with Jira to open tickets related to test failures. P.S. Can be implemented today with webhooks on a Run level.
Andrew Goldis 3 months ago
Integration
Integration with Linear
Enable integration with Linear to open tickets related to test failures. P.S. Can be implemented today with webhooks on a Run level.
Andrew Goldis 3 months ago
Integration
Integration with Linear
Enable integration with Linear to open tickets related to test failures. P.S. Can be implemented today with webhooks on a Run level.
Andrew Goldis 3 months ago
Integration
Email notifications with run results
Send an email similar to Slack and Teams reports as an email notification with passed, failed tests and a link to the currents run in it.
Andrew Goldis 4 months ago
Integration
Email notifications with run results
Send an email similar to Slack and Teams reports as an email notification with passed, failed tests and a link to the currents run in it.
Andrew Goldis 4 months ago
Integration
Add Test Notes via Dashboard
Allow users to add / and edit notes. The notes can be attached to: test recording - will be visible only for specific recording test definition - will show up for all relevant test recordings
Andrew Goldis 5 months ago
Add Test Notes via Dashboard
Allow users to add / and edit notes. The notes can be attached to: test recording - will be visible only for specific recording test definition - will show up for all relevant test recordings
Andrew Goldis 5 months ago
In Progress
Evaluation and Quarantine Mode
Allow ignoring certain tests and prevent certain failures from failing executions by adding test to βquarantineβ list.
Andrew Goldis 5 months ago
In Progress
Evaluation and Quarantine Mode
Allow ignoring certain tests and prevent certain failures from failing executions by adding test to βquarantineβ list.
Andrew Goldis 5 months ago
In Progress
Single Slack notification for all groups in a run
Some teams have 10+ groups per run and getting a notification for each is quite annoying. A solution should send one notification when a run completes and the results should include all the groups.
Andrew Goldis 6 months ago
In Progress
Single Slack notification for all groups in a run
Some teams have 10+ groups per run and getting a notification for each is quite annoying. A solution should send one notification when a run completes and the results should include all the groups.
Andrew Goldis 6 months ago
In Progress
GitHub Commit Status: Single check for all groups
Having a commit status check for each project run can be too noisy. Some customers are only interested in knowing whether any of the groups failed. Desired: A single check "Current Tests" for all 3 groups. Notes: This might be a more common scenario with users that have one project for their main test suites, and additional projects for "setup"/"teardown". Having the groups' execution details outlined in the PR comment is useful and shouldn't be changed.
walter@currents.dev 8 months ago
Integration
In Progress
GitHub Commit Status: Single check for all groups
Having a commit status check for each project run can be too noisy. Some customers are only interested in knowing whether any of the groups failed. Desired: A single check "Current Tests" for all 3 groups. Notes: This might be a more common scenario with users that have one project for their main test suites, and additional projects for "setup"/"teardown". Having the groups' execution details outlined in the PR comment is useful and shouldn't be changed.
walter@currents.dev 8 months ago
Integration