alfresco-content-app/e2e/playwright/README.md

Playwright E2E Tests

This directory contains end-to-end (E2E) tests for the Alfresco Content Application using Playwright.

Table of Contents

• Running Tests Locally
• Running Tests in CI
• Viewing Test Results
• Browser Support
• Excluding Tests
• Test Configuration
• Troubleshooting

Running Tests Locally

Prerequisites

1. Install dependencies:

   npm ci
   

2. Install Playwright browsers:

   # Install Chromium only (default)
   npm run ci:playwright:install
   
   # Install all browsers (Chromium, Chrome, Firefox, WebKit, MS Edge)
   npm run ci:playwright:install:all
   

   Note: Chrome/Chromium and MS Edge use real browser binaries. Firefox uses Playwright's patched version. Safari/WebKit uses emulation (not supported in CI).

Running a Specific Test Suite

Each test suite can be run independently using Nx:

# Run authentication tests
nx run authentication-e2e:e2e

# Run viewer tests
nx run viewer-e2e:e2e

# Run search tests
nx run search-e2e:e2e

Running with Options

You can run tests with various configurations:

# Run in UI mode (interactive)
nx run authentication-e2e:e2e --configuration=ui

# Run in headed mode (see browser)
nx run authentication-e2e:e2e --configuration=headed

# Run in debug mode
nx run authentication-e2e:e2e --configuration=debug

Running with Specific Browser

Set the PLAYWRIGHT_BROWSER environment variable:

# Run with Firefox
PLAYWRIGHT_BROWSER=firefox nx run authentication-e2e:e2e

# Run with WebKit
PLAYWRIGHT_BROWSER=webkit nx run authentication-e2e:e2e

# Run with MS Edge
PLAYWRIGHT_BROWSER=msedge nx run authentication-e2e:e2e

# Run with Chrome (default)
PLAYWRIGHT_BROWSER=chrome nx run authentication-e2e:e2e

Running All Tests

To run all test suites, you can use the CI command:

# Set the E2E_TARGET environment variable to the suite name
E2E_TARGET=authentication npm run ci:e2e

Running Tests in CI

Pull Request Workflow

When a pull request is opened, synchronized, or reopened against master or develop branches, the following happens:

1. Lint - Code linting checks
2. Build - Application build
3. Unit Tests - Unit test execution
4. E2E Playwright Tests - All test suites run across multiple browsers

Test Matrix

The CI runs tests in a matrix configuration:

• Browsers: all, chromium, firefox, webkit, msedge
• Test Suites: All available test suites

Each combination of browser and test suite runs in parallel, creating multiple test jobs.

Example Job Name Format

E2E | chromium | authentication | Playwright
E2E | firefox | viewer | Playwright

Scheduled Workflows

Tests also run on a schedule:

• Schedule: Every weekday at 12:00 UTC (0 12 * * 1-5)
• Configuration: Same as pull request workflow

CI Environment Variables

The following environment variables are set in CI:

BASE_URL: http://localhost
ADMIN_EMAIL: admin
ADMIN_PASSWORD: admin
PLAYWRIGHT_E2E_HOST: http://localhost:4200
PLAYWRIGHT_BROWSER: ${{ matrix.browser }}
MAXINSTANCES: 2
RETRY_COUNT: 2

CI Test Execution Flow

1. Deploy Local ACS
2. Install Dependencies (npm ci)
3. Install Playwright Browsers
4. Start Application
5. Run Tests
6. Collect Artifacts (reports, screenshots, videos, traces)

Viewing Test Results

When tests run in CI, results are available in two places: GitHub Actions and Report Portal.

Viewing Results in GitHub Actions

1. Navigate to the repository's Actions tab and select the workflow run
2. Find the specific test job (e.g., E2E | chromium | authentication | Playwright)
3. View test output and logs in the job output
4. Download test artifacts on failure:
   • Screenshots, videos (.webm), and traces (.zip)
   • Analyze traces using: npx playwright show-trace <trace-file.zip>
5. Application logs are available as artifacts named logs-<browser>-<suite-name>

Viewing Results in Report Portal

CI e2e test results are automatically reported to Report Portal for analysis and historical tracking. The Report Portal URL is available in the GitHub Actions workflow run under "Run e2e Playwright" section.

Sample URL format: https://reportportal-dev.studio.dev.experience.hyland.com/ui/#alfresco-content-app/launches/all/<launch-id>

• Project name: alfresco-content-app
• Tests are reported only in CI environments (not locally)

Browser Support

The tests support the following browsers:

• Chromium (default) - Uses real Google Chrome browser
• Firefox - Uses Playwright's patched version
• WebKit - Safari-based browser (uses emulation, not supported in CI)
• MS Edge - Uses real Microsoft Edge browser

Browser-specific test exclusions are supported (see Excluding Tests).

Excluding Tests

How It Works

Test exclusion is handled through exclude.tests.json files located in each test suite directory. The exclusion system:

1. Reads the exclude.tests.json file for the current test suite
2. Checks the PLAYWRIGHT_BROWSER environment variable to determine the current browser
3. Applies exclusions based on:
   • all - Excludes tests for all browsers
   • chromium - Excludes tests only for Chromium
   • firefox - Excludes tests only for Firefox
   • webkit - Excludes tests only for WebKit
   • msedge - Excludes tests only for MS Edge

Exclusion File Format

Each exclude.tests.json file follows this structure:

{
  "all": {
    "XAT-4370": "https://hyland.atlassian.net/browse/ACS-5479"
  },
  "firefox": {
    "XAT-17773": "https://hyland.atlassian.net/browse/ACS-10917"
  },
  "webkit": {
    "XAT-5417": "https://hyland.atlassian.net/browse/ACS-10917"
  }
}

Adding Test Exclusions

Example:

{
  "all": {
    "XAT-4370": "https://hyland.atlassian.net/browse/ACS-5479"
  },
  "firefox": {
    "XAT-17773": "https://hyland.atlassian.net/browse/ACS-10917"
  }
}

How Exclusions Are Applied

• The exclusion system uses regex patterns to match test IDs
• Tests matching excluded IDs are skipped during execution
• Console output shows which tests are excluded and why
• Example console output:

  [ 🎭 Playwright Excludes - Authentication ] ❌ Tests excluded for browser 'firefox' because of 🐛 : XAT-17773 (from keys: all, firefox)
  

Best Practices

• Always include a Jira ticket - Link to the bug or issue tracking the problem
• Use specific browser exclusions - Only exclude tests from browsers where they fail
• Review exclusions regularly - Remove exclusions when bugs are fixed
• Document the reason - The Jira ticket URL serves as documentation

Test Configuration

Global Configuration

Global test configuration is defined in projects/aca-playwright-shared/src/base-config/playwright.config.ts:

• Timeout: Global test timeout
• Retries: 2 retries on CI, 0 locally
• Workers: 3 parallel workers
• Headless: true on CI, configurable locally
• Traces: Retained on failure
• Videos: Retained on failure
• Screenshots: Captured only on failure

Suite-Specific Configuration

Each test suite has its own playwright.config.ts that:

• Extends the global configuration
• Applies test exclusions via grepInvert
• Defines suite-specific projects

Environment Variables

Variable	Description	Default
PLAYWRIGHT_BROWSER	Browser to use for tests	chromium
PLAYWRIGHT_E2E_HOST	Base URL for the application	http://localhost:4200
PLAYWRIGHT_HEADLESS	Run in headless mode	true (CI) / false (local)

Troubleshooting

Tests Fail Locally

1. Check application is running:

   npm start
   

   Ensure the app is accessible at http://localhost:4200

2. Verify browser installation:

   npx playwright install
   

3. Check environment variables:

   echo $PLAYWRIGHT_BROWSER
   echo $PLAYWRIGHT_E2E_HOST
   

CI Failures

When tests fail in CI:

1. Navigate to the failed job in GitHub Actions
2. Download test artifacts (screenshots, videos, traces) from the job's Artifacts section
3. Analyze traces using: npx playwright show-trace <trace-file.zip>

Additional Debugging Steps

1. Verify environment - Ensure all required services (ACS) are deployed correctly
2. Check environment variables - Verify all required secrets and variables are set
3. Review related logs - Check ACS deployment logs and application logs

Debug Mode

Run tests in debug mode to step through execution:

nx run authentication-e2e:e2e --configuration=debug

This will:

• Open Playwright Inspector
• Allow step-by-step execution
• Show browser DevTools

Additional Resources

• Project-specific test utilities: projects/aca-playwright-shared/