Elevating Flutter Test Reports with Allure

Testing is a crucial part of application development, ensuring software behaves as expected. As testers, we aim to provide clear reports that document test results, highlight failures, and include visual evidence such as screenshots. These reports help developers, project managers, product owners, stakeholders, and testers to assess stability and make informed decisions.

‍

For Flutter integration testing, well-structured reports are even more critical. Traditional Flutter end-to-end automation testing reports often include dense logs that are counterintuitive and lack actionable insights to analyze and share. Instead, reports should prioritize clarity, visual representation, and actionable insights, making them accessible to both technical and non-technical audiences.

‍

We identified an opportunity to amplify how existing Flutter UI testing frameworks present their results. This blog introduces how reports typically look today, highlights the reporting needs our team faced, and walks through a solution that elevates test reporting with greater control and clarity.

Reports That Work for Every Audience

We wanted to enhance the communication of test results across teams. Our goal was to create a reporting solution that delivers clear, detailed, and well-structured test execution data, accessible to both technical and non-technical audiences. While existing tools—such as Flutter Integration Tests, Maestro, Fluttium, and Patrol—offer valuable capabilities, we identified opportunities to further enhance the experience by building a more intuitive and stakeholder-friendly reporting solution.

Key requirements for the reporting solution:

1. Step-by-Step Breakdown: The report should provide a detailed sequence of test steps, clearly marking passed and failed steps. This helps pinpoint issues and understand the test flow.
   
   
2. Non-Technical Readability: The report must be easily understandable by PMs, clients, and other stakeholders, avoiding excessive technical jargon while maintaining clarity.
   
   
3. Visual Evidence: Screenshots or other visual elements should be included at key steps, especially where failures occur, to provide immediate context.
   
   
4. Framework Compatibility: The solution should support both Flutter Integration Tests and Patrol.

Current Solutions

Before building a custom solution, we explored how existing tools handle test reporting. We tested the most widely used UI testing frameworks in the Flutter ecosystem: Maestro, Fluttium, Flutter Integration Test, and Patrol, using a basic Flutter sample app to evaluate how each presents test results.

‍

While each tool has its strengths, we found that none fully meet our needs: accessible to both developers and non-technical stakeholders.

‍

‍

Examples

Below are a few sample outputs to illustrate how each tool presents test results.

Flutter Integration Test – Console Output

Failure case (verbose, technical logs):

Patrol – Console Output

Failure stops execution and shows a test summary:

Maestro – HTML Report and Console

Terminal shows step-by-step output, and HTML/JUnit report shows high-level result only (no step breakdown):

Introducing Allure

Allure is an open-source framework designed to generate rich, interactive, and user-friendly test reports. Originally built for Java-based testing frameworks like JUnit and TestNG, it has since evolved to support a broad range of ecosystems—including Python, JavaScript, and Kotlin. While Flutter isn’t natively supported, Allure’s flexible data format allows us to integrate it manually by exporting test results in a compatible structure.

‍

What makes Allure compelling is its ability to transform raw test execution data into a structured, shareable HTML report that’s not only informative but easy to navigate. This elevates the experience for both developers and non-technical stakeholders, helping teams better understand test behavior and failures.

How does it work?

Allure consumes test result data stored in JSON files, following a specific format to ensure compatibility. Each test case is represented as a JSON object with key properties such as:

• name: The name of the test case.
• status: The result of the test (passed, failed, broken, skipped).
• steps: A list of step objects, each containing:
  
  • name: Step description.
  • status: Step result.
  • start and stop: Timestamps for execution (milliseconds).
• attachments: Optional artifacts such as screenshots, logs, or videos.
• parameters: Input parameters used in the test case.
• uuid: A unique identifier for the test.

These JSON file names should follow a consistent pattern, using the uuid-result.json format to ensure uniqueness and prevent conflicts.

‍

Allure then aggregates this structured data and converts it into an interactive HTML report that enhances test result readability and debugging efficiency.

‍

For more details on the test result file structure, visit the Allure documentation.

‍

Note: In this blog, we will showcase a subset of Allure’s capabilities. The framework supports many additional features not covered here—see the official documentation for more.

Allure for Flutter: Our Initial Effort

To meet our requirements while building a scalable solution, we developed a framework-agnostic custom Allure adapter. This approach lets us enhance any Flutter UI testing framework by gaining full control over what gets captured and reported—free from the constraints of built-in tools.

‍

This marks an initial effort to bring Allure support to Flutter, designed to work with integration_test, Patrol, or any other setup you might use.

Want to see official Flutter support in Allure?

As of today, Allure does not officially support a Flutter/Dart adapter. However, we saw the potential and built a custom proof of concept (POC) to bridge the gap. Our goal is to inspire the community and demonstrate what’s possible.

‍

If you'd like to see first-class Flutter support in Allure, consider adding your voice to the discussion on this GitHub issue. The more people contribute or show interest, the better the chances of making it happen!

Introducing the TestResults class

We designed the TestResults class to wrap and log test execution steps, making it possible to track, manage, and save test results.

‍

This class helps structure test execution details by recording test names, timestamps, and statuses while providing utility methods to register, update, and manage test steps.

‍

Additionally, TestResults generates structured test reports in JSON format, making it compatible with Allure reporting.

‍

You can see the implementation of the class in integration_test/utils/test_results.dart on our Allure-Reports repo.

How to integrate with your existing tests or new ones

When a test starts, an instance of TestResults is created with the test name. This automatically initializes the report with:

• The test name (name field).
• A default failed status.
• A start timestamp indicating when the test began.

‍

Each test step is logged using the addStep() method, which:

• Registers a new step with a default status (in progress).
• Executes the provided function, the step.
• Updates the step’s status to passed or failed based on execution outcome.
• Adds timestamps to track execution duration.

‍

Note: For more granular control, the registerStep() and updateStep() methods allow direct manipulation of test steps without executing a function with the actual step.

‍

When we reach the end of the test without any exceptions being thrown, we call the passTest() method to mark the test as passed.

‍

Let’s see what a test would look like:

‍

For this example, we’re using a simple Flutter app with the integration_test package, you can see the implementation on integration_test/test/app_test.dart.

‍

void main() {
  IntegrationTestWidgetsFlutterBinding.ensureInitialized();

  const testName = 'Tap on the floating action button, verify counter';
  late final TestResults report;
  late final String testUUID;
  setUp(() {
    report = TestResults(testName);
    testUUID = report.generateAllureReportId();
  });

  testWidgets(testName, (tester) async {
    await report.addStep('Load app widget', () async {
      await tester.pumpWidget(MyApp());
    });

    await report.addStep('Verify counter starts at 0', () async {
      expect(find.text('0'), findsOneWidget);
    });

    final actionButtonFinder = find.byKey(ValueKey('increment'));
    await report.addStep('Finds the floating action button to tap on', () {
      expect(actionButtonFinder, findsOneWidget);
    });

    await report.addStep('Tap on the floating action button', () async {
      await tester.tap(actionButtonFinder);
    });

    await report.addStep('Pump and settle the widget', () async {
      await tester.pumpAndSettle();
    });

    await report.addStep('Verify counter increments by 1', () {
      expect(find.text('1'), findsOneWidget);
    });

    report.passTest();
  });

  tearDown(() async {
    await report.uploadReportToGoogleCloudStorage(testUUID);
  });
}

‍

The resulting JSON after the test execution looks like this:

‍

{
    "name": "Tap on the floating action button, verify counter",
    "status": "passed",
    "start": 1745503452538,
    "steps": [
        {
            "name": "Load app widget",
            "status": "passed",
            "start": 1745503452675,
            "stop": 1745503452876
        },
        {
            "name": "Verify counter starts at 0",
            "status": "passed",
            "start": 1745503452876,
            "stop": 1745503452889
        },
        {
            "name": "Finds the floating action button to tap on",
            "status": "passed",
            "start": 1745503452889,
            "stop": 1745503452892
        },
        {
            "name": "Tap on the floating action button",
            "status": "passed",
            "start": 1745503452892,
            "stop": 1745503452928
        },
        {
            "name": "Pump and settle the widget",
            "status": "passed",
            "start": 1745503452928,
            "stop": 1745503453637
        },
        {
            "name": "Verify counter increments by 1",
            "status": "passed",
            "start": 1745503453637,
            "stop": 1745503453638
        }
    ],
    "stop": 1745503453638
}

‍

As you can see, the output contains the test execution details, including the test name, overall status, timestamps, and a detailed list of steps with their statuses and execution times.

Saving test results in JSON

Since the TestResults class runs on Flutter, you have the flexibility to store test execution results either locally on your machine, or on the cloud by uploading them to whatever service you like.

‍

We chose the second option, uploading them to Google Cloud. It allows us to access the results from anywhere and enables some additional capabilities such as an automated GitHub workflow.

‍

Feel free to replace the storage logic with any other service of your choice, or implement a local solution for storing and retrieving test results.

‍

Current storage logic is in integration_test/utils/google_cloud_service.dart.

Retrieving test results and generating the Allure report locally

In our solution, once the JSON result file is saved in Google Cloud Storage, we retrieve them to generate the Allure report. To do so, we store them in a designated allure-results/ folder and then feed them to Allure.

‍

Generating the Allure Report

There are two ways to generate the Allure report:

1. Serve the report locally Using the command below, Allure will generate the report in a temporary directory and launch a local server to display it:
   
   

allure serve allure-results/

‍

2. Generate a single HTML file If you want a shareable, self-contained HTML report, Allure provides you with the following command:

allure generate --single-file allure-results/

‍

3. There are two ways to open the report: 
   • You can double-click to open the single HTML file
   • You can also launch the local server from the single HTML file with allure open
   Here are some images showing the Allure report look:

Conclusion

Current Flutter UI automation frameworks offer valuable features, but we saw an opportunity to go beyond their existing capabilities. Specifically, we needed a solution that delivers both detailed execution insight and shareable reports that resonate with technical and non-technical stakeholders alike.

‍

By integrating Allure, we’ve taken a significant step toward closing that gap—creating clear, structured, and visually rich test reports. This improves not only the debugging experience but also team collaboration and decision-making, helping Flutter teams communicate results more effectively. 🚀

Next Steps

Include visual evidence

Currently, our solution does not provide visual evidence, but Allure makes this possible. If we capture a screenshot during a test, Allure can embed it directly into the report, making debugging easier and test results more insightful.

‍

By integrating screenshots, logs, and additional artifacts, we can enhance report clarity and provide visual confirmation of test failures. This step will further improve collaboration between developers, testers, and stakeholders, ensuring faster issue resolution and better test analysis.