Your First Test Suite

A test suite is a structured collection of multiple test cases that you can run as a group. Currently, you can run all the tests in a suite in sequence, on one or more devices in parallel. Suites allow you to test your app more extensively, and enable easy repeatability of tests and ensuring optimal coverage.

A suite is structured into 3 task types:

1. Setup: This should be used for the steps required to get to the app feature that you want to test.(If you want to test the checkout flow, the setup will include steps like searching for an item, adding it to cart, and navigating to cart.)

2. Test: These should be the actual tests you want to run.(If you're testing checkout, these would be steps to test the flow with single item, multiple items, different payment methods, delivery types etc.)

3. Teardown: These should be used as reset steps in between or at the end of the suite.(Steps like navigating back to cart after each checkout attempt, or clearing the cart and closing the app at the end of the test.)

Creating a new Test Suite

To create a new test suite, follow the steps:

1. Click on Suites in the left navigation panel.

2. Click on the Create Suite button.

3. Give your suite a name (e.g., Login flow validation) and a description (e.g., Verify login flow including sign-up, password reset, and logout).

4. Add your tasks to the suite.

To add a task to a suite, you can either add a previously run task directly from the recent tasks window, or you can create a new task from scratch by clicking on the Add Setup Task (gear icon), Add Test Task (test tube icon), or Add Teardown Task (power icon) buttons.

You can define execution configurations like vision, reasoning, max steps etc. for all the tasks inside a suite by clicking on the configurations button at the top right, or you can also define these individually for a task by clicking on the caret icon to expand it.

Once done, you can click on the Save button to save the test suite.

Sequential and Isolated Suites

Any test suite can either be a sequential suite or an isolated suite. This specifies how the different tasks within the suite are linked with each other.

If the tasks in a suite depend on each other and must run in a strict order, the suite should be marked as sequential. For example, if Task 1 handles login, Task 2 handles search, Task 3 navigates to the cart, and Task 4 completes checkout, the flow only makes sense when these tasks execute one after another.

If all tasks are complete, independent flows, the suite should be marked as isolated. For example, if Task 1 covers the entire journey from app launch to checkout with one payment method, and the other tasks do the same journey with different payment methods or locations, none of them rely on each other, so they can run independently.

The option to distribute tasks across multiple devices when running will only be available for isolated suites, because sequential suites will require all tasks to be executed on each device.

Running a suite

To run a suite, click on the Run button inside the suite and select the device you want to run it on. Once created, your suite will appear under Runs → Suites in the sidebar, where you can monitor the live execution of the test suite.