Skip to content

Commit

Permalink
style(Docs): Fix formatting of code within docs
Browse files Browse the repository at this point in the history
  • Loading branch information
@apatelia
apatelia committed Oct 20, 2024
1 parent cf75819 commit 2c0c803
Show file tree
Hide file tree
Showing 2 changed files with 89 additions and 76 deletions.
64 changes: 31 additions & 33 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -137,26 +137,26 @@ Replace directory with the path to the directory containing the CTRF reports you

The test object in the report includes the following [CTRF properties](https://ctrf.io/docs/schema/test):

| Name | Type | Required | Details |
| -------------- | ---------------- | -------- | -------------------------------------------------------------------------------------------- |
| `name` | String | Required | The name of the test. |
| `status` | String | Required | The outcome of the test. One of:`passed`, `failed`, `skipped`, `pending`, `other`. |
| `duration` | Number | Required | The time taken for the test execution, in milliseconds. |
| `start` | Number | Optional | The start time of the test as a Unix epoch timestamp. |
| `stop` | Number | Optional | The end time of the test as a Unix epoch timestamp. |
| `suite` | String | Optional | The suite or group to which the test belongs. |
| `message` | String | Optional | The failure message if the test failed. |
| `trace` | String | Optional | The stack trace captured if the test failed. |
| `rawStatus` | String | Optional | The original playwright status of the test before mapping to CTRF status. |
| `tags` | Array of Strings | Optional | The tags retrieved from the test name |
| `type` | String | Optional | The type of test (e.g.,`api`, `e2e`). |
| `filepath` | String | Optional | The file path where the test is located in the project. |
| `retries` | Number | Optional | The number of retries attempted for the test. |
| `flaky` | Boolean | Optional | Indicates whether the test result is flaky. |
| `browser` | String | Optional | The browser used for the test. |
| `screenshot` | String | Optional | A base64 encoded screenshot taken during the test. |
| `steps` | Array of Objects | Optional | Individual steps in the test, especially for BDD-style testing. |
| `extra` | Object | Optional | Custom data relevant to the test. |
| Name | Type | Required | Details |
| ------------ | ---------------- | -------- | ---------------------------------------------------------------------------------- |
| `name` | String | Required | The name of the test. |
| `status` | String | Required | The outcome of the test. One of:`passed`, `failed`, `skipped`, `pending`, `other`. |
| `duration` | Number | Required | The time taken for the test execution, in milliseconds. |
| `start` | Number | Optional | The start time of the test as a Unix epoch timestamp. |
| `stop` | Number | Optional | The end time of the test as a Unix epoch timestamp. |
| `suite` | String | Optional | The suite or group to which the test belongs. |
| `message` | String | Optional | The failure message if the test failed. |
| `trace` | String | Optional | The stack trace captured if the test failed. |
| `rawStatus` | String | Optional | The original playwright status of the test before mapping to CTRF status. |
| `tags` | Array of Strings | Optional | The tags retrieved from the test name |
| `type` | String | Optional | The type of test (e.g.,`api`, `e2e`). |
| `filepath` | String | Optional | The file path where the test is located in the project. |
| `retries` | Number | Optional | The number of retries attempted for the test. |
| `flaky` | Boolean | Optional | Indicates whether the test result is flaky. |
| `browser` | String | Optional | The browser used for the test. |
| `screenshot` | String | Optional | A base64 encoded screenshot taken during the test. |
| `steps` | Array of Objects | Optional | Individual steps in the test, especially for BDD-style testing. |
| `extra` | Object | Optional | Custom data relevant to the test. |

## BDD styled tests

Expand All @@ -166,30 +166,28 @@ The test object in the report includes the following [CTRF properties](https://c

Some features require additional setup or usage considerations.


### Test steps

Test steps are included in the `steps` property of a `Test` object. However, this `steps` property includes only top level steps. This means any children steps i.e. implementation details, are not included in it.

Children steps are included in the `extra` property as `childSteps`.


Every step has following properties of its own.

| Name | Type | Required | Details |
| ---------- | ------ | -------- | -------------------------------------------------------------------------------------------- |
| `name` | String | Required | The name/title of the step. |
| `status` | String | Required | The outcome of the step. One of:`passed`, `failed`, `skipped`, `pending`, `other`. |
| `extra` | Object | Optional | Custom data relevant to the step. |
| Name | Type | Required | Details |
| -------- | ------ | -------- | ---------------------------------------------------------------------------------- |
| `name` | String | Required | The name/title of the step. |
| `status` | String | Required | The outcome of the step. One of:`passed`, `failed`, `skipped`, `pending`, `other`. |
| `extra` | Object | Optional | Custom data relevant to the step. |

Apart from including child steps, `extra` property is also used to report a steps' category, execution duration and the location in a source file. `extra` object contains following additional properties generated specifically for this JSON report.

| Name | Type | Details |
| -------------- | --------------- | --------------------------------------------------------------------------------------------- |
| `category` | String | The category of the step. One of:`hook`, `expect`, `pw:api`, `test.step` |
| `duration` | String | The execution duration of the step, in milliseconds. |
| `location` | Object | Location the step in test suite, including file name, line number and column number. |
| `childSteps` | Array of object | Child steps of the current step, if any. Child steps have the same properties that of a step. |
| Name | Type | Details |
| ------------ | --------------- | --------------------------------------------------------------------------------------------- |
| `category` | String | The category of the step. One of:`hook`, `expect`, `pw:api`, `test.step` |
| `duration` | String | The execution duration of the step, in milliseconds. |
| `location` | Object | Location the step in test suite, including file name, line number and column number. |
| `childSteps` | Array of object | Child steps of the current step, if any. Child steps have the same properties that of a step. |

#### Include test steps only

Expand Down
101 changes: 58 additions & 43 deletions docs/test-steps-only-examples.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,11 @@ Below are some examples of how the report will look based on the `testStepsOnly`
### Simple test - No `test.step()` used.

```typescript
test("has title @title", async ({ page }) => {
await page.goto("https://playwright.dev/");
test('has title @title', async ({ page }) => {
await page.goto('https://playwright.dev/')

await expect(page).toHaveTitle(/Playwright/);
});
await expect(page).toHaveTitle(/Playwright/)
})
```

A simple test like above, will be reported as:
Expand Down Expand Up @@ -176,22 +176,27 @@ A simple test like above, will be reported as:
```ts
// beforeEach hook, will be included in CTRF report.
test.beforeEach(async ({ page }) => {
await page.goto("https://playwright.dev/");
});

test("with nested steps", { tag: [ "@steps", "@nested-steps" ] }, async ({ page }) => {

await test.step("Verify page title contains Playwright", async () => {
await expect.soft(page).toHaveTitle(/Playwright/);
});

// No test.step() usage for this step, but it will be included in CTRF report.
await page.getByRole("link", { name: "Get started" }).click();

await test.step("Verify Installation heading is visible", async () => {
await expect.soft(page.getByRole("heading", { name: "Installation" })).toBeVisible();
});
});
await page.goto('https://playwright.dev/')
})

test(
'with nested steps',
{ tag: ['@steps', '@nested-steps'] },
async ({ page }) => {
await test.step('Verify page title contains Playwright', async () => {
await expect.soft(page).toHaveTitle(/Playwright/)
})

// No test.step() usage for this step, but it will be included in CTRF report.
await page.getByRole('link', { name: 'Get started' }).click()

await test.step('Verify Installation heading is visible', async () => {
await expect
.soft(page.getByRole('heading', { name: 'Installation' }))
.toBeVisible()
})
}
)
```

A test having `test.step()` as steps, will be reported as:
Expand Down Expand Up @@ -416,6 +421,7 @@ A test having `test.step()` as steps, will be reported as:
```

### BDD Style test - using [Playwright-bdd](https://vitalets.github.io/playwright-bdd/#/)

```gherkin
@login
Feature: User Login
Expand Down Expand Up @@ -727,11 +733,11 @@ A BDD styled test like the one above, will be reported as:
### Simple test - No `test.step()` used.

```ts
test("has title @title", async ({ page }) => {
await page.goto("https://playwright.dev/");
test('has title @title', async ({ page }) => {
await page.goto('https://playwright.dev/')

await expect(page).toHaveTitle(/Playwright/);
});
await expect(page).toHaveTitle(/Playwright/)
})
```

Simple test above will be reported as:
Expand Down Expand Up @@ -766,25 +772,31 @@ Simple test above will be reported as:
```ts
// beforeEach hook, will NOT be included in CTRF report.
test.beforeEach(async ({ page }) => {
await page.goto("https://playwright.dev/");
});

test("with nested steps", { tag: [ "@steps", "@nested-steps" ] }, async ({ page }) => {
await test.step("Navigate to Playwright homepage", async () => {
await page.goto("https://playwright.dev/");
});

await test.step("Verify page title contains Playwright", async () => {
await expect(page).toHaveTitle(/Playwright/);
});

// No test.step() used for this step, it will NOT be included in CTRF report.
await page.getByRole("link", { name: "Get started" }).click();

await test.step("Verify Installation heading is visible", async () => {
await expect(page.getByRole("heading", { name: "Installation" })).toBeVisible();
});
});
await page.goto('https://playwright.dev/')
})

test(
'with nested steps',
{ tag: ['@steps', '@nested-steps'] },
async ({ page }) => {
await test.step('Navigate to Playwright homepage', async () => {
await page.goto('https://playwright.dev/')
})

await test.step('Verify page title contains Playwright', async () => {
await expect(page).toHaveTitle(/Playwright/)
})

// No test.step() used for this step, it will NOT be included in CTRF report.
await page.getByRole('link', { name: 'Get started' }).click()

await test.step('Verify Installation heading is visible', async () => {
await expect(
page.getByRole('heading', { name: 'Installation' })
).toBeVisible()
})
}
)
```

A test with `test.step()` usage, will be reported as:
Expand Down Expand Up @@ -855,6 +867,7 @@ Feature: User Login
When the User tries to login with "locked_out_user" as username and "secret_sauce" as password
Then the User should see a locked out error message
```

A BDD styled test like the one above, is reported as shown below.

> ⚠️ Notice that the step in the 'Background' hook is not included. Similarly, any steps or fixtures executed as part of 'After' hooks will be omitted as well.
Expand Down Expand Up @@ -908,5 +921,7 @@ A BDD styled test like the one above, is reported as shown below.
}
}
```

## BDD styled tests with [Cucumber](https://cucumber.io/docs/guides/overview/)

BDD styled tests that use cucumber as test runner are not supported. Only [playwright-bdd](https://vitalets.github.io/playwright-bdd/#/) is supported because it uses [Playwright Test](https://playwright.dev/docs/test-configuration) as a test runner to execute the tests.

0 comments on commit 2c0c803

Please sign in to comment.