Skip to content

Commit

Permalink
simplify test image build by removing TEST_IMAGE_SRC
Browse files Browse the repository at this point in the history
  • Loading branch information
reubenmiller committed Nov 25, 2024
1 parent 5501cd1 commit 676afd3
Show file tree
Hide file tree
Showing 3 changed files with 2 additions and 5 deletions.
1 change: 0 additions & 1 deletion .github/workflows/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,6 @@ jobs:
env:
COMPOSE_PROJECT_NAME: ci_${{ matrix.job.image }}_${{github.run_id}}_${{github.run_attempt || '1'}}
TEST_IMAGE: ${{ matrix.job.image }}
TEST_IMAGE_SRC: ${{ matrix.job.image }}

strategy:
fail-fast: false
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ TEDGE_C8Y_OPERATIONS_AUTO_LOG_UPLOAD=always

#### Testing

The system tests are writing using the [RobotFramework](https://robotframework.org/) with some custom thin-edge.io libraries. Generally the test framework will spin up a new container engine environment (defined by the `TEST_IMAGE` and `TEST_IMAGE_SRC` variables). In the test itself, a new instance of the **tedge-container-bundle** will be created which the test then uses to check the specified functionality. Using this setup does bring a but if complexity into the setup, however it is necessary to ensure that the **tedge-container-bundle** can be tested against multiple container engine environments (e.g. docker, podman and different versions of each), whilst it also provides a test environment which does not pollute your host's container engine environment.
The system tests are writing using the [RobotFramework](https://robotframework.org/) with some custom thin-edge.io libraries. Generally the test framework will spin up a new container engine environment (defined by the `TEST_IMAGE` variable). In the test itself, a new instance of the **tedge-container-bundle** will be created which the test then uses to check the specified functionality. Using this setup does bring a but if complexity into the setup, however it is necessary to ensure that the **tedge-container-bundle** can be tested against multiple container engine environments (e.g. docker, podman and different versions of each), whilst it also provides a test environment which does not pollute your host's container engine environment.

The following tools are required to run the tests:

Expand Down
4 changes: 1 addition & 3 deletions justfile
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@ RELEASE_VERSION := env_var_or_default("RELEASE_VERSION", `date +'%Y%m%d.%H%M'`)

# Test Variables
TEST_IMAGE := env_var_or_default("TEST_IMAGE", "debian-systemd-docker-cli")
TEST_IMAGE_SRC := env_var_or_default("TEST_IMAGE_SRC", "debian-systemd-docker-cli")


# Initialize a dotenv file for usage with a local debugger
Expand All @@ -23,7 +22,6 @@ init-dotenv:
@echo "Recreating .env file..."
@echo "DEVICE_ID=$DEVICE_ID" > .env
@echo "TEST_IMAGE=$IMAGE" >> .env
@echo "TEST_IMAGE_SRC=$IMAGE_SRC" >> .env
@echo "C8Y_BASEURL=$C8Y_BASEURL" >> .env
@echo "C8Y_USER=$C8Y_USER" >> .env
@echo "C8Y_PASSWORD=$C8Y_PASSWORD" >> .env
Expand Down Expand Up @@ -64,7 +62,7 @@ lint *ARGS:
# Build test images
build-test: build-test-bundles
echo "Creating test infrastructure image"
[ -d "./test-images/{{TEST_IMAGE_SRC}}" ] && docker build --load -t {{TEST_IMAGE}} -f ./test-images/{{TEST_IMAGE_SRC}}/Dockerfile . || docker pull "{{TEST_IMAGE}}"
[ -d "./test-images/{{TEST_IMAGE}}" ] && docker build --load -t {{TEST_IMAGE}} -f ./test-images/{{TEST_IMAGE}}/Dockerfile . || docker pull "{{TEST_IMAGE}}"

build-test-bundles:
echo "Building tedge-container-bundle images"
Expand Down

6 comments on commit 676afd3

@github-actions
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Robot Results

✅ Passed ❌ Failed ⏭️ Skipped Total Pass % ⏱️ Duration
14 0 1 14 100 2m15.212294s

Passed Tests

Name ⏱️ Duration Suite
Grace period to allow container to startup 5.054 s Operations
tedge-container-plugin service is up 0.147 s Operations
Get Logfile Request 2.348 s Operations
Get Configuration File 4.688 s Operations
Execute Shell Command 2.370 s Operations
Install application using docker compose 17.414 s Operations
Get Container Logs 2.361 s Operations
Trigger self update via local command 21.762 s Self-Update
Self update should only update if there is a new image 15.104 s Self-Update
Self update using software update operation 43.501 s Self-Update
Rollback when trying to install a non-tedge based image 17.405 s Self-Update
Cloud Connection is Online 0.161 s Telemetry
Service status 0.269 s Telemetry
Sends measurements 2.578 s Telemetry

@github-actions
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Robot Results

✅ Passed ❌ Failed ⏭️ Skipped Total Pass % ⏱️ Duration
14 0 1 14 100 2m15.583103s

Passed Tests

Name ⏱️ Duration Suite
Grace period to allow container to startup 5.047 s Operations
tedge-container-plugin service is up 0.146 s Operations
Get Logfile Request 2.338 s Operations
Get Configuration File 4.676 s Operations
Execute Shell Command 2.357 s Operations
Install application using docker compose 17.393 s Operations
Get Container Logs 2.365 s Operations
Trigger self update via local command 21.472 s Self-Update
Self update should only update if there is a new image 16.839 s Self-Update
Self update using software update operation 42.554 s Self-Update
Rollback when trying to install a non-tedge based image 17.454 s Self-Update
Cloud Connection is Online 0.158 s Telemetry
Service status 0.263 s Telemetry
Sends measurements 2.471 s Telemetry

@github-actions
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Robot Results

✅ Passed ❌ Failed ⏭️ Skipped Total Pass % ⏱️ Duration
14 0 1 14 100 2m16.373752s

Passed Tests

Name ⏱️ Duration Suite
Grace period to allow container to startup 5.047 s Operations
tedge-container-plugin service is up 0.177 s Operations
Get Logfile Request 2.439 s Operations
Get Configuration File 4.860 s Operations
Execute Shell Command 2.437 s Operations
Install application using docker compose 15.649 s Operations
Get Container Logs 2.448 s Operations
Trigger self update via local command 22.605 s Self-Update
Self update should only update if there is a new image 15.630 s Self-Update
Self update using software update operation 43.750 s Self-Update
Rollback when trying to install a non-tedge based image 18.269 s Self-Update
Cloud Connection is Online 0.190 s Telemetry
Service status 0.296 s Telemetry
Sends measurements 2.534 s Telemetry

@github-actions
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Robot Results

✅ Passed ❌ Failed ⏭️ Skipped Total Pass % ⏱️ Duration
14 0 1 14 100 2m26.075228s

Passed Tests

Name ⏱️ Duration Suite
Grace period to allow container to startup 5.046 s Operations
tedge-container-plugin service is up 0.197 s Operations
Get Logfile Request 2.517 s Operations
Get Configuration File 5.226 s Operations
Execute Shell Command 2.525 s Operations
Install application using docker compose 18.174 s Operations
Get Container Logs 2.553 s Operations
Trigger self update via local command 23.082 s Self-Update
Self update should only update if there is a new image 18.098 s Self-Update
Self update using software update operation 44.151 s Self-Update
Rollback when trying to install a non-tedge based image 21.033 s Self-Update
Cloud Connection is Online 0.223 s Telemetry
Service status 0.482 s Telemetry
Sends measurements 2.728 s Telemetry

@github-actions
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Robot Results

✅ Passed ❌ Failed ⏭️ Skipped Total Pass % ⏱️ Duration
14 0 1 14 100 2m35.572047s

Passed Tests

Name ⏱️ Duration Suite
Grace period to allow container to startup 5.031 s Operations
tedge-container-plugin service is up 0.235 s Operations
Get Logfile Request 2.573 s Operations
Get Configuration File 5.165 s Operations
Execute Shell Command 2.586 s Operations
Install application using docker compose 20.578 s Operations
Get Container Logs 2.585 s Operations
Trigger self update via local command 25.595 s Self-Update
Self update should only update if there is a new image 20.475 s Self-Update
Self update using software update operation 44.062 s Self-Update
Rollback when trying to install a non-tedge based image 23.284 s Self-Update
Cloud Connection is Online 0.229 s Telemetry
Service status 0.409 s Telemetry
Sends measurements 2.722 s Telemetry

@github-actions
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Robot Results

✅ Passed ❌ Failed ⏭️ Skipped Total Pass % ⏱️ Duration
14 0 1 14 100 2m50.520845s

Passed Tests

Name ⏱️ Duration Suite
Grace period to allow container to startup 5.045 s Operations
tedge-container-plugin service is up 0.248 s Operations
Get Logfile Request 4.717 s Operations
Get Configuration File 5.097 s Operations
Execute Shell Command 2.566 s Operations
Install application using docker compose 18.263 s Operations
Get Container Logs 4.736 s Operations
Trigger self update via local command 26.394 s Self-Update
Self update should only update if there is a new image 22.671 s Self-Update
Self update using software update operation 50.089 s Self-Update
Rollback when trying to install a non-tedge based image 27.168 s Self-Update
Cloud Connection is Online 0.253 s Telemetry
Service status 0.425 s Telemetry
Sends measurements 2.804 s Telemetry

Please sign in to comment.