Testing

Motivation

The testing suite is a fundamental part of Anthias, ensuring its stability and quality.

Unit Testing

• Unit tests are automated tests that verify individual components or units of the codebase, such as functions or classes.
• They ensure the correctness and expected behavior of these isolated parts.
• Unit tests are crucial for early bug detection and refactoring confidence.

Example

The following code snippet demonstrates a unit test for an empty playlist scenario:

class TestEmptyPl(ViewerTestCase):
          
              @mock.patch('viewer.SERVER_WAIT_TIMEOUT', 0)
              @mock.patch('viewer.start_loop', side_effect=noop)
              @mock.patch('viewer.view_image', side_effect=noop)
              @mock.patch('viewer.view_webpage', side_effect=noop)
              @mock.patch('viewer.setup', side_effect=noop)
              def test_empty(
                  self,
                  mock_setup,
                  mock_view_webpage,
                  mock_view_image,
                  mock_start_loop,
              ):
                  m_asset_list = mock.Mock()
                  m_asset_list.return_value = ([], None)
          
                  with mock.patch.object(self.u, 'generate_asset_list', m_asset_list):
                      self.u.main()
          
                      m_asset_list.assert_called_once()
                      mock_setup.assert_called_once()
                      mock_view_webpage.assert_called_once()
                      self.assertEqual(mock_view_image.call_count, 2)
                      mock_start_loop.assert_called_once()
          

Integration Testing

• Integration tests verify the interactions between different components of the system.
• They test the flow of data and communication between modules, ensuring they work together as expected.

Example

The following code snippet demonstrates an integration test:

$ docker compose \
          -f docker-compose.test.yml \
          exec anthias-test bash ./bin/prepare_test_environment.sh -s
          
          # Integration and non-integration tests should be run separately as the
          # former doesn't run as expected when run together with the latter.
          
          $ docker compose \
          -f docker-compose.test.yml \
          exec anthias-test ./manage.py test --exclude-tag=integration
          
          $ docker compose \
          -f docker-compose.test.yml \
          exec anthias-test ./manage.py test --tag=integration
          

Manual Testing

• Manual testing involves real-world interaction with the application.
• It helps identify usability issues, performance bottlenecks, and edge-case scenarios that unit and integration tests might miss.
• A comprehensive manual testing checklist is available at docs/qa-checklist.md.

Example

The following is an example of a manual test from the checklist:

• Disable Default assets: Make sure that the assets are deleted from the list of active assets. Also make sure that the assets are not being displayed on the screen. docs/qa-checklist.md

End-to-End Testing

• End-to-end (E2E) testing focuses on the entire application workflow.
• E2E tests simulate real user interactions, ensuring that the application behaves correctly from start to finish.
• E2E tests can be complex to implement, but they provide valuable insights into real-world performance and stability.

Example

The following code snippet shows an example of a test for a webpage asset:

start_date = new Date(2014, 4, 6, 14, 20, 0, 0);
                end_date = new Date();
                end_date.setMonth(end_date.getMonth() + 2)
                asset = new Anthias.Asset({
                  asset_id: 2
                  duration: "8"
                  end_date: end_date
                  is_enabled: true
                  mimetype: 'webpage'
                  name: 'Test'
                  start_date: start_date
                  uri: 'https://anthias.screenly.io'
                })
          
                it "should be active if enabled and date is in range", ->
                  expect(asset.active()).toBe true
          
                it "should be inactive if disabled and date is in range", ->
                  asset.set 'is_enabled', false
                  expect(asset.active()).toBe false
          
                it "should be inactive if enabled and date is out of range", ->
                  asset.set 'is_enabled', true
                  asset.set 'start_date', asset.get 'end_date'
                  expect(asset.active()).toBe false
          

CI Integration

• The testing suite is integrated into the continuous integration (CI) pipeline for continuous quality assurance.
• The CI pipeline runs automated tests for each commit to ensure the codebase remains stable and functional.

Codebase Structure

• The testing codebase is located within the tests directory.
• Individual test modules are organized based on the components they test.
• The tests/test_viewer.py module is an example of a unit test module for the viewer.py component.

          
          ## Top-Level Directory Explanations
          
          <a class='local-link directory-link' data-ref=".github/" href="#.github/">.github/</a> - This directory contains GitHub-specific configuration files and workflows for the project.
          
          <a class='local-link directory-link' data-ref=".github/workflows/" href="#.github/workflows/">.github/workflows/</a> - This directory contains YAML files defining continuous integration and deployment workflows for GitHub Actions.
          
          <a class='local-link directory-link' data-ref="ansible/" href="#ansible/">ansible/</a> - Ansible is an open-source configuration management and automation tool. This directory contains Ansible playbooks and roles for managing and configuring the project.
          
          <a class='local-link directory-link' data-ref="ansible/roles/" href="#ansible/roles/">ansible/roles/</a> - This directory contains Ansible roles, which are reusable collections of tasks and configurations for managing specific aspects of a system.
          
          <a class='local-link directory-link' data-ref="ansible/roles/network/" href="#ansible/roles/network/">ansible/roles/network/</a> - This role manages network configurations.
          
          <a class='local-link directory-link' data-ref="ansible/roles/splashscreen/" href="#ansible/roles/splashscreen/">ansible/roles/splashscreen/</a> - This role manages the configuration of a splash screen.
          
          <a class='local-link directory-link' data-ref="anthias_app/" href="#anthias_app/">anthias_app/</a> - This directory contains the main application codebase for the project, likely written in Django.
          
          <a class='local-link directory-link' data-ref="anthias_django/" href="#anthias_django/">anthias_django/</a> - This directory may contain additional Django-specific configuration files and code.
          
          <a class='local-link directory-link' data-ref="api/" href="#api/">api/</a> - This directory contains the API codebase for the project.
          
          <a class='local-link directory-link' data-ref="api/serializers/" href="#api/serializers/">api/serializers/</a> - This directory contains Django REST Framework serializers for converting data between Python objects and JSON.
          
          <a class='local-link directory-link' data-ref="bin/" href="#bin/">bin/</a> - This directory contains executable scripts for the project.
          
          <a class='local-link directory-link' data-ref="lib/" href="#lib/">lib/</a> - This directory contains reusable Python modules and libraries for the project.
          
          <a class='local-link directory-link' data-ref="static/" href="#static/">static/</a> - This directory contains static files, such as images, CSS, and JavaScript, that are served directly to the user by the web server.
          
          <a class='local-link directory-link' data-ref="static/js/" href="#static/js/">static/js/</a> - This directory contains JavaScript files.
          
          <a class='local-link directory-link' data-ref="static/spec/" href="#static/spec/">static/spec/</a> - This directory contains test files for the static files.
          
          <a class='local-link directory-link' data-ref="static/spec/jasmine/" href="#static/spec/jasmine/">static/spec/jasmine/</a> - This directory contains Jasmine test files.
          
          <a class='local-link directory-link' data-ref="templates/" href="#templates/">templates/</a> - This directory contains HTML templates used to render dynamic content.
          
          <a class='local-link directory-link' data-ref="tests/" href="#tests/">tests/</a> - This directory contains test files for the project.
          
          <a class='local-link directory-link' data-ref="tools/" href="#tools/">tools/</a> - This directory contains tools and scripts used to develop and maintain the project.
          
          <a class='local-link directory-link' data-ref="tools/image_builder/" href="#tools/image_builder/">tools/image_builder/</a> - This tool likely builds and optimizes images for the project.
          
          <a class='local-link directory-link' data-ref="website/" href="#website/">website/</a> - This directory contains the website codebase.
          
          <a class='local-link directory-link' data-ref="website/bin/" href="#website/bin/">website/bin/</a> - This directory contains website executable scripts.
          
          <a class='local-link directory-link' data-ref="webview/" href="#webview/">webview/</a> - This directory likely contains configuration files and code for a webview component.