Testing and Debugging for Anthias
Motivation:
Thorough testing and debugging are essential for ensuring the stability and reliability of Anthias. This outline covers various testing and debugging methods employed in the project.
Running Unit Tests:
Running unit tests is a good way to ensure the code is working as expected. docs/qa-checklist.md
Manual Testing:
Some bugs might not be caught by unit tests. Manual testing is necessary to identify and resolve them.
Manual Testing Checklist:
- Make sure that the device is connected to the internet (e.g., via Ethernet).
- Turn on the device and wait for the splash page to appear.
- Make sure that the splash page is being displayed properly.
- Use the provided IP address and make sure that entering it in a browser will redirect you to the web UI home page (the assets page).
- Add assets (image, video, or webpage) and make them active by toggling the switch.
- Make sure that these assets are shown on the screen.
- Disable the assets and make sure that the screen in standby mode, which means that it displays the Anthias standby page.
- Change a duration for any asset and make sure that it is being displayed for the specified duration.
- Change a the start and end dates and make sure that the asset is being displayed only during the specified period.
- Turn on some assets and change their order (by dragging and dropping them). Make sure that the assets are being displayed in the correct order.
- Try to change a name of any asset.
- Disable Default assets and make sure that the assets are deleted from the list of active assets. Also make sure that the assets are not being displayed on the screen.
- Enable Shuffle playlist. Activate some assets and make sure that the assets are being displayed in random order.
- Enable Use 24-hour clock. Go to the assets page and make sure that the time field uses correct format.
- Enable any video asset with sounds and choose HDMI for the Audio output. Make sure that the sound works.
- Enable any video file with sounds and choose 3.5mm jack for the Audio output. Make sure that the sound works.
- Choose any format for the Date format. Go to the assets page and make sure that the date field uses the correct format.
- Turn on some assets. Click on Previous asset and on Next asset. Make sure that the screen displays the asset that comes before or after the current asset.
- Click on the download button near any asset. The asset should be downloaded into your computer.
Debugging Anthias WebView:
The Anthias WebView is a custom-built web browser based on the Qt toolkit framework. docs/developer-documentation.md
Enabling QT Debugging:
To enable QT debugging, you can use the following command:
export QT_LOGGING_RULES=qt.qpa.*=true
Additional Debugging Settings:
export QT_LOGGING_DEBUG=1
export QT_LOGGING_RULES="*.debug=true"
export QT_QPA_EGLFS_DEBUG=1
docs/developer-documentation.md
Testing with Selenium and Splinter:
The following Python packages are used for testing:
future==0.18.3
mock==3.0.5
pep8==1.7.1
selenium==3.141.0
splinter==0.14.0
time-machine==2.15.0
unittest-parametrize==1.4.0
requirements/requirements.dev.txt
Example Unit Test (tests/test_viewer.py):
class ViewerTestCase(unittest.TestCase):
def setUp(self):
self.original_splash_delay = viewer.SPLASH_DELAY
viewer.SPLASH_DELAY = 0
self.u = viewer
self.m_scheduler = mock.Mock(name='m_scheduler')
self.p_scheduler = mock.patch.object(
self.u, 'Scheduler', self.m_scheduler)
self.m_cmd = mock.Mock(name='m_cmd')
self.p_cmd = mock.patch.object(self.u.sh, 'Command', self.m_cmd)
self.m_killall = mock.Mock(name='killall')
self.p_killall = mock.patch.object(
self.u.sh, 'killall', self.m_killall)
self.m_reload = mock.Mock(name='reload')
self.p_reload = mock.patch.object(
self.u, 'load_settings', self.m_reload)
self.m_sleep = mock.Mock(name='sleep')
self.p_sleep = mock.patch.object(self.u, 'sleep', self.m_sleep)
self.m_loadb = mock.Mock(name='load_browser')
self.p_loadb = mock.patch.object(self.u, 'load_browser', self.m_loadb)
def tearDown(self):
self.u.SPLASH_DELAY = self.original_splash_delay
Example Unit Test (tests/test_settings.py):
class SettingsTest(TestCase):
def setUp(self):
if not os.path.exists(CONFIG_DIR):
os.mkdir(CONFIG_DIR)
self.orig_getenv = os.getenv
os.getenv = getenv
def tearDown(self):
shutil.rmtree(CONFIG_DIR)
os.getenv = self.orig_getenv
def test_parse_settings(self):
with fake_settings(settings1) as (mod_settings, settings):
self.assertEquals(settings['player_name'], 'new player')
self.assertEquals(settings['show_splash'], False)
self.assertEquals(settings['shuffle_playlist'], True)
self.assertEquals(settings['debug_logging'], True)
self.assertEquals(settings['default_duration'], 45)
def test_default_settings(self):
with fake_settings(empty_settings) as (mod_settings, settings):
self.assertEquals(
settings['player_name'],
mod_settings.DEFAULTS['viewer']['player_name'])
self.assertEquals(
settings['show_splash'],
mod_settings.DEFAULTS['viewer']['show_splash'])
self.assertEquals(
settings['shuffle_playlist'],
mod_settings.DEFAULTS['viewer']['shuffle_playlist'])
self.assertEquals(
settings['debug_logging'],
mod_settings.DEFAULTS['viewer']['debug_logging'])
self.assertEquals(
settings['default_duration'],
mod_settings.DEFAULTS['viewer']['default_duration'])
def broken_settings_should_raise_value_error(self):
with self.assertRaises(ValueError):
with fake_settings(broken_settings) as (mod_settings, settings):
pass
def test_save_settings(self):
with fake_settings(settings1) as (mod_settings, settings):
settings.conf_file = CONFIG_DIR + '/new.conf'
settings['default_duration'] = 35
settings['verify_ssl'] = True
settings.save()
with open(CONFIG_DIR + '/new.conf') as f:
saved = f.read()
with fake_settings(saved) as (mod_settings, settings):
# changes saved?
self.assertEqual(settings['default_duration'], 35)
self.assertEqual(settings['verify_ssl'], True)
# no out of thin air changes?
self.assertEqual(settings['audio_output'], 'hdmi')
Example Unit Test (tests/test_utils.py):
class URLHelperTest(TestCase):
def test_url_1(self):
self.assertTrue(url_fails(url_fail))
def test_url_2(self):
self.assertFalse(url_fails(url_redir))
def test_url_3(self):
self.assertFalse(url_fails(uri_))
JavaScript Tests (static/spec/jasmine/jasmine.js):
/**
* Top level namespace for Jasmine, a lightweight JavaScript BDD/spec/testing framework.
*
* @namespace
*/
/**
* @private
*/
/**
* Use jasmine.undefined instead of undefined, since undefined is just
* a plain old variable and may be redefined by somebody else.
*
* @private
*/
/**
* Show diagnostic messages in the console if set to true
*
*/
/**
* Default interval in milliseconds for event loop yields (e.g. to allow network activity or to refresh the screen with the HTML-based runner). Small values here may result in slow test running. Zero means no updates until all tests have completed.
*
*/
/**
* Maximum levels of nesting that will be included when an object is pretty-printed
*/
/**
* Default timeout interval in milliseconds for waitsFor() blocks.
*/
/**
* By default exceptions thrown in the context of a test are caught by jasmine so that it can run the remaining tests in the suite.
* Set to false to let the exception bubble up in the browser.
*
*/
/**
* Allows for bound functions to be compared. Internal use only.
*
* @ignore
* @private
* @param base {Object} bound 'this' for the function
* @param name {Function} function to find
*/
/**
* Getter for the Jasmine environment. Ensures one gets created
*/
/**
* @ignore
* @private
* @param value
* @returns {Boolean}
*/
/**
* @ignore
* @private
* @param value
* @returns {Boolean}
*/
/**
* @ignore
* @private
* @param value
* @returns {Boolean}
*/
/**
* @ignore
* @private
* @param {String} typeName
* @param value
* @returns {Boolean}
*/
/**
* Pretty printer for expecations. Takes any object and turns it into a human-readable string.
*
* @param value {Object} an object to be outputted
* @returns {String}
*/
/**
* Returns true if the object is a DOM Node.
*
* @param {Object} obj object to check
* @returns {Boolean}
*/
/**
* Returns a matchable 'generic' object of the class type. For use in expecations of type when values don't matter.
*
* @example
* // don't care about which function is passed in, as long as it's a function
* expect(mySpy).toHaveBeenCalledWith(jasmine.any(Function));
*
* @param {Class} clazz
* @returns matchable object of the type clazz
*/
/**
* Returns a matchable subset of a JSON object. For use in expectations when you don't care about all of the
* attributes on the object.
*
* @example
* // don't care about any other attributes than foo.
* expect(mySpy).toHaveBeenCalledWith(jasmine.objectContaining({foo: "bar"});
*
* @param sample {Object} sample
* @returns matchable object for the sample
*/
/**
* Jasmine Spies are test doubles that can act as stubs, spies, fakes or when used in an expecation, mocks.
*
* Spies should be created in test setup, before expectations. They can then be checked, using the standard Jasmine
* expectation syntax. Spies can be checked if they were called or not and what the calling params were.
*
* A Spy has the following fields: wasCalled, callCount, mostRecentCall, and argsForCall (see docs).
*
* Spies are torn down at the end of every spec.
*
* Note: Do not call new jasmine.Spy() directly - a spy must be created using spyOn, jasmine.createSpy or jasmine.createSpyObj.
*
* @example
* // a stub
* var myStub = jasmine.createSpy('myStub'); // can be used anywhere
*
* // spy example
* var foo = {
* not: function(bool) { return !bool; }
* }
*
* // actual foo.not will not be called, execution stops
* spyOn(foo, 'not');
// foo.not spied upon, execution will continue to implementation
* spyOn(foo, 'not').andCallThrough();
*
* // fake example
* var foo = {
* not: function(bool) { return !bool; }
* }
*
* // foo.not(val) will return val
* spyOn(foo, 'not').andCallFake(function(value) {return value;});
*
* // mock example
* foo.not(7 == 7);
* expect(foo.not).toHaveBeenCalled();
* expect(foo.not).toHaveBeenCalledWith(true);
*
* @constructor
* @see spyOn, jasmine.createSpy, jasmine.createSpyObj
* @param {String} name
*/
/**
* Tells a spy to call through to the actual implemenatation.
*
* @example
* var foo = {
* bar: function() { // do some stuff }
* }
*
* // defining a spy on an existing property: foo.bar
* spyOn(foo, 'bar').andCallThrough();
*/
/**
* For setting the return value of a spy.
*
* @example
* // defining a spy from scratch: foo() returns 'baz'
* var foo = jasmine.createSpy('spy on foo').andReturn('baz');
*
* // defining a spy on an existing property: foo.bar() returns 'baz'
* spyOn(foo, 'bar').andReturn('baz');
*
* @param {Object} value
*/
/**
* For throwing an exception when a spy is called.
*
* @example
* // defining a spy from scratch: foo() throws an exception w/ message 'ouch'
* var foo = jasmine.createSpy('spy on foo').andThrow('baz');
*
* // defining a spy on an existing property: foo.bar() throws an exception w/ message 'ouch'
* spyOn(foo, 'bar').andThrow('baz');
*
* @param {String} exceptionMsg
*/
/**
* Calls an alternate implementation when a spy is called.
*
* @example
* var baz = function() {
* // do some stuff, return something
* }
* // defining a spy from scratch: foo() calls the function baz
* var foo = jasmine.createSpy('spy on foo').andCall(baz);
*
* // defining a spy on an existing property: foo.bar() calls an anonymnous function
* spyOn(foo, 'bar').andCall(function() { return 'baz';} );
*
* @param {Function} fakeFunc
*/
/**
* Resets all of a spy's the tracking variables so that it can be used again.
*
* @example
* spyOn(foo, 'bar');
*
* foo.bar();
*
* expect(foo.bar.callCount).toEqual(1);
*
* foo.bar.reset();
*
* expect(foo.bar.callCount).toEqual(0);
*/
/**
* Determines whether an object is a spy.
*
* @param {jasmine.Spy|Object} putativeSpy
* @returns {Boolean}
*/
/**
* Creates a more complicated spy: an Object that has every property a function that is a spy. Used for stubbing something
* large in one call.
*
* @param {String} baseName name of spy class
* @param {Array} methodNames array of names of methods to make spies
*/
/**
* All parameters are pretty-printed and concatenated together, then written to the current spec's output.
*
* Be careful not to leave calls to jasmine.log in production code.
*/
/**
* Function that installs a spy on an existing object's method name. Used within a Spec to create a spy.
*
* @example
* // spy example
* var foo = {
* not: function(bool) { return !bool; }
* }
* spyOn(foo, 'not'); // actual foo.not will not be called, execution stops
*
* @see jasmine.createSpy
* @param obj
* @param methodName
* @return {jasmine.Spy} a Jasmine spy that can be chained with all spy methods
*/
/**
* Creates a Jasmine spec that will be added to the current suite.
*
* // TODO: pending tests
*
* @example
* it('should be true', function() {
* expect
## Top-Level Directory Explanations
<a class='local-link directory-link' data-ref=".github/" href="#.github/">.github/</a> - This directory contains GitHub-specific configuration files and workflows for the project.
<a class='local-link directory-link' data-ref=".github/workflows/" href="#.github/workflows/">.github/workflows/</a> - This directory contains YAML files defining continuous integration and deployment workflows for GitHub Actions.
<a class='local-link directory-link' data-ref="ansible/" href="#ansible/">ansible/</a> - Ansible is an open-source configuration management and automation tool. This directory contains Ansible playbooks and roles for managing and configuring the project.
<a class='local-link directory-link' data-ref="ansible/roles/" href="#ansible/roles/">ansible/roles/</a> - This directory contains Ansible roles, which are reusable collections of tasks and configurations for managing specific aspects of a system.
<a class='local-link directory-link' data-ref="ansible/roles/network/" href="#ansible/roles/network/">ansible/roles/network/</a> - This role manages network configurations.
<a class='local-link directory-link' data-ref="ansible/roles/splashscreen/" href="#ansible/roles/splashscreen/">ansible/roles/splashscreen/</a> - This role manages the configuration of a splash screen.
<a class='local-link directory-link' data-ref="ansible/roles/system/" href="#ansible/roles/system/">ansible/roles/system/</a> - This role manages system-level configurations.
<a class='local-link directory-link' data-ref="anthias_app/" href="#anthias_app/">anthias_app/</a> - This directory contains the main application codebase for the project, likely written in Django.
<a class='local-link directory-link' data-ref="anthias_django/" href="#anthias_django/">anthias_django/</a> - This directory may contain additional Django-specific configuration files and code.
<a class='local-link directory-link' data-ref="api/" href="#api/">api/</a> - This directory contains the API codebase for the project.
<a class='local-link directory-link' data-ref="bin/" href="#bin/">bin/</a> - This directory contains executable scripts for the project.
<a class='local-link directory-link' data-ref="lib/" href="#lib/">lib/</a> - This directory contains reusable Python modules and libraries for the project.
<a class='local-link directory-link' data-ref="static/" href="#static/">static/</a> - This directory contains static files, such as images, CSS, and JavaScript, that are served directly to the user by the web server.
<a class='local-link directory-link' data-ref="static/spec/" href="#static/spec/">static/spec/</a> - This directory contains test files for the static files.
<a class='local-link directory-link' data-ref="static/spec/jasmine/" href="#static/spec/jasmine/">static/spec/jasmine/</a> - This directory contains Jasmine test files.
<a class='local-link directory-link' data-ref="templates/" href="#templates/">templates/</a> - This directory contains HTML templates used to render dynamic content.
<a class='local-link directory-link' data-ref="tests/" href="#tests/">tests/</a> - This directory contains test files for the project.
<a class='local-link directory-link' data-ref="tools/" href="#tools/">tools/</a> - This directory contains tools and scripts used to develop and maintain the project.
<a class='local-link directory-link' data-ref="tools/image_builder/" href="#tools/image_builder/">tools/image_builder/</a> - This tool likely builds and optimizes images for the project.
<a class='local-link directory-link' data-ref="website/" href="#website/">website/</a> - This directory contains the website codebase.
<a class='local-link directory-link' data-ref="website/bin/" href="#website/bin/">website/bin/</a> - This directory contains website executable scripts.
<a class='local-link directory-link' data-ref="webview/" href="#webview/">webview/</a> - This directory likely contains configuration files and code for a webview component.
<a class='local-link directory-link' data-ref="webview/src/" href="#webview/src/">webview/src/</a> - This directory contains webview source code.
