Automated Testing

Frappe Framework provides built-in tools for writing and running automated tests. These tests help verify application functionality, prevent regressions, and ensure your custom apps continue working correctly as the codebase evolves.

Testing Rules

• Test files can be placed anywhere inside your application’s repository.
• Every test file must begin with test_ and use the .py extension.
• For DocType tests, Frappe automatically creates dependent test records for linked DocTypes.
• For regular Python modules, simply create unit tests inside files prefixed with test_.

Writing DocType Tests

Whenever a new DocType is created in Developer Mode, Frappe generates a boilerplate test file named test_{doctype}.py.

A good test should:

• Create any required test records.
• Prepare test dependencies inside setUp().
• Clean up resources if required.
• Verify expected behaviour using assertions.

Typical test structure:

import unittest

class TestEvent(unittest.TestCase):

    def setUp(self):
        # Create required test data
        pass

    def tearDown(self):
        # Cleanup if required
        pass

    def test_example(self):
        self.assertTrue(True)

Writing Tests for Bench Commands

Bench commands can also be tested automatically. Command tests should inherit from BaseTestCommands together with Python’s unittest.TestCase.

This allows you to execute Bench commands and validate:

• Exit status codes.
• Console output.
• Error messages.
• Command arguments.

Example:

class TestCommands(BaseTestCommands, unittest.TestCase):

    def test_execute(self):
        self.execute("bench --site {site} execute frappe.db.get_database_size")
        self.assertEqual(self.returncode, 0)

Installing Test Dependencies

Many applications include additional development packages inside dev-requirements.txt. Before running tests, install these dependencies using:

bench setup requirements --dev

Running Tests

Frappe provides several options for executing tests depending on what you want to validate.

Checking Whether Tests Are Running

Sometimes your application needs to behave differently during automated tests. You can determine whether code is executing inside a test environment by checking the global variable:

if not frappe.in_test:
    notify_error_to_user()

Older Frappe versions also support frappe.flags.in_test, although frappe.in_test is the preferred approach.

Running Tests in Parallel

Large applications can contain hundreds of test files. Running them sequentially may significantly increase CI execution time. Frappe supports parallel test execution across multiple build machines.

Basic Parallel Execution

Split tests evenly across multiple CI instances using:

bench --site [sitename] --app [app-name] run-parallel-tests \
--build-id <build-number> \
--total-builds <total-builds>

Each build receives a different subset of test files.

Parallel Testing with Test Orchestrator

Instead of splitting tests equally, the Test Orchestrator dynamically assigns the next available test to whichever CI instance becomes free first. This helps balance workloads when some tests take much longer than others.

bench --site [sitename] --app [app-name] run-parallel-tests --use-orchestrator

To use this feature, the following environment variables must be configured:

• CI_BUILD_ID
• ORCHESTRATOR_URL

Parallel Testing Comparison