Skip to content

Testing Your Changes

Framework and Structure

  • We use pytest as our testing framework. Do not use the unittest package.
  • Test files should mirror the package structure:
project_root/
├── amee_utils/
│   ├── utils/
│   │   ├── __init__.py
│   │   └── some_utility.py
│   ├── feature_generator/
│   │   ├── __init__.py
│   │   └── generator.py
│   ├── writer.py
│   └── loader.py
├── tests/
│   ├── utils/
│   │   ├── __init__.py
│   │   └── test_some_utility.py
│   ├── feature_generator/
│   │   ├── __init__.py
│   │   └── test_generator.py
│   ├── test_writer_UCWriter.py
│   ├── test_writer_DeltaWriter.py
│   └── test_loader.py
  • Test files should be prefixed with test_
  • The name should match the module being tested
  • Example: writer.py → test_writer_UCWriter.py and test_writer_DeltaWriter.py

Writing Tests

  • Use fixtures for common setup and teardown

  • Tests that require Spark can use the existing spark fixture in conftest.py:

    conftest.py
    """Default utility fixtures for pytest."""

import pytest
from pyspark.sql import SparkSession


@pytest.fixture(scope="module")
def spark():
    spark_builder = SparkSession.builder.appName("Test")
    spark_builder.config("spark.master", "local[1]")
    spark = spark_builder.getOrCreate()
    yield spark
    spark.stop()

    This spark fixture can then be used in any test module as follows:

    test_module_a.py
    import pytest
# ... (other imports if necessary)
def test_basic_action_1(spark):
    # A test using the spark fixture
    ...

Running tests

Running all tests

You can run all existing tests and your added tests by running the following command: 

uv run pytest tests/

All existing tests should pass without any alteration. If an existing test requires alteration, this should be investigated and the change should be marked as a breaking change in the Pull Request.

Running tests with code coverage

To make sure your tests are covering the code sufficiently, please run the tests with code coverage: 

uv run pytest --cov=amee_utils --cov-report=term-missing tests/

Make sure that the coverage does not drop significantly, and that your additional code is appropriately covered.

Documentation Changes

Info

We use NumPy formatting for docstrings. Please make sure your docstrings are appropriately formatted in order to reflect correctly in the API Reference section.

To preview your changes made to the documentation, and to test that the updated documentation reflects appropriately, please run the following: 

mkdocs serve

You can then open a local version of the documents in your browser (localhost), and thoroughly review your changes

Tip

Please be sure to check the API Reference for your added code to make sure your docstrings have pulled through correctly.