:orphan: Testing Guide ============= Testing Philosophy ------------------ Minimum Requirement +++++++++++++++++++ + Demonstrate that a particular functionality is working + Show that the effect of new code is understood, and less likely to interact with the rest of code base in unexpected ways + Validate that a bug has been fixed and we can detect the problem in the future What's the Benefit to the Team? +++++++++++++++++++++++++++++++ + Reading test code allows others to more easily understand and work with our code + Ensures a consistent level of functionality testing for all subsequent changes + Speeds up future changes because we can instantly test to see if a change works as expected How do we Test if a Feature Works? ++++++++++++++++++++++++++++++++++ Since we are dealing with data, it helps to have a toy model which has the following characteristics: + Has enough variation to test the different cases which arise due to branching in the code (e.g., include generators of different types to test that the code handles scaling for the right type of generators) + Yet is still small enough to be able to verify overall correctness manually + Particular values in the data set are easy to work with and confirm manually (e.g., use `5` instead of `4.624754`) + Includes just the data needed for the code to run - in testing it is better to have an error if the code ever attempts to change something unexpectedly Prerequisite for Writing Tests ------------------------------ In order to be discoverable by ``pytest``, all the test files need to be prefixed with **test**. In addition, within the test files themselves test functions should have name starting with ``test``, e.g., ``test_scale_capacity_argument_type``, where ``scale_capacity`` is the function being tested and ``argument_type`` is the test that is performed on this function. Examples -------- Let's Start Easy ++++++++++++++++ You develop a new feature and you need a helper function that calculates the square root of a number. The function will be declared in a ``helpers`` module and the tests will be written in a ``test_helpers`` module located in a **tests** folder at the same level as the ``helpers`` module. Three tests should be written for this function: + Test the argument type + Test the argument value + Test the returned value The function would read: .. code-block:: python def calculate_square_root(number): """Calculate the nth root of a number. :param int/float number: positive number. :return: (*float*) -- principal square root of number. :raises TypeError: if number has wrong type. :raises ValueError: if number is negative. """ if not isinstance(number, (int, float)): raise TypeError("number must be a int or float") if number < 0: raise ValueError("number must be positive") return number ** (1.0 / 2) and the tests will look like: .. code-block:: python def test_calculate_square_root_argument_type(): arg = ((4), [16], {9}) for a in arg: with pytest.raises(TypeError): calculate_square_root(a) def test_calculate_square_root_argument_value(): with pytest.raises(ValueError): calculate_square_root(-9) def test_calculate_square_root(): assert calculate_square_root(64) == 8.0 assert calculate_square_root(49.0) == 7.0 assert calculate_square_root(36.0) != 5 Mock Objects ++++++++++++ Let's now consider a more complex example. One of the simulation outputs is the power generated by a generator in the grid in a given hour. We would like to summarize this result by calculating the total power generated in an hour per bus. To do so, we need to relate each generator to a connected bus. This information is enclosed in the ``Grid`` class. The function will read: .. code-block:: python def summarize_plant_to_bus(pg, grid): """Take a plant-column data frame and sum to a bus-column data frame. :param pandas.DataFrame pg: indices are UTC timestamp and columns are plant id in grid. :param powersimdata.input.grid.Grid grid: a Grid instance. :return: (*pandas.DataFrame*) -- indices as input data frame, columns are buses. """ # build a data frame mapping plant id to bus id. all_buses_in_grid = grid.plant["bus_id"] # keep only the rows matching the column of the pg data frame buses_in_df = all_buses_in_grid.loc[pg.columns] # transpose the pg data frame to get the plant id as indices, group all rows whose # indices correspond to the same bus id and sum the values in these rows for each # timestamp (column). Finally, transpose back. bus_data = pg.T.groupby(buses_in_df).sum().T return bus_data To test this function we need a couple of [mock objects]. One for the ``pg`` data frame and one for the `grid` object. We have developed a catalog of mock objects in the **powersimdata/tests/**. We show below how the ``MockGrid`` object can be used to test the ``summarize_plant_to_bus`` function: .. code-block:: python import pandas as pd from numpy.testing import assert_array_equal from powersimdata.tests.mock_grid import MockGrid # plant_id is the index mock_plant = { "plant_id": ["A", "B", "C", "D"], "bus_id": [1, 1, 2, 3], } # bus_id is the index mock_bus = { "bus_id": [1, 2, 3], "lat": [47.6, 37.8, 40.7], "lon": [122.3, 122.4, 74], } mock_pg = pd.DataFrame( {"A": [1, 2, 3, 4], "B": [1, 2, 4, 8], "C": [1, 1, 2, 3], "D": [1, 3, 5, 7]}, index=pd.date_range(start="2018-04-24", freq="H", periods=4), ) grid = MockGrid({"plant": mock_plant, "bus": mock_bus}) def test_summarize_plant_to_bus(): expected_return = pd.DataFrame( {1: [2, 4, 7, 12], 2: [1, 1, 2, 3], 3: [1, 3, 5, 7]}, index=pd.date_range(start="2018-04-24", freq="H", periods=4), ) bus_data = summarize_plant_to_bus(mock_pg, grid) assert_array_equal(bus_data.to_numpy(), expected_return.to_numpy()) .. _mock objects: https://en.wikipedia.org/wiki/Mock_object