Introduction

When talking about software maintainability, we always imply that you can extend or change a given software (even by another person than the original maintainer) without breaking the code or introducing bugs. This premise assumes that there is a good test suite making sure that your software still works as intended after the change (or with a new version of an external dependency). For example, the complexity of the software in the BIOfid software framework, both in the backend and the frontend, can only be tamed by applying tests that make sure that the code works as intended - now and in the future.

When testing your software, you try to isolate your tests as well as possible¹. However, what if your application depends on an external API (e.g. a database, a microservice, an external Web-API)? The return value of this external API may change frequently or is not reachable (making all your tests fail). Hence, when testing and debugging an application, we want to exclude external influences like these to get 100% reproducibility.

In one of the previous posts, my colleague from the FID African Studies introduced the django-orcid app, which also has an external API dependency. To enhance reusability and maintainability, we agreed that I provide some tests for the app. Still, writing tests for existing code can be very hard (and is no fun). In general, I recommend using Test-Driven Development, because this approach leads among others to less coupled code, eventually increasing maintainability.

In this post, we will create two integration tests² for the django-orcid app, using its authentication backend to log into our Django website. Normally, this procedure would call the ORCID server for authentication data – a behaviour we want to avoid for our tests. After having all tests passing, we will make the tests easy to understand and make adding further tests a joy. Although the latter may sound trivial, this is an important consideration!

I assume that you have a fair idea of why tests are important and how tests in Django are written (if not here is a starting point).

Testing Setup

Django comes per default with a testing framework extending the possibilities of the Python core library unittest. Although there are packages further extending the Django testing possibilities (e.g. pytest-django), we will stick here with the default Django testing framework.

If you use additional packages for testing, make sure to separate their installation from the packages that are necessary for running the main application. I recommend using a setup.py with the following additional configuration:

1
2
3
4
5
6

    ...
    extra_require={
        'dev': [
            'pytest-django'
        ]
    }

This allows to install the development environment only when using pip install django-orcid['dev']. Alternatively, you can put all testing related packages into a requirements-dev.txt. In other programming languages, the compilers also allow such configurations (e.g. Java’s maven has a ’test’ scope for exactly this purpose). Separating your testing dependencies from your main application, reduces the risk of license collisions/problems in your published applications.

Writing Tests

Testing a failing login

Before we check for a successful login, we need to make sure that not everybody will get access to our website via the django-orcid authentication. Hence, first we test that any simple GET request will return a failed login HTML.

The first test looks like this and pass:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

from django.test import TestCase, RequestFactory

from django_orcid import views


class TestLogin(TestCase):
    """ Tests the application with a running test database in the background. """
    def test_failing_login(self):
        authentication_url = accounts/auth'

        # Create a mock request with the provided data
        request_factory = RequestFactory()
        request = request_factory.get(authentication_url)

        # Test the view!
        response = views.auth(request)

        self.assertIn('Login Failed', str(response.content))

Be aware that this test does not call the URL itself, but the RequestFactory only generates a request object, which is handed to the django-orcid authentication view.

By using the base testing class LiveServerTestCase, Django would start a test server in the background automatically and we could request the authentication URL via the client member of this testing class. But that would be out of the scope of our test goal - which is to test, if we can login or not.

That was the easy part! The login in the test above fails, because no token for authentication was provided. If we would provide a token, Django would try to connect to the ORCID server and complain that we do not provide the correct credentials for any action on the ORCID server. However, if we want to test a successful login, we need to work around this problem. To do so, in our case, we need to use mock.patch from the unittest core library.

Alternatives to mock.patch

Using mock.patch in tests is often seen as a “code smell”. You may hear developers say that when you use mock.patch, you messed up something in your software design already. Although I think that mock.patch has its use cases, I generally try to avoid it. Just remember that with great power comes great responsibility and you should have appropriate background knowledge on how mock.patch works and when (not) to use it.

I want to point out briefly three ways to avoid mock.patch in a environment where the class under testing is initialised by the framework (Django in our case). In this case, a better implementation would be to provide an abstract base class or a Protocol class. From this class, we could derive a class only for testing purpose, which provides all the functionality and methods we want to test. Only the method which makes the call to the external dependency would change in this case. In the testing setup or in a specific Django testing configuration, you could now configure the usage of this testing class for the app under testing.

By doing so, we not only have more flexibility in our tests (by simply adding another derived testing class providing [fixed] mock data), but subsequently can add other classes that implement different data retrieval approaches (e.g. a class not using requests but ZeroMQ) in the production code. These different implementations would need each an own suite of test cases that only test the methods of the specific class in a unittest approach. This way, we could separate testing the data flow within the app from the tests of the actually used implementations.

Another approach, that is very similar, would be the usage of dependency injection. Only in this case, instead of creating multiple classes doing all more or less the same and only differing in their way calling the external API, you have only one “working” class that is provided with an object as dependency that is actually making the call to the external API. In the test, we would then provide the class with a dependency that simply returns mock data.

The third alternative to mock.patch is to derive a testing class from the class under test by inheritance. In this child class, we could now overwrite the function that is making the actual HTTP call and return a dummy response. However, again we would need control over the object instantiation, which we do not have in the approach I showed above.

Testing a successful login

To write a test that successfully logs into Django without calling the ORCID server, we apply mock.patch and come up with something like this (which passes):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43

from unittest import mock

from django.contrib.sessions.middleware import SessionMiddleware
from django.test import TestCase, RequestFactory

from django_orcid import views
from django_orcid.tests.fixtures import (
    orcid_get_records_data,
    orcid_post_response_data,
    MockResponse
)


class TestLogin(TestCase):
    def test_failing_login(self):
        ...

    @mock.patch(target='requests.get', autospec=True)
    @mock.patch(target='requests.post', autospec=True)
    def test_successful_login(self, mock_post_request: mock.MagicMock, mock_get_request: mock.MagicMock):
        # Provide the mocks with dummy data
        mock_post_request.return_value = MockResponse(json_data=orcid_post_response_data(),
                                                      status_code=200)
        mock_get_request.return_value = MockResponse(json_data=orcid_get_records_data(), status_code=200)

        token = 'dummy-token-123'
        authentication_url = accounts/auth'

        # Create a mock request with the provided data
        request_factory = RequestFactory()
        request = request_factory.get(authentication_url, data={'code': token},
                                      **{'HTTP_X_FORWARD_FOR': '123.456.7.89'})

        # The request object needs a session and a COOKIES property that will be added by the
        # SessionMiddleware
        middleware = SessionMiddleware()
        middleware.process_request(request)
        request.session.save()

        # Test the view!
        response = views.auth(request)

        self.assertIn('You successfully logged in', str(response.content))

With the @mock.patch decorators, we patched all occurrences of requests.get and requests.post calls and provide their respective “mocks”³ with test data. Additionally, most of the testing methods and classes are put into a fixtures.py (which you can find here) file to keep the test file as clear as possible.

I created a MockResponse class that the mocks use as pseudo ORCID response data. They will return these objects when requests.get or requests.post are called with any (!) argument, respectively. A dataclass like MockResponse allows total and easy control of the data flow. Moreover, in future tests it could aid by e.g. providing malicious response data and check our application for security.

Cleaning Up

Okay! The tests work, but they are both super ugly and barely maintainable. Writing tests and especially extending existing tests should be fun (otherwise people will not write them ;) ). So, the last step is to clean up the tests to make them as clear and as short as possible.

The cleaned version of the above code could look like this (find the complete code on GitHub):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61

class TestLogin(TestCase):
    """Tests the application with a running test database in the background."""

    authentication_url = "/accounts/auth"

    def test_failing_login(self):
        request = create_get_request(self.authentication_url)
        response = views.auth(request)

        self.assertIn("Login Failed", str(response.content))

    def test_successful_login(self):
        request = create_get_request(orcid_authentication_request_parameters)
        response = views.auth(request)

        self.assertIn("You successfully logged in", str(response.content))

    def setUp(self) -> None:
        self.django_orcid_test_settings = DjangoOrcidTestSettings()
        self.django_orcid_test_settings.setup()

        self.request_factory = RequestFactory()

        self.mocks = []
        for mock_parameters in orcid_patches:
            return_value = (
                mock_parameters["return_value"]
                if "return_value" in mock_parameters
                else None
            )
            patcher = mock.patch(**mock_parameters)

            if return_value is not None:
                patcher.return_value = return_value

            patcher.start()
            self.mocks.append(patcher)

            # This makes sure that the patch will be stopped, even if there is an exception
            self.addCleanup(patcher.stop)

    def tearDown(self) -> None:
        self.django_orcid_test_settings.restore()


orcid_patches = [
    {
        "target": "requests.get",
        "auto_spec": True,
        "return_value": MockResponse(
            json_data=orcid_get_records_data(), status_code=200
        ),
    },
    {
        "target": "requests.post",
        "auto_spec": True,
        "return_value": MockResponse(
            json_data=orcid_post_response_data(), status_code=200
        ),
    },
]

To achieve clear tests, I use the setUp method of the TestCase class that is called before every test. Into the setUp method, I pushed all preparations that may be also of use for other tests of the same class. self.mocks now holds default mocks for reuse in all tests of this class. self.mocks allows us to manipulate e.g. the data returned by the mock within single tests, if necessary.

You see that by moving many repeatable tasks into separate methods, the two created tests melt down to 3 lines of code, making them very clear in what they do. This shortness makes them very easy to understand and allows you to add more tests even in 3 months, when you forgot about the details.

The setUp method is intentionally at the bottom of the class. If your test suite throws an error and you open the file with the failing test, you do not want to read over the setUp first, but directly want to skim the tests.

Incidentally, both tests also fulfill a testing “dogma”: That there should be only one assert statement per test. In my opinion, this should be striven for but should not hinder you from adding two or three asserts into a single test. However, in this case, you may consider using a custom assert method that you provide only with an object and the expected data.

Conclusion

In the above example, I provided you with an idea of how to make your application maintainable and extensible even when facing an external API. By creating these tests, you are even able to test you application compatibility with different dependency versions via tox⁴.