1

tl;dr I am looking for the best practice for testing a flask app which uses external APIs under its routes, and probably this is where I have to simulate the external APIs using mock servers in the testing stage.


Ok, all I have got searching pytest-mock and similar packages is that I can mock a function and have the results prepared for test assertions. But unfortunately I cannot find a best-practice explanation for my situation:

I have a flask app, under some of the routes of which I use external APIs and I want to use mock servers replacing those real external API URLs. I have come to two apparently fair solutions:

Using test URLs when testing

What I have got so far is that I can set different URLs dynamically by app.config so that in my conftest.py I can invoke using test URLs, and in reality, my app will be launched by the main URLs. Now my question here is how I can run a mock server automatically in the pytest fixture setup stage to occupy the test URLs localhost ports. I said automatically because I can naively run a manual mock app in background myself, which is silly and useless for automated testing purposes

Using a mock function instead of sending the request to the main URL

I have realized that I can setup a mocker which is a function returning my desired value, and use it alongside calling my to-be-tested app. but the point is I want to use the mock inside my app, not comparing the returning value with a mock generated value

So where am I and what should I do?!

Here is a sample:

My flask app:

app = Flask(__name__)

@app.route('/test')
def test():
    value = request.args.get('dummy_val')

    # using external APIs
    # This is where I need mock server
    output = requests.get(
         'http://external.api/some/url',
         params={'val':value}
    )
    return output*2

my test file:

def test_can_double_value(test_client):
    result = test_client('/test', query_string={'dummy_val':'foo'})
    # test if status code == 200
    # test if string equals to something
    # blah blah blah
6
  • If you want to mock requests, you could use responses.
    – jonrsharpe
    Jan 5, 2019 at 9:10
  • @jonrsharpe what is the behavior of the @response.activate? if I decorate my test function, will my flask function also get the effect?
    – Alireza
    Jan 5, 2019 at 9:14
  • It would be pretty pointless if it only applied to requests made from the test function, no? I'd recommend trying it out and seeing whether it suits your purpose.
    – jonrsharpe
    Jan 5, 2019 at 9:19
  • @jonrsharpe It is successful if I add responses to the test itself. any idea how to make a fixture which enables responses feature? decorating fixture doesn't work
    – Alireza
    Jan 5, 2019 at 10:28
  • There's a section in the readme specifically on using responses as a pytest fixture, did you try that?
    – jonrsharpe
    Jan 5, 2019 at 10:28

2 Answers 2

1

Thanks to jonrsharpe I now am aware that this can be done using responses library. Now if we want to mock an API in the fixture, it can be done as follows:

in conftest.py

import response
from pytest import fixture

@fixture
def client():
    # add response
    responses.add(responses.GET,
        'http://mock.api/v',
        json={'key': 'value'},
        status=200
    )
    # do fixture stuff -> here it is yielding app test client
    app.config['TESTING'] = True
    client = app.test_client() 

    yield client

in test_module.py

import response

@response.activate
def test_can_connect(client):
    result = client.get('/test')
    assert result.status_code == 200

and in app.py

@app.route('/test')
def test():
    res = requests.get('http://mock.api/v')
    # blah blah

Comment in responses documentation it is suggested that we add a response in the test function. But as I wanted to add it in a fixture, this is how I did it.

5
  • That seems deeply suspect. Firstly, you have some typos. But more importantly the fixture should be providing the mock handler, not the client itself (you can have a second fixture for that). Secondly, explicitly telling the code it's being tested seems like a very bad idea - you end up testing the test branches rather than the production code. Finally, the point of setting the response in the test rather than the fixture is that: 1. you can have different responses in different tests (e.g. to ensure that errors and successes are both handled correctly); and 2. the test can be read on its own.
    – jonrsharpe
    Jan 5, 2019 at 10:53
  • it is right that the response can be added in a new fixture, but I did not experience any limit in setting the response in the test client fixture... I thought it would be a good idea to add the response in a fixture because many test functions are to be tested with mock servers, and I don't want to repeat myself. Am I right?
    – Alireza
    Jan 5, 2019 at 10:59
  • Your comment about explicitly telling the main code is probably because I used the 'mock.api/v' url in my flask app, but it was only a figure of speech! I meant any URL that is real and I wanted to mock it in the test stage by pytest
    – Alireza
    Jan 5, 2019 at 11:01
  • There is a limit, it's that you can't change the response anywhere. If you have one single response that's used in most of your test cases, by all means make that a specific fixture, but in general you don't need that many tests for one specific upstream response. And by "explicitly telling the code" I meant app.config['TESTING'] = True; apparently that's to "disable the error catching during request handling", so if you don't have conditionals in your own code based on it that's probably not a problem.
    – jonrsharpe
    Jan 5, 2019 at 11:05
  • Regarding the response adding in the fixture, you are right about the area of effect of the fixture, however, I may be able to control it by setting the scope of the fixture to function and do the response.remove in teardown stage of the fixture. Generally speaking, I did it because I wanted the same connection over the majority of the tests... and about the app.config part, you are right, it is a situation to be careful about. thanks a lot
    – Alireza
    Jan 5, 2019 at 11:35
1

Bit late to answer, but I have stumbled across the same problem just now. Previous answers seems to be solving the problem in practice, but if you think about your problem in theory, I think you have 2 options.

Either:

  • mock the interface on 'requests' level, which works fine, but exposes your test code to implementation detail, as suggested before me
  • create adapters for these services in your code, and replace them with your mocks/stubs in your test code.
  • using a test fixture, you can start your own webserver for the process using the code from this gist: https://gist.github.com/eruvanos/f6f62edb368a20aaa880e12976620db8

The benefit of the last approach is quite straightforward. You can leave your code as is when testing, no stubbing/mocking, and create the mocks outside your system, where you have free control over mocks, and don't have to worry about actual implementation (like before, you have to KNOW that the service is called via requests).

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.