8

I'm trying to test some python code that uses boto. I'd rather not try and do an integration test against AWS, so I am trying to mock it out with moto, and it's not behaving as I'd expect.

Here's the test code:

import io
import boto3

from moto import mock_ec2
from unittest.mock import patch
from argparse import Namespace
from awswl import commands


@mock_ec2
@patch('awswl.externalip.get_external_ip', return_value='192.0.2.1')
def test_list_command_lists_ipv4_and_ipv6_cidrs(exip_method):
    # Given
    options = Namespace()
    options.sgid = "sg-123456"

    ec2 = boto3.resource('ec2')
    sg = ec2.create_security_group(
        Description='Security Group for SSH Whitelisting',
        GroupName='SSH Whitelist'
    )
    print("Created security group: {0}".format(sg.GroupId))

And the error:

Testing started at 16:46 ...
/path/.virtualenvs/awswl-Ir8BWU8l/bin/python "/path/Library/Application Support/JetBrains/Toolbox/apps/PyCharm-P/ch-0/173.4301.16/PyCharm.app/Contents/helpers/pycharm/_jb_pytest_runner.py" --target test_commands.py::test_list_command_lists_ipv4_and_ipv6_cidrs
Launching py.test with arguments test_commands.py::test_list_command_lists_ipv4_and_ipv6_cidrs in /path/awswl/tests

============================= test session starts ==============================
platform darwin -- Python 3.6.4, pytest-3.4.2, py-1.5.2, pluggy-0.6.0
rootdir: /path/awswl, inifile:
collected 1 item
test_commands.py F
tests/test_commands.py:22 (test_list_command_lists_ipv4_and_ipv6_cidrs)
exip_method = <MagicMock name='get_external_ip' id='4547595848'>

    @mock_ec2
    @patch('awswl.externalip.get_external_ip', return_value='192.0.2.1')
    def test_list_command_lists_ipv4_and_ipv6_cidrs(exip_method):
        # Given
        options = Namespace()
        options.sgid = "sg-123456"

>       ec2 = boto3.resource('ec2')

test_commands.py:30: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../../../../.virtualenvs/awswl-Ir8BWU8l/lib/python3.6/site-packages/boto3/__init__.py:92: in resource
    return _get_default_session().resource(*args, **kwargs)
../../../../../.virtualenvs/awswl-Ir8BWU8l/lib/python3.6/site-packages/boto3/session.py:389: in resource
    aws_session_token=aws_session_token, config=config)
../../../../../.virtualenvs/awswl-Ir8BWU8l/lib/python3.6/site-packages/boto3/session.py:263: in client
    aws_session_token=aws_session_token, config=config)
../../../../../.virtualenvs/awswl-Ir8BWU8l/lib/python3.6/site-packages/botocore/session.py:861: in create_client
    client_config=config, api_version=api_version)
../../../../../.virtualenvs/awswl-Ir8BWU8l/lib/python3.6/site-packages/botocore/client.py:76: in create_client
    verify, credentials, scoped_config, client_config, endpoint_bridge)
../../../../../.virtualenvs/awswl-Ir8BWU8l/lib/python3.6/site-packages/botocore/client.py:285: in _get_client_args
    verify, credentials, scoped_config, client_config, endpoint_bridge)
../../../../../.virtualenvs/awswl-Ir8BWU8l/lib/python3.6/site-packages/botocore/args.py:45: in get_client_args
    endpoint_url, is_secure, scoped_config)
../../../../../.virtualenvs/awswl-Ir8BWU8l/lib/python3.6/site-packages/botocore/args.py:111: in compute_client_args
    service_name, region_name, endpoint_url, is_secure)
../../../../../.virtualenvs/awswl-Ir8BWU8l/lib/python3.6/site-packages/botocore/client.py:358: in resolve
    service_name, region_name)
../../../../../.virtualenvs/awswl-Ir8BWU8l/lib/python3.6/site-packages/botocore/regions.py:122: in construct_endpoint
    partition, service_name, region_name)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <botocore.regions.EndpointResolver object at 0x10f1a3908>
partition = OrderedDict([('defaults', OrderedDict([('hostname', '{service}.{region}.{dnsSuffix}'), ('protocols', ['https']), ('sig...1', OrderedDict()), ('us-east-2', OrderedDict()), ('us-west-1', OrderedDict()), ('us-west-2', OrderedDict())]))]))]))])
service_name = 'ec2', region_name = None

    def _endpoint_for_partition(self, partition, service_name, region_name):
        # Get the service from the partition, or an empty template.
        service_data = partition['services'].get(
            service_name, DEFAULT_SERVICE_DATA)
        # Use the partition endpoint if no region is supplied.
        if region_name is None:
            if 'partitionEndpoint' in service_data:
                region_name = service_data['partitionEndpoint']
            else:
>               raise NoRegionError()
E               botocore.exceptions.NoRegionError: You must specify a region.

../../../../../.virtualenvs/awswl-Ir8BWU8l/lib/python3.6/site-packages/botocore/regions.py:135: NoRegionError
                                                       [100%]

=================================== FAILURES ===================================
_________________ test_list_command_lists_ipv4_and_ipv6_cidrs __________________

exip_method = <MagicMock name='get_external_ip' id='4547595848'>

    @mock_ec2
    @patch('awswl.externalip.get_external_ip', return_value='192.0.2.1')
    def test_list_command_lists_ipv4_and_ipv6_cidrs(exip_method):
        # Given
        options = Namespace()
        options.sgid = "sg-123456"

>       ec2 = boto3.resource('ec2')

test_commands.py:30: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../../../../.virtualenvs/awswl-Ir8BWU8l/lib/python3.6/site-packages/boto3/__init__.py:92: in resource
    return _get_default_session().resource(*args, **kwargs)
../../../../../.virtualenvs/awswl-Ir8BWU8l/lib/python3.6/site-packages/boto3/session.py:389: in resource
    aws_session_token=aws_session_token, config=config)
../../../../../.virtualenvs/awswl-Ir8BWU8l/lib/python3.6/site-packages/boto3/session.py:263: in client
    aws_session_token=aws_session_token, config=config)
../../../../../.virtualenvs/awswl-Ir8BWU8l/lib/python3.6/site-packages/botocore/session.py:861: in create_client
    client_config=config, api_version=api_version)
../../../../../.virtualenvs/awswl-Ir8BWU8l/lib/python3.6/site-packages/botocore/client.py:76: in create_client
    verify, credentials, scoped_config, client_config, endpoint_bridge)
../../../../../.virtualenvs/awswl-Ir8BWU8l/lib/python3.6/site-packages/botocore/client.py:285: in _get_client_args
    verify, credentials, scoped_config, client_config, endpoint_bridge)
../../../../../.virtualenvs/awswl-Ir8BWU8l/lib/python3.6/site-packages/botocore/args.py:45: in get_client_args
    endpoint_url, is_secure, scoped_config)
../../../../../.virtualenvs/awswl-Ir8BWU8l/lib/python3.6/site-packages/botocore/args.py:111: in compute_client_args
    service_name, region_name, endpoint_url, is_secure)
../../../../../.virtualenvs/awswl-Ir8BWU8l/lib/python3.6/site-packages/botocore/client.py:358: in resolve
    service_name, region_name)
../../../../../.virtualenvs/awswl-Ir8BWU8l/lib/python3.6/site-packages/botocore/regions.py:122: in construct_endpoint
    partition, service_name, region_name)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <botocore.regions.EndpointResolver object at 0x10f1a3908>
partition = OrderedDict([('defaults', OrderedDict([('hostname', '{service}.{region}.{dnsSuffix}'), ('protocols', ['https']), ('sig...1', OrderedDict()), ('us-east-2', OrderedDict()), ('us-west-1', OrderedDict()), ('us-west-2', OrderedDict())]))]))]))])
service_name = 'ec2', region_name = None

    def _endpoint_for_partition(self, partition, service_name, region_name):
        # Get the service from the partition, or an empty template.
        service_data = partition['services'].get(
            service_name, DEFAULT_SERVICE_DATA)
        # Use the partition endpoint if no region is supplied.
        if region_name is None:
            if 'partitionEndpoint' in service_data:
                region_name = service_data['partitionEndpoint']
            else:
>               raise NoRegionError()
E               botocore.exceptions.NoRegionError: You must specify a region.

../../../../../.virtualenvs/awswl-Ir8BWU8l/lib/python3.6/site-packages/botocore/regions.py:135: NoRegionError
=========================== 1 failed in 1.80 seconds ===========================
Process finished with exit code 0

It looks like it's invoking boto, not mocking it out with Moto, and I'm getting a NoRegionError because there's no region/profile specified.

What am I doing wrong? I assume it's me, but I haven't figured out how. ;)

1 Answer 1

7
+50

It looks like the problem as the test output points out might be related to not specifying any region when creating the resource with boto3. From your test output:

botocore.exceptions.NoRegionError: You must specify a region.

Checking also moto's tests, looks like the region is always specified there as well.

Simply initialising the resource with an example region should fix your problem:

boto3.resource('ec2', region_name='eu-central-1')
2
  • 2
    You're right -- I had mistakenly assumed: (a) Moto didn't need a region. (b) the fact that the code throwing the exception was Boto meant that Moto mocking somehow wasn't working. However, as soon as I set os.environ['AWS_DEFAULT_REGION'], the mocking starts working as expected. Mar 21, 2018 at 14:49
  • 1
    The way moto mocking integrates is actually fairly deep in the boto stack -- it's more like having mock service (although the http part is mocked, not just the remote end). Mar 23, 2018 at 16:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.