How to use mock_open with json.load()?

Question

I'm trying to get a unit test working that validates a function that reads credentials from a JSON-encoded file. Since the credentials themselves aren't fixed, the unit test needs to provide some and then test that they are correctly retrieved.

Here is the credentials function:

def read_credentials():
    basedir = os.path.dirname(__file__)
    with open(os.path.join(basedir, "authentication.json")) as f:
        data = json.load(f)
        return data["bot_name"], data["bot_password"]

and here is the test:

def test_credentials(self):
    with patch("builtins.open", mock_open(
        read_data='{"bot_name": "name", "bot_password": "password"}\n'
    )):
        name, password = shared.read_credentials()
    self.assertEqual(name, "name")
    self.assertEqual(password, "password")

However, when I run the test, the json code blows up with a decode error. Looking at the json code itself, I'm struggling to see why the mock test is failing because json.load(f) simply calls f.read() then calls json.loads().

Indeed, if I change my authentication function to the following, the unit test works:

def read_credentials():
    # Read the authentication file from the current directory and create a
    # HTTPBasicAuth object that can then be used for future calls.
    basedir = os.path.dirname(__file__)
    with open(os.path.join(basedir, "authentication.json")) as f:
        content = f.read()
        data = json.loads(content)
        return data["bot_name"], data["bot_password"]

I don't necessarily mind leaving my code in this form, but I'd like to understand if I've got something wrong in my test that would allow me to keep my function in its original form.

Stack trace:

Traceback (most recent call last):
  File "test_shared.py", line 56, in test_credentials
shared.read_credentials()
  File "shared.py", line 60, in read_credentials
data = json.loads(content)
  File "/home/philip/.local/share/virtualenvs/atlassian-webhook-basic-3gOncDp4/lib/python3.6/site-packages/flask/json/__init__.py", line 205, in loads
return _json.loads(s, **kwargs)
  File "/usr/lib/python3.6/json/__init__.py", line 367, in loads
return cls(**kw).decode(s)
  File "/usr/lib/python3.6/json/decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.6/json/decoder.py", line 357, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Answer 1

I had the same issue and got around it by mocking json.load and builtins.open:

import json
from unittest.mock import patch, MagicMock

# I don't care about the actual open
p1 = patch( "builtins.open", MagicMock() )

m = MagicMock( side_effect = [ { "foo": "bar" } ] )
p2 = patch( "json.load", m )

with p1 as p_open:
    with p2 as p_json_load:
        f = open( "filename" )
        print( json.load( f ) ) 

Result:

{'foo': 'bar'}

Answer 2

I had the exact same issue and solved it. Full code below, first the function to test, then the test itself.

The original function I want to test loads a json file that is structured like a dictionary, and checks to see if there's a specific key-value pair in it:

def check_if_file_has_real_data(filepath):
    with open(filepath, "r") as f:
        data = json.load(f)
    if "fake" in data["the_data"]:
        return False
    else:
        return True

But I want to test this without loading any actual file, exactly as you describe. Here's how I solved it:

from my_module import check_if_file_has_real_data

import mock

@mock.patch("my_module.json.load")
@mock.patch("my_module.open")
def test_check_if_file_has_real_data(mock_open, mock_json_load):
    mock_json_load.return_value = dict({"the_data": "This is fake data"})
    assert check_if_file_has_real_data("filepath") == False

    mock_json_load.return_value = dict({"the_data": "This is real data"})
    assert check_if_file_has_real_data("filepath") == True

The mock_open object isn't called explicitly in the test function, but if you don't include that decorator and argument you get a filepath error when the with open part of the check_if_file_has_real_data function tries to run using the actual open function rather than the MagicMock object that's been passed into it.

Then you overwrite the response provided by the json.load mock with whatever you want to test.