How Do You Use JSON in Python: Let’s Find Out

The Python json module allows to use data in JSON format in your applications. Do you want to convert a Python object into JSON or viceversa? This is the right module to do that.

The Python json module provides functions to convert a Python object into JSON (dump() and dumps()), this conversion is called serialization or encoding. The json module also provides functions to convert JSON data into a Python object (load() and loads()), this is called deserialization or decoding.

In this tutorial we will learn how to use the Python json module to perform data serialization and deserialization using the JSON format.

Let’s start by looking at two functions provided by the json module: dumps() and loads().

What is JSON Dumps in Python?

The json.dumps() function converts a Python object into a string in JSON format.

The process of converting a Python object into a JSON string is called serialization. This process is also called encoding.

For example, let’s define a Python dictionary:

>>> user = {'name': 'John', 'email': 'test@email.com'}
>>> type(user)
<class 'dict'>

And see what output we get when we pass it to the json.dumps() function.

>>> import json
>>> user_json = json.dumps(user)
>>> print(user_json)
{"name": "John", "email": "test@email.com"}
>>> type(user_json)
<class 'str'> 

The dumps function has converted (serialized) the dictionary into a JSON string.

The ‘s’ in the name of the dumps function refers to the fact that this function works with strings (as opposed as the dump() function that works on files).

What is JSON Loads in Python?

The process of converting a JSON string into a Python object is called deserialization. This process is also called decoding.

The json.loads() function performs deserialization of a JSON string into a Python object.

Take the JSON string we have generated in the previous section and convert it back into a dictionary:

>>> user = json.loads(user_json)
>>> print(user)
{'name': 'John', 'email': 'test@email.com'}
>>> type(user)
<class 'dict'> 

In the same way, we have seen with the json.dumps() function, the json.loads() function name ends with the letter ‘s’.

That’s because it applies to JSON strings as opposed to JSON files (for that you would use the json.load() function).

How Do You Read a JSON File in Python?

The Python json module provides the load() function to read the content of a JSON file and convert it into a Python object.

The behavior is the same as the loads() function with the only difference being that loads() applies to strings and load() to files.

Here is how you can read JSON data from a file. I have created a file called posts.json with the following content:

[
  { 
    "userId": 1,
    "postId": 1,
    "title": "How to Convert a JSON Object into a Python Object"
  },
  { 
    "userId": 1,
    "postId": 2,
    "title": "How to Convert a Python Object into a JSON Object"
  }
] 

Let’s say I want to read the title of the first post.

Firstly, let’s convert the JSON in the file into a Python object using load(). To open the file we will use the with open context manager that closes the file automatically when it’s not needed anymore.

>>> import json
>>> with open('posts.json') as json_file:
...     data = json.load(json_file)
... 
>>> print(data)
[{'userId': 1, 'postId': 1, 'title': 'How to Convert a JSON Object into a Python Object'}, {'userId': 1, 'postId': 2, 'title': 'How to Convert a Python Object into a JSON Object'}]
>>> print(type(data))
<class 'list'> 

We have converted (or decoded or deserialized) our data into a list.

Let’s confirm that the elements of the list are dictionaries:

>>> print(type(data[0]))
<class 'dict'> 

That’s good, this means that to get the title of the first post we can simply access the ‘title’ key of the data[0] dictionary.

>>> print(data[0]['title'])
How to Convert a JSON Object into a Python Object 

How Do You Write a JSON File in Python?

We have seen how to read JSON data from a file and convert it into a Python object.

How can you do the opposite?

The Python json module provides the dump() function that writes a Python object to a file in JSON format.

Let’s start from the Python object created in the previous section:

>>> data = [{'userId': 1, 'postId': 1, 'title': 'How to Convert a JSON Object into a Python Object'}, {'userId': 1, 'postId': 2, 'title': 'How to Convert a Python Object into a JSON Object'}] 

Instead of simply writing this data to a JSON file we will modify the data first.

As a result we want to have a JSON object in which both items contains a new empty attribute called “body”.

Add the “body” key to both dictionaries inside our Python list and then write the object to a file using the dump() function.

>>> data[0]['body'] = ''
>>> data[1]['body'] = ''
>>> print(data)
[{'userId': 1, 'postId': 1, 'title': 'How to Convert a JSON Object into a Python Object', 'body': ''}, {'userId': 1, 'postId': 2, 'title': 'How to Convert a Python Object into a JSON Object', 'body': ''}]
>>> with open('posts.json') as json_file:
...     json.dump(data, json_file)
... 
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "/opt/anaconda3/lib/python3.8/json/__init__.py", line 180, in dump
    fp.write(chunk)
io.UnsupportedOperation: not writable 

Using with open and the dump() function we write to the posts.json file. Notice that the dump() function takes the Python object as first argument and the file object as second argument.

However, we get an exception back related to the fact that the file object is not writable.

That’s because we have to explicitly open the file in write mode.

>>> with open('posts.json', 'w') as json_file:
...     json.dump(data, json_file)
... 
>>>  

All good this time!

Let’s read the content of the updated JSON file to confirm both items have the “body” attribute:

>>> with open('posts.json') as json_file:
...     print(json_file.read())
... 
[{"userId": 1, "postId": 1, "title": "How to Convert a JSON Object into a Python Object", "body": ""}, {"userId": 1, "postId": 2, "title": "How to Convert a Python Object into a JSON Object", "body": ""}] 

Yes, the JSON file has been updated as we expected!

JSON Conversion Table in Python

The conversion of a JSON object into a Python object follows the conventions in the table below:

JSON	Python
object	dict
array	list
string	str
number (int)	int
number (real)	float
true	True
false	False
null	None

Let’s demonstrate with some examples the mappings shown in the JSON conversion table.

We have already seen before that often when a JSON string gets deserialized the result is a dictionary.

For example, let’s deserialize the following JSON array:

>>> weekdays_json = '["Monday", "Tuesday", "Wednesday", "Thursday", "Friday"]' 

To convert this array into a Python object we can use the loads() function.

>>> import json
>>> weekdays = json.loads(weekdays_json)
>>> print(weekdays)
['Monday', 'Tuesday', 'Wednesday', 'Thursday', 'Friday']
>>> print(type(weekdays))
<class 'list'> 

You can see that after decoding a JSON array we got back a Python list.

>>> print(weekdays[2])
Wednesday 

Let’s see what happens if a JSON object contains an array as one of its values.

>>> json_value = '{"weekdays": ["Monday", "Tuesday", "Wednesday", "Thursday", "Friday"]}'
>>> value = json.loads(json_value)
>>> print(value)
 {'weekdays': ['Monday', 'Tuesday', 'Wednesday', 'Thursday', 'Friday']}
>>> print(type(value))
<class 'dict'> 

As shown in the conversion table the JSON object becomes a dictionary.

At the same time the value inside our JSON object mapped to the key “weekdays” was a JSON array and now it’s a Python list.

>>> print(type(value['weekdays']))
<class 'list'> 

Converting Boolean, Numeric and Null Values from JSON to Python

Let’s use another object to show the JSON-to-Python conversion for numbers, booleans and null values.

>>> json_value = '{"value1": true, "value2": false, "value3": 1, "value4": 3.4, "value5": null}'
>>> value = json.loads(json_value)
>>> print(value)
{'value1': True, 'value2': False, 'value3': 1, 'value4': 3.4, 'value5': None} 

You can see that after decoding our JSON object the following happens:

• true becomes True.
• false becomes False.
• int and real numbers become Python int and float numbers.
• null becomes None.

>>> print(type(value['value1']))
<class 'bool'>
>>> print(type(value['value2']))
<class 'bool'>
>>> print(type(value['value3']))
<class 'int'>
>>> print(type(value['value4']))
<class 'float'>
>>> print(type(value['value5']))
<class 'NoneType'> 

Now, how can we go back from the Python object to the JSON string?

We can use the json.dumps() function…

>>> print(value)
{'value1': True, 'value2': False, 'value3': 1, 'value4': 3.4, 'value5': None}
>>> json_value = json.dumps(value)
>>> print(json_value)
{"value1": true, "value2": false, "value3": 1, "value4": 3.4, "value5": null} 

Et voilà, we got back the original JSON object.

How to Pretty Print JSON Data

In the previous section we have seen an example of JSON object where every value is printed on a single line:

>>> print(json_value)
{"value1": true, "value2": false, "value3": 1, "value4": 3.4, "value5": null}

This could work if the JSON object doesn’t contain too many attributes and if its values are not too long.

But, imagine if you have a JSON object with 50 attributes and values that could be arrays with multiple elements.

At that point, it would become really hard to read the JSON object when represented in a single line.

To solve this problem there is the optional argument indent that we can pass to the dumps() function when we encode a Python object into JSON.

Have a look at how the encoded JSON changes if we pass three different values for indent: 0, 1, and 2.

>>> print(json.dumps(value, indent=0))
{
"value1": true,
"value2": false,
"value3": 1,
"value4": 3.4,
"value5": null
} 

>>> print(json.dumps(value, indent=1))
{
 "value1": true,
 "value2": false,
 "value3": 1,
 "value4": 3.4,
 "value5": null
} 

>>> print(json.dumps(value, indent=2))
{
  "value1": true,
  "value2": false,
  "value3": 1,
  "value4": 3.4,
  "value5": null
} 

When we pass the indent argument to the json.dumps() function two things happen:

1. The JSON object is displayed on multiple lines and hence is more readable.
2. Every attribute of the JSON object is preceded by several whitespaces that depend on the value of indent.

Separators For JSON Objects in Python

The json.dumps and json.dump functions also accept an optional argument called separators that is a tuple with the following format:

(item_separator, key_separator)

As you can imagine changing item and key separators completely changes the way a JSON object looks.

The default separators value used in Python to encode objects into JSON is (‘, ‘, ‘: ‘).

Notice how both the default item and key separator contain a whitespace. This is the JSON we have generated so far using the default separators and indent set to 2:

>>> print(json.dumps(value, indent=2))
{
  "value1": true,
  "value2": false,
  "value3": 1,
  "value4": 3.4,
  "value5": null
} 

Now notice the difference if we change the value of the separators argument.

We can remove the whitespace to obtain a more compact JSON representation:

>>> print(json.dumps(value, indent=2, separators=(',',':')))
{
  "value1":true,
  "value2":false,
  "value3":1,
  "value4":3.4,
  "value5":null
} 

Sorting the Keys of a JSON Object

In some cases, it might be handy to be able to sort the keys in a JSON object.

Imagine you have a JSON object that has 100 keys…

…ordering them would make the object more readable.

Here is how we can sort the keys of a JSON object.

For instance, let’s start with the following Python object converted into a JSON object:

>>> user = {'name': 'John', 'surname': 'Red', 'age':35}
>>> print(json.dumps(user))
{"name": "John", "surname": "Red", "age": 35} 

You can see that the keys of the JSON object are not sorted.

To sort the keys of a JSON object when encoding it from a Python object you can use the argument sort_keys, a boolean whose default value is False.

>>> print(json.dumps(user, sort_keys=True))
{"age": 35, "name": "John", "surname": "Red"} 

As a result, this time the JSON attributes are sorted alphabetically.

Using a For Loop to Print Elements in a JSON Array

Let’s say we have the following JSON file called users.json that contains details about all the users who subscribed to your service:

{
  "users": [
    {
      "id": 1,
      "first_name": "John",
      "last_name": "Red",
      "email": "johnred@example.com"
    },
    {
      "id": 2,
      "first_name": "Mike",
      "last_name": "Yellow",
      "email": "mikeyellow@example.com"
    },
    {
      "id": 3,
      "first_name": "Jane",
      "last_name": "Green",
      "email": "janegreen@example.com"
    }
  ]
} 

How can we use a Python for loop to go through every user in the JSON array?

First of all, we have to convert the content of the JSON file into a Python object. To do that we will use the json.load() function.

>>> import json
>>> with open('users.json') as json_file:
...     data = json.load(json_file)
... 
>>> print(data)
{'users': [{'id': 1, 'first_name': 'John', 'last_name': 'Red', 'email': 'johnred@example.com'}, {'id': 2, 'first_name': 'Mike', 'last_name': 'Yellow', 'email': 'mikeyellow@example.com'}, {'id': 3, 'first_name': 'Jane', 'last_name': 'Green', 'email': 'janegreen@example.com'}]}
>>> print(type(data))
<class 'dict'> 

Using load() we have created a Python object called data that we can then use to read the details about each user.

The Python object is a dictionary, to go through each user we have to access the ‘users’ key first. Then loop through it using a for loop:

>>> for user in data['users']:
...     print(user)
... 
{'id': 1, 'first_name': 'John', 'last_name': 'Red', 'email': 'johnred@example.com'}
{'id': 2, 'first_name': 'Mike', 'last_name': 'Yellow', 'email': 'mikeyellow@example.com'}
{'id': 3, 'first_name': 'Jane', 'last_name': 'Green', 'email': 'janegreen@example.com'} 

Makes sense?

How to Get JSON From an API Using Python

But, how would you use what we have learned so far in practice?

JSON is the most common data format used for APIs nowadays, this means that knowing how to read it and write it allows you to integrate multiple systems that exchange data in JSON format.

Let’s have a look at how you would read JSON data returned by a public API.

For this example, we will use the following API that returns dog breeds:

https://dog.ceo/api/breeds/list/all

Note: this API could change in the future but the underlying concepts to retrieve data from it remain the same.

If you open this URL in your browser you will see the API response…

…but how can we retrieve the response using Python?

We can use the urllib.request module to perform a GET request to the API endpoint:

>>> import urllib.request
>>> response = urllib.request.urlopen('https://dog.ceo/api/breeds/list/all').read().decode()

Let’s print the first 200 characters of the response using the slicing operator:

>>> print(response[:200])
{"message":{"affenpinscher":[],"african":[],"airedale":[],"akita":[],"appenzeller":[],"australian":["shepherd"],"basenji":[],"beagle":[],"bluetick":[],"borzoi":[],"bouvier":[],"boxer":[],"brabancon":[ 

Now, we can use our knowledge in manipulating JSON data to convert the response coming from the API into a Python object using json.loads().

>>> import json
>>> data = json.loads(response)
>>> print(type(data))
<class 'dict'>
>>> print(data['message']['terrier'])
['american', 'australian', 'bedlington', 'border', 'dandie', 'fox', 'irish', 'kerryblue', 'lakeland', 'norfolk', 'norwich', 'patterdale', 'russell', 'scottish', 'sealyham', 'silky', 'tibetan', 'toy', 'westhighland', 'wheaten', 'yorkshire'] 

As a result, we got back a dictionary and we can now access any data we need.

Conclusion

Now you have the knowledge to use the json module to perform:

• serialization (encoding): convert a Python object into a JSON object.
• deserialization (decoding): convert a JSON object into a Python object.

We have learned that to work with JSON strings you can use json.dumps() and json.loads(). To work with JSON files you would use json.dump() and json.load().

You also know how to pretty print a JSON object to improve its readability and how to read the JSON response coming from an API.

JSON is one of the formats for exchanging data between systems and it doesn’t depend on the programming language the systems are built with.

Python also provides another way to serialize and deserialize your data, have a look at the Pickle module if you haven’t already.

Related article: read this CodeFatherTech tutorial about Python JSONPath to learn how to extract data from JSON documents.