How to merge multiple json files into one file in python

Question

I want to merge multiple json files into one file in python. The thing that I want to do is if there are several .json files like:

# temp1.json
[{'num':'1', 'item':'smartphone','data':'2019-01-01'},
{'num':'2', 'item':'smartphone','data':'2019-01-02'},
{'num':'3', 'item':'smartphone','data':'2019-01-03'},
{'num':'4', 'item':'smartphone','data':'2019-01-04'}]

# temp2.json
[{'num':'5', 'item':'smartphone','data':'2019-01-05'},
{'num':'6', 'item':'smartphone','data':'2019-01-06'},
{'num':'7', 'item':'smartphone','data':'2019-01-07'}]

# temp3.json
[{'num':'8', 'item':'smartphone','data':'2019-01-08'},
{'num':'9', 'item':'smartphone','data':'2019-01-09'},
{'num':'10', 'item':'smartphone','data':'2019-01-10'},
{'num':'11', 'item':'smartphone','data':'2019-01-11'},
{'num':'12', 'item':'smartphone','data':'2019-01-12'}]

The result.json files I want to get should look like:

# result.json
[{'num':'1', 'item':'smartphone','data':'2019-01-01'},
{'num':'2', 'item':'smartphone','data':'2019-01-02'},
{'num':'3', 'item':'smartphone','data':'2019-01-03'},
{'num':'4', 'item':'smartphone','data':'2019-01-04'},
{'num':'5', 'item':'smartphone','data':'2019-01-05'},
{'num':'6', 'item':'smartphone','data':'2019-01-06'},
{'num':'7', 'item':'smartphone','data':'2019-01-07'},
{'num':'8', 'item':'smartphone','data':'2019-01-08'},
{'num':'9', 'item':'smartphone','data':'2019-01-09'},
{'num':'10', 'item':'smartphone','data':'2019-01-10'},
{'num':'11', 'item':'smartphone','data':'2019-01-11'},
{'num':'12', 'item':'smartphone','data':'2019-01-12'}]

The result.json files I got is:

# result.json
[[{'num':'1', 'item':'smartphone','data':'2019-01-01'},
{'num':'2', 'item':'smartphone','data':'2019-01-02'},
{'num':'3', 'item':'smartphone','data':'2019-01-03'},
{'num':'4', 'item':'smartphone','data':'2019-01-04'}],
[{'num':'5', 'item':'smartphone','data':'2019-01-05'},
{'num':'6', 'item':'smartphone','data':'2019-01-06'},
{'num':'7', 'item':'smartphone','data':'2019-01-07'}],
[{'num':'8', 'item':'smartphone','data':'2019-01-08'},
{'num':'9', 'item':'smartphone','data':'2019-01-09'},
{'num':'10', 'item':'smartphone','data':'2019-01-10'},
{'num':'11', 'item':'smartphone','data':'2019-01-11'},
{'num':'12', 'item':'smartphone','data':'2019-01-12'}]]

I used the code to merge .json files from here and changed it very slightly like below:

files=['my.json','files.json',...,'name.json']

def merge_JsonFiles(filename):
    result = list()
    for f1 in filename:
        with open(f1, 'r') as infile:
            result.append(json.load(infile))

    with open('counseling3.json', 'w') as output_file:
        json.dump(result, output_file)

merge_JsonFiles(files)

I already read several related questions, but there is no answer I need. Can anyone help me?

Answer 1

You should use extend instead of append. It will add the items of the passed list to result instead of a new list:

files=['my.json','files.json',...,'name.json']

def merge_JsonFiles(filename):
    result = list()
    for f1 in filename:
        with open(f1, 'r') as infile:
            result.extend(json.load(infile))

    with open('counseling3.json', 'w') as output_file:
        json.dump(result, output_file)

merge_JsonFiles(files)

Answer 2

import json
import pandas as pd

with open('example1.json') as f1:               # open the file
    data1 = json.load(f1)

with open('example2.json') as f2:                # open the file       
    data2 = json.load(f2)
    
df1 = pd.DataFrame([data1])                      # Creating DataFrames
df2 = pd.DataFrame([data2])                      # Creating DataFrames

MergeJson = pd.concat([df1, df2], axis=1)         # Concat DataFrames

MergeJson.to_json("MergeJsonDemo.json")          # Writing Json

Answer 3

files=['my.json','files.json',...,'name.json']

with open('merged_file_name.json', "w") as outfile:
   outfile.write('{}'.format('\n'.join([open(f, "r").read() for f in files])))

Answer 4

There is another way.Just loads the json text in those files as a python list, and add them together. Code as below.

# temp1.json
json_a = [{'num':'1', 'item':'smartphone','data':'2019-01-01'},
{'num':'2', 'item':'smartphone','data':'2019-01-02'},
{'num':'3', 'item':'smartphone','data':'2019-01-03'},
{'num':'4', 'item':'smartphone','data':'2019-01-04'}]

# temp2.json
json_b = [{'num':'5', 'item':'smartphone','data':'2019-01-05'},
{'num':'6', 'item':'smartphone','data':'2019-01-06'},
{'num':'7', 'item':'smartphone','data':'2019-01-07'}]

# temp3.json
json_c = [{'num':'8', 'item':'smartphone','data':'2019-01-08'},
{'num':'9', 'item':'smartphone','data':'2019-01-09'},
{'num':'10', 'item':'smartphone','data':'2019-01-10'},
{'num':'11', 'item':'smartphone','data':'2019-01-11'},
{'num':'12', 'item':'smartphone','data':'2019-01-12'}]

print(json_a + json_b + json_c)

Output:

[{'num': '1', 'item': 'smartphone', 'data': '2019-01-01'},
 {'num': '2', 'item': 'smartphone', 'data': '2019-01-02'},
 {'num': '3', 'item': 'smartphone', 'data': '2019-01-03'},
 {'num': '4', 'item': 'smartphone', 'data': '2019-01-04'},
 {'num': '5', 'item': 'smartphone', 'data': '2019-01-05'},
 {'num': '6', 'item': 'smartphone', 'data': '2019-01-06'},
 {'num': '7', 'item': 'smartphone', 'data': '2019-01-07'},
 {'num': '8', 'item': 'smartphone', 'data': '2019-01-08'},
 {'num': '9', 'item': 'smartphone', 'data': '2019-01-09'},
 {'num': '10', 'item': 'smartphone', 'data': '2019-01-10'},
 {'num': '11', 'item': 'smartphone', 'data': '2019-01-11'},
 {'num': '12', 'item': 'smartphone', 'data': '2019-01-12'}]

Answer 5

If the goal is to merge several JSON files, try the following:

def combine_jsons():
    file_list = ['first.json', 'second.json',... ,'last.json']
    all_data_dict = {}
    for json_file in file_list:
       with open(json_file,'r+') as file:
          # First we load existing data into a dict.
       file_data = json.load(file)
       all_data_dict.update(file_data)
    with open('merged_data.json', "w") as outfile:# save to json file
        json.dump(all_data_dict, outfile)

If you it must be a list for some reason just change the last line

json.dump([all_data_dict], outfile)

Based on How do I merge two dictionaries in a single expression (take union of dictionaries)?