'Pagination not working in Python Session.put()

I am trying to upload a file to a website (that has an inbuilt API) using the following code. The code reads a list of medical codes/diagnoses codes etc. (1 column in a text file) and uploads it to the required page.

Issue: After uploading the file, I noticed that the number pages is not coming out properly. There can be up to 4000 codes (lines) in the file. The code list page in the website will show 20 lines per page, which means, I would expect at least 200 pages to be there after uploading. This is not happening. I am not sure what is the mistake that I am doing.

Also, I am new to Python (primarily SAS) and have been working on automating bits and pieces of code. One such automation is this exercise. Here, the goal is to upload multiple files to the said URL. Today the team is uploading them one by one manually. With the knowledge I picked up from tutorials and other sources, I was able to come up with this.

import requests
import json
import os
import random
import pandas as pd
import time

token = os.environ.get("USER_TOKEN")
user_id = os.environ.get("USER_ID")
user_name = os.environ.get("USER_NAME")

headers = {"X-API-Key": token}
url = 'https://XXXXXXXXXXXX.com/api/code_lists'

session=requests.session()
cl = session.get(url, headers=headers).json()

def uploading_files(file,name,kind,coding_system,rand_id):
    df = pd.read_table(file, converters={0:str}, header=None)
    print("Came In")
    CODES = df[0].astype('str').tolist()
    codes = {"codes": CODES}
    new_cl = {"_id": rand_id, "name": name, "project_group": "TEST BETA", "kind": kind,
              "coding_system": coding_system, "user": user_id, "creator": user_name, "creation_method": "Upload", "is_category_mapping": False,
              "assoc_users": [], "global": True, "readonly": False, "description": "", "num_codes": len(CODES)}

    request_json = json.dumps(new_cl)
    print(request_json)

    codes_json = json.dumps(codes)
    print(codes_json)

    session.post(url, data=request_json)
    session.put(url + '/' + rand_id, data=codes_json)

text_Files= os.listdir(r'C://Users//XXXXXXXXXXXXX//data')

for i in text_Files:
    if ".txt" in i:
        x=i.split("_")
        file='C://Users//XXXXXXXXXXXXX//data//' + i
        name=""
        for j in i[:-4]:
            if j!="_":
                name+=j
            elif j=="_":
                name+=" "
        kind=x[2]
        coding_system=x[3][:-4]

        rand_id = "".join(random.choice("0123456789abcdef") for i in range(24))

        print("-------------START-----------------")
        print("file : ", file)
        print("name : ", name)
        print("kind : ", kind)
        print("coding system : ", coding_system)
        print("Rand_Id : ", rand_id)
        uploading_files(file, name, kind, coding_system, rand_id)
        time.sleep(2)
        print("---------------END---------------")
        print("")
        break ''' to upload only 1 file in the directory'''

Example data in the file (testfile.txt)

C8900 C8901 C8902 C8903 C8904 C8905 C8906 C8907 C8908 C8909 C8910 C8911 C8912 C8913 C8914 C8918 C8919 C8920 C8921 C8922 C8923 C8924 C8925 C8926 C8927 C8928 C8929 C8930 C8931 C8932 C8933 C8934 C8935 C8936 C9723 C9744 C9762 C9763 C9803 D0260

Sample Data Snapshot

Wrong Representation

Expected



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source