'Append new data into an existing frame and upload to sheets Python

I'm connected to my APIs client, sent the credentials, I made the request, I asked the API for data and put it to a DF.

Then, I have to upload this data to a sheet, so then this sheet is gonna be connected to PowerBI as a datasource in order to develop a dashboard and monitor some KPIs and so on..

Simple and common ETL process. BUT: to be honest, I'm a rookie and I'm doing my best.

Above here is just code to connect to the API, here is where the "extraction" begins

if response_page.status_code == 200:
    if page == 1 :
        df = pd.DataFrame(json.loads(response_page.content)["list"])
    else :
        df2 = pd.DataFrame(json.loads(response_page.content)["list"])
        
        df = df.append(df2)

Then I just pick up but I need:

columnas = ['orderId','totalValue','paymentNames']
df2 = df[columnas]
df2

This is what the DF looks like:

example: this df is which I need to append the new data

Then I've just connected to Sheets here, send the credentials, open the sheet("carrefourMetodosDePago") and the page("transacciones")

sa = gspread.service_account(filename="service_account.json")
sh = sa.open("carrefourMetodosDePago")
wks = sh.worksheet("transacciones")

The magic begins:

wks.update([df2.columns.values.tolist()] + df2.values.tolist())

With this sentence I upload what the picture shows, to the sheet!

I need the new data that generates the API to be appended/merged/concatenated to the current data so the code upload the current data PLUS the new everytime I run it and so forth.

How can I do that? Should I use a for loop and iterate over every new data en append it to the sheet?

This is the best I could have done, I think I reached my turning point here...

If I explained myself wrong just let me know. If you reach up to here just let me thank you to give me some time :)



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source