Category "pandas"

Flatten list of dictionaries in dataframe

I'm pulling data with Facebook Insights API and there are nested columns in the data I pull. I tried separating them by index but failed. column I want to split

BeautifulSoup getting href of a list with ++ 10k records

BeautifulSoup getting href of a list with ++ records I have the following soup: <a href="some_url">next</a> <span class="class">...</span&g

pandas dataframe function mean() not working correctly to ignore nan values

By default, the mean() method should ignore the nan value, but for my case, it didn't work. It still takes the nan value. a = np.array([1,9]) b = np.array([3,na

Working with a multiindex dataframe, to get summation results over a boolean column, based on a condition from another column

We have a multiindex dataframe that looks like: date condition_1 condition_2 item1 0 2021-06-10 06:30:00+00:00

Install requirements.txt uses a different path

Fellow users, I am trying to install a requirements.txt file (via cmd prompt: "pip install -r "path\to\file"). The problem is that my system returns an OSError

Efficiently upsert thousands of records to mongodb

We have a database containing around ~20 Million records with an indexed field order_id. Every day, after each hour, we receive an incremental update, starting

How to generate an equivalent of pandas.DataFrame.describe in bigquery SQL?

In Pandas, you can get an overview of the table (mean, quantiles, count and count of unique values) for all columns by using the following method: pandas.DataFr

How to combine two dataframes into one like this, using pandas and python?

Please see the picture here. I have two data frames and i need to convert it into single one, using merge or concat method and i am unable to do so. Can our com

How to combine two dataframes into one like this, using pandas and python?

Please see the picture here. I have two data frames and i need to convert it into single one, using merge or concat method and i am unable to do so. Can our com

Splitting a record into 12 months based on the date in pandas dataframe

I have the data in the below format stored in a pandas dataframe PolicyNumber InceptionDate 1 2017-12-28 00:00:00.0 https://i.stack.imgur.com/pE

Merge two dfs with multiple entries of same value in joining column

I have two data frames. The first is input which looks like the following: Merchant SKU Quantity Per Box NOB Shipment Status id_using_regex prepped_by_in

How to convert the dummy variable columns in to several columns?

I know how to unstack rows into columns, but how to deal with the following dataframe? date dummy avg lable 1-19 1 20 l1 1-19 0 40 l1 1-27 1 100 l2 1-27 0 140

How to insert nulls into a SQL Server table

I have the following dataframe: data = [['Alex', 182.2],['Bob', 183.2],['Clarke', 188.4], ['Kelly', NA]] df = pd.DataFrame(data, columns = ['Name', 'Height'])

Convert txt file, with variable categories, to dictionary and pandas df

I've converted a txt file that has a fixed number of variables, for every entry, to a dict and df. For example, if every entry in the txt file has a Date entry

Iterating through rows in a dataframe

I have a dataframe of 12 different teams with their own statistics. My objective is to repeat an entire series of steps for one team, and so on, until the last

Fill in uneven sized lists in Python

I have a 2D-List contains unequal size lengths, like this: lst = [[1,2,3],[-1,2,4],[0,2],[2,-3,6]] I use this code to insert a 0 if element size less 3: newlis

How to import data with dates as index from excel with pandas

I am importing the data with this command df = pd.read_excel('C:/Users/Me/Data.xlsx', sheet_name='Prices') and this is the result: The date is a common column

Unnest json dict to rows in pandas

I have the following dataset from a json file: mydf = pd.DataFrame({ 'load': { 0: {'id': '100','name': 'Joe'}, 1: {'id': '101','name': 'Ann'}, 2: {'id': '1

Modifying overlapping time period to include 1 day difference

I am trying to modify the overlapping time period problem so that if there is 1 day difference between dates, it should still be counted as an overlap. As long

How to locate print output? or convert it into jpeg?

I'm trying to show more than one dataframe with using tkinter. There are 2 options for me, showing dataframe directly by using print() and saving dataframe as j