Category "dataframe"

Pandas: how can I generate "year-month" format column (period)?

In [20]: df.head() Out[20]: year month capital sales income profit debt 0 2000 6 -19250379.0 37924704.0 -4348337.0 25

How to parse this JSON which starts with two square brackets?

I have a JSON File that starts with two square brackets. How do i parse the data from it? The type of the JSON is class 'list'. I have gone though many Stackove

Inplace Forward Fill on a multi-level column dataframe

I have the following dataframe: arrays = [['bar', 'bar', 'baz', 'baz', 'foo', 'foo', 'qux', 'qux'], ['one', 'two', 'one', 'two', 'one', 'two', 'one', 'two']]

Creating an empty Pandas DataFrame, then filling it?

I'm starting from the pandas DataFrame docs here: http://pandas.pydata.org/pandas-docs/stable/dsintro.html I'd like to iteratively fill the DataFrame with valu

Count groups of consecutive 1s in pandas

I have a list of '1's and '0s' and I would like to calculate the number of groups of consecutive '1's. mylist = [0,0,1,1,0,1,1,1,1,0,1,0] Doing it by hand g

How to apply a function to two columns of Pandas dataframe

Suppose I have a df which has columns of 'ID', 'col_1', 'col_2'. And I define a function : f = lambda x, y : my_function_expression. Now I want to apply the f

pandas row values to column headers

I have a daraframe like this df = pd.DataFrame({'id1':[1,1,1,1,2,2,2],'id2':[1,1,1,1,2,2,2],'value':['a','b','c','d','a','b','c']}) id1 id2 value 0 1

Python Pandas replace NaN in one column with value from corresponding row of second column

I am working with this Pandas DataFrame in Python. File heat Farheit Temp_Rating 1 YesQ 75 N/A 1 NoR 115 N/A

joblib.Memory and pandas.DataFrame inputs

I've been finding that joblib.Memory.cache results in unreliable caching when using dataframes as inputs to the decorated functions. Playing around, I found tha

joblib.Memory and pandas.DataFrame inputs

I've been finding that joblib.Memory.cache results in unreliable caching when using dataframes as inputs to the decorated functions. Playing around, I found tha

Pandas split dataframe column for every character

i have multiple dataframe columns which look like this: Day1 0 DDDDDDDDDDBBBBBBAAAAAAAAAABBBBBBDDDDDDDDDDDDDDDD 1 DDDDDDDDDDBBBB

Get element in each cluster

I've got this following code which extract 2 feature(tempo & slotID) from csv file and plot kmeans clustering based on this 2 features. df = pd.read_csv("pr

Python pandas DataFrame from first and last row of csv

All - I am looking to create a pandas DataFrame from only the first and last lines of a very large csv. The purpose of this exercise is to be able to easily g

Pandas: sum DataFrame rows for given columns

I have the following DataFrame: In [1]: df = pd.DataFrame({'a': [1, 2, 3], 'b': [2, 3, 4], 'c': ['dd', 'ee', 'ff'],

Percentage difference between any two columns of pandas dataframe

I would like to have a function defined for percentage diff calculation between any two pandas columns. Lets say that my dataframe is defined by: R1 R2 R3

pyspark get element from array Column of struct based on condition

I have a spark df with the following schema: |-- col1 : string |-- col2 : string |-- customer: struct | |-- smt: string | |-- attributes: array (null

pandas combine two columns with null values

I have a df with two columns and I want to combine both columns ignoring the NaN values. The catch is that sometimes both columns have NaN values in which case

Set MultiIndex of an existing DataFrame in pandas

I have a DataFrame that looks like Emp1 Empl2 date Company 0 0 0 2012-05-01 apple 1 0 1 2012-05-29

check if pair of values is in pair of columns in pandas

Basically, I have latitude and longitude (on a grid) in two different columns. I am getting fed two-element lists (could be numpy arrays) of a new coordinate se

How to remove the border of Pandas dataframe?

When I use pandas dataframe to excel, the border of the header will be generated automatically. When I use styleframe to excel, the border of the whole table wi