When using df_hdr_join.count() > 0 in when statement, it gives an error 'condition should be a Column'. I tried following. df_result = df.withColumn('NUM', w
How to transform a list of dictionary into a table. Here is the table: [{'wow': 1, 'item': 1, 'money': 1}, {'best': 1, 'sock': 1, 'saved': 1, 'found'
We use Synapse Notebooks to perform data transformations and load the data into fact and dimension tables within our ADLSG2 data lake. We are disappointed with
I'm doing a simple group by operation, trying to compare group means. As you can see below, I have selected specific columns from a larger dataframe, from which
I used .write_ipc from Polars to store as a feather file. It turns out that the numerical strings have been saved as integers. So I need to convert the columns
I am working with pandas and I was wondering if there is a difference based on which statistical functions are applied as shown in the below examples and if the
I use the following script to measure the average RGB color of the picture in a selected path. I tried to make 1 dataframe with pd.concat but it doesn't work ou
I have a dataset that looks like this: main_id time_stamp aaa 2019-05-29 08:16:05+05
This is my dataframe: ID number Date purchase 1 2022-05-01 1 2021-03-03 1 2020-01-03 2 2019-01-03 2 2018-01-03 I want to get a horizontal dataframe with alle
working on NLP problem I ended up with a big features dataset dfMethod Out[2]: c0000167 c0000294 c0000545 ... c4721555 c4759703 c4759772 0
I'm pulling data with Facebook Insights API and there are nested columns in the data I pull. I tried separating them by index but failed. column I want to split
We have a multiindex dataframe that looks like: date condition_1 condition_2 item1 0 2021-06-10 06:30:00+00:00
I have various columns in Spark DataFrame, they are nested json columns. In configuration i will provide a list of columns and fields to remove from json. For e
Please see the picture here. I have two data frames and i need to convert it into single one, using merge or concat method and i am unable to do so. Can our com
Please see the picture here. I have two data frames and i need to convert it into single one, using merge or concat method and i am unable to do so. Can our com
I have the data in the below format stored in a pandas dataframe PolicyNumber InceptionDate 1 2017-12-28 00:00:00.0 https://i.stack.imgur.com/pE
I have two data frames. The first is input which looks like the following: Merchant SKU Quantity Per Box NOB Shipment Status id_using_regex prepped_by_in
I know how to unstack rows into columns, but how to deal with the following dataframe? date dummy avg lable 1-19 1 20 l1 1-19 0 40 l1 1-27 1 100 l2 1-27 0 140
i created a data frame using polars. when datas are inserted, dtype of the coulmn automatically changes to what inserted. (i think its a feature of polars?) but
I am importing the data with this command df = pd.read_excel('C:/Users/Me/Data.xlsx', sheet_name='Prices') and this is the result: The date is a common column