pandas to_dict with python native datetime type and not timestamp

Question

I have a pandas DataFrame df that contains Timesatamp columns.

I wish to create an iterator of rows (either via the iter.. methods or via to_dict) from df where the Timesatamp values are python datetime.

I have tried doing this

for col in df.select_dtypes(['datetime']):
        df[col] = df[col].dt.to_pydatetime()

however it seems like the columns is still Timesatamp when using the above mentioned iterator methods. Is there a 'batch'y way to achieve this apart from manualy converting each values when its iterated upon?

---

example

df = pd.DataFrame({'d': pd.date_range('2018-01-01', freq='12h', periods=2), 'a':[1,2]})
for col in df.select_dtypes(['datetime']):
    df[col] = df[col].dt.to_pydatetime()
print(df.to_dict('records'))

the output:

[{'d': Timestamp('2018-01-01 00:00:00'), 'a': 1}, {'d': Timestamp('2018-01-01 12:00:00'), 'a': 2}]

the desired output:

[{'d': datetime.datetime(2018, 1, 1, 0, 0), 'a': 1}, {'d': datetime.datetime(2018, 1, 1, 12, 0), 'a': 2}]

Answer 1

You can try

df[col] = pd.Series(df[col].dt.to_pydatetime(), dtype = object)

instead of

df[col] = df[col].dt.to_pydatetime()

Answer 2

Try it:

df["d"]=df.d.apply(lambda t: t.date())                                                                              
df.d.to_dict()                                                                                                      

{0: datetime.date(2018, 1, 1), 1: datetime.date(2018, 1, 2)}

Answer 3

somewhat relevant but if you want to dump data into a json, convert the dataframe into a json

json_df = df.to_json(orient='records')

then output the data into a new json file

with open('out.json', 'w') as outfile:
    json.dump(json.loads(json_df), outfile)