How to append row from itertuples to dataframe withouth losing the index in Python?

Question

I have the following problem:

I have a DataFrame df which looks like this:

       eqpid  recpid   queuetime           trackintime         trackouttime
3723   A      XYZ      2017-01-01 03:14:58 2017-01-04 03:43:28 2017-01-04 03:43:33
...    ...    ...      ...                 ...                 ...

I am iterating through this DataFrame with itertuples() (I checked vectorization & .apply(), does not work here unfortunately). Now, beside other operations: I want to append the row (which is a namedtuple in my understanding) to another DataFrame with the same columns and keep the initial index, so it looks like this:

       eqpid  recpid   queuetime           trackintime         trackouttime
...    ...    ...      ...                 ...                 ...
3723   A      XYZ      2017-01-01 03:14:58 2017-01-04 03:43:28 2017-01-04 03:43:33

In theory, the code should look something like this:

temp_df = pd.DataFrame(columns=df.columns)
for row in df.itertuples():
    ...
    temp_df.append(row)

But this does not work, the temp_df remains empty. I also tried something like this:

temp_df = pd.DataFrame(columns=df.columns)
for row in df.itertuples():
    ...
    temp = pd.DataFrame([row],columns = row._fields)
    temp.set_index('Index', inplace = True)
    temp_df.append(temp)

But even though when I print temp it looks like:

Index  eqpid  recpid   queuetime           trackintime         trackouttime
3723   A      XYZ      2017-01-01 03:14:58 2017-01-04 03:43:28 2017-01-04 03:43:33

The temp_df remains empty. Can someone give me a hint what to do or what my mistake is?

Thank you all in advance!

Answer 1

Try to use 'iterrows' which returns rows as a tuple of index,series:

for idx,ser in df.iterrows():
    ...
    temp_df = temp_df.append(ser)

The series itself contains the index, too, so the index alignment works.

Answer 2

Assign the row to the dataframe

temp_df = pd.DataFrame()
for row in df.itertuples(index=False):
    temp_df = temp_df.append(pd.DataFrame([row],columns = row._fields), ignore_index=True)

Specify index=False to ignore the index, ignore_index=True to reset the index counter