Pandas wide to long bringing empty DataFrame

Question

I was working in a pretty simple task: applying wide_to_long to a DataFrame, but every time I ran it, I got an empty DataFrame. I was almost sure I was doing it the right way, so I went to the documentation and tried to apply the example shown there, and it also brought an empty DataFrame! This is the sample code:

import pandas as pd

df = pd.DataFrame({
    'famid': [1, 1, 1, 2, 2, 2, 3, 3, 3],
    'birth': [1, 2, 3, 1, 2, 3, 1, 2, 3],
    'ht_one': [2.8, 2.9, 2.2, 2, 1.8, 1.9, 2.2, 2.3, 2.1],
    'ht_two': [3.4, 3.8, 2.9, 3.2, 2.8, 2.4, 3.3, 3.4, 2.9]
})

l = pd.wide_to_long(df, stubnames='ht', i=['famid', 'birth'], j='age', sep='_', suffix='\\w')
l.shape

Output:

Out[2]: (0, 3)

I tried updating Pandas, but it didn't help. What could be happening?

Answer 1

That dataframe is the same used in Pandas' official documentation on pandas.wide_to_long.

For this specific case, that have non-integers as suffixes, use suffix=r'\w+',

l = pd.wide_to_long(df, stubnames='ht', i=['famid', 'birth'], j='age',
                    sep='_', suffix=r'\w+')

And if one prints the shape of l, one gets the following

print(l.shape)
[Out]: (18, 1)

Or, as @BENY suggests, suffix='\\w+'

l = pd.wide_to_long(df, stubnames='ht', i=['famid', 'birth'], j='age',
                    sep='_', suffix='\\w+')

And if one prints the shape of l one gets the following

print(l.shape)
[Out]: (18, 1)