Trying to concatenate dataframes with numpy and recursion

Question

I have a list of pandas dataframes, which I am trying to concatenate into one dataframe using recursion and numpy.

def recur(framelist, index=0, result=0):


    if index == len(framelist)-1:
        return result

    else:
        return recur(np.concatenate((framelist[index],framelist[index+1])))

My intention with the above is to pass the dataframe list to the recur function. The base case is when the end of the list is reached. the functionality is to concat all pairs of dataframes

However I get an error that 0 dimensional arrays cannot be concatenated

Answer 1

To work out what's going on it's a good idea to walk through it step by step.

You say your initial call to recur passes in a list of panda dataframes. You don't show the creation of them, but let's say they're something like...

framelist = [
    pd.DataFrame(np.array([1, 2, 3])),
    pd.DataFrame(np.array([4, 5])),
    pd.DataFrame(np.array([6, 7])),
    pd.DataFrame(np.array([8]))
    ]

So, first time through it concatenates the first two entries from framelist as numpy arrays.

[[1], [2],  [3]]  and [[4], [5]]

This will result in a numpy ndarray which looks like:

[[1], [2], [3], [4], [5]]

This result is passed into recur() as the new framelist

Second time through it concatenates the first two entries from framelist.

[1] and [2]

This will result in a numpy array which looks like:

[1, 2]

This result is passed into recur() as the new framelist

Third time through it concatenates the first two entries from framelist.

1 and 2

These are simply numbers, not arrays, so you see the error '0 dimensional arrays cannot be concatenated'

Here's an example of how to do the concatenation with recursion. You don't need to keep track using any kind of index parameter. Just keep taking the first off the list and pass the remainder into recur. When you get to the point where there's only 1 left in the list, that gets passed back up and concatenated with the previous one. The result is passed back up and concatenated with the previous one, etc.

def recur(framelist):
    # keep going until there's just 1 left.
    if len(framelist) == 1:
        return framelist[0]

    return np.concatenate((framelist[0], recur(framelist[1:])))

print(recur(framelist))