Debugging Numpy VisibleDeprecationWarning (ndarray from ragged nested sequences)

Question

Since NumPy version 19.0, one must specify dtype=object when creating an array from "ragged" sequences. I'm faced with a large number of array calls from my own code and Pandas using threading, line-by-line debugging led me nowhere.

I'd like to figure out which call resulted in VisibleDeprecationWarning in my own code or a call from Pandas. How would I be able to debug this? I've been looking through the source and I cannot see this warning getting called in Python (only in numpy.core._multiarray_umath.cp38-win_amd64.pyd).

Answer 1

With a function that creates a ragged array:

In [60]: def foo(): 
    ...:     print('one') 
    ...:     x = np.array([[1],[1,2]]) 
    ...:     return x 
    ...:                                                                                             
In [61]: foo()                                                                                       
one
/usr/local/bin/ipython3:3: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray
  # -*- coding: utf-8 -*-
Out[61]: array([list([1]), list([1, 2])], dtype=object)

I get the warning, but also the expected result.

I can control the warnings.

For example to turn if off:

In [68]: np.warnings.filterwarnings('ignore', category=np.VisibleDeprecationWarning)                 
In [69]: foo()                                                                                       
one
Out[69]: array([list([1]), list([1, 2])], dtype=object)

Or to raise an error:

In [70]: np.warnings.filterwarnings('error', category=np.VisibleDeprecationWarning)                  
In [71]: foo()                                                                                       
one
---------------------------------------------------------------------------
VisibleDeprecationWarning                 Traceback (most recent call last)
<ipython-input-71-c19b6d9633cf> in <module>
----> 1 foo()

<ipython-input-60-6ad21d9e07b4> in foo()
      1 def foo():
      2     print('one')
----> 3     x = np.array([[1],[1,2]])
      4     return x
      5 

VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray

The error gives a traceback telling me where the warning was raised.

There may be ways of refining the warning filter to catch just this one, and not others of the same category. I haven't used this mechanism much.

Read np.warnings.filterwarnings docs for more details.

Answer 2

b2 = np.array(
    [[1, 2, 3, 4, 5, 6, 7, 8, 9, 10], [11, 12, 13, 14, 15, 16, 17, 18]],
    dtype=object,
)

Reference to the above example will clear the warning. You must specify dtype=object.

Answer 3

This warning is caused by deprecated API of NumPy version 1.19 or higher, you may continue using it and just suppress the warning:

import warnings
warnings.filterwarnings("ignore", category=np.VisibleDeprecationWarning) 

Answer 4

I faced the np.VisibleDeprecationWarning while stacking lists containing audio data from WAV files. This problem occurred because audio files had different lengths. So, the lists that I needed to stack together into one numpy array also had varying lengths.

Ignoring or suppressing this warning did not give the desired stacked np array so, I made all the audio files of the same length by using pydub.AudioSegment as mentioned in this answer.

This resolved the warning.

Answer 5

You can add dtype=object when you create your numpy array as :

numpy.array([[1,2,3],[4,5,6]], dtype=object)

or if you change a list or a tuple called 'a' to a numpy array code as:

numpy.asarray(a,dtype=object)

This helps you to avoid the warning.