How to fix Python error "...failed to map segment from shared object" appearing when I try to import NumPy library on GCP?

Question

I've recently started to use Google Cloud Platform and I run my python scripts in Cloud Shell within Linux environment.

By running one of the scripts that is using pandas library I ran into very unpleasant error which is:

    Traceback (most recent call last):
  File "pandas_excercises.py", line 1, in <module>
    import pandas as pd
  File "/home/dann_frol/.local/lib64/python3.6/site-packages/pandas/__init__.py", line 17, in <module>
    "Unable to import required dependencies:\n" + "\n".join(missing_dependencies)
ImportError: Unable to import required dependencies:
numpy:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
was
installed.
We have compiled some common reasons and troubleshooting tips at:

    https://numpy.org/devdocs/user/troubleshooting-importerror.html
Please note and check the following:
  * The Python version is: Python3.6 from "/usr/bin/python3"
  * The NumPy version is: "1.19.0"
and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.
Original error was: /home/dann_frol/.local/lib64/python3.6/site-packages/numpy/core/_multiarray_umath.cpython-36m-x86_64-linux-gnu.so: failed to map segment from shared object

I've checked both python and numpy versions and they seem to match to the ones described in the error message. I've tried to reinstall both pandas and numpy using python3 -m pip uninstall pandas and python3 -m pip install pandas , however it didn't help.

When I run the code, I use the following command: python3 pandas_excercises.py

Some info about the OS:

  Operating System: Container-Optimized OS from Google
            Kernel: Linux 4.19.114+
      Architecture: x86-64

What could be the possible issue and how to fix this error?

Thank you, any help is appreciated.

Answer 1

I'm pretty sure that in case of Python libraries' natives, the message failed to map segment from shared object means that the .so file isn't marked executable. (ls -l says e.g. -rw-r--r-- instead of -rwxr-xr-x.)

The only way (that I can think of) how it can be so after a sane pip install is if filesystem is mounted with noexec flag. (Which was the case on my setup.) You can quickly check for it using mount command, it will be listed among other flags inside braces if it is set.

How to remove that noexec flag is another question.

Answer 2

Container-Optimized OS (COS) has several limitations that could be a reason that your program does not work.

COS is an operating system optimized for running Docker containers. But it lacks many of components you have in a typical Linux distribution. This distribution components and especially a kernel are quite different from the generic Linux installation.

Container-Optimized OS Overview => Limitations:

Container-Optimized OS has limited or no support for some of the common features you may be familiar with on other operating systems:

• Container-Optimized OS does not include a package manager; as such, you'll be unable to install software packages directly on an instance.
• Container-Optimized OS does not support execution of non-containerized applications.
• The Container-Optimized OS kernel is locked down; you'll be unable to install third-party kernel modules or drivers.

Container-Optimized OS > OS Security > Immutable root filesystem and verified boot:

The Container-Optimized OS root filesystem is always mounted as read-only. Additionally, its checksum is computed at build time and verified by the kernel on each boot. Additionally, several other mounts are non-executable by default. See Filesystem for details.

Additional information: Container-Optimized OS Overview