'Installing scipy and scikit-learn on apple m1

The installation on the m1 chip for the following packages: Numpy 1.21.1, pandas 1.3.0, torch 1.9.0 and a few other ones works fine for me. They also seem to work properly while testing them. However when I try to install scipy or scikit-learn via pip this error appears:

ERROR: Failed building wheel for numpy

Failed to build numpy

ERROR: Could not build wheels for numpy which use PEP 517 and cannot be installed directly

Why should Numpy be build again when I have the latest version from pip already installed?

Every previous installation was done using python3.9 -m pip install ... on Mac OS 11.3.1 with the apple m1 chip.

Maybe somebody knows how to deal with this error or if its just a matter of time.

Solution 1:^[1]

UPDATE: `scikit-learn` now works via pip ?

Just first brew install openblas - it has instructions for different processors (wikipedia)

brew install openblas
export OPENBLAS=$(/opt/homebrew/bin/brew --prefix openblas)
export CFLAGS="-falign-functions=8 ${CFLAGS}"
# ^ no need to add to .zshrc, just doing this once.
pip install scikit-learn # ==0.24.1 if you want

Worked great on M1 Pro ?

Extra details

Pip downloaded the source from Pipy, then built the wheel targeting MacOS X 12.0, and arm64 (apple silicon): scikit_learn-1.0.1-cp38-cp38-macosx_12_0_arm64.whl.

Building wheels for collected packages: scikit-learn
  Building wheel for scikit-learn (pyproject.toml) ... done
  Created wheel for scikit-learn: filename=scikit_learn-1.0.1-cp38-cp38-macosx_12_0_arm64.whl size=6364030 sha256=0b0cc9a21af775e0c8077ee71698ff62da05ab62efc914c5c15cd4bf97867b31
Successfully built scikit-learn
Installing collected packages: scipy, scikit-learn
Successfully installed scikit-learn-1.0.1 scipy-1.7.3

Note on Pipy: we usually download either a pre-built wheel (yay, this is excellent for reliable distribution and ensuring compatability). Or, if no prebuilt wheel exists (sad) then we download a tar.gz and build it ourselves. This happens because the authors don't publish a prebuilt wheel to Pipy, but more and more people are adding this to their CI (github actions) workflow. Building the wheel ourselves takes more cpu time, and is generally less reliable but works in this case.

Here we are downloading a pre-built wheel that has very few limitations: it works for any version of python 3, for any os, for any architecture (like amd64 or arm64): click-8.0.3-py3-none-any.whl

Collecting click>=7.0
  Downloading click-8.0.3-py3-none-any.whl

Here apparently we had no wheel available, so we have to build it ourselves with setuptools running setup.py.

Collecting grpcio>=1.28.1
  Downloading grpcio-1.42.0.tar.gz (21.3 MB)
     |????????????????????????????????| 21.3 MB 12.7 MB/s
  Preparing metadata (setup.py) ... done

## later in the process it installs using setuptools 
Running setup.py install for grpcio ... done

Good luck and happy piping.

Solution 2:^[2]

Finally managed to sort this using below steps:

install python 3.9 using brew? (brew install [email protected])
setup PATH to pick brew python3 first (export PATH=opt/homebrew/bin:$PATH)
Run below commands to install scipy and scikit-learn



    >> /opt/homebrew/bin/brew install openblas
    >> export OPENBLAS=$(/opt/homebrew/bin/brew --prefix openblas)
    >> export CFLAGS="-falign-functions=8 ${CFLAGS}"
    >> git clone https://github.com/scipy/scipy.git
    >> cd scipy
    >> git submodule update --init
    >> /opt/homebrew/bin/pip3 install .
    >> /opt/homebrew/bin/pip3 install scikit-learn

go to the folder where virtual env needs to be created and run python3 -m venv --system-site-packages .venv

Solution 3:^[3]

Please see this note of scikit-learn about

Installing on Apple Silicon M1 hardware

The recently introduced macos/arm64 platform (sometimes also known as macos/aarch64) requires the open source community to upgrade the build configuation and automation to properly support it.

At the time of writing (January 2021), the only way to get a working installation of scikit-learn on this hardware is to install scikit-learn and its dependencies from the conda-forge distribution, for instance using the miniforge installers:

https://github.com/conda-forge/miniforge

The following issue tracks progress on making it possible to install scikit-learn from PyPI with pip:

https://github.com/scikit-learn/scikit-learn/issues/19137

Solution 4:^[4]

I do not know why numpy would be built again (maybe scipy has other version requirements?), but I can report on a way how to install scipy and scikit-learn. I just had success using the following sequence of commands, base on Python 3.8.11 (especially scipy compilation took some time, so this process is not a quick one, unfortunately):

# SciPy:
python -m pip install --no-cache --no-use-pep517 pythran cython pybind11 gast"==0.4.0"
pyenv rehash
python -m pip install --no-cache --no-binary :all: --no-use-pep517 scipy"==1.7.1"

# Scikit-Learn
python -m pip install --no-use-pep517 scikit-learn"==0.24.2"

So the information on the scikit-learn homepage about the conda installation path being necessary is outdated, it seems.

There were the Apple developer tools (XCode, xcode-select) and some other packages installed via homebrew, and in a pyenv virtualenv environment before, which might also be relevant, so I add them here below:

brew install openblas [email protected] pkg-config pyenv pyenv-virtualenv
python -m pip install numpy==1.19.5

I needed the older numpy version because there is also tensorflow installed in this environment which is not compatible with newer numpy versions; I do not know whether your newer numpy could contribute to the problems you encountered.

It appears that there is a lot currently changing with respect to Apple M1 support, so this might become still easier soon.

Solution 5:^[5]

As of March 14 (pie day!) 2022, all I had to do was type the following in a conda environment:

conda install --channel=conda-forge scikit-learn

That was it. I am running a MacBook Pro (13", M1, 2020) with MacOS 11.6, and python 3.8 in the conda environment.

Solution 6:^[6]

I guess Scipy and Numpy is not directly supported on arm64 processor (MAC M1 OS BigSur and higher versions) which requires the users to manually install and build the dependencies. I figured a work around by creating a virtual environment with Python version 3.8. Once the final releases are rolled out, this changes can be removed easily without any conflict.

Setup a python virtual environment with Python version 3.8.

python3.8 -m venv <environment_name>

Activate the environment

source <environment_name>/bin/activate

Install numpy and scipy

pip3 install numpy
pip3 install scipy

Solution 7:^[7]

I'm chiming in here since I'm installing all of this using a Docker container. Host machine is M1 MBP running Monterey 12.0.1 - I found the following solution in this thread on GitHub. It seems the scipy team will have an update soon.

Make sure you are on Python 3.9 and using HomeBrew, run brew install openblas. Once that is done you can add the following lines to your Dockerfile:

RUN OPENBLAS="/opt/homebrew/opt/openblas" CFLAGS="-falign-functions=8 ${CFLAGS}" pip3 install scipy
RUN OPENBLAS="/opt/homebrew/opt/openblas" CFLAGS="-falign-functions=8 ${CFLAGS}" pip3 install scikit-learn

Solution 8:^[8]

The best way to do this is using Miniforge . Confirmed workiing for Macbook M1 and OS 12(monterey)

On macbook M1, you can install miniforge using the command brew install miniforge.
Once this is installed, you can create a virtual environment using the command conda create -n .venv python
activate the environment using the command conda activate .venv
Install scikit-learn to the environment using conda install scikit-learn

The official issue tracker related to the scikit-learn installation is in their github issues . Some of the solutions suggested there is working for Mac OS 11, but not for Mac OS 12.
I finally found the working solution here

Solution 9:^[9]

Apple M1 Pro, macOS 12.1

Python 3.9.10 (main, Jan 15 2022, 11:40:53) 
[Clang 13.0.0 (clang-1300.0.29.3)] on darwin
pip 22.0.3

install numpy and scipy

brew install openblas gfortran
OPENBLAS="$(brew --prefix openblas)" pip install numpy==1.19.3
OPENBLAS="$(brew --prefix openblas)" pip install scipy==1.7.2

install scikit-learn (0.21.3)

pip install cython
brew install libomp
export CC=/usr/bin/clang
export CXX=/usr/bin/clang++
export CPPFLAGS="$CPPFLAGS -Xpreprocessor -fopenmp"
export CFLAGS="$CFLAGS -I/opt/homebrew/Cellar/libomp/13.0.1/include"
export CXXFLAGS="$CXXFLAGS -I/opt/homebrew/Cellar/libomp/13.0.1/include"
export LDFLAGS="$LDFLAGS -L/opt/homebrew/Cellar/libomp/13.0.1/lib -lomp"
export DYLD_LIBRARY_PATH=/opt/homebrew/Cellar/libomp/13.0.1/lib
pip install scikit-learn==0.21.3

Solution 10:^[10]

I solved this by using poetry and setting python restricted dependencies scipy = { version = "^1.8.0", python = "~3.9" } in pyproject.toml.

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution	Source
Solution 1
Solution 2	FossilBlade
Solution 3	afsharov
Solution 4	Numenor
Solution 5
Solution 6	Dharman
Solution 7	jfunk
Solution 8	emilpmp
Solution 9	menrfa
Solution 10	olavis

'Installing scipy and scikit-learn on apple m1

Solution 1:[1]

UPDATE: scikit-learn now works via pip ?

Extra details

Solution 2:[2]

Solution 3:[3]

Solution 4:[4]

Solution 5:[5]

Solution 6:[6]

Solution 7:[7]

Solution 8:[8]

Solution 9:[9]

Solution 10:[10]