'pipenv locking for NumPy fails on Python3.9 and pip21.0.1
When I try to install pandas via pipenv I get a bunch of numpy errors after the locking process. When I try to install bs4 via pipenv it works fine, but for some reason, it fails with pandas. I have tried deleting and reinstalling pipenv and removing the environment, but nothing has worked.
I am using Python 3.9.1 and pip 21.0.1.
I am on macOS Big Sur.
Here is the code snippet of the errors I get. The errors occur during the locking process (writing to pipenv.lock):
Copying numpy.egg-info to build/bdist.macosx-11-x86_64/wheel/numpy-1.20.0-py3.9.egg-info
running install_scripts
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/pipenv/patched/notpip/_vendor/pep517/_in_process.py", line 257, in <module>
main()
File "/usr/local/lib/python3.9/site-packages/pipenv/patched/notpip/_vendor/pep517/_in_process.py", line 240, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/usr/local/lib/python3.9/site-packages/pipenv/patched/notpip/_vendor/pep517/_in_process.py", line 181, in build_wheel
return _build_backend().build_wheel(wheel_directory, config_settings,
File "/private/var/folders/d9/737381n90cv4_jc2pnx1vc5w0000gn/T/pip-build-env-9iqn62cr/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 211, in build_wheel
return self._build_with_temp_dir(['bdist_wheel'], '.whl',
File "/private/var/folders/d9/737381n90cv4_jc2pnx1vc5w0000gn/T/pip-build-env-9iqn62cr/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 197, in _build_with_temp_dir
self.run_setup()
File "/private/var/folders/d9/737381n90cv4_jc2pnx1vc5w0000gn/T/pip-build-env-9iqn62cr/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 248, in run_setup
super(_BuildMetaLegacyBackend,
File "/private/var/folders/d9/737381n90cv4_jc2pnx1vc5w0000gn/T/pip-build-env-9iqn62cr/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 142, in run_setup
exec(compile(code, __file__, 'exec'), locals())
File "setup.py", line 513, in <module>
setup_package()
File "setup.py", line 505, in setup_package
setup(**metadata)
File "/private/var/folders/d9/737381n90cv4_jc2pnx1vc5w0000gn/T/pip-install-o5k4mrv9/numpy/numpy/distutils/core.py", line 169, in setup
return old_setup(**new_attr)
File "/private/var/folders/d9/737381n90cv4_jc2pnx1vc5w0000gn/T/pip-build-env-9iqn62cr/overlay/lib/python3.9/site-packages/setuptools/__init__.py", line 165, in setup
return distutils.core.setup(**attrs)
File "/usr/local/Cellar/[email protected]/3.9.1_6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/local/Cellar/[email protected]/3.9.1_6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/usr/local/Cellar/[email protected]/3.9.1_6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/private/var/folders/d9/737381n90cv4_jc2pnx1vc5w0000gn/T/pip-build-env-9iqn62cr/overlay/lib/python3.9/site-packages/wheel/bdist_wheel.py", line 328, in run
impl_tag, abi_tag, plat_tag = self.get_tag()
File "/private/var/folders/d9/737381n90cv4_jc2pnx1vc5w0000gn/T/pip-build-env-9iqn62cr/overlay/lib/python3.9/site-packages/wheel/bdist_wheel.py", line 252, in get_tag
plat_name = get_platform(self.bdist_dir)
File "/private/var/folders/d9/737381n90cv4_jc2pnx1vc5w0000gn/T/pip-build-env-9iqn62cr/overlay/lib/python3.9/site-packages/wheel/bdist_wheel.py", line 48, in get_platform
result = calculate_macosx_platform_tag(archive_root, result)
File "/private/var/folders/d9/737381n90cv4_jc2pnx1vc5w0000gn/T/pip-build-env-9iqn62cr/overlay/lib/python3.9/site-packages/wheel/macosx_libfile.py", line 356, in calculate_macosx_platform_tag
assert len(base_version) == 2
AssertionError
########### EXT COMPILER OPTIMIZATION ###########
Platform :
Architecture: x64
Compiler : clang
CPU baseline :
Requested : 'min'
Enabled : SSE SSE2 SSE3
Flags : -msse -msse2 -msse3
Extra checks: none
CPU dispatch :
Requested : 'max -xop -fma4'
Enabled : SSSE3 SSE41 POPCNT SSE42 AVX F16C FMA3 AVX2 AVX512F AVX512CD AVX512_KNL AVX512_SKX AVX512_CLX AVX512_CNL AVX512_ICL
Generated :
:
SSE41 : SSE SSE2 SSE3 SSSE3
Flags : -msse -msse2 -msse3 -mssse3 -msse4.1
Extra checks: none
Detect : SSE SSE2 SSE3 SSSE3 SSE41
: numpy/core/src/umath/_umath_tests.dispatch.c
:
SSE42 : SSE SSE2 SSE3 SSSE3 SSE41 POPCNT
Flags : -msse -msse2 -msse3 -mssse3 -msse4.1 -mpopcnt -msse4.2
Extra checks: none
Detect : SSE SSE2 SSE3 SSSE3 SSE41 POPCNT SSE42
: build/src.macosx-11-x86_64-3.9/numpy/core/src/_simd/_simd.dispatch.c
:
AVX2 : SSE SSE2 SSE3 SSSE3 SSE41 POPCNT SSE42 AVX F16C
Flags : -msse -msse2 -msse3 -mssse3 -msse4.1 -mpopcnt -msse4.2 -mavx -mf16c -mavx2
Extra checks: none
Detect : AVX F16C AVX2
: numpy/core/src/umath/_umath_tests.dispatch.c
:
(FMA3 AVX2) : SSE SSE2 SSE3 SSSE3 SSE41 POPCNT SSE42 AVX F16C
Flags : -msse -msse2 -msse3 -mssse3 -msse4.1 -mpopcnt -msse4.2 -mavx -mf16c -mfma -mavx2
Extra checks: none
Detect : AVX F16C FMA3 AVX2
: build/src.macosx-11-x86_64-3.9/numpy/core/src/_simd/_simd.dispatch.c
:
AVX512F : SSE SSE2 SSE3 SSSE3 SSE41 POPCNT SSE42 AVX F16C FMA3 AVX2
Flags : -msse -msse2 -msse3 -mssse3 -msse4.1 -mpopcnt -msse4.2 -mavx -mf16c -mfma -mavx2 -mavx512f
Extra checks: AVX512F_REDUCE
Detect : AVX512F
: build/src.macosx-11-x86_64-3.9/numpy/core/src/_simd/_simd.dispatch.c
:
AVX512_SKX : SSE SSE2 SSE3 SSSE3 SSE41 POPCNT SSE42 AVX F16C FMA3 AVX2 AVX512F AVX512CD
Flags : -msse -msse2 -msse3 -mssse3 -msse4.1 -mpopcnt -msse4.2 -mavx -mf16c -mfma -mavx2 -mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq
Extra checks: AVX512BW_MASK AVX512DQ_MASK
Detect : AVX512_SKX
: build/src.macosx-11-x86_64-3.9/numpy/core/src/_simd/_simd.dispatch.c
CCompilerOpt._cache_write[796] : write cache to path -> /private/var/folders/d9/737381n90cv4_jc2pnx1vc5w0000gn/T/pip-install-o5k4mrv9/numpy/build/temp.macosx-11-x86_64-3.9/ccompiler_opt_cache_ext.py
########### CLIB COMPILER OPTIMIZATION ###########
Platform :
Architecture: x64
Compiler : clang
CPU baseline :
Requested : 'min'
Enabled : SSE SSE2 SSE3
Flags : -msse -msse2 -msse3
Extra checks: none
CPU dispatch :
Requested : 'max -xop -fma4'
Enabled : SSSE3 SSE41 POPCNT SSE42 AVX F16C FMA3 AVX2 AVX512F AVX512CD AVX512_KNL AVX512_SKX AVX512_CLX AVX512_CNL AVX512_ICL
Generated : none
CCompilerOpt._cache_write[796] : write cache to path -> /private/var/folders/d9/737381n90cv4_jc2pnx1vc5w0000gn/T/pip-install-o5k4mrv9/numpy/build/temp.macosx-11-x86_64-3.9/ccompiler_opt_cache_clib.py
----------------------------------------
ERROR: Failed building wheel for numpy
Failed to build numpy
ERROR: Could not build wheels for numpy which use PEP 517 and cannot be installed directly
----------------------------------------
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/pipenv/resolver.py", line 764, in <module>
main()
File "/usr/local/lib/python3.9/site-packages/pipenv/resolver.py", line 758, in main
_main(parsed.pre, parsed.clear, parsed.verbose, parsed.system, parsed.write,
File "/usr/local/lib/python3.9/site-packages/pipenv/resolver.py", line 741, in _main
resolve_packages(pre, clear, verbose, system, write, requirements_dir, packages, dev)
File "/usr/local/lib/python3.9/site-packages/pipenv/resolver.py", line 702, in resolve_packages
results, resolver = resolve(
File "/usr/local/lib/python3.9/site-packages/pipenv/resolver.py", line 684, in resolve
return resolve_deps(
File "/usr/local/lib/python3.9/site-packages/pipenv/utils.py", line 1395, in resolve_deps
results, hashes, markers_lookup, resolver, skipped = actually_resolve_deps(
File "/usr/local/lib/python3.9/site-packages/pipenv/utils.py", line 1108, in actually_resolve_deps
resolver.resolve()
File "/usr/local/lib/python3.9/site-packages/pipenv/utils.py", line 823, in resolve
results = self.resolver.resolve(max_rounds=environments.PIPENV_MAX_ROUNDS)
File "/usr/local/lib/python3.9/site-packages/pipenv/patched/piptools/resolver.py", line 180, in resolve
has_changed, best_matches = self._resolve_one_round()
File "/usr/local/lib/python3.9/site-packages/pipenv/patched/piptools/resolver.py", line 268, in _resolve_one_round
their_constraints.extend(self._iter_dependencies(best_match))
File "/usr/local/lib/python3.9/site-packages/pipenv/patched/piptools/resolver.py", line 383, in _iter_dependencies
dependencies = self.repository.get_dependencies(ireq)
File "/usr/local/lib/python3.9/site-packages/pipenv/patched/piptools/repositories/pypi.py", line 229, in get_dependencies
legacy_results = self.get_legacy_dependencies(ireq)
File "/usr/local/lib/python3.9/site-packages/pipenv/patched/piptools/repositories/pypi.py", line 350, in get_legacy_dependencies
results, ireq = self.resolve_reqs(
File "/usr/local/lib/python3.9/site-packages/pipenv/patched/piptools/repositories/pypi.py", line 306, in resolve_reqs
results = resolver._resolve_one(reqset, ireq)
File "/usr/local/lib/python3.9/site-packages/pipenv/patched/notpip/_internal/legacy_resolve.py", line 339, in _resolve_one
abstract_dist = self._get_abstract_dist_for(req_to_install)
File "/usr/local/lib/python3.9/site-packages/pipenv/patched/notpip/_internal/legacy_resolve.py", line 287, in _get_abstract_dist_for
abstract_dist = self.preparer.prepare_linked_requirement(req)
File "/usr/local/lib/python3.9/site-packages/pipenv/patched/notpip/_internal/operations/prepare.py", line 508, in prepare_linked_requirement
abstract_dist = _get_prepared_distribution(
File "/usr/local/lib/python3.9/site-packages/pipenv/patched/notpip/_internal/operations/prepare.py", line 95, in _get_prepared_distribution
abstract_dist.prepare_distribution_metadata(finder, build_isolation)
File "/usr/local/lib/python3.9/site-packages/pipenv/patched/notpip/_internal/distributions/sdist.py", line 38, in prepare_distribution_metadata
self._setup_isolation(finder)
File "/usr/local/lib/python3.9/site-packages/pipenv/patched/notpip/_internal/distributions/sdist.py", line 66, in _setup_isolation
self.req.build_env.install_requirements(
File "/usr/local/lib/python3.9/site-packages/pipenv/patched/notpip/_internal/build_env.py", line 202, in install_requirements
call_subprocess(args, spinner=spinner)
File "/usr/local/lib/python3.9/site-packages/pipenv/patched/notpip/_internal/utils/subprocess.py", line 242, in call_subprocess
raise InstallationError(exc_msg)
pipenv.patched.notpip._internal.exceptions.InstallationError: Command errored out with exit status 1: /Users/kaiznanji/.local/share/virtualenvs/Kalgo-wH0XXFPH/bin/python /usr/local/lib/python3.9/site-packages/pipenv/patched/notpip install --ignore-installed --no-user --prefix /private/var/folders/d9/737381n90cv4_jc2pnx1vc5w0000gn/T/pip-build-env-4nfnxp9e/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- setuptools wheel 'Cython>=0.29.21,<3' 'numpy==1.16.5; python_version=='"'"'3.7'"'"' and platform_system!='"'"'AIX'"'"'' 'numpy==1.17.3; python_version=='"'"'3.8'"'"' and platform_system!='"'"'AIX'"'"'' 'numpy==1.16.5; python_version=='"'"'3.7'"'"' and platform_system=='"'"'AIX'"'"'' 'numpy==1.17.3; python_version=='"'"'3.8'"'"' and platform_system=='"'"'AIX'"'"'' 'numpy; python_version>='"'"'3.9'"'"'' Check the logs for full command output.
Pipfile
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"
[packages]
bs4 = "*"
requests = "*"
pandas = "*"
joblib = "*"
[dev-packages]
[requires]
python_version = "3.9"
Solution 1:[1]
I face this problem constantly. Sometimes this solution works, but not all the time. Run this in your terminal:
export SYSTEM_VERSION_COMPAT=1
If someone else understands why this works sometimes, please feel free to comment or edit my answer.
Solution 2:[2]
The stacktrace from the original poster as well as mine included an error in bdist_wheel.py. Opening that exact file, it was importing 'distutils' to call 'distutils.util.get_platform()'. But when I started a python shell to see what that call returned on my machine, the active version of distutils didn't have 'util' subpackage. That's a clue that the python version I'm using is not the version being used by pipenv to build libraries.
That led me to figure out what version of python I really wanted to use and make sure pipenv was using that as well as currently installed packages:
> which python
/usr/local/Caskroom/miniforge/base/bin/python
>pipenv install --python=/usr/local/Caskroom/miniforge/base/bin/python --site-packages
This works so far and may help some folks, however specifying the python version does not work for pipenv lock so it's probably not a complete solution.
Ultimately I probably need to go back to basics and use pyenv to properly manage the python version and keep things consistent.
Solution 3:[3]
Running this first solved it for me: export SYSTEM_VERSION_COMPAT=1
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|---|
| Solution 1 | Dani |
| Solution 2 | LisaD |
| Solution 3 | Christopher Broderick |