Compare packages between two anaconda installations

Question

I had two versions of Python, with Anaconda, installed in my Mac, 3.5 and 3.7. Is there any way to know what packages I had installed in 3.5 that are not in 3.7, like those that you can install with pip (pulp, wordcloud, and alike).

What I have tried so far is using command line with this:

diff -rq anaconda/.../python3.5/site-packages anaconda3/.../python3.7/site-packages

This shows the differences between the directories but shows a lot of information of repeated packages and core modules. How can I find the differences in packages between two Anaconda versions?

Answer 1

You can list all the packages installed under a specific environment using conda list. Also, you can export the list to a file with the flag --export. For example, if your environments are named python3 and pytorch_p36, you can do the following:

conda list -n python3 --export > python3-packages.txt
conda list -n pytorch_p36 --export > pytorch_p36-packages.txt
diff python3-packages.txt pytorch_p36-packages.txt

The command also accepts the flag --json, which will export the data in a json format, suitable for automatic processing.

Answer 2

Thanks dataista for your solution for Linux. For Windows diff does not work, but in Windows one can use FC:

conda list -n python3 --export > python3-packages.txt
conda list -n pytorch_p36 --export > pytorch_p36-packages.txt
FC python3-packages.txt pytorch_p36-packages.txt

Answer 3

A recent development now provides the ability to compare a conda environment against a requirements file. The only mention of this I've found so far is in a pull request that was merged back on July 16, 2020. I can verify that the compare command works in conda version 4.8.4. It should be noted that only differences are listed in the output.

Usage:

(ProcMCD43A1) C:\Users\justincase>conda compare environment.yml --json
[
  "argon2-cffi found but mismatch. Specification pkg: argon2-cffi==20.1.0=py37he774522_1, Running pkg: argon2-cffi==20.1.0=py37h4ab8f01_1",
  "attrs found but mismatch. Specification pkg: attrs==19.3.0=py_0, Running pkg: attrs==20.2.0=pyh9f0ad1d_0",
  "backcall found but mismatch. Specification pkg: backcall==0.2.0=py_0, Running pkg: backcall==0.2.0=pyh9f0ad1d_0",
  "bleach found but mismatch. Specification pkg: bleach==3.1.5=py_0, Running pkg: bleach==3.2.0=pyh9f0ad1d_0",
  "brotlipy found but mismatch. Specification pkg: brotlipy==0.7.0=py37he774522_1000, Running pkg: brotlipy==0.7.0=py37h4ab8f01_1000",
  "bzip2 found but mismatch. Specification pkg: bzip2==1.0.8=he774522_0, Running pkg: bzip2==1.0.8=he774522_3",
  "cffi found but mismatch. Specification pkg: cffi==1.14.0=py37h7a1dbc1_0, Running pkg: cffi==1.14.3=py37h26f1ce3_0",
  "cfitsio found but mismatch. Specification pkg: cfitsio==3.470=he774522_5, Running pkg: cfitsio==3.470=hbbe6aef_6",
  "cftime found but mismatch. Specification pkg: cftime==1.2.1=py37h2a96729_0, Running pkg: cftime==1.2.1=py37h44b1f71_0",
  "chardet found but mismatch. Specification pkg: chardet==3.0.4=py37_1003, Running pkg: chardet==3.0.4=py37hc8dfbb8_1006",
  "click not found",
  "click-plugins not found",
  "cligj not found",
  "cryptography found but mismatch. Specification pkg: cryptography==2.9.2=py37h7a1dbc1_0, Running pkg: cryptography==3.1=py37h26f1ce3_0",
  "curl found but mismatch. Specification pkg: curl==7.67.0=h2a8f88b_0, Running pkg: curl==7.71.1=h4b64cdc_5",
  "cycler found but mismatch. Specification pkg: cycler==0.10.0=py37_0, Running pkg: cycler==0.10.0=py_2",
  "descartes not found",
  "entrypoints found but mismatch. Specification pkg: entrypoints==0.3=py37_0, Running pkg: entrypoints==0.3=py37hc8dfbb8_1001",
  "fiona not found",
  "freexl found but mismatch. Specification pkg: freexl==1.0.5=hfa6e2cd_0, Running pkg: freexl==1.0.5=hd288d7e_1002",
  "gdal found but mismatch. Specification pkg: gdal==3.0.2=py37hdf43c64_0, Running pkg: gdal==3.1.2=py37h6ddc196_1",
  "geopandas not found",
  "geos found but mismatch. Specification pkg: geos==3.8.0=h33f27b4_0, Running pkg: geos==3.8.1=he025d50_0",
  "geotiff found but mismatch. Specification pkg: geotiff==1.5.1=h5770a2b_1, Running pkg: geotiff==1.6.0=h09e6dc1_1",
  "h5py not found",
  "hdf4 found but mismatch. Specification pkg: hdf4==4.2.13=h712560f_2, Running pkg: hdf4==4.2.13=hf8e6fe8_1003",
  "hdf5 found but mismatch. Specification pkg: hdf5==1.10.4=h7ebc959_0, Running pkg: hdf5==1.10.6=nompi_he0bbb20_101",
   ...
]

Answer 4

Using mini conda 4.11, with the new compare command, I'm going to compare a named environment with a path environment.

? conda --version
conda 4.11.0

First list the environments I have

? conda env list
# conda environments:
#
base                  *  /Users/me/miniforge3
experiments              /Users/me/miniforge3/envs/experiments
tfm1                     /Users/me/miniforge3/envs/tfm1
                         /Users/me/path/project/env

Now I want to compare the conda env in the directory ./path/project/env with the named env tfm1 (my tensorflow env)... First I switch to the tfm1 env and export its package list:

? conda activate tfm1
? conda env export -f /Users/me/path/tfm1_env.yml

Next I switch to the other env, in the directory:

? conda activate /Users/me/path/project/env

And finally use the new compare command:

? conda compare /Users/me/path/tfm1_env.yml

black not found
certifi found but mismatch. Specification pkg: certifi==2021.10.8=py38h10201cd_1, Running pkg: certifi==2021.10.8=pypi_0
click not found
dataclasses not found
executing found but mismatch. Specification pkg: executing==0.8.2=pyhd8ed1ab_0, Running pkg: executing==0.8.3=pyhd8ed1ab_0
flit-core found but mismatch. Specification pkg: flit-core==3.6.0=pyhd8ed1ab_0, Running pkg: flit-core==3.7.1=pyhd8ed1ab_0
fonttools found but mismatch. Specification pkg: fonttools==4.29.1=py38hea4295b_0, Running pkg: fonttools==4.31.2=py38h33210d7_0
fribidi not found
...

The documentation is not too clear on this, but "Specification pkg" refers to a package found in the environment whose specification we exported to a file ("tfm1" in my example). "Running pkg" is, of course, the one I activated.

Even less clear, the "not found" refers to the active environment: the packages "black", "fribidi", etc, are in my tfm1 (specified) but not in my project/env (active) environment.