Ansible cannot pickle 'module' object

Question

Running the playbook with

$ ansible-playbook test-playbook.yml

Getting error response

ERROR! Unexpected Exception, this is probably a bug: cannot pickle 'module' object

Also have tried shell: "echo text", same error.

Ansible version 2.9, Python 3.8, running on Macos.

test-playbook.yml:

---
  - name: run on localhost
    hosts: localhost
    gather_facts: false
    connection: local

    tasks:

      - name: some name
        debug:  
          msg: "some text"
...

Full output:

$ ansible-playbook test-playbook.yml 
[WARNING]: No inventory was parsed, only implicit localhost is available

[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does
not match 'all'


PLAY [run on localhost] *************************************************************************************

TASK [some name] ********************************************************************************************
ERROR! Unexpected Exception, this is probably a bug: cannot pickle 'module' object
to see the full traceback, use -vvv

Verbose output is not helpful:

$ ansible-playbook test-playbook.yml -vvv
ansible-playbook 2.9.0
  config file = None
  configured module search path = ['/Users/jprusakova/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/ansible
  executable location = /Library/Frameworks/Python.framework/Versions/3.8/bin/ansible-playbook
  python version = 3.8.0 (v3.8.0:fa919fdf25, Oct 14 2019, 10:23:27) [Clang 6.0 (clang-600.0.57)]
No config file found; using defaults
host_list declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
Skipping due to inventory source not existing or not being readable by the current user
script declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
auto declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
Skipping due to inventory source not existing or not being readable by the current user
yaml declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
Skipping due to inventory source not existing or not being readable by the current user
ini declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
Skipping due to inventory source not existing or not being readable by the current user
toml declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
[WARNING]: No inventory was parsed, only implicit localhost is available

[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does
not match 'all'


PLAYBOOK: test-playbook.yml *********************************************************************************
1 plays in test-playbook.yml

PLAY [run on localhost] *************************************************************************************
META: ran handlers

TASK [some name] ********************************************************************************************
task path: /Users/jprusakova/ViaControl/temp-ansible/test-playbook.yml:9
ERROR! Unexpected Exception, this is probably a bug: cannot pickle 'module' object
the full traceback was:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.8/bin/ansible-playbook", line 123, in <module>
    exit_code = cli.run()
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/ansible/cli/playbook.py", line 127, in run
    results = pbex.run()
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/ansible/executor/playbook_executor.py", line 169, in run
    result = self._tqm.run(play=play)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/ansible/executor/task_queue_manager.py", line 240, in run
    play_return = strategy.run(iterator, play_context)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/ansible/plugins/strategy/linear.py", line 310, in run
    self._queue_task(host, task, task_vars, play_context)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/ansible/plugins/strategy/__init__.py", line 343, in _queue_task
    worker_prc.start()
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/ansible/executor/process/worker.py", line 96, in start
    return super(WorkerProcess, self).start()
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/context.py", line 283, in _Popen
    return Popen(process_obj)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 47, in _launch
    reduction.dump(process_obj, fp)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
TypeError: cannot pickle 'module' object

Ansible version

$ ansible --version
ansible 2.9.0
  config file = None
  configured module search path = [...]
  ansible python module location = /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/ansible
  executable location = /Library/Frameworks/Python.framework/Versions/3.8/bin/ansible
  python version = 3.8.0 (v3.8.0:fa919fdf25, Oct 14 2019, 10:23:27) [Clang 6.0 (clang-600.0.57)]

Answer 1

UPDATE: Using the latest version of Ansible in my venv fixed the problem (replacing ansible==2.9 with ansible in requirements.txt)

$ ansible all -vvv -a 'uptime' -i hosts.yaml
ansible [core 2.12.1]
  config file = None
  configured module search path = ['/Users/jsherman/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/jsherman/Desktop/Code/ansible-plays/ansible-env/lib/python3.9/site-packages/ansible
  ansible collection location = /Users/jsherman/.ansible/collections:/usr/share/ansible/collections
  executable location = /Users/jsherman/Desktop/Code/ansible-plays/ansible-env/bin/ansible
  python version = 3.9.9 (main, Nov 21 2021, 03:22:47) [Clang 12.0.0 (clang-1200.0.32.29)]
  jinja version = 3.0.3
  libyaml = True

I'm seeing the same issue, also on macOS. I was able to reproduce the issue using python 3.9.9 and 3.10.1 (both from Homebrew), and the OS-installed python3 binary (3.8.2)

Using python2 on macOS fixes it for me, as does using python3 on Ubuntu instead.

I'm wondering if there's something wrong with python3 on macOS specifically that's causing this

Answer 2

I had the same issue today, my install details:

$ ansible --version
ansible 2.9.0
  config file = None
  configured module search path = ['/Users/mark/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/mark/.pyvenv/ansible/lib/python3.9/site-packages/ansible
  executable location = /Users/mark/.pyvenv/ansible/bin/ansible
  python version = 3.9.6 (default, Jun 29 2021, 05:25:02) [Clang 12.0.5 (clang-1205.0.22.9)]
$ ansible app -m ping
ERROR! Unexpected Exception, this is probably a bug: cannot pickle 'module' object
to see the full traceback, use -vvv

I was unable to resolve with Python3, but using a virtual environment for Python2 it did work:

$ ansible --version
/Users/mark/.pyenv/py2ansible/lib/python2.7/site-packages/ansible/parsing/vault/__init__.py:41: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in the next release.
  from cryptography.exceptions import InvalidSignature
ansible 2.9.0
  config file = None
  configured module search path = [u'/Users/mark/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/mark/.pyenv/py2ansible/lib/python2.7/site-packages/ansible
  executable location = /Users/mark/.pyenv/py2ansible/bin/ansible
  python version = 2.7.16 (default, May  8 2021, 11:48:02) [GCC Apple LLVM 12.0.5 (clang-1205.0.19.59.6) [+internal-os, ptrauth-isa=deploy
$ ansible app -m ping
/Users/mark/.pyenv/py2ansible/lib/python2.7/site-packages/ansible/parsing/vault/__init__.py:41: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in the next release.
  from cryptography.exceptions import InvalidSignature
m1-base | SUCCESS => {
    "changed": false,
    "ping": "pong"
}

Maybe not ideal, but at least I can carry on. If you did manage to solve, please post your solution

To create the virtual env on MacBookPro:

pip install virtualenv
python -m virtualenv ~/.pyenv/py2ansible
source ~/.pyenv/py2ansible/bin/activate

Answer 3

I had the same issue on Mac (Python 3.10.4)

I was able to resolve it by installing ansible v 2.9.8

ansible 2.9.8
  config file = None
  configured module search path = ['/Users/smanciot/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/ansible
  executable location = /Library/Frameworks/Python.framework/Versions/3.10/bin/ansible
  python version = 3.10.4 (v3.10.4:9d38120e33, Mar 23 2022, 17:29:05) [Clang 13.0.0 (clang-1300.0.29.30)]