How to activate python virtual environment on remote machine with ansible?

Question

I'm learning Vagrant and Ansible, I'm trying to setup a local development environment for a basic flask app in ubuntu20.04 with Nginx.

my vagrantfile looks like this:

Vagrant.configure("2") do |config|
  config.vm.define :ubuntuserver do | ubuntuserver |
    ubuntuserver.vm.box = "bento/ubuntu-20.04"
    ubuntuserver.vm.hostname = "ubuntuserver"
    ubuntuserver.vm.provision :ansible do | ansible |
      ansible.playbook = "development.yml"
    end
    ubuntuserver.vm.network "private_network", ip:"10.11.1.105"
    ubuntuserver.vm.network "forwarded_port", guest: 80, host: 8080
    ubuntuserver.vm.network "public_network", bridge: "en1: Wi-Fi (AirPort)"
    ubuntuserver.vm.provider :virtualbox do |vb|
      vb.memory = "1024"
    end
    ubuntuserver.vm.synced_folder "./shared", "/var/www"
  end
end

my ansible-playbook like so:

-
  name: local env
  hosts: ubuntuserver
  tasks:
    - name: update and upgrade apt packages
      become: yes
      apt: 
        upgrade: yes
        update_cache: yes

    - name: install software properties common
      apt:
        name: software-properties-common
        state: present

    - name: install nginx
      become: yes
      apt:
        name: nginx
        state: present
        update_cache: yes

    - name: ufw allow http
      become: yes
      community.general.ufw:
        rule: allow
        name: "Nginx HTTP"
    
    - name: installing packages for python env
      become: yes
      apt:
        name: 
          - python3-pip
          - python3-dev
          - build-essential
          - libssl-dev
          - libffi-dev
          - python3-setuptools
          - python3-venv
        update_cache: yes
    
    - name: Create app directory if it does not exist
      ansible.builtin.file:
        path: /var/www/app
        state: directory
        mode: '0774'

    - name: Install virtualenv via pip
      become: yes
      pip:
        name: virtualenv
        executable: pip3

    - name: Set python virual env
      command:
        cmd: virtualenv /var/www/app/ -p python3
        creates: "/var/www/app/"

    - name: Install requirements
      pip:
        requirements: /var/www/requirements.txt
        virtualenv: /var/www/app/appenv
        virtualenv_python: python3

My playbook fails at the next task with error:

- name: Activate /var/www/app/appenv
      become: yes
      command: source /var/www/app/appenv/bin/activate

fatal: [ubuntuserver]: FAILED! => {"changed": false, "cmd": "source /var/www/app/appenv/bin/activate", "msg": "[Errno 2] No such file or directory: b'source'", "rc": 2}

Rest of the playbook

   
    - name: ufw allow 5000
      become: yes
      community.general.ufw:
        rule: allow
        to_port: 5000
    
    - name: Run app
      command: python3 /var/www/app/appenv/app.py

From what I understand from this thread, The "source" command must be used from inside the vagrant machine. (I tried solutions from the thread but couldn't get it to work) If I ssh into the vagrant machine and execute the three last commands of my playbook manually:

source /var/www/app/appenv/bin/activate
sudo ufw allow 5000
python3 /var/www/app/appenv/app.py

my basic flask app is running on port 5000 at the IP set in the vagrantfile 10.11.1.105

My questions are:

How can I get the playbook to work and not have to ssh into the machine to accomplish the same?

Is my approach even correct, knowing that my end goal is to replicate in the vagrant machine a similar environment to what would be the production environment and develop the flask app from my local machine in the synced folder?

to give a maximum of information, if one wants to reproduce this. I also have a shared/app/appenv/app.py file containing the basic flask app

from flask import Flask
app = Flask(__name__)

@app.route("/")
def hello():
    return "<h1 style='color:blue'>Hello There!</h1>"

if __name__ == "__main__":
    app.run(host='0.0.0.0')

and shared/requirements.txt file

wheel
uwsgi
flask

Answer 1

This question was coming from a misconception about python venv. I thought that in order to install packages inside the virtual env it had to be activated. for example:

source env/bin/activate
pip install package_name

I understood later that I can install packages in the venv without activating it by doing:

env/bin/pip install package_name

So the solution with ansible is not to activate the venv to install packages but instead

- name: "Install python packages with the local instance of pip"
   shell: "{{virtualenv_path}}/bin/pip3 install package_name"
   become: no

or better even with the pip module and the packages in a requirements.txt file:

- name: Install project requirements in venv
  pip:
    requirements: '{{project_path}}/requirements.txt'
    virtualenv: '{{virtualenv_path}}'
    virtualenv_python: python3

Answer 2

I wrote this article for a Linux computer with Python 3.x. In this scenario, this is your Ansible development machine. First, verify the installed Python version and path:

# check Python version
$ python3 -V
Python 3.6.8

$ which python3
/usr/bin/python3

I recommend setting up a directory for the virtual environment:

$ mkdir python-venv
$ cd !$

Create a new virtual environment

$ python3 -m venv ansible2.9
$ ls
ansible2.9

activate python venv

$ source ansible2.9/bin/activate
(ansible2.9)$ python3 -V
Python 3.6.8

upgrade pip

(ansible2.9)$ python3 -m pip install --upgrade pip

Install Ansible in a virtual environment

(ansible2.9)$ python3 -m pip install ansible==2.9
(ansible2.9)$ which ansible
~/python-venv/ansible2.9/bin/ansible

Verify your new installation:

 (ansible2.9)$ ansible --version
    ansible 2.9.0
      config file = /etc/ansible/ansible.cfg
      configured module search path = ['/home/devops/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/devops/python-venv/ansible2.9/lib64/python3.6/site-packages/ansible
  executable location = /home/devops/python-venv/ansible2.9/bin/ansible
  python version = 3.6.8 (default, Jan 09 2021, 10:57:11) [GCC 8.3.1 20191121 (Red Hat 8.3.1-5)]

Install Ansible roles or collections

(ansible2.9)$ ansible-galaxy collection install \
  kubernetes.core:==1.2.1 -p collections

Deactivate a Python virtual environment

(ansible2.9)$ deactivate

Create another Python virtual environment for Ansible 3.0

$ python3 -m venv ansible3.0
$ ls -1
ansible2.9
ansible3.0
$ source ansible3.0/bin/activate
(ansible3.0)$ which python
~/python-venv/ansible3.0/bin/python
(ansible3.0)$ python3 -m pip install --upgrade pip
(ansible3.0)$ python3 -m pip install ansible==3.0