How to avoid error "conda --version: conda not found" in az ml run --submit-script command?

Question

I would like to run a test script on an existing compute instance of Azure using the Azure Machine Learning extension to the Azure CLI:

az ml run submit-script test.py --target compute-instance-test --experiment-name test_example --resource-group ex-test-rg

I get a Service Error with the following error message:

Unable to run conda package manager. AzureML uses conda to provision python\nenvironments from a dependency specification. To manage the python environment\nmanually instead, set userManagedDependencies to True in the python environment\nconfiguration. To use system managed python environments, install conda from:\nhttps://conda.io/miniconda.html

But when I connect to the compute instance through the Azure portal and select the default Python kernel, conda --version prints 4.5.12. So conda is effectively already installed on the compute instance. This is why I do not understand the error message.

Further information on the azure versions:

  "azure-cli": "2.12.1",
  "azure-cli-core": "2.12.1",
  "azure-cli-telemetry": "1.0.6",
  "extensions": {
    "azure-cli-ml": "1.15.0"
  }

The image I use is:

mcr.microsoft.com/azure-cli:latest

Can somebody please explain as to why I am getting this error and help me resolve the error? Thank you!

EDIT: I tried to update the environment in which the az ml run-command is run. Essentially this is my GitLab job. The installation of miniconda is a bit complicated as the azure-cli uses an alpine Linux image (reference: Installing miniconda on alpine linux fails). I replaced some names with ... and cut out some irrelevant pieces of code.

test:
  image: 'mcr.microsoft.com/azure-cli:latest'
  script:
    - echo "Download conda"
    - apk --update add bash curl wget ca-certificates libstdc++ glib
    - wget -q -O /etc/apk/keys/sgerrand.rsa.pub https://raw.githubusercontent.com/sgerrand/alpine-pkg-node-bower/master/sgerrand.rsa.pub
    - curl -L "https://github.com/sgerrand/alpine-pkg-glibc/releases/download/2.23-r3/glibc-2.23-r3.apk" -o glibc.apk
    - apk del libc6-compat
    - apk add glibc.apk
    - curl -L "https://github.com/sgerrand/alpine-pkg-glibc/releases/download/2.23-r3/glibc-bin-2.23-r3.apk" -o glibc-bin.apk 
    - apk add glibc-bin.apk 
    - curl -L "https://github.com/andyshinn/alpine-pkg-glibc/releases/download/2.25-r0/glibc-i18n-2.25-r0.apk" -o glibc-i18n.apk
    - apk add --allow-untrusted glibc-i18n.apk 
    - /usr/glibc-compat/bin/localedef -i en_US -f UTF-8 en_US.UTF-8 
    - /usr/glibc-compat/sbin/ldconfig /lib /usr/glibc/usr/lib
    - rm -rf glibc*apk /var/cache/apk/*
    - echo "yes" | curl -sSL https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh -o miniconda.sh
    - echo "Install conda"
    - (echo -e "\n"; echo "yes"; echo -e "\n"; echo "yes") | bash -bfp miniconda.sh
    - echo "Installing Azure Machine Learning Extension"
    - az extension add -n azure-cli-ml
    - echo "Azure Login"
    - az login
    - az account set --subscription ...
    - az configure --defaults group=...
    - az ml folder attach -w ... 
    - az ml run submit-script test.py --target ... --experiment-name hello_world --resource-group ...

Answer 1

One needs to pass the --workspace-name argument to be able to run it on Azure's compute target and not on the local compute target:

az ml run submit-script test.py --target compute-instance-test --experiment-name test_example --resource-group ex-test-rg --workspace-name test-ws

Answer 2

You need conda in your base image for container based environment. You can extend the base image by installing conda using base_dockerfile instead of base_image https://docs.microsoft.com/en-us/python/api/azureml-core/azureml.core.environment.dockersection?view=azure-ml-py or, which if that works for you, use one of the AzureML base docker images.

If you do not need any python dependencies on top your base image you can set user_managed_dependencies to True and base image will be used as is and no additional dependencies will be installed

https://docs.microsoft.com/en-us/python/api/azureml-core/azureml.core.environment.pythonsection?view=azure-ml-py

Answer 3

Use:

runconfig.environment.python.user_managed_dependencies = True

That should solve the issue