'Python and conda on EFS drive use cases?
I'm working in a JupyterLab environment running on AWS with EFS storage. When I try to create a new conda environment, it takes nearly 30 minutes just to install the base environment. conda create -n my_env python=3.8 take 30 minutes and if I install anything else (like tensorflow) it takes another half hour or more.
I've tested a similar environment with EBS storage which runs MUCH faster. No huge surprise that it is faster, but very surprising how much faster - less than a minute for the base environment and maybe a minute to install tensorflow, matplotlib, pandas, and numpy.
Is this expected? Or do data scientists operate on EFS drives and I'm simply doing it wrong? I'm not a DevOps engineer and don't have access to the specific configuration of the EFS drive, so unfortunately I cannot provide any of those details. But, is there a use case similar to mine that doesn't take an hour to simply install needed packages into my environment?
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|