问题描述
我有一个 conda 环境,在一个 bash 终端中,带有一个 Intel Python Distribution 解释器.但是,在导入包时,它们是从系统默认 Python 的用户目录中导入的,而不是从环境中导入的.查看 pandas 包的版本差异和 __spec__ 来源.
I have a conda environment, in a bash terminal, with an Intel Python Distribution interpreter. However, when importing packages, they are imported from what looks to be the user directory of the system default Python, not the environment. Take a look at the version discrepancy and the __spec__ origin of the pandas package.
~ $ conda activate idp ~ $ which python ~/anaconda3/envs/idp/bin/python ~ $ python Python 3.6.8 |Intel Corporation| (default, Mar 1 2019, 00:10:45) [GCC 4.8.2 20140120 (Red Hat 4.8.2-15)] on linux Type "help", "copyright", "credits" or "license" for more information. Intel(R) Distribution for Python is brought to you by Intel Corporation. Please check out: http://www.51sjk.com/Upload/Articles/1/0/336/336396_20221114145048694.jpg >>> import pandas >>> pandas.__version__ '0.22.0' >>> pandas.__spec__ ModuleSpec(name='pandas', loader=<_frozen_importlib_external.SourceFileLoader object at 0x7f509e184ba8>, origin='/home/torstein/.local/lib/python3.6/site-packages/pandas/__init__.py', submodule_search_locations=['/home/torstein/.local/lib/python3.6/site-packages/pandas']) >>> ~ $ conda list | head -n 3 # packages in environment at /home/torstein/anaconda3/envs/idp: # # Name Version Build Channel ~ $ conda list | grep pandas pandas 0.24.1 py36_3 intel ~ $ conda env list # conda environments: # base /home/torstein/anaconda3 idp * /home/torstein/anaconda3/envs/idp py36 /home/torstein/anaconda3/envs/py36
当使用 base 环境时,这不会发生;从正确的路径导入包(例如 pandas):
When using the base environment, this does NOT happen; packages (e.g. pandas) are imported from the correct path:
~ $ conda deactivate ~ $ conda env list # conda environments: # base * /home/torstein/anaconda3 idp /home/torstein/anaconda3/envs/idp py36 /home/torstein/anaconda3/envs/py36 ~ $ which python ~/anaconda3/bin/python ~ $ python Python 3.7.0 (default, Oct 9 2018, 10:31:47) [GCC 7.3.0] :: Anaconda custom (64-bit) on linux Type "help", "copyright", "credits" or "license" for more information. >>> import pandas pan>>> pandas.__version__ '0.23.4' >>> pandas.__spec__ ModuleSpec(name='pandas', loader=<_frozen_importlib_external.SourceFileLoader object at 0x7fad808a8e80>, origin='/home/torstein/anaconda3/lib/python3.7/site-packages/pandas/__init__.py', submodule_search_locations=['/home/torstein/anaconda3/lib/python3.7/site-packages/pandas'])
.bashrc的相关部分(路径中没有显式设置anaconda):
The relevant part of .bashrc (no anaconda explicitly set in path):
export PATH="/home/torstein/.cargo/bin:$PATH" export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/opt/intel/lib/intel64_lin:/opt/intel/compilers_and_libraries_2018.0.128/linux/mkl/lib/intel64_lin/" # >>> conda initialize >>> # !! Contents within this block are managed by 'conda init' !! __conda_setup="$('/home/torstein/anaconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)" if [ $? -eq 0 ]; then eval "$__conda_setup" else if [ -f "/home/torstein/anaconda3/etc/profile.d/conda.sh" ]; then . "/home/torstein/anaconda3/etc/profile.d/conda.sh" else export PATH="/home/torstein/anaconda3/bin:$PATH" fi fi unset __conda_setup
分别为 base 和 idp 环境生成这些 $PATH:
Which yields these $PATHs, for the base and idp envs respectively:
~ $ echo $PATH /home/torstein/anaconda3/bin:/home/torstein/anaconda3/condabin:/home/torstein/.cargo/bin:/home/torstein/.cargo/bin:/home/torstein/anaconda3/bin:/home/torstein/.cargo/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/home/torstein/.local/bin:/home/torstein/bin ~ $ conda activate idp ~ $ echo $PATH /home/torstein/anaconda3/envs/idp/bin:/home/torstein/anaconda3/condabin:/home/torstein/.cargo/bin:/home/torstein/.cargo/bin:/home/torstein/anaconda3/bin:/home/torstein/.cargo/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/home/torstein/.local/bin:/home/torstein/bin
我做要导入的pandas位于这里,应该是:
The pandas that I do want to import is located here, where it should be:
/home/torstein/anaconda3/envs/idp/lib/python3.6/site-packages/pandas
推荐答案
诊断
PATH 中似乎有(或曾经有)另一个 Python 3.6,我怀疑 Conda 依赖解析器最终以某种方式将一些包解析为这个替代 site-packages 并且无意中将这个目录包含在 sys.path 中.这似乎是一个已知问题.
Diagnosis
There appears to be (or have been) another Python 3.6 in the PATH, and I suspect that somehow the Conda dependency resolver ended up resolving some packages to this alternative site-packages and inadvertently including this directory in sys.path. This appears to be a known issue.
我相信这是因为 pandas 模块是从这里加载的:
The reason I believe this is because the pandas module is being loaded from here:
/home/torstein/.local/lib/python3.6/site-packages/pandas
/home/torstein/.local/lib/python3.6/site-packages/pandas
如果你签入 Python
If you check in Python
import sys sys.path
我希望这应该显示上面的目录.
I expect that this should show the above directory.
由于据报道 PYTHONPATH 是空的(应该是这样!),因此不能对这种错误加载负责,因此我认为是 Conda 以某种方式配置了 env.
Since it was reported that PYTHONPATH is empty (as it should be!), that can't be responsible for this misloading, hence I think it was Conda that somehow configured the env this way.
此外,您的 Python 3.7 环境未受影响这一事实可能是因为您无法跨不同的次要版本加载模块.
Also, the fact that your Python 3.7 env is unaffected is likely because you can't load modules across different minor versions.
您需要以某种方式摆脱这种依赖.有一些事情可以尝试
Somehow you need to get rid of that dependency. There are a few things to try
- 从您的 PATH 中删除该 /home/torstein/.local/.不过,这可能会导致其他问题.大概你在 PATH 中有它,因为你有其他非开发的东西在那里.
- 专门转储该 site-packages 目录.在评论中,有人说这是不再使用的全局 Python 安装的残留物,因此摆脱它似乎是一件好事.不过,请备份它,以防有其他依赖项.
- 在导入模块之前从 sys.path 中清除此路径.不确定一种干净的方式来做到这一点.
- Remove that /home/torstein/.local/ from your PATH. This could cause other issues though. Presumably you have it in PATH because you have other non-development things in there.
- Dump specifically that site-packages directory. In comments, it was stated that this is residual from a global Python installation no longer in use, so it seems like a good thing to just get rid of. Do back it up, though, in case there are other dependencies on it.
- Clear this path from sys.path before importing modules. Not sure of a clean way to do this.
就个人而言,我想删除它并创建新的环境.可能很难知道你是如何与这个目录绑定在一起的,所以我会谨慎地假设其他包对那里的内容没有隐藏的依赖关系.
Personally, I'd want to delete it and create new envs. It can be hard to know how tied into this directory you are, so I'd be wary of assuming that other packages don't somehow have hidden dependencies on what is in there.
GitHub 问题推荐的解决方法是添加以下环境变量,
The recommended workaround from the GitHub issue is to add the following environment variable,
export PYTHONNOUSERSITE=True
这会阻止 Conda 加载其他本地 site-packages 目录.有了这个,你一开始就不应该遇到这个问题.
which prevents Conda from loading other local site-packages directories. With this, you shouldn't have encountered the problem in the first place.