jedi-vim is not aware of pyenv path #869

frank777777777 · 2018-08-06T15:16:01Z

Issue

I am using pyenv to switch between difference python version. The autocomplete and goto function only uses modules from system python. For example, while I am in a pyenv virtual environment, I installed beautifulsoup module, jedi-vim does not find my module in my virtual environment, it searches for module in system site-packages instead.
As you can see my verbose JediDebugInfo, my python2 environment is linking to pyenv, while python3 environment is linking to system python.
In vim, !python --version and !pip --version works, its just jedi seems not finding my pyenv.

Steps to reproduce

Output of “:verbose JediDebugInfo”

##### jedi-vim version

 - jedi-vim git version: 0.9.0-57-g132d850-dirty
 - jedi git submodule status:  a79a1fbef57564d031c59f47def20a6d9adf3439 pythonx/jedi (v0.12.1)
 - parso git submodule status:  e6bc924fbabbfa3090044e475c91a7655b9c6e9b pythonx/parso (v0.3.1)

##### Global Python

Using Python version 3 to access Jedi.
 - global sys.version: `3.5.2 (default, Nov 23 2017, 16:37:01), [GCC 5.4.0 20160609]`
 - global site module: `/usr/lib/python3.5/site.py`

##### Jedi

 - path: `/home/yili/.vim/bundle/jedi-vim/pythonx/jedi/jedi/__init__.py`
 - version: 0.12.1

##### Jedi environment: <Environment: 3.6.5 in /usr>

 - executable: /usr/bin/python3.6
 - sys_path:
    - `/usr/lib/python36.zip`
    - `/usr/lib/python3.6`
    - `/usr/lib/python3.6/lib-dynload`
    - `/usr/local/lib/python3.6/dist-packages`
    - `/usr/lib/python3/dist-packages`

##### Known environments

 - <Environment: 3.6.5 in /usr> (/usr/bin/python3.6)
 - <Environment: 3.5.2 in /usr> (/usr/bin/python3.5)
 - <Environment: 2.7.15 in /home/yili/.pyenv> (/home/yili/.pyenv/shims/python2.7)

##### Settings

g:jedi#documentation_command = 'J' (default: 'K')
g:jedi#usages_command = 'n' (default: 'n')
g:jedi#completions_enabled = 0 (default: 1)

omnifunc=python3complete#Complete
Last set from /usr/share/vim/vim81/ftplugin/python.vim
completeopt=preview,menuone
Last set from ~/.vim/bundle/YouCompleteMe/autoload/youcompleteme.vim


#### :version

VIM - Vi IMproved 8.1 Included patches: 1-155
Modified by user vimrc file: "$HOME/.vimrc"
2nd user vimrc file: "~/.vim/vimrc"
user exrc file: "$HOME/.exrc"
defaults file: "$VIMRUNTIME/ fall-back for $VIM: Compilation: gcc -c ormat-security -U_FORTIFY_SOURCE Linking: gcc -Wl,-Bsymbolic-functions nsl -lselinux -lacl -lattr -lgpm -ldl dl -lutil -lm (2018 May 18, compiled Jul 6 2018 08:05:07)
i/l/email-protection#34445f5319425d591959555d5a40555d5a51464774585d4740471a55585d5b405c1a5051565d555a1a5b4653">[email protected]
i/l/email-protection#21514a460c57484c0c4c40484f5540484f445352614d485255520f404d484e55490f45444348404f0f4e5346">[email protected]
Features included (+) or not (-):
+job -mouse_sysmouse +scrollbind +vertsplit
+jumplist +mouse_urxvt +signs +virtualedit
+keymap +mouse_xterm +smartindent +visual
+lambda +multi_byte +startuptime +visualextra
+langmap +multi_lang +statusline +viminfo
+libcall -mzscheme -sun_workshop +vreplace
+linebreak +netbeans_intg +syntax +wildignore
+lispindent +num64 +tag_binary +wildmenu
+listcmds +packages +tag_old_static +windows
+localmap +path_extra -tag_any_white +writebackup
-lua -perl -tcl -X11
+menu +persistent_undo +termguicolors -xfontset
+mksession +postscript +terminal -xim
+modify_fname +printer +terminfo -xpm
+mouse +profile +termresponse -xsmp
-mouseshape -python +textobjects -xterm_clipboard
+mouse_dec +python3 +timers -xterm_save
+mouse_gpm +quickfix +title
-mouse_jsbterm +reltime -toolbar
+mouse_netterm +rightleft +user_commands
+mouse_sgr -ruby +vartabs
defaults.vim"
"/usr/share/vim"
-I. -Iproto -DHAVE_CONFIG_H -Wdate-time -g -O2 -fPIE -fstack-protector-strong -Wformat -Werror=f
-D_FORTIFY_SOURCE=1
-fPIE -pie -Wl,-z,relro -Wl,-z,now -Wl,--as-needed -o vim -lm -ltinfo -l
-L/usr/lib/python3.5/config-3.5m-x86_64-linux-gnu -lpython3.5m -lpthread -l

The text was updated successfully, but these errors were encountered:

blueyed · 2018-08-06T16:02:27Z

Try let g:jedi#force_py_version = '2.7.15'.

Also see #836.

frank777777777 · 2018-08-06T17:06:17Z

Try let g:jedi#force_py_version = '2.7.15'.

I am trying to use python 3 not python2...
I just realized that, if I install python3.6 in pyenv, jedi will refer the python executable to pyenv. If I use python 3.7 in pyenv, jedi will ignore pyenv and use system python executable instead. It seems to me that jedi has hardcoded that it only look for python 3.6 and 3.5, because python 3.5 executable still link to system python. Here is the verbose JediDebugInfo after I install python3.6 in pyenv:

##### Jedi environment: <Environment: 3.6.0 in /home/yili/.pyenv>

 - executable: /home/yili/.pyenv/shims/python3.6
 - sys_path:
    - `/home/yili/.pyenv/versions/3.6.0/lib/python36.zip`
    - `/home/yili/.pyenv/versions/3.6.0/lib/python3.6`
    - `/home/yili/.pyenv/versions/3.6.0/lib/python3.6/lib-dynload`
    - `/home/yili/.pyenv/versions/3.6.0/lib/python3.6/site-packages`

##### Known environments

 - <Environment: 3.6.0 in /home/yili/.pyenv> (/home/yili/.pyenv/shims/python3.6)
 - <Environment: 3.5.2 in /usr> (/usr/bin/python3.5)
 - <Environment: 2.7.15 in /home/yili/.pyenv> (/home/yili/.pyenv/shims/python2.7)

The executable refers to pyenv python, works fine. The sys_path refers to pyenv site-package, works fine. Omni-completion and goto works perfect for modules install in pyenv site package.

However, when I create a virtualenv in pyenv like pyenv virtualenv 3.6.0 scrapper, pyenv local scrapper, the sys_path refer back to system site package. Here is the :verbose JediDebugInfo:

##### Jedi environment: <Environment: 3.6.5 in /home/yili/.pyenv>

 - executable: /home/yili/.pyenv/shims/python3.6
 - sys_path:
    - `/usr/lib/python36.zip`
    - `/usr/lib/python3.6`
    - `/usr/lib/python3.6/lib-dynload`
    - `/usr/local/lib/python3.6/dist-packages`
    - `/usr/lib/python3/dist-packages`

##### Known environments

 - <Environment: 3.6.5 in /home/yili/.pyenv> (/home/yili/.pyenv/shims/python3.6)
 - <Environment: 3.5.2 in /usr> (/usr/bin/python3.5)
 - <Environment: 2.7.15 in /home/yili/.pyenv> (/home/yili/.pyenv/shims/python2.7)

The executable stills uses pyenv, but the sys_path link back to system site-package, omni-completion and goto does not work anymore.

I am running out ideas.

blueyed · 2018-08-06T17:32:02Z

Thanks for the input.

Please consider trying #836 and see if that makes it better already - otherwise check the source to see what actually happens.
Currently venv support is very basic, and I do not remember the state of 836 really, but it should allow to switch and tab-complete them actually.

blueyed · 2018-08-06T17:36:37Z

Re "/home/daniel/.pyenv/shims/scrapper": it is using "Environment: 3.6.5 in /home/yili/.pyenv" there then.
IIRC jedi looks not for "python" first, but with versions appended.. at least it used to do.
Digging into the source (based on #836) is the best you can do currently I guess - if you want to help fix this.
Otherwise try to adjust g:jedi#force_py_version to make it pick up the version you want, but that is also only rudimentary on master.

frank777777777 · 2018-08-06T18:01:44Z

Isn't g:jedi#force_py_version just to specify the python executable version only? I mean it is already picking up the correct python version in pyenv if I install 3.6.0.
It is the sys-path that is not incorrect. I saw in #836 you mentioned use let g:jedi#environment_paths = [$HOME . '/.pyenv/versions]', is it implemented yet?

blueyed · 2018-08-06T18:29:17Z

It does a bit more on master already:

jedi-vim/pythonx/jedi_vim.py

Lines 166 to 185 in 132d850

    
           vim_force_python_version = vim_eval("g:jedi#force_py_version") 
        
           if use_cache and vim_force_python_version == current_environment[0]: 
        
               return current_environment[1] 
        
           environment = None 
        
           if vim_force_python_version == "auto": 
        
               environment = jedi.api.environment.get_cached_default_environment() 
        
           else: 
        
               force_python_version = vim_force_python_version 
        
               if '0000' in force_python_version or '9999' in force_python_version: 
        
                   # It's probably a float that wasn't shortened. 
        
                   try: 
        
                       force_python_version = "{:.1f}".format(float(force_python_version)) 
        
                   except ValueError: 
        
                       pass 
        
               elif isinstance(force_python_version, float): 
        
                   force_python_version = "{:.1f}".format(force_python_version) 
        
               try: 
        
                   environment = jedi.get_system_environment(force_python_version)

you mentioned use let g:jedi#environment_paths = [$HOME . '/.pyenv/versions]', is it implemented yet?

I think so:

jedi-vim/pythonx/jedi_vim.py

Lines 218 to 223 in c06c8ef

    
           def get_known_environments(): 
        
               """Get known Jedi environments.""" 
        
               paths = vim_eval("g:jedi#environment_paths") 
        
               envs = list(jedi.api.environment.find_virtualenvs(paths=paths, safe=False)) 
        
               envs.extend(jedi.api.environment.find_system_environments()) 
        
               return envs

frank777777777 · 2018-08-06T19:18:16Z

wait, is this still a working progress and not committed to master yet? Because what I see in jedi-vim/pythonx/jedi_vim.py, it is still:

196 def get_known_environments():
197     """Get known Jedi environments."""
198     envs = list(jedi.api.environment.find_virtualenvs())
199     envs.extend(jedi.api.environment.find_system_environments())
200     return envs

blueyed · 2018-08-06T19:22:06Z

Yes, it is WIP - an open PR.
To get it working for you you could also hardcode something there, of course.

blueyed · 2018-08-13T15:46:40Z

davidhalter/jedi#1199
#872

statham-stone · 2019-07-22T10:35:49Z

change all "safe=True" to "safe=False" in "~/.vim/bundle/jedi-vim/pythonx/jedi/jedi/api/environment.py" solved the problem for me.

bo5o · 2019-09-16T09:26:44Z

I am experiencing a similar issue.

I have system python version 3.6.8 installed and use pyenv to manage multiple python installations. When I create a virtual environment with python -m venv .venv jedi-vim is not able to provide completions. This only happens if I set the python version to anything other than the system version (e.g. via pyenv local 3.7.4). I am using ALE to provide completions.

`:verbose JediDebugInfo`

#### Jedi-vim debug information

##### jedi-vim version

 - jedi-vim git version: 0.10.0-3-g4f2499e
 - jedi git submodule status:  005f69390c4b1a3b864e7373cfcbdaf95e65411d pythonx/jedi (v0.15.1)
 - parso git submodule status:  c0ace63a6905500e18d8ac06af1a0a9057e63498 pythonx/parso (v0.5.1)

##### Global Python

Using Python version 3 to access Jedi.
 - global sys.version: `3.6.8 (default, Aug 20 2019, 17:12:48), [GCC 8.3.0]`
 - global site module: `/usr/lib/python3.6/site.py`

##### Jedi

 - path: `~/.local/share/nvim/plugged/jedi-vim/pythonx/jedi/jedi/__init__.py`
 - version: 0.15.1

##### Jedi environment: <Environment: 3.7.4 in /tmp/myapp/.venv>

 - executable: /tmp/myapp/.venv/bin/python
 - sys_path:
    - `~/.pyenv/versions/3.7.4/lib/python37.zip`
    - `~/.pyenv/versions/3.7.4/lib/python3.7`
    - `~/.pyenv/versions/3.7.4/lib/python3.7/lib-dynload`
    - `/tmp/myapp/.venv/lib/python3.7/site-packages`

##### Known environments

 - <Environment: 3.7.4 in /tmp/myapp/.venv> (/tmp/myapp/.venv/bin/python)
 - <Environment: 3.7.4 in ~/.pyenv/versions/3.7.4> (~/.pyenv/versions/3.7.4/bin/python3.7)
 - <Environment: 3.6.8 in /usr> (/usr/bin/python3.6)
 - <Environment: 2.7.15 in /usr> (/usr/bin/python2.7)

##### Settings

g:jedi#completions_enabled = 0 (default: 1)

  omnifunc=python3complete#Complete
        Zuletzt gesetzt in /tmp/.mount_nvimt3iaAF/usr/share/nvim/runtime/ftplugin/python.vim line 41
  completeopt=noinsert,menuone,noselect
        Zuletzt gesetzt in ~/dotfiles/nvim/.config/nvim/init.vim line 178

#### :version

NVIM v0.4.0-2014-g554566cce
Build type: RelWithDebInfo
LuaJIT 2.0.5
Features: +acl +iconv +tui

Although it recognizes the correct environment (which is in /tmp/myapp/.venv) and the correct executable (which is python3.7 in ~/.pyenv/versions) I can not autocomplete packages I install in the virtual environment. Weirdly I can only autocomplete packages installed with system python.

blueyed · 2019-09-16T13:42:26Z

@cbows
With Neovim check :checkhealth provider to see which Python it uses.

bo5o · 2019-09-16T14:36:32Z

health#provider#check
========================================================================
## Clipboard (optional)
  - OK: Clipboard tool found: xclip

## Python 2 provider (optional)
  - INFO: pyenv: Path: ~/.pyenv/libexec/pyenv
  - INFO: pyenv: $PYENV_ROOT is not set. Infer from `pyenv root`.
  - INFO: pyenv: Root: ~/.pyenv
  - INFO: Using: g:python_host_prog = "/usr/bin/python2"
  - WARNING: $VIRTUAL_ENV exists but appears to be inactive. This could lead to unexpected results.
    - ADVICE:
      - If you are using Zsh, see: http://vi.stackexchange.com/a/7654
  - INFO: Executable: /usr/bin/python2
  - INFO: Python version: 2.7.15
  - INFO: pynvim version: 0.3.2

## Python 3 provider (optional)
  - INFO: pyenv: Path: ~/.pyenv/libexec/pyenv
  - INFO: pyenv: $PYENV_ROOT is not set. Infer from `pyenv root`.
  - INFO: pyenv: Root: ~/.pyenv
  - INFO: Using: g:python3_host_prog = "/usr/bin/python3"
  - WARNING: $VIRTUAL_ENV exists but appears to be inactive. This could lead to unexpected results.
    - ADVICE:
      - If you are using Zsh, see: http://vi.stackexchange.com/a/7654
  - INFO: Executable: /usr/bin/python3
  - INFO: Python version: 3.6.8
  - INFO: pynvim version: 0.3.2

I don't understand the warning WARNING: $VIRTUAL_ENV exists but appears to be inactive. This could lead to unexpected results.. I have activated the environment with source .venv/bin/activate as usual. I am using zsh but the advice didn't help.

If I unset g:python3_host_prog and let nvim figure out the executable, it finds the correct version from pyenv root, but it doesn't solve the problem (also I would have to install pynvim for every version I have installed).

blueyed · 2019-09-16T14:42:45Z

WARNING: $VIRTUAL_ENV exists but appears to be inactive. This could lead to unexpected results

This means it is not using the virtualenv's python (which might be fine of course).

As for jedi-vim: there's a host Python (the one Neovim uses), and it will use virtualenvs then.
Jedi/jedi-vim uses /tmp/myapp/.venv/bin/python - what completions are you missing from there?

blueyed · 2019-09-16T14:44:08Z

maybe you are running into an issue where the host (3.6) cannot handle syntax for the venv (3.7)?

bo5o · 2019-09-16T15:29:16Z

I use a fresh Python executable (no site packages) installed with pyenv (not system python) to create a virtual environment (e.g. /tmp/myapp/.venv).
I install packages in this environment.
I open a fresh python file and type import ... (no special syntax) but my packages are not listed in the completions.

The only packages I do see, are the ones I have installed with my system python, which is a different version of python and was not used to create the virtual environment and should therefore be out of scope.

bo5o · 2019-09-16T15:37:08Z

FYI: Completion works great in ipython with the same version of jedi and the exact same environment setup.

blueyed · 2019-09-16T17:10:49Z

@cbows
And JediDebugInfo says that it uses your env then?

From your initial output (where it looks like it), but please make sure that's still the case:

Jedi environment: <Environment: 3.7.4 in /tmp/myapp/.venv>

blueyed · 2019-09-16T17:11:22Z

@cbows
For what it's worth, this would have been better off in a new issue.

bo5o · 2019-09-16T17:18:15Z

The JediDebugInfo of my initial post applies to the very scenario I just depicted.

I will file a new issue with a detailed description tomorrow.

bo5o · 2019-09-17T07:07:51Z

@blueyed
I did some further investigation and the problem does not seem to be caused by jedi-vim.
With the following vimrc, completions work fine.

call plug#begin('~/.local/share/nvim/plugged')
Plug 'davidhalter/jedi-vim'
call plug#end()

set nocompatible

let script_dir = fnamemodify(expand('<sfile>'), ':h')
let &runtimepath .= ','.script_dir.','.script_dir.'/after'

syntax on
filetype plugin indent on

So the issue does not exist as far as jedi-vim is concerned.
I previously said

I am using ALE to provide completions.

This is of course nonsense. I mixed up names, sorry. What I actually meant was: I am using ncm2/ncm2-jedi to provide completions. And the problem only arises if I add this plugin to the vimrc. I filed an issue on their tracker and hope I will get some answer. If you have any further ideas, they are greatly appreciated. Otherwise, I thank you for your help so far.

blueyed · 2019-09-17T08:51:24Z

@cbows
Ok, cool. For reference: ncm2/ncm2-jedi#23

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

jedi-vim is not aware of pyenv path #869

jedi-vim is not aware of pyenv path #869

frank777777777 commented Aug 6, 2018

blueyed commented Aug 6, 2018

frank777777777 commented Aug 6, 2018

blueyed commented Aug 6, 2018

blueyed commented Aug 6, 2018

frank777777777 commented Aug 6, 2018

blueyed commented Aug 6, 2018

frank777777777 commented Aug 6, 2018

blueyed commented Aug 6, 2018

blueyed commented Aug 13, 2018 •

edited

Loading

statham-stone commented Jul 22, 2019

bo5o commented Sep 16, 2019

blueyed commented Sep 16, 2019

bo5o commented Sep 16, 2019

blueyed commented Sep 16, 2019 •

edited

Loading

blueyed commented Sep 16, 2019

bo5o commented Sep 16, 2019

bo5o commented Sep 16, 2019

blueyed commented Sep 16, 2019

Jedi environment: <Environment: 3.7.4 in /tmp/myapp/.venv>

blueyed commented Sep 16, 2019

bo5o commented Sep 16, 2019

bo5o commented Sep 17, 2019

blueyed commented Sep 17, 2019

jedi-vim is not aware of pyenv path #869

jedi-vim is not aware of pyenv path #869

Comments

frank777777777 commented Aug 6, 2018

Issue

Steps to reproduce

Output of “:verbose JediDebugInfo”

blueyed commented Aug 6, 2018

frank777777777 commented Aug 6, 2018

blueyed commented Aug 6, 2018

blueyed commented Aug 6, 2018

frank777777777 commented Aug 6, 2018

blueyed commented Aug 6, 2018

frank777777777 commented Aug 6, 2018

blueyed commented Aug 6, 2018

blueyed commented Aug 13, 2018 • edited Loading

statham-stone commented Jul 22, 2019

bo5o commented Sep 16, 2019

:verbose JediDebugInfo

blueyed commented Sep 16, 2019

bo5o commented Sep 16, 2019

blueyed commented Sep 16, 2019 • edited Loading

blueyed commented Sep 16, 2019

bo5o commented Sep 16, 2019

bo5o commented Sep 16, 2019

blueyed commented Sep 16, 2019

Jedi environment: <Environment: 3.7.4 in /tmp/myapp/.venv>

blueyed commented Sep 16, 2019

bo5o commented Sep 16, 2019

bo5o commented Sep 17, 2019

blueyed commented Sep 17, 2019

blueyed commented Aug 13, 2018 •

edited

Loading

`:verbose JediDebugInfo`

blueyed commented Sep 16, 2019 •

edited

Loading