'How do I make my plugins work with `set encoding=utf-8` in Vim run from Windows PowerShell?

What I want

I want to use the a plugin for Vim and I use the famous plugin manager pathogen for this (https://github.com/tpope/vim-pathogen).

The plugin is vim-ps1 (https://github.com/PProvost/vim-ps1) and provides syntax coloring for Windows Powershell files whose file extension is *.ps1.

My config

In order to install my plugin, I just cloned it into the bundle directory. I'm on my office computer which runs Windows, so my vimfiles directory (instead of the .vim linux version) looks like this:

    vimfiles
     ├── autoload
     │    └── pathogen.vim
     ├── bundle
     │    ├── nerdtree
     │    └── vim-ps1
     ├── spell
     ├── template
     └── vimrc

So there are two plugins, vim-ps1 which I want to install, and nerdtree, added there only to be sure my problems aren't from the plugin (I made the assumption that there is particular problem with nerdtree).

The vimrc file is loaded by the _vimrc file because I want to version my settings. More information about this here https://stackoverflow.com/a/18203545/7964523

    runtime vimrc

The problem

With `set encoding=utf-8`, I can't use plugins

When I open a *.sp1 nothing happens. Here is the content of my vimrc file for the debugging:

    runtime! debian.vim
    set nocompatible

    execute pathogen#infect("~/vimfiles/bundle/{}")
    syntax on
    filetype plugin indent on

    set encoding=utf-8

The output of :scriptnames is

      1: ~\_vimrc
      2: ~\vimfiles\vimrc
      3: ~\vimfiles\autoload\pathogen.vim
      4: C:\Program Files (x86)\Vim\vim81\syntax\syntax.vim
      5: C:\Program Files (x86)\Vim\vim81\syntax\synload.vim
      6: C:\Program Files (x86)\Vim\vim81\syntax\syncolor.vim
      7: C:\Program Files (x86)\Vim\vim81\filetype.vim
      8: ~\vimfiles\bundle\vim-ps1\ftdetect\ps1.vim
      9: ~\vimfiles\bundle\vim-ps1\ftdetect\ps1xml.vim
     10: ~\vimfiles\bundle\vim-ps1\ftdetect\xml.vim
     11: C:\Program Files (x86)\Vim\vim81\ftplugin.vim
     12: C:\Program Files (x86)\Vim\vim81\indent.vim
     13: C:\Program Files (x86)\Vim\vim81\plugin\getscriptPlugin.vim
     14: C:\Program Files (x86)\Vim\vim81\plugin\gzip.vim
     15: C:\Program Files (x86)\Vim\vim81\plugin\logiPat.vim
     16: C:\Program Files (x86)\Vim\vim81\plugin\manpager.vim
     17: C:\Program Files (x86)\Vim\vim81\plugin\matchparen.vim
     18: C:\Program Files (x86)\Vim\vim81\plugin\netrwPlugin.vim
     19: C:\Program Files (x86)\Vim\vim81\plugin\rrhelper.vim
     20: C:\Program Files (x86)\Vim\vim81\plugin\spellfile.vim
     21: C:\Program Files (x86)\Vim\vim81\plugin\tarPlugin.vim
     22: C:\Program Files (x86)\Vim\vim81\plugin\tohtml.vim
     23: C:\Program Files (x86)\Vim\vim81\plugin\vimballPlugin.vim
     24: C:\Program Files (x86)\Vim\vim81\plugin\zipPlugin.vim
     25: C:\Program Files (x86)\Vim\vim81\scripts.vim
     26: C:\Program Files (x86)\Vim\vim81\syntax\vim.vim
     27: C:\Program Files (x86)\Vim\vim81\ftplugin\vim.vim
     28: C:\Program Files (x86)\Vim\vim81\indent\vim.vim

You can see that there are some files from the vim-ps1 directory loaded. The output of ":set runtimepath? is

    runtimepath=
    ~\vimfiles\bundle\nerdtree,
    ~\vimfiles\bundle\vim-ps1,
    ~/vimfiles,
    C:\Program Files (x86)\Vim/vimfiles,
    C:\Program Files (x86)\Vim\vim81,
    C:\Program Files (x86)\Vim/vimfiles/after,
    ~/vimfiles/after

You can also see that nerdtree and vim-sp1 folders are there, but none of them works.

Without `set encoding=utf-8`, I have trouble with characters display

Now I remove the line set encoding=utf-8 from my vimrcfile. I open the *.sp1, and the syntax coloring feature works! Nerdtree works too.

But I now have trouble with the display of some characters, like the character code ac (checked with the vim command ga) interpreted as ¬ in utf-8 and ¼ in latin-1.

The output of :scriptnames provides more lines including those related to the syntax plugin (see 46, 47, 48):

      1: ~\_vimrc
      2: ~\vimfiles\vimrc
      3: ~\vimfiles\autoload\pathogen.vim
      4: C:\Program Files (x86)\Vim\vim81\syntax\syntax.vim
      5: C:\Program Files (x86)\Vim\vim81\syntax\synload.vim
      6: C:\Program Files (x86)\Vim\vim81\syntax\syncolor.vim
      7: C:\Program Files (x86)\Vim\vim81\filetype.vim
      8: ~\vimfiles\bundle\vim-ps1\ftdetect\ps1.vim
      9: ~\vimfiles\bundle\vim-ps1\ftdetect\ps1xml.vim
     10: ~\vimfiles\bundle\vim-ps1\ftdetect\xml.vim
     11: C:\Program Files (x86)\Vim\vim81\ftplugin.vim
     12: C:\Program Files (x86)\Vim\vim81\indent.vim
     13: ~\vimfiles\bundle\nerdtree\plugin\NERD_tree.vim
     14: ~\vimfiles\bundle\nerdtree\autoload\nerdtree.vim
     15: ~\vimfiles\bundle\nerdtree\lib\nerdtree\path.vim
     16: ~\vimfiles\bundle\nerdtree\lib\nerdtree\menu_controller.vim
     17: ~\vimfiles\bundle\nerdtree\lib\nerdtree\menu_item.vim
     18: ~\vimfiles\bundle\nerdtree\lib\nerdtree\key_map.vim
     19: ~\vimfiles\bundle\nerdtree\lib\nerdtree\bookmark.vim
     20: ~\vimfiles\bundle\nerdtree\lib\nerdtree\tree_file_node.vim
     21: ~\vimfiles\bundle\nerdtree\lib\nerdtree\tree_dir_node.vim
     22: ~\vimfiles\bundle\nerdtree\lib\nerdtree\opener.vim
     23: ~\vimfiles\bundle\nerdtree\lib\nerdtree\creator.vim
     24: ~\vimfiles\bundle\nerdtree\lib\nerdtree\flag_set.vim
     25: ~\vimfiles\bundle\nerdtree\lib\nerdtree\nerdtree.vim
     26: ~\vimfiles\bundle\nerdtree\lib\nerdtree\ui.vim
     27: ~\vimfiles\bundle\nerdtree\lib\nerdtree\event.vim
     28: ~\vimfiles\bundle\nerdtree\lib\nerdtree\notifier.vim
     29: ~\vimfiles\bundle\nerdtree\autoload\nerdtree\ui_glue.vim
     30: ~\vimfiles\bundle\nerdtree\nerdtree_plugin\exec_menuitem.vim
     31: ~\vimfiles\bundle\nerdtree\nerdtree_plugin\fs_menu.vim
     32: ~\vimfiles\bundle\nerdtree\nerdtree_plugin\vcs.vim
     33: C:\Program Files (x86)\Vim\vim81\plugin\getscriptPlugin.vim
     34: C:\Program Files (x86)\Vim\vim81\plugin\gzip.vim
     35: C:\Program Files (x86)\Vim\vim81\plugin\logiPat.vim
     36: C:\Program Files (x86)\Vim\vim81\plugin\manpager.vim
     37: C:\Program Files (x86)\Vim\vim81\plugin\matchparen.vim
     38: C:\Program Files (x86)\Vim\vim81\plugin\netrwPlugin.vim
     39: C:\Program Files (x86)\Vim\vim81\plugin\rrhelper.vim
     40: C:\Program Files (x86)\Vim\vim81\plugin\spellfile.vim
     41: C:\Program Files (x86)\Vim\vim81\plugin\tarPlugin.vim
     42: C:\Program Files (x86)\Vim\vim81\plugin\tohtml.vim
     43: C:\Program Files (x86)\Vim\vim81\plugin\vimballPlugin.vim
     44: C:\Program Files (x86)\Vim\vim81\plugin\zipPlugin.vim
     45: C:\Program Files (x86)\Vim\vim81\scripts.vim
     46: ~\vimfiles\bundle\vim-ps1\syntax\ps1.vim
     47: ~\vimfiles\bundle\vim-ps1\ftplugin\ps1.vim
     48: ~\vimfiles\bundle\vim-ps1\indent\ps1.vim

No change of the output of :set runtimepath?

What I try to overcome this

I manage to identify the trigger (see above).

I moved the line set encoding=utf-8

the problem remains the same whatever its position after the line that executes pathogen

above the line that executes pathogen, I got this error:

E117: Unknown function: pathogen#infect
E15: Invalid expression: pathogen#infect("~/vimfiles/bundle/{}")

so it is worst.

How do I make my plugins work with `set encoding=utf-8`?

Edit1

As I'm running Vim 8.1, I applied the solution suggested by @Matt. The problem is still here but is not linked to pathogen.

Adding set fileencodings=utf-8,default changed nothing.

I ran from my terminal vim -Vdebug and analysed the output file debug. My user name contains the letter é coded e9 in hexadecimal unicode, and at some point in the debugging file I've got

    Searching for "C:\Users\Aur<e9>lien/vimfiles/pack/*/start/*"

See the <e9> part. It is four separate characters <, e, 9 and > (I can put my cursor over each of them).

Without set fileencodings=utf-8,default and set encoding=utf-8, the output of the debugging file is correct (except the slashes and backslashes but it works):

    Searching for "C:\Users\Aurélien/vimfiles/pack/*/start/*"

This explains why it doesn't work with set encoding=utf-8, but I can't explain this behaviour.

Solution 1:^[1]

The solution is to modify a parameter in Windows in order to use UTF-8 properly. It is under "Control Panel" > "Clock and Region" > "Region". Under the Tab "Administrative", click on "Change system locale...". You need to have administrator privileges for this.

Change setting part 1:

Then check the box before "Beta: Use Unicode UTF-8 for worldwide language support" (see the screenshot below).

Change setting part 2:

I don't know exactly what this parameter changes. There is actually a question about it on Stack Overflow: What does "Beta: Use Unicode UTF-8 for worldwide language support" actually do?. It does not have any answers yet.

An older thread on Microsoft forum does not provide more information: https://social.msdn.microsoft.com/Forums/windowsdesktop/en-US/6f9f1695-9d64-43f3-8527-6b895599233a/use-unicode-utf8-for-worldwide-language-support

Solution 2:^[2]

Look like you're running Vim 8, so you don't need Pathogen at all. Just move your plugins under ~/vimfiles/pack/bundle/start and you're done. Read :help packages for more info.

Also, in my experience setting set encoding=utf-8 as the very first line (not counting comments) in your config saves many troubles.

Solution 3:^[3]

I found a solution: running Vim from a different terminal than Windows Powershell. The terminal I used for this is the one that ships with Git for Windows (https://gitforwindows.org/). As @Matt mentioned, it is based on mintty.

Changing the code page of Windows PowerShell from the default 850 (latin-1) to 65001 (utf-8) has no effect. I don't have to use Windows PowerShell, but another solution would be helpful for people who need to.

Solution 4:^[4]

I was struggling with this problem, too (my USERNAME & USERPROFILE contain a Unicode letter). For me the following worked:

set encoding=utf-8
set fileencoding=utf-8
set termencoding=utf-8

let &runtimepath = '~/vimfiles,' . &runtimepath

" the rest of the script

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution	Source
Solution 1	Zoe stands with Ukraine
Solution 2	Matt
Solution 3	quelinau
Solution 4	Åukasz Nojek

'How do I make my plugins work with `set encoding=utf-8` in Vim run from Windows PowerShell?

What I want

My config

The problem

With set encoding=utf-8, I can't use plugins

Without set encoding=utf-8, I have trouble with characters display

What I try to overcome this

How do I make my plugins work with set encoding=utf-8?

Edit1

Solution 1:[1]

Solution 2:[2]

Solution 3:[3]

Solution 4:[4]