'On Windows why does "poetry shell" remove/lose shell history features?

In the Windows default command shell (cmd.exe) you can access the history of typed commands using various controls (up arrow for most recent command etc).

When you spawn a new virtual environment using the "poetry shell" command you have a new shell spawned, but the history commands (like up arrow and F7) are non-functional, and if you run doskey /history it gives you the command history from before you ran the "poetry shell" command.

If you run cmd.exe to spawn another shell, this also has history disabled.

Any ideas as to why this might happen or how you get a command shell with history disabled?

Thanks

windowsshellpython-poetry


Solution 1:[1]

This is definitely one of the pain points in using Poetry. I don't know what causes this. My workaround: activate the virtual environment Poetry creates manually instead. This avoids whatever Poetry does to interfere with cmd.exe's history. To do this, I ask Poetry where it created its venv, then copy and paste that plus Scripts\activate to enter a working venv. For example:

C:\>poetry env info

Virtualenv
Python:         3.9.8
Implementation: CPython
Path:           C:\Users\xxx\AppData\Local\pypoetry\Cache\virtualenvs\
runestone-poetry-project-kVed82-L-py3.9
Valid:          True

System
Platform: win32
OS:       nt
Python:   C:\Users\xxx\AppData\Local\Programs\Python\Python39

C:\>C:\Users\xxx\AppData\Local\pypoetry\Cache\virtualenvs\
runestone-poetry-project-kVed82-L-py3.9\Scripts\activate
(runestone-poetry-project-kVed82-L-py3.9) C:\>

Another workaround: use poetry run pwsh to run PowerShell instead; its history works properly (but, of course, PowerShell is very different than cmd).

Solution 2:[2]

I think the problem you are talking about should be a venv bug. You can reproduce it with the following command:

lenovo@LENOVO-PC D:\gym\works\opencv-python-demo
# .venv\Scripts\activate.bat

(.venv) lenovo@LENOVO-PC D:\gym\works\opencv-python-demo
# python test_subprocess_open_cmd.py

The contents of the "test_subprocess_open_cmd.py" file are as follows:


import os
import subprocess as subprocess

exe = subprocess.Popen([os.environ.get("COMSPEC")])
exe.communicate()
print(exe.returncode)

Similar code appears in the poetry source code.

Solution 3:[3]

This thread explains the issue nicely. Applying set_console_history_info() as defined at the bottom of the thread, I was able to do a quick proof-of-concept hack of my copy of Poetry to see if I could get a nicer cmd.exe when running poetry shell. And it seems to work.

I posted my p.o.c. here.

Solution 4:[4]

This is a know bug that has various portions of the fix scattered around different forums. Through no knowledge of my own, I combined the bits and made an easily movable fix that I use throughout my poetry projects. In your poetry project's root folder, add these files:

fix_powershell.py:

import ctypes
import collections
from ctypes import wintypes

# Powershell and Poetry don't work well together, and you lose console history (up/down arrows).
# Run this from your shell prior to launching the "poetry run python manage.py shell" and they'll work!
# https://discuss.python.org/t/interactive-command-history-in-session-started-with-subprocess-on-windows/3701/5

kernel32 = ctypes.WinDLL('kernel32', use_last_error=True)

HISTORY_NO_DUP_FLAG = 1

class CONSOLE_HISTORY_INFO(ctypes.Structure):
    _fields_ = (('cbSize', wintypes.UINT),
                ('HistoryBufferSize', wintypes.UINT),
                ('NumberOfHistoryBuffers', wintypes.UINT),
                ('dwFlags', wintypes.DWORD))

    def __init__(self, *args, **kwds):
        super().__init__(ctypes.sizeof(self), *args, **kwds)

ConsoleHistoryInfo = collections.namedtuple('ConsoleHistoryInfo', 
    'bufsize nbuf flags')

def get_console_history_info():
    info = CONSOLE_HISTORY_INFO()
    if not kernel32.GetConsoleHistoryInfo(ctypes.byref(info)):
        raise ctypes.WinError(ctypes.get_last_error())
    return ConsoleHistoryInfo(info.HistoryBufferSize, 
            info.NumberOfHistoryBuffers, info.dwFlags)

def set_console_history_info(bufsize=512, nbuf=32,
      flags=HISTORY_NO_DUP_FLAG):
    info = CONSOLE_HISTORY_INFO(bufsize, nbuf, flags)
    if not kernel32.SetConsoleHistoryInfo(ctypes.byref(info)):
        raise ctypes.WinError(ctypes.get_last_error())

print(get_console_history_info())
set_console_history_info()
print(get_console_history_info())

And shell.ps1:

python fix_powershell.py
$oldtitle = $host.ui.RawUI.WindowTitle
try
{
    $host.ui.RawUI.WindowTitle="Django Shell (running)"
    poetry run python manage.py shell
}
finally
{
    $host.ui.RawUI.WindowTitle=$oldtitle
}

Then just launch your shell with .\shell.ps1, and your history will work properly.

The poetry folk know about the issue, but seem to be of the position that it is a Microsoft issue, and (understandably) seem not to want to bloat up their project with platform specific hacks...

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 Bryan A. Jones
Solution 2 gym603
Solution 3 Jeff Robbins
Solution 4 Dan K

Related Questions