Restarting a thread in Python

Question

I'm trying to make threaded flight software for a project in Python 3.4, in which I need threads to restart themselves in case an I/O error occurs during a sensor read or another fluke crash like that. Therefore I am working on making a watchdog to check if threads have died and restarting them.

At first I attempted to just check if the thread was no longer alive and restart it, which did this:

>>> if not a_thread.isAlive():
...     a_thread.start()
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "c:\Python34\lib\threading.py", line 847, in start
    raise RuntimeError("threads can only be started once")
RuntimeError: threads can only be started once

This behaviour makes sense from the standpoint of threadingand Python itself, but makes my job harder. So I implemented a solution using a dictionary to store the initial thread and copy it to a new object and start it when necessary. Unfortunately this doesn't work either. Here's a basic example:

import threading
import logging
import queue
import time
from copy import copy, deepcopy

def a():
    print("I'm thread a")
def b():
    print("I'm thread b")

# Create thread objects
thread_dict = {
'a': threading.Thread(target=a, name='a'),
'b': threading.Thread(target=b, name='b')
}

threads = [copy(t) for t in thread_dict.values()]

for t in threads:
    t.start()
for i in range(len(threads)):
    if not threads[i].isAlive():
        temp = thread_dict[threads[i].name]
        threads[i] = deepcopy(temp)
        threads[i].start()
    thread(i).join(5)

which returns:

I'm thread a
I'm thread b
Traceback (most recent call last):
  File "main_test.py", line 25, in <module>
    threads[i] = deepcopy(temp)
  File "c:\Python34\lib\copy.py", line 182, in deepcopy
    y = _reconstruct(x, rv, 1, memo)
  ... (there's about 20 lines of traceback within copy)
  File "c:\Python34\lib\copyreg.py", line 88, in __newobj__
    return cls.__new__(cls, *args)
TypeError: object.__new__(_thread.lock) is not safe, use _thread.lock.__new__()

So apparently threading objects are not safe to copy... Is there anyway to restart threads short of recreating the entire object?

Answer 1

There's no reason to let your threads die.

If they're actually crashing, your whole program will crash.

If they're just raising exceptions, you can just catch the exceptions.

If they're returning normally, you can just not do that.

You can even trivially wrap a thread function to restart itself on exception or return:

def threadwrap(threadfunc):
    def wrapper():
        while True:
            try:
                threadfunc()
            except BaseException as e:
                print('{!r}; restarting thread'.format(e))
            else:
                print('exited normally, bad thread; restarting')
    return wrapper

thread_dict = {
    'a': threading.Thread(target=wrapper(a), name='a'),
    'b': threading.Thread(target=wrapper(b), name='b')
}    

Problem solved.

---

You cannot restart a thread.

Most platforms have no way to do so.

And conceptually, it doesn't make any sense. When a thread finished, its stack is dead; its parent is flagged or signaled; once it's joined, its resources are destroyed (including kernel-level resources like its process table entry). The only way to restart it would be to create a whole new set of everything. Which you can already do by creating a new thread.

So, just do it. If you really don't want to handle the exceptions internally, just store the construction arguments and use them to start a new thread.

You can even create your own subclass that hangs onto them for you:

class RestartableThread(threading.Thread):
    def __init__(self, *args, **kwargs):
        self._args, self._kwargs = args, kwargs
        super().__init__(*args, **kwargs)
    def clone(self):
        return RestartableThread(*self._args, **self._kwargs)

And now it's easy to "copy" the thread (with the semantics you wanted):

if not a_thread.is_alive():
    a_thread = a_thread.clone()

---

Yes, threading.Thread objects are not safe to copy

What would you expect to happen? At best, you'd get a different wrapper around the same OS-level thread object, so you'd fool Python into not noticing that you're trying to do the illegal, possibly segfault-inducing things it was trying to stop you from doing.

Answer 2

As "figs" said, you should rather handle the exceptions inside the Thread than trying to restart it. see the exception documentation here : https://docs.python.org/2/tutorial/errors.html

It is much more simple and pythonic to do so.

Answer 3

Here is example how to completely restart the thread. maybe it is not good solution, but it works for my purpose very well.

#!/usr/bin/python3

import threading
from time import sleep

def thread1(thread_id, number):
    thr = threading.currentThread()
    print("[thread id:%d] start function" % (thread_id))
    while True:
       if getattr(thr, "need_stop", False):
          print("[thread id:%d] Thread was stopped by external signal number"%(thread_id))
          exit(0)
       print("[thread id:%d] Number: %d<- "%(thread_id, number))
       sleep(1)

def main():
    thread_1 = []
    thread_2 = []
    for i in range(10):
       sleep(0.5)
       if i in [0,1,4,6]:
          if len(thread_2) != 0:
              for thr2 in thread_2:
                  thr2.need_stop = True
                  thr2.join()
                  thread_2.remove(thr2)

          if 'thr_1' not in locals():
              thr_1 = threading.Thread(target=thread1, args=([1, i]))
          if thr_1.is_alive() is False:
              try:
                thr_1.start()
                thread_1.append(thr_1)
              except Exception as e:
                del thr_1
                thr_1 = threading.Thread(target=thread1, args=([1, i]))
                thr_1.start()
                thread_1.append(thr_1)
       else:
          if len(thread_1) != 0:
              for thr1 in thread_1:
                  thr1.need_stop = True
                  thr1.join()
                  thread_1.remove(thr1)

          if 'thr_2' not in locals():
              thr_2 = threading.Thread(target=thread1, args=([2, i]))
          if thr_2.is_alive() is False:
              try:
                thr_2.start()
                thread_2.append(thr_2)
              except Exception as e:
                del thr_2
                thr_2 = threading.Thread(target=thread1, args=([2, i]))
                thr_2.start()
                thread_2.append(thr_2)

    # finish all threads
    if len(thread_2) != 0:
        for thr2 in thread_2:
            thr2.need_stop = True
            thr2.join()
            thread_2.remove(thr2)
    if len(thread_1) != 0:
        for thr1 in thread_1:
            thr1.need_stop = True
            thr1.join()
            thread_1.remove(thr1)
  

if __name__ == '__main__':
   main()

The gist of the code is if the thread is already running then don't touch it. In the output your can see that there are no numbers 1,3 because the thread already running. if the

output:

$ python3 test_thread.py
[thread id:1] start function
[thread id:1] Number: 0<-
[thread id:1] Number: 0<-
[thread id:1] Thread was stopped by external signal number
[thread id:2] start function
[thread id:2] Number: 2<-
[thread id:2] Number: 2<-
[thread id:2] Thread was stopped by external signal number
[thread id:1] start function
[thread id:1] Number: 4<-
[thread id:1] Thread was stopped by external signal number
[thread id:2] start function
[thread id:2] Number: 5<-
[thread id:2] Thread was stopped by external signal number
[thread id:1] start function
[thread id:1] Number: 6<-
[thread id:1] Thread was stopped by external signal number
[thread id:2] start function
[thread id:2] Number: 7<-
[thread id:2] Number: 7<-
[thread id:2] Thread was stopped by external signal number

Answer 4

If you want to check if thread got killed and then want to start the same thread , then set if first to None.

if not a_thread.isAlive():
    a_thread = None
    a_thread .Thread(target=threadFunc)
    a_thread.start()