'Joining multiple thread in Python is not blocking the main

Hello I'm trying to block the main until all the threads are done. Apparently even by executing the .join() method, the main sometime doesn't wait all the thread since I see in the output randomly one thread with the result == to none (see below):

Result of T0 =  {'values': {'thread_id': 0, 'Array_inspected': array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]), 'Status': 'SUCCESS'}}
Result of T1 =  {'values': {'thread_id': 1, 'Array_inspected': array([10, 11, 12, 13, 14, 15, 16, 17, 18, 19]), 'Status': 'SUCCESS'}}
Result of T2 =  {'values': {'thread_id': 2, 'Array_inspected': array([20, 21, 22, 23, 24, 25, 26, 27, 28, 29]), 'Status': 'SUCCESS'}}
Result of T3 =  {'values': {'thread_id': 3, 'Array_inspected': array([30, 31, 32, 33, 34, 35, 36, 37, 38, 39]), 'Status': 'SUCCESS'}}
Result of T4 =  {'values': {'thread_id': 4, 'Array_inspected': array([40, 41, 42, 43, 44, 45, 46, 47, 48, 49]), 'Status': 'SUCCESS'}}
Result of T5 =  {'values': {'thread_id': 5, 'Array_inspected': array([50, 51, 52, 53, 54, 55, 56, 57, 58, 59]), 'Status': 'SUCCESS'}}
Result of T6 =  {'values': {'thread_id': 6, 'Array_inspected': array([60, 61, 62, 63, 64, 65, 66, 67, 68, 69]), 'Status': 'SUCCESS'}}
Result of T7 =  None
Result of T8 =  {'values': {'thread_id': 8, 'Array_inspected': array([80, 81, 82, 83, 84, 85, 86, 87, 88, 89]), 'Status': 'SUCCESS'}}
Result of T9 =  {'values': {'thread_id': 9, 'Array_inspected': array([90, 91, 92, 93, 94, 95, 96, 97, 98, 99]), 'Status': 'SUCCESS'}}

This implies that the main finished while T7 was still running.

This is my source code and I'm not sure what I'm doing wrong:

from threading import Thread
from datetime import datetime
from typing import List
import time
import numpy as np

TEST_ARRAY_SIZE = 100
THREAD_POOL_SIZE = 10

def init_array() -> List:
    result = []
    for x in range(TEST_ARRAY_SIZE):
        result.append(x)
    
    return result


def do_calculations(thread_id: int, input, result):
    print(f"Starting calculation for thread id {thread_id} @{datetime.now()}")
    tmp_result = {
        "values": {
            "thread_id": thread_id,
            "Array_inspected": input,
            "Status": "SUCCESS"
            }
    }
    result.insert(thread_id, tmp_result)

start_time = datetime.now()
# print(start_time, 'Before creating tasks.')

array_to_inspect = init_array()
# print(array_to_inspect)

splits = np.array_split(array_to_inspect, THREAD_POOL_SIZE)
# print("SPLITTED", splits)

thread_pool = []
thread_result_array = [None] * THREAD_POOL_SIZE

print(splits)

for x in range(len(splits)):
    thread_pool.append(Thread(target=do_calculations, args=[x, splits[x], thread_result_array]))
    thread_pool[x].start()

for x in range(len(splits)):
    # print(thread_pool[x])
    thread_pool[x].join()

for x in range(len(splits)):
    print(f"Result of T{x} = ", thread_result_array[x])

end_time = datetime.now()

print(f'Total time elapsed: {end_time - start_time} seconds')


Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source