Category "multiprocessing"

Split work between writing output from .exe and reading the output

I have a simple .exe downloaded from the apple app store. It gives real-time updates on crypto prices and their percentage change. I am extracting percentage ch

Is it more profitable to read files too large line by line or read all files in one step with pandas Dataframe, maybe?

I have run my script in an instance of 18Gb of ram, 4 CPU, and 20 Gb of a disk in both use cases My use case is (read line by line): Read line by line and proce

Multiprocessing uproot approach

I just started looking into parallelizing the analysis of rootfiles, i.e. trees. I have worked with RDataFrames where implicit multiprocessing can be enabled wi

Memory error when running scipy with multiprocess under Windows 10

I write a program that will use concurrent futures ProcessPoolExecutor. When I set the worker number to 2, it runs smooth. But after I increase the worker numbe

How to explain these multiprocessing execution results?

I met such a problem: it has some mutually independent big tasks, and each task has some following mutually independent small tasks that depend on the result of

Trying multiprocessing but the program dies before it even starts

I have this code: from multiprocessing import Process, cpu_count def readplayerinfo(): y=0 Gameon = True while Gameon: y+=1 print('

Delegate task from Multiprocess to Sub Process in Python

Hi currently I am using python telepot to: Listen to incoming message Based on the message it will do a long operation to generate some image. Problem: The mess

TypeError: cannot pickle '_thread.lock' object , EOFError: Ran out of input

Trying to use multiprocessing for running a watchdog for folder as a seperate process. The watchdog should start running as a seperate process so that other fun

How should I pass a function that runs an external file to be able to use multiprocessing?

I recently saw a module that allows me to run my code simultaneously which happens to be what I need. However, as I was testing with function, I ran into some e

Implementing multiprocessing for a nested loop calculation (python)

I am trying to implement multiprocessing to speed up a nested loop. Following is a psudo code of what happens: result = [] z = [] a1 = array of values a2 = arra

Error 409 occurred in telebot when using multiprocessing [exe file]

I want to start another process, it will start, if I write to telegram bot "start". But right after this occurring the ERROR: 2022-04-11 11:16:13,602 (__init__.

How to use onnxruntime parallel with flask?

Created a server that want to run a session of onnxruntime parallel. First question, will be used multi-threads or multi-processings? Try to use multi-threads,

Multiprocessing Pool implementation

I'm doing some intense computation and I'd like to speed up the process using all the computational power available (8 cores on my PC). I'm doing some calculati

How to share datafram from multiprocess to main process?

Here's the 2nd version coding I'm using now( it's from Booboo), it takes about 17mins to return query result, and data could be transfer to patrent process. fro

Make codes run at once for all users

Is it possible to listen to each users contract events from the database at once I've tried but it only works for one user.. Helps will be appreciated Web3 pyth

How to use multiprocessing for sequential memory hungry tasks?

I am trying to execute a series of tasks that use a lot of memory because of the objects sizes. Basically here are the steps : a = building_function() saving_to

Difference between Isolate vs Thread vs Process?

As we know flutter app runs in an isolates. Somewhere I read that isolates are not system processes. So what really an isolate is and how it's different from pr

Python multiprocessing, parent process hangs at recv after child process raises exception before send

Python is definitely one of the worst languages for parallel processing. Just finna get the parent process to throw an error when the child process fails. Inste

SLURM nodes, tasks, cores, and cpus

Would someone be able to clarify what each of these things actually are? From what I gathered, nodes are computing points within the cluster, essentially a sing

why run program with boost::process::spawn with modifed PATH environment failed?

the code as follow: #include <boost/process.hpp> #include <iostream> int main() { // cmd // std::string cmd{"g++.exe"}; // ok std::str