Is there a way to take advantage of multiple CPU cores with asyncio?

Question

I've created a simple HTTP Server with python and asyncio. But, I have read that asyncio-based servers can only take advantage of one CPU core. I am trying to fix this with multiprocessing and it doesn't work. When I try to create a process, It gives me an error saying that it cannot create the process from _winapi. This is my code so far:

def serve_forever(self, host, port):
    srv, loop = self.init(host, port)
    print()
    if self.name:
        print('* Serving App {}'.format(self.name))
    print('* Serving On http://{host}:{port}'.format(host=host, port=port))
    print('* Press <CTRL-C> To Quit')
    workers = []
    try:
        for i in range(mp.cpu_count()-1):
            p = mp.Process(target=loop.run_forever)
            workers.append(p)
            p.start()
        loop.run_forever()
    except KeyboardInterrupt:
        print()
        for p in workers:
            p.terminate()
        srv.close()
        loop.run_until_complete(srv.wait_closed())
        loop.close()
        pass

By the way, the self.init function works.

Answer 1

I think, possibly, you're getting a little mixed up between parallel and concurrent programming. At first glance these may appear similar, but you'll quickly realise they are very different.

Asyncio helps with concurrency, all that means is that you can write your code in a non-blocking fashion. In other words, for I/O operations that take time to respond, such as network calls or disk access, you can have a particular piece of code not block your process as it waits for a response. This frees up CPU cycles for other async parts of your code in the same thread.

Parallel programming involves delegating small parts of some higher level task to multiple processes or threads and (usually) collecting and merging the results once they are all done.

Here's three scenarios to help differentiate:

You could write a server program such that every request received is handled by a new thread. That thread may be 100% blocking, so if it makes a network call, or reads a file from disk, it will wait until the I/O task has completed. But this is ok, because it's within it's own thread and the operating system will take care of switching which threads run when, on what cores, etc., so other threads will get a chance to run while that one is waiting for I/O. The downside to this is that there are resource overheads to threads, and the OS doesn't have perfect knowledge of what's happening within the threads, it's just doing it's best to make sure they all get a fair turn.

Another version of the server could be written in a concurrent fashion. Here only one thread is used, but the thread has detailed knowledge about what is blocking and what is executing (asyncio to the rescue), so you can write code that only handles one request at a time, but while a given request is waiting for data it lets another request do some processing, switching between tasks while others are blocked, all within the same thread/process. This is a much more efficient use of resources, but it generally only suits high I/O workloads, such as a simple server that reads/writes to a DB. It wouldn't be great for a server that has to do lots of big computations for each request as there wouldn't be I/O events mid-computation to trigger task switching.

A third scenario is where you combine these two concepts. This is useful to help scale an async server that handles a lot of I/O operations. It could also be used for one that needs to handle lots of connections and long running tasks, where tasks are delegated to threads, or other more complex configurations, but realistically it's most useful for scaling.

Asyncio has some built in support for subprocesses.

I highly recommend reading this article, it's very good.