Threads Not Able To Reduce The Run Time Of Two Function When Run At Once

I have two functions f1 and f2 which increment an integer specific number of times in a loop inside these two functions. Two ways I call these functions. 1) One by one, that is fir

Solution 1:

In python, only a single thread can run at a time, because of the GIL(Global Interpreter Lock). What is a GIL?. So running threads for cpu intensive operation is not very useful in python. But threads are great for I/O. I hope, i clarified :)

Assuming python3, you could use ProcessPoolExecutor from concurrent.futures like,

$ cat cpuintense.py
import time
from concurrent.futures import ProcessPoolExecutor


deff1(a):
    for i inrange(1,100000000):
        a+=1return a

deff2(a):
    for i inrange(1,100000000):
        a+=1return a

defrun_in_sequence(a):
    start = time.time()
    f1(a)
    f2(a)
    end = time.time()
    print(f'[Sequential] Took {end-start} seconds')

defrun_in_parallel(a):
    with ProcessPoolExecutor(max_workers=2) as pool:
        start = time.time()
        fut1 = pool.submit(f1, a)
        fut2 = pool.submit(f2, a)
        for fut in (fut1, fut2):
            print(fut.result())
        end = time.time()
        print(f'[Parallel] Took {end-start} seconds')


if __name__ == '__main__':
    a = 0
    run_in_sequence(a)
    run_in_parallel(a)

Output:

$ python3 cpuintense.py
[Sequential] Took 6.838468790054321 seconds
9999999999999999
[Parallel] Took 3.488879919052124 seconds

Note: The if __name__ == '__main__' guard is required for windows. From the docs the reason is,

Since Windows lacks os.fork() it has a few extra restrictions:

Safe importing of main module

Make sure that the main module can be safely imported by a newPython interpreter without causing unintended side effects(such a starting a new process).

For example, under Windows running the following module would fail with a RuntimeError:

from multiprocessing import Process

def foo():
    print 'hello'

p = Process(target=foo)
p.start()

Instead one should protect the “entry point” of the program by using if__name__== '__main__': as follows:

from multiprocessing import Process, freeze_support

def foo():
    print 'hello'if__name__== '__main__':
    freeze_support()
    p = Process(target=foo)
    p.start()

(The freeze_support() line can be omitted if the program will be run normally instead of frozen.)

This allows the newly spawned Python interpreter to safely import the module and then run the module’s foo() function.

Similar restrictions apply if a pool or manager is created in the main module.