Fabian Stanke

More parrallel programming

As a faithful follower of the Fedora Planet, today I stumbled upon a post about parallel programming in Python. Having made similar experiences myself, I would like to add another alternative for parallel programming in Python. I could have posted this in the comments of the original post, but this way the formatting is nicer.

My point is, that Parallel Python is a really nice library, but the functionality (at least at the level demonstrated here) is also provided by the multiprocessing module included with Python.

Here is my slightly modified implementation of the same program:

#!/usr/bin/python
 
"""
Another asynchronous python example
"""
 
import multiprocessing
import time
 
def background_stuff(num):
  time.sleep(5)
  return "%s I'm done" % num
 
if __name__ == "__main__":
    print "Start at:" , time.asctime(time.localtime())
    pool = multiprocessing.Pool()
 
    print "Start doing something"
    it = pool.imap(background_stuff, [(1,), (2,)])
 
    print "Do something..."
    print " ... do something else..."
 
    print it.next()
    print it.next()
 
    print "End at:", time.asctime(time.localtime())

2 Kommentare

  1. Hey!

    Thanks for this post, I didn’t know about multiprocessing which nicely gets rid of the dependency to an external library.

    However you changed a little bit the spirit of the script. In the example I was giving you could submit two different functions easily which is no the case with imap(), but I guess there would be a way to do it by reading the module’s documentation 😉

    Anyway, thanks for the info!

  2. You’re welcome!

    Here is how it works with different functions:

    import multiprocessing
import time

def background_stuff(num):
	time.sleep(5)
	return "%s I'm done" % num

def more_background_stuff(num):
	time.sleep(5)
	return "%s I'm also done" % num

if __name__ == "__main__":
	print "Start at:" , time.asctime(time.localtime())
	pool = multiprocessing.Pool()

	print "Start doing something"
	r1 = pool.apply_async(background_stuff, (1,))
	r2 = pool.apply_async(more_background_stuff, (2,))

	print "Do something..."
	print " ... do something else..."

	print r1.get()
	print r2.get()

	print "End at:", time.asctime(time.localtime())

    Of course, reading the docs is a good idea nevertheless 😉

Kommentare sind geschlossen.