1 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> 2 <html xmlns="http://www.w3.org/1999/xhtml" lang="en-gb"> 3 <head> 4 <meta content="text/html; charset=UTF-8" http-equiv="content-type" /> 5 <title>pprocess - Tutorial</title> 6 </head> 7 <body> 8 9 <h1>pprocess - Tutorial</h1> 10 11 <p>The <code>pprocess</code> 12 module provides several mechanisms for running Python code concurrently 13 in several processes. The most straightforward way of making a program 14 parallel-aware - that is, where the program can take advantage of more 15 than one processor to simultaneously process data - is to use the 16 <code>pmap</code> function. Consider the following Python code:</p> 17 18 <pre> 19 t = time.time() 20 21 sequence = [] 22 for i in range(0, N): 23 for j in range(0, N): 24 sequence.append((i, j)) 25 26 results = map(calculate, sequence) 27 28 print "Time taken:", time.time() - t 29 for i in range(0, N): 30 for result in results[i*N:i*N+N]: 31 print result, 32 print 33 </pre> 34 35 <p>(This code in context with <code>import</code> statements and functions is found in the <code>examples/simple_map.py</code> file.)</p> 36 37 <p>Here, we initialise a sequence of inputs for an N by N grid, perform a 38 calculation on each element, then print the resulting sequence as a 39 grid. Since the <code>map</code> function performs the calculations sequentially, even if the <code>calculate</code> 40 function could be invoked independently for each input value, we have 41 to wait for each calculation to complete before initiating a new 42 one.</p> 43 44 <p>In order to reduce the processing time - to speed the code up, 45 in other words - we can make this code use several processes instead of 46 just one. Here is the modified code:</p> 47 48 <pre> 49 t = time.time() 50 51 sequence = [] 52 for i in range(0, N): 53 for j in range(0, N): 54 sequence.append((i, j)) 55 56 results = <strong>pprocess.pmap</strong>(calculate, sequence<strong>, limit=limit</strong>) 57 58 print "Time taken:", time.time() - t 59 for i in range(0, N): 60 for result in results[i*N:i*N+N]: 61 print result, 62 print 63 </pre> 64 65 <p>(This code in context with <code>import</code> statements and functions is found in the <code>examples/simple_pmap.py</code> file.)</p> 66 67 <p>By replacing usage of the <code>map</code> function with the <code>pprocess.pmap</code> 68 function, and specifying the limit on the number of processes to be active at any 69 given time, several calculations can now be performed in parallel.</p> 70 71 </body> 72 </html>