Introduction to Parallel Programming

using Python Presented by: Samah Gad

July 11, 2013

Friday, July 12, 13

Road MapMotivationForking ProcessesThreadsInterprocess Communication - OverviewThe multiprocessing Module - Overview

Friday, July 12, 13

Motivation

Problem:Most computers spend a lot of time doing nothing.Majority of the modern CPU’s capacity is often spent in an idle state.

Friday, July 12, 13

Motivation -Cont.Solution:

Running more than one program at a time.

Dividing the CPU attention among a set of tasks.Parallel Processing, Multiprocessing, or Multitasking.

Friday, July 12, 13

Parallel Processing in Python

Two main ways to run tasks:Process forks Spawned threads

Python built-in tools like: os.fork, threading, queue, and multiprocessing.Third Party domains offers more advanced tools.

Friday, July 12, 13

Forking ProcessesTraditional ways to structure parallel tasks.Straight forward way to start an independent program.What is forking?

Copying programs.Python Module - os.fork

Friday, July 12, 13

Example 1

Friday, July 12, 13

Example 2

Friday, July 12, 13

Threads

Another way to start activities running at the same time.Lightweight processes

Run within the same single process.

Friday, July 12, 13

Threads - Advantages:

PerformanceSimplicityShared global memoryPortability

Friday, July 12, 13

Python ModulesPython Modules:

_thread modulethreading modules

Both modules provide tools for synchronizing access to shared objects with locks.

Friday, July 12, 13

The _thread Module

Start new independent threads of execution within a process.Doesn't support OOPPlatform independent module.

Friday, July 12, 13

Example 3

Friday, July 12, 13

Example 4

Friday, July 12, 13

Synchronizing access to shared objects and names

What is the problem? Objects and namespaces in a process that span the life of threads are shared by all spawned threads.

Solution: Threads automatically come with a cross-task communications

Friday, July 12, 13

Example 5

Friday, July 12, 13

Threading Module

Internally uses the _thread module to implement objects that represent threads and common synchronization tools.Manage threads with high-level class-based objects.

Friday, July 12, 13

Interprocess Communication - Overview

Other solutions don’t support cross-program communication Sockets, Pipes, and SignalsEnable performing Inter-Process Communication (IPC)

Friday, July 12, 13

The multiprocessing Module - Overview

Provide the best of processes and threads.Platform independent.Uses processes instead of threads.Provide synchronizations tools.Leverage the capacity of multiple processors.

Friday, July 12, 13

Reference

Title: Programming PythonAuthor: Mark LutzPublisher:O'Reilly Media; Fourth Edition edition (January 7, 2011)

Friday, July 12, 13