Hank Childs, University of OregonJan. 30th, 2014

CIS 610: Vector Models for Data-Parallel Computing

Class Organization

• Who will lecture next week?

• Alternate ideas for organizing lectures?

Material for Today’s Lecture

A.K.A, “Data Parallel Bible” according to Chris Sewell

Material & figures in this lecture almost

uniformly come from this dissertation

History Lesson via Google

Scholar

Machine Models

• Machine models you likely know about– Random access machine (RAM) model– Turing machine model

• This thesis defines a new model: parallel vector model – Defined in terms of machine architecture

Parallel Vector Models

Machine specifics

• V-RAM = S-RAM + vector memory & vector processor

• Each instruction of the vector processor operates on– Fixed number of vectors from the vector memory– Possibly scalars from scalar memory

• Example vector instruction: sum the elements of two vectors

Premise

• Claim: parallel vector models are the right abstraction to connect theory, languages, and architecture

• Specifically:– Can be mapped onto a broad variety of architectures– Can serve as algorithmic models to analyze

complexity– Can serve as instruction sets for virtual machine for

higher-level programming languages

Complexity

• Two main notions of complexity– Step complexity• # of steps executed by a program

– Element complexity• Sum over all steps of the lengths of all vectors

manipulated in each step

• Important: all data-parallel primitives considered have to finish in O(log N) time for N elements.

Complexity Example

• Step #1:– Sum two vectors of size 10

• Step #2:– Square vector of length 100

What is Step Complexity?What is Element Complexity?

2120

Some New Vector Instructions…

Scan Instruction

Segmented Instructions

It is a little fuzzy to me how to do this in practice…

Quicksort

Quicksort pseudo-code

OK, let’s try to do quicksort.

Rest of Book

Rest of Book