i. sorting networks · batcher’s sorting network counting networks i. sorting networks...

I. Sorting NetworksThomas Sauerwald

Easter 2015

Outline

Outline of this Course

Introduction to Sorting Networks

Batcher’s Sorting Network

Counting Networks

I. Sorting Networks Outline of this Course 2

(Tentative) List of Topics

Algorithms (I, II) Complexity Theory Advanced Algorithms

I. Sorting Networks (Sorting, Counting, Load Balancing)II. Matrix Multiplication (Serial and Parallel)III. Linear Programming (Formulating, Applying and Solving)IV. Approximation Algorithms: Covering ProblemsV. Approximation Algorithms via Exact AlgorithmsVI. Approximation Algorithms: Travelling Salesman ProblemVII. Approximation Algorithms: Randomisation and RoundingVIII. Approximation Algorithms: MAX-CUT Problem

Closely follow the book and use the samenumberring of theorems/lemmas etc.

Outline

Counting Networks

I. Sorting Networks Introduction to Sorting Networks 2

Overview: Sorting Networks

we already know several (comparison-based) sorting algorithms:Insertion sort, Bubble sort, Merge sort, Quick sort, Heap sort

execute one operation at a time

can handle arbitrarily large inputs

sequence of comparisons is not set in advance

(Serial) Sorting Algorithms

only perform comparisons

can only handle inputs of a fixed size

sequence of comparisons is set in advance

Comparisons can be performed in parallel

Sorting Networks

Allows to sort n numbersin sublinear time!

Simple concept, but surprisingly deep and complex theory!

Sorting Networks

Comparison Networks

A comparison network consists solely of wires and comparators:

comparator is a device with, on given two inputs, x and y , returns twooutputs x ′ and y ′wire connect output of one comparator to the input of anotherspecial wires: n input wires a1, a2, . . . , an and n output wires b1, b2, . . . , bn

Comparison Network

27.1 Comparison networks 705

comparator

(a) (b)

x ! = min(x, y)x ! = min(x, y)

y! = max(x, y)y! = max(x, y)

Figure 27.1 (a) A comparator with inputs x and y and outputs x ! and y!. (b) The same comparator,drawn as a single vertical line. Inputs x = 7, y = 3 and outputs x ! = 3, y! = 7 are shown.

A comparison network is composed solely of wires and comparators. A compara-tor, shown in Figure 27.1(a), is a device with two inputs, x and y, and two outputs,x ! and y!, that performs the following function:

x ! = min(x, y) ,

y! = max(x, y) .

Because the pictorial representation of a comparator in Figure 27.1(a) is toobulky for our purposes, we shall adopt the convention of drawing comparators assingle vertical lines, as shown in Figure 27.1(b). Inputs appear on the left andoutputs on the right, with the smaller input value appearing on the top output andthe larger input value appearing on the bottom output. We can thus think of acomparator as sorting its two inputs.

We shall assume that each comparator operates in O(1) time. In other words,we assume that the time between the appearance of the input values x and y andthe production of the output values x ! and y! is a constant.

A wire transmits a value from place to place. Wires can connect the outputof one comparator to the input of another, but otherwise they are either networkinput wires or network output wires. Throughout this chapter, we shall assumethat a comparison network contains n input wires a1, a2, . . . , an , through whichthe values to be sorted enter the network, and n output wires b1, b2, . . . , bn , whichproduce the results computed by the network. Also, we shall speak of the inputsequence "a1, a2, . . . , an# and the output sequence "b1, b2, . . . , bn#, referring tothe values on the input and output wires. That is, we use the same name for both awire and the value it carries. Our intention will always be clear from the context.

Figure 27.2 shows a comparison network, which is a set of comparators inter-connected by wires. We draw a comparison network on n inputs as a collectionof n horizontal lines with comparators stretched vertically. Note that a line doesnot represent a single wire, but rather a sequence of distinct wires connecting vari-ous comparators. The top line in Figure 27.2, for example, represents three wires:input wire a1, which connects to an input of comparator A; a wire connecting thetop output of comparator A to an input of comparator C; and output wire b1, whichcomes from the top output of comparator C . Each comparator input is connected

operates in O(1)

Convention: use the same name for both a wire and its value.

A sorting network is a comparison network whichworks correctly (that is, it sorts every input)

Comparison Networks

A comparison network consists solely of wires and comparators:comparator is a device with, on given two inputs, x and y , returns twooutputs x ′ and y ′

wire connect output of one comparator to the input of anotherspecial wires: n input wires a1, a2, . . . , an and n output wires b1, b2, . . . , bn

Comparison Network

comparator

(a) (b)

x ! = min(x, y) ,

y! = max(x, y) .

operates in O(1)

Comparison Networks

A comparison network consists solely of wires and comparators:comparator is a device with, on given two inputs, x and y , returns twooutputs x ′ and y ′

wire connect output of one comparator to the input of anotherspecial wires: n input wires a1, a2, . . . , an and n output wires b1, b2, . . . , bn

Comparison Network

comparator

(a) (b)

x ! = min(x, y) ,

y! = max(x, y) .

operates in O(1)

Comparison Networks

A comparison network consists solely of wires and comparators:comparator is a device with, on given two inputs, x and y , returns twooutputs x ′ and y ′wire connect output of one comparator to the input of another

special wires: n input wires a1, a2, . . . , an and n output wires b1, b2, . . . , bn

Comparison Network

comparator

(a) (b)

x ! = min(x, y) ,

y! = max(x, y) .

operates in O(1)

Comparison Networks

A comparison network consists solely of wires and comparators:comparator is a device with, on given two inputs, x and y , returns twooutputs x ′ and y ′wire connect output of one comparator to the input of anotherspecial wires: n input wires a1, a2, . . . , an and n output wires b1, b2, . . . , bn

Comparison Network

comparator

(a) (b)

x ! = min(x, y) ,

y! = max(x, y) .

operates in O(1)

Comparison Networks

Comparison Network

comparator

(a) (b)

x ! = min(x, y) ,

y! = max(x, y) .

operates in O(1)

Comparison Networks

Comparison Network

comparator

(a) (b)

x ! = min(x, y) ,

y! = max(x, y) .

operates in O(1)

Example of a Comparison Network (Figure 27.2)

A horizontal line representsa sequence of distinct wires

This network is in fact a sorting network!

depth 0 1 1 2 2 3

Depth of a wire:

Input wire has depth 0

If a comparator has two inputs of depths dx and dy , then outputs havedepth max{dx , dy}+ 1

Maximum depth of an outputwire equals total running time

Interconnections between comparatorsmust be acyclic

Tracing back a path must never cycle back onitself and go through the same comparator twice.

depth 0 1 1 2 2 3

Depth of a wire:

depth 0 1 1 2 2 3

Depth of a wire:

depth 0 1 1 2 2 3

Depth of a wire:

depth 0 1 1 2 2 3

Depth of a wire:

depth 0 1 1 2 2 3

Depth of a wire:

depth 0 1 1 2 2 3

Depth of a wire:

depth 0 1 1 2 2 3

Depth of a wire:

depth 0 1 1 2 2 3

Depth of a wire:

depth 0 1 1 2 2 3

Depth of a wire:

depth 0 1 1 2 2 3

Depth of a wire:

depth 0 1 1 2 2 3

Depth of a wire:

depth 0 1 1 2 2 3

Depth of a wire:

Interconnections between comparatorsmust be acyclic X

depth 0 1 1 2 2 3

Depth of a wire:

depth 0 1 1 2 2 3

Depth of a wire:

depth 0 1 1 2 2 3

Depth of a wire:

depth 0 1 1 2 2 3

Depth of a wire:

depth 0 1 1 2 2 3

Depth of a wire:

depth 0 1 1 2 2 3

Depth of a wire:

depth 0 1 1 2 2 3

Depth of a wire:

depth 0 1 1 2 2 3

Depth of a wire:

depth 0 1 1 2 2 3

Depth of a wire:

depth 0 1 1 2 2 3

Depth of a wire:

depth 0 1 1 2 2 3

Depth of a wire:

depth 0 1 1 2 2 3

Depth of a wire:

depth 0 1 1 2 2 3

Depth of a wire:

depth 0 1 1 2 2 3

Depth of a wire:

depth 0 1 1 2 2 3

Depth of a wire:Input wire has depth 0

depth 0 1 1 2 2 3

0 1 1 2 2 3

depth 0

1 1 2 2 3

depth 0 1

1 2 2 3

depth 0 1 1

depth 0 1 1 2

depth 0 1 1 2 2

depth 0 1 1 2 2 3

Zero-One Principle

Zero-One Principle: A sorting networks works correctly on arbitrary in-puts if it works correctly on binary inputs.

If a comparison network transforms the input a = 〈a1, a2, . . . , an〉 intothe output b = 〈b1, b2, . . . , bn〉, then for any monotonically increasingfunction f , the network transforms f (a) = 〈f (a1), f (a2), . . . , f (an)〉 intof (b) = 〈f (b1), f (b2), . . . , f (bn)〉.

Lemma 27.1

Zero-One Principle

Lemma 27.1

Zero-One Principle

Lemma 27.1

710 Chapter 27 Sorting Networks

min( f (x), f (y)) = f (min(x, y))

max( f (x), f (y)) = f (max(x, y))

Figure 27.4 The operation of the comparator in the proof of Lemma 27.1. The function f ismonotonically increasing.

To prove the claim, consider a comparator whose input values are x and y. Theupper output of the comparator is min(x, y) and the lower output is max(x, y).Suppose we now apply f (x) and f (y) to the inputs of the comparator, as is shownin Figure 27.4. The operation of the comparator yields the value min( f (x), f (y))on the upper output and the value max( f (x), f (y)) on the lower output. Since fis monotonically increasing, x ! y implies f (x) ! f (y). Consequently, we havethe identities

min( f (x), f (y)) = f (min(x, y)) ,

max( f (x), f (y)) = f (max(x, y)) .

Thus, the comparator produces the values f (min(x, y)) and f (max(x, y)) whenf (x) and f (y) are its inputs, which completes the proof of the claim.We can use induction on the depth of each wire in a general comparison network

to prove a stronger result than the statement of the lemma: if a wire assumes thevalue ai when the input sequence a is applied to the network, then it assumes thevalue f (ai) when the input sequence f (a) is applied. Because the output wires areincluded in this statement, proving it will prove the lemma.For the basis, consider a wire at depth 0, that is, an input wire ai . The result

follows trivially: when f (a) is applied to the network, the input wire carries f (ai).For the inductive step, consider a wire at depth d, where d " 1. The wire is theoutput of a comparator at depth d, and the input wires to this comparator are at adepth strictly less than d. By the inductive hypothesis, therefore, if the input wiresto the comparator carry values ai and a j when the input sequence a is applied,then they carry f (ai ) and f (a j ) when the input sequence f (a) is applied. Byour earlier claim, the output wires of this comparator then carry f (min(ai , a j ))and f (max(ai , a j )). Since they carry min(ai , a j ) and max(ai , a j ) when the inputsequence is a, the lemma is proved.

As an example of the application of Lemma 27.1, Figure 27.5(b) shows the sort-ing network from Figure 27.2 (repeated in Figure 27.5(a)) with the monotonicallyincreasing function f (x) = #x/2$ applied to the inputs. The value on every wireis f applied to the value on the same wire in Figure 27.2.When a comparison network is a sorting network, Lemma 27.1 allows us to

prove the following remarkable result.

Zero-One Principle

Lemma 27.1

If a comparison network with n inputs sorts all 2n possible sequencesof 0’s and 1’s correctly, then it sorts all sequences of arbitrary numberscorrectly.

Theorem 27.2 (Zero-One Principle)

Proof of the Zero-One Principle

Proof:

For the sake of contradiction, suppose the network does not correctly sort.

Let a = 〈a1, a2, . . . , an〉 be the input with ai < aj , but the network places aj

before ai in the output

Define a monotonically increasing function f as:

f (x) =

{0 if x ≤ ai ,1 if x > ai .

Since the network places aj before ai , by the previous lemma

⇒ f (aj ) is placed before f (ai )

But f (aj ) = 1 and f (ai ) = 0, which contradicts the assumption that thenetwork sorts all sequences of 0’s and 1’s correctly

Proof:

f (x) =

{0 if x ≤ ai ,1 if x > ai .

Proof:

f (x) =

{0 if x ≤ ai ,1 if x > ai .

Proof:

f (x) =

{0 if x ≤ ai ,1 if x > ai .

Proof:

f (x) =

{0 if x ≤ ai ,1 if x > ai .

Proof:

f (x) =

{0 if x ≤ ai ,1 if x > ai .

Proof:

f (x) =

{0 if x ≤ ai ,1 if x > ai .

Proof:

f (x) =

{0 if x ≤ ai ,1 if x > ai .

Since the network places aj before ai , by the previous lemma⇒ f (aj ) is placed before f (ai )

Proof:

f (x) =

{0 if x ≤ ai ,1 if x > ai .

Since the network places aj before ai , by the previous lemma⇒ f (aj ) is placed before f (ai )

Some Basic (Recursive) Sorting Networks

n − 1n

n-wire Sorting Network ???

Bubble Sort

n − 1n

n-wire Sorting Network

???Insertion Sort

These are Sorting Networks, but with depth Θ(n).

n − 1n

Bubble Sort

n − 1n

???Insertion Sort

n − 1n

Bubble Sort

n − 1n

n-wire Sorting Network ???

Insertion Sort

n − 1n

Bubble Sort

n − 1n

Insertion Sort

n − 1n

Bubble Sort

n − 1n

Insertion Sort

Outline

Counting Networks

I. Sorting Networks Batcher’s Sorting Network 9

Bitonic Sequences

A sequence is bitonic if it monotonically increases and then monoton-ically decreases, or can be circularly shifted to become monotonicallyincreasing and then monotonically decreasing.

Bitonic Sequence

Sequences of one or two numbers are defined to be bitonic.

Examples:

〈1, 4, 6, 8, 3, 2〉〈6, 9, 4, 2, 3, 5〉〈9, 8, 3, 2, 4, 6〉

binary sequences:

Bitonic Sequences

Bitonic Sequence

Examples:

〈1, 4, 6, 8, 3, 2〉〈6, 9, 4, 2, 3, 5〉〈9, 8, 3, 2, 4, 6〉

binary sequences:

Bitonic Sequences

Bitonic Sequence

Examples:

〈1, 4, 6, 8, 3, 2〉〈6, 9, 4, 2, 3, 5〉〈9, 8, 3, 2, 4, 6〉

binary sequences:

Bitonic Sequences

Bitonic Sequence

Examples:

〈1, 4, 6, 8, 3, 2〉 ?

〈6, 9, 4, 2, 3, 5〉〈9, 8, 3, 2, 4, 6〉

binary sequences:

Bitonic Sequences

Bitonic Sequence

Examples:

〈1, 4, 6, 8, 3, 2〉 X

〈6, 9, 4, 2, 3, 5〉〈9, 8, 3, 2, 4, 6〉

binary sequences:

Bitonic Sequences

Bitonic Sequence

Examples:

〈1, 4, 6, 8, 3, 2〉 X〈6, 9, 4, 2, 3, 5〉 ?

〈9, 8, 3, 2, 4, 6〉

binary sequences:

Bitonic Sequences

Bitonic Sequence

Examples:

〈1, 4, 6, 8, 3, 2〉 X〈6, 9, 4, 2, 3, 5〉 X

〈9, 8, 3, 2, 4, 6〉

binary sequences:

Bitonic Sequences

Bitonic Sequence

Examples:

〈1, 4, 6, 8, 3, 2〉 X〈6, 9, 4, 2, 3, 5〉 X〈9, 8, 3, 2, 4, 6〉 ?

binary sequences:

Bitonic Sequences

Bitonic Sequence

Examples:

〈1, 4, 6, 8, 3, 2〉 X〈6, 9, 4, 2, 3, 5〉 X〈9, 8, 3, 2, 4, 6〉 X

binary sequences:

Bitonic Sequences

Bitonic Sequence

Examples:

〈1, 4, 6, 8, 3, 2〉 X〈6, 9, 4, 2, 3, 5〉 X〈9, 8, 3, 2, 4, 6〉 X〈4, 5, 7, 1, 2, 6〉 ?

binary sequences:

Bitonic Sequences

Bitonic Sequence

Examples:

〈1, 4, 6, 8, 3, 2〉 X〈6, 9, 4, 2, 3, 5〉 X〈9, 8, 3, 2, 4, 6〉 X

((((((〈4, 5, 7, 1, 2, 6〉

binary sequences:

Bitonic Sequences

Bitonic Sequence

Examples:

〈1, 4, 6, 8, 3, 2〉 X〈6, 9, 4, 2, 3, 5〉 X〈9, 8, 3, 2, 4, 6〉 X

((((((〈4, 5, 7, 1, 2, 6〉binary sequences: ?

Bitonic Sequences

Bitonic Sequence

Examples:

〈1, 4, 6, 8, 3, 2〉 X〈6, 9, 4, 2, 3, 5〉 X〈9, 8, 3, 2, 4, 6〉 X

((((((〈4, 5, 7, 1, 2, 6〉binary sequences: 0i1j0k , or, 1i0j1k , for i, j, k ≥ 0.

Towards Bitonic Sorting Networks

A half-cleaner is a comparison network of depth 1 in which input wire i iscompared with wire i + n/2 for i = 1, 2, . . . , n/2.

Half-Cleaner

We always assume that n is even.

If the input to a half-cleaner is a bitonic sequence of 0’s and 1’s, then theoutput satisfies the following properties:

both the top half and the bottom half are bitonic,

every element in the top is not larger than any element in the bottom,

at least one half is clean.

Lemma 27.3

27.3 A bitonic sorting network 713

0011100

0000101

bitonic

bitonic,clean

bitonic

0011111

0010111

bitonic

bitonic,clean

bitonic

Figure 27.7 The comparison network HALF-CLEANER[8]. Two different sample zero-one inputand output values are shown. The input is assumed to be bitonic. A half-cleaner ensures that ev-ery output element of the top half is at least as small as every output element of the bottom half.Moreover, both halves are bitonic, and at least one half is clean.

even.) Figure 27.7 shows HALF-CLEANER[8], the half-cleaner with 8 inputs and8 outputs.When a bitonic sequence of 0’s and 1’s is applied as input to a half-cleaner, the

half-cleaner produces an output sequence in which smaller values are in the tophalf, larger values are in the bottom half, and both halves are bitonic. In fact, atleast one of the halves is clean—consisting of either all 0’s or all 1’s—and it is fromthis property that we derive the name “half-cleaner.” (Note that all clean sequencesare bitonic.) The next lemma proves these properties of half-cleaners.

Lemma 27.3If the input to a half-cleaner is a bitonic sequence of 0’s and 1’s, then the outputsatisfies the following properties: both the top half and the bottom half are bitonic,every element in the top half is at least as small as every element of the bottomhalf, and at least one half is clean.

Proof The comparison network HALF-CLEANER[n] compares inputs i andi + n/2 for i = 1, 2, . . . , n/2. Without loss of generality, suppose that the in-put is of the form 00 . . . 011 . . . 100 . . . 0. (The situation in which the input is ofthe form 11 . . . 100 . . . 011 . . . 1 is symmetric.) There are three possible cases de-pending upon the block of consecutive 0’s or 1’s in which the midpoint n/2 falls,and one of these cases (the one in which the midpoint occurs in the block of 1’s) isfurther split into two cases. The four cases are shown in Figure 27.8. In each caseshown, the lemma holds.

Half-Cleaner

Lemma 27.3

0011100

0000101

bitonic

bitonic,clean

bitonic

0011111

0010111

bitonic

bitonic,clean

bitonic

Half-Cleaner

Lemma 27.3

0011100

0000101

bitonic

bitonic,clean

bitonic

0011111

0010111

bitonic

bitonic,clean

bitonic

Half-Cleaner

Lemma 27.3

0011100

0000101

bitonic

bitonic,clean

bitonic

0011111

0010111

bitonic

bitonic,clean

bitonic

Half-Cleaner

Lemma 27.3

0011100

0000101

bitonic

bitonic,clean

bitonic

0011111

0010111

bitonic

bitonic,clean

bitonic

Half-Cleaner

Lemma 27.3

0011100

0000101

bitonic

bitonic,clean

bitonic

0011111

0010111

bitonic

bitonic,clean

bitonic

Half-Cleaner

Lemma 27.3

0011100

0000101

bitonic

bitonic,clean

bitonic

0011111

0010111

bitonic

bitonic,clean

bitonic

Half-Cleaner

Lemma 27.3

0011100

0000101

bitonic

bitonic,clean

bitonic

0011111

0010111

bitonic

bitonic,clean

bitonic

Proof of Lemma 27.3

W.l.o.g. assume that the input is of the form 0i1j0k , for some i, j, k ≥ 0.

This suggests a recursive approach, since it nowsuffices to sort the top and bottom half separately.

Proof of Lemma 27.3

W.l.o.g. assume that the input is of the form 0i1j0k , for some i, j, k ≥ 0.714 Chapter 27 Sorting Networks

bitonic,clean

bitonic

divide compare combine

top top

bottom bottom

1bitonic

top top

bottom bottom

0bitonic

top top

bottom bottom

0bitonic

top top

bottom bottom

bitonic,clean

bitonic

bitonic,clean

bitonic

bitonic,clean

bitonic

Figure 27.8 The possible comparisons in HALF-CLEANER[n]. The input sequence is assumedto be a bitonic sequence of 0’s and 1’s, and without loss of generality, we assume that it is of theform 00 . . . 011 . . . 100 . . . 0. Subsequences of 0’s are white, and subsequences of 1’s are gray. Wecan think of the n inputs as being divided into two halves such that for i = 1, 2, . . . , n/2, inputs iand i + n/2 are compared. (a)–(b) Cases in which the division occurs in the middle subsequenceof 1’s. (c)–(d) Cases in which the division occurs in a subsequence of 0’s. For all cases, everyelement in the top half of the output is at least as small as every element in the bottom half, bothhalves are bitonic, and at least one half is clean.

Proof of Lemma 27.3

W.l.o.g. assume that the input is of the form 0i1j0k , for some i, j, k ≥ 0.714 Chapter 27 Sorting Networks

bitonic,clean

bitonic

top top

bottom bottom

1bitonic

top top

bottom bottom

0bitonic

top top

bottom bottom

0bitonic

top top

bottom bottom

bitonic,clean

bitonic

bitonic,clean

bitonic

bitonic,clean

bitonic

Proof of Lemma 27.3

bitonic,clean

bitonic

top top

bottom bottom

1bitonic

top top

bottom bottom

0bitonic

top top

bottom bottom

0bitonic

top top

bottom bottom

bitonic,clean

bitonic

bitonic,clean

bitonic

bitonic,clean

bitonic

Proof of Lemma 27.3

bitonic,clean

bitonic

top top

bottom bottom

1bitonic

top top

bottom bottom

0bitonic

top top

bottom bottom

0bitonic

top top

bottom bottom

bitonic,clean

bitonic

bitonic,clean

bitonic

bitonic,clean

bitonic

The Bitonic Sorter27.3 A bitonic sorting network 715

00001011

00111000

00000111

(b)(a)

bitonic sorted

BITONIC-SORTER[n/2]

HALF-CLEANER[n]

BITONIC-SORTER[n/2]

Figure 27.9 The comparison network BITONIC-SORTER[n], shown here for n = 8. (a) The re-cursive construction: HALF-CLEANER[n] followed by two copies of BITONIC-SORTER[n/2] thatoperate in parallel. (b) The network after unrolling the recursion. Each half-cleaner is shaded. Sam-ple zero-one values are shown on the wires.

The bitonic sorter

By recursively combining half-cleaners, as shown in Figure 27.9, we can builda bitonic sorter, which is a network that sorts bitonic sequences. The first stageof BITONIC-SORTER[n] consists of HALF-CLEANER[n], which, by Lemma 27.3,produces two bitonic sequences of half the size such that every element in thetop half is at least as small as every element in the bottom half. Thus, we cancomplete the sort by using two copies of BITONIC-SORTER[n/2] to sort the twohalves recursively. In Figure 27.9(a), the recursion has been shown explicitly, andin Figure 27.9(b), the recursion has been unrolled to show the progressively smallerhalf-cleaners that make up the remainder of the bitonic sorter. The depth D(n) ofBITONIC-SORTER[n] is given by the recurrence

D(n) =!0 if n = 1 ,D(n/2) + 1 if n = 2k and k ! 1 ,

whose solution is D(n) = lg n.Thus, a zero-one bitonic sequence can be sorted by BITONIC-SORTER, which

has a depth of lg n. It follows by the analog of the zero-one principle given asExercise 27.3-6 that any bitonic sequence of arbitrary numbers can be sorted bythis network.

Exercises

27.3-1How many zero-one bitonic sequences of length n are there?

Recursive Formula for depth D(n):

D(n) =

{0 if n = 1,D(n/2) + 1 if n = 2k .

Henceforth we will alwaysassume that n is a power of 2.

BITONIC-SORTER[n] has depth log n and sorts any zero-one bitonic sequence.

00001011

00111000

00000111

(b)(a)

bitonic sorted

BITONIC-SORTER[n/2]

HALF-CLEANER[n]

BITONIC-SORTER[n/2]

The bitonic sorter

Exercises

D(n) =

{0 if n = 1,D(n/2) + 1 if n = 2k .

00001011

00111000

00000111

(b)(a)

bitonic sorted

BITONIC-SORTER[n/2]

HALF-CLEANER[n]

BITONIC-SORTER[n/2]

The bitonic sorter

Exercises

D(n) =

{0 if n = 1,D(n/2) + 1 if n = 2k .

00001011

00111000

00000111

(b)(a)

bitonic sorted

BITONIC-SORTER[n/2]

HALF-CLEANER[n]

BITONIC-SORTER[n/2]

The bitonic sorter

Exercises

D(n) =

{0 if n = 1,D(n/2) + 1 if n = 2k .

Merging Networks

can merge two sorted input sequences into one sorted outputsequences

will be based on a modification of BITONIC-SORTER[n]

Merging Networks

Basic Idea:

consider two given sequences X = 00000111, Y = 00001111

concatenating X with Y R (the reversal of Y )⇒ 0000011111110000

This sequence is bitonic!

Hence in order to merge the sequences X and Y , it suf-fices to perform a bitonic sort on X concatenated with Y R .

Merging Networks

Basic Idea:

Merging Networks

Basic Idea:

Merging Networks

Basic Idea:

Merging Networks

Basic Idea:

Merging Networks

Basic Idea:

Construction of a Merging Network (1/2)

Given two sorted sequences 〈a1, a2, . . . , an/2〉 and 〈an/2+1, an/2+2, . . . , an〉

We know it suffices to bitonically sort 〈a1, a2, . . . , an/2, an, an−1, . . . , an/2+1〉Recall: first half-cleaner of BITONIC-SORTER[n] compares i and n/2 + i

⇒ First part of MERGER[n] compares inputs i and n − i for i = 1, 2, . . . , n/2Remaining part is identical to BITONIC-SORTER[n]27.4 A merging network 717

00110001

00001101

a1a2a3a4a5a6a7a8

b1b2b3b4b5b6b7b8

bitonic

sorted

00111000

00001011

a2a3a4

b1b2b3b4

bitonic

Figure 27.10 Comparing the first stage of MERGER[n] with HALF-CLEANER[n], for n = 8.(a) The first stage of MERGER[n] transforms the two monotonic input sequences !a1, a2, . . . , an/2"and !an/2+1, an/2+2, . . . , an" into two bitonic sequences !b1, b2, . . . , bn/2" and !bn/2+1, bn/2+2,. . . , bn". (b) The equivalent operation for HALF-CLEANER[n]. The bitonic input sequence!a1, a2, . . . , an/2#1, an/2, an , an#1, . . . , an/2+2, an/2+1" is transformed into the two bitonic se-quences !b1, b2, . . . , bn/2" and !bn , bn#1, . . . , bn/2+1".

We can construct MERGER[n] by modifying the first half-cleaner of BITONIC-SORTER[n]. The key is to perform the reversal of the second half of the inputsimplicitly. Given two sorted sequences !a1, a2, . . . , an/2" and !an/2+1, an/2+2,. . . , an" to be merged, we want the effect of bitonically sorting the sequence!a1, a2, . . . , an/2, an, an#1, . . . , an/2+1". Since the first half-cleaner of BITONIC-SORTER[n] compares inputs i and n/2 + i , for i = 1, 2, . . . , n/2, we make thefirst stage of the merging network compare inputs i and n # i + 1. Figure 27.10shows the correspondence. The only subtlety is that the order of the outputs fromthe bottom of the first stage of MERGER[n] are reversed compared with the orderof outputs from an ordinary half-cleaner. Since the reversal of a bitonic sequenceis bitonic, however, the top and bottom outputs of the first stage of the mergingnetwork satisfy the properties in Lemma 27.3, and thus the top and bottom can bebitonically sorted in parallel to produce the sorted output of the merging network.The resulting merging network is shown in Figure 27.11. Only the first stage of

MERGER[n] is different from BITONIC-SORTER[n]. Consequently, the depth ofMERGER[n] is lg n, the same as that of BITONIC-SORTER[n].

Exercises

27.4-1Prove an analog of the zero-one principle for merging networks. Specifically, showthat a comparison network that can merge any two monotonically increasing se-

Lemma 27.3 still applies, since the reversal of a bitonic sequence is bitonic.

Given two sorted sequences 〈a1, a2, . . . , an/2〉 and 〈an/2+1, an/2+2, . . . , an〉We know it suffices to bitonically sort 〈a1, a2, . . . , an/2, an, an−1, . . . , an/2+1〉

Recall: first half-cleaner of BITONIC-SORTER[n] compares i and n/2 + i⇒ First part of MERGER[n] compares inputs i and n − i for i = 1, 2, . . . , n/2

Remaining part is identical to BITONIC-SORTER[n]27.4 A merging network 717

00110001

00001101

a1a2a3a4a5a6a7a8

b1b2b3b4b5b6b7b8

bitonic

sorted

00111000

00001011

a2a3a4

b1b2b3b4

bitonic

Exercises

Given two sorted sequences 〈a1, a2, . . . , an/2〉 and 〈an/2+1, an/2+2, . . . , an〉We know it suffices to bitonically sort 〈a1, a2, . . . , an/2, an, an−1, . . . , an/2+1〉Recall: first half-cleaner of BITONIC-SORTER[n] compares i and n/2 + i

00110001

00001101

a1a2a3a4a5a6a7a8

b1b2b3b4b5b6b7b8

bitonic

sorted

00111000

00001011

a2a3a4

b1b2b3b4

bitonic

Exercises

⇒ First part of MERGER[n] compares inputs i and n − i for i = 1, 2, . . . , n/2

Remaining part is identical to BITONIC-SORTER[n]27.4 A merging network 717

00110001

00001101

a1a2a3a4a5a6a7a8

b1b2b3b4b5b6b7b8

bitonic

sorted

00111000

00001011

a2a3a4

b1b2b3b4

bitonic

Exercises

Remaining part is identical to BITONIC-SORTER[n]

27.4 A merging network 717

00110001

00001101

a1a2a3a4a5a6a7a8

b1b2b3b4b5b6b7b8

bitonic

sorted

00111000

00001011

a2a3a4

b1b2b3b4

bitonic

Exercises

00110001

00001101

a1a2a3a4a5a6a7a8

b1b2b3b4b5b6b7b8

bitonic

sorted

00111000

00001011

a2a3a4

b1b2b3b4

bitonic

Exercises

00110001

00001101

a1a2a3a4a5a6a7a8

b1b2b3b4b5b6b7b8

bitonic

sorted

00111000

00001011

a2a3a4

b1b2b3b4

bitonic

Exercises

00110001

00001101

a1a2a3a4a5a6a7a8

b1b2b3b4b5b6b7b8

bitonic

sorted

00111000

00001011

a2a3a4

b1b2b3b4

bitonic

Exercises

00101111

00110111

00011111

(b)(a)

sorted

BITONIC-SORTER[n/2]

Figure 27.11 A network that merges two sorted input sequences into one sorted output sequence.The network MERGER[n] can be viewed as BITONIC-SORTER[n]with the first half-cleaner altered tocompare inputs i and n! i+1 for i = 1, 2, . . . , n/2. Here, n = 8. (a) The network decomposed intothe first stage followed by two parallel copies of BITONIC-SORTER[n/2]. (b) The same network withthe recursion unrolled. Sample zero-one values are shown on the wires, and the stages are shaded.

quences of 0’s and 1’s can merge any two monotonically increasing sequences ofarbitrary numbers.

27.4-2How many different zero-one input sequences must be applied to the input of acomparison network to verify that it is a merging network?

27.4-3Show that any network that can merge 1 item with n ! 1 sorted items to produce asorted sequence of length n must have depth at least lg n.

27.4-4 !Consider a merging network with inputs a1, a2, . . . , an , for n an exact power of 2,in which the two monotonic sequences to be merged are "a1, a3, . . . , an!1# and"a2, a4, . . . , an#. Prove that the number of comparators in this kind of mergingnetwork is "(n lg n). Why is this an interesting lower bound? (Hint: Partition thecomparators into three sets.)

27.4-5 !Prove that any merging network, regardless of the order of inputs, requires"(n lg n) comparators.

Construction of a Sorting Network

1. BITONIC-SORTER[n]sorts any bitonic sequencedepth log n

2. MERGER[n]

merges two sorted input sequencesdepth log n

Main Components

SORTER[n] is defined recursively:If n = 2k , use two copies of SORTER[n/2] tosort two subsequences of length n/2 each.Then merge them using MERGER[n].If n = 1, network consists of a single wire.

can be seen as a parallel version of merge sort

00001011

00111000

00000111

(b)(a)

bitonic sorted

BITONIC-SORTER[n/2]

HALF-CLEANER[n]

BITONIC-SORTER[n/2]

The bitonic sorter

Exercises

00101111

00110111

00011111

(b)(a)

sorted

BITONIC-SORTER[n/2]

10101000

00000111

01010100

00110001

1 2 2 3 4 4 4 4 5 5 6depth(c)

(a) (b)

SORTER[n/2]

MERGER[n] MERGER[8]

MERGER[2]

SORTER[n/2]

MERGER[2]

MERGER[4]

Figure 27.12 The sorting network SORTER[n] constructed by recursively combining merging net-works. (a) The recursive construction. (b)Unrolling the recursion. (c) Replacing the MERGER boxeswith the actual merging networks. The depth of each comparator is indicated, and sample zero-onevalues are shown on the wires.

27.5-2Show that the depth of SORTER[n] is exactly (lg n)(lg n + 1)/2.

27.5-3Suppose that we have 2n elements !a1,a2, . . . ,a2n" and wish to partition them intothe n smallest and the n largest. Prove that we can do this in constant additionaldepth after separately sorting !a1, a2, . . . , an" and !an+1, an+2, . . . , a2n".

27.5-4 !Let S(k) be the depth of a sorting network with k inputs, and let M(k) be thedepth of a merging network with 2k inputs. Suppose that we have a sequence of nnumbers to be sorted and we know that every number is within k positions of itscorrect position in the sorted order. Show that we can sort the n numbers in depthS(k) + 2M(k).

2. MERGER[n]merges two sorted input sequencesdepth log n

Main Components

00001011

00111000

00000111

(b)(a)

bitonic sorted

BITONIC-SORTER[n/2]

HALF-CLEANER[n]

BITONIC-SORTER[n/2]

The bitonic sorter

Exercises

00101111

00110111

00011111

(b)(a)

sorted

BITONIC-SORTER[n/2]

10101000

00000111

01010100

00110001

1 2 2 3 4 4 4 4 5 5 6depth(c)

(a) (b)

SORTER[n/2]

MERGER[n] MERGER[8]

MERGER[2]

SORTER[n/2]

MERGER[2]

MERGER[4]

Main Components

00001011

00111000

00000111

(b)(a)

bitonic sorted

BITONIC-SORTER[n/2]

HALF-CLEANER[n]

BITONIC-SORTER[n/2]

The bitonic sorter

Exercises

00101111

00110111

00011111

(b)(a)

sorted

BITONIC-SORTER[n/2]

10101000

00000111

01010100

00110001

1 2 2 3 4 4 4 4 5 5 6depth(c)

(a) (b)

SORTER[n/2]

MERGER[n] MERGER[8]

MERGER[2]

SORTER[n/2]

MERGER[2]

MERGER[4]

Main Components

00001011

00111000

00000111

(b)(a)

bitonic sorted

BITONIC-SORTER[n/2]

HALF-CLEANER[n]

BITONIC-SORTER[n/2]

The bitonic sorter

Exercises

00101111

00110111

00011111

(b)(a)

sorted

BITONIC-SORTER[n/2]

10101000

00000111

01010100

00110001

1 2 2 3 4 4 4 4 5 5 6depth(c)

(a) (b)

SORTER[n/2]

MERGER[n] MERGER[8]

MERGER[2]

SORTER[n/2]

MERGER[2]

MERGER[4]

Unrolling the Recursion (Figure 27.12)720 Chapter 27 Sorting Networks

10101000

00000111

01010100

00110001

1 2 2 3 4 4 4 4 5 5 6depth(c)

(a) (b)

SORTER[n/2]

MERGER[n] MERGER[8]

MERGER[2]

SORTER[n/2]

MERGER[2]

MERGER[4]

10101000

00000111

01010100

00110001

1 2 2 3 4 4 4 4 5 5 6depth(c)

(a) (b)

SORTER[n/2]

MERGER[n] MERGER[8]

MERGER[2]

SORTER[n/2]

MERGER[2]

MERGER[4]

10101000

00000111

01010100

00110001

1 2 2 3 4 4 4 4 5 5 6depth(c)

(a) (b)

SORTER[n/2]

MERGER[n] MERGER[8]

MERGER[2]

SORTER[n/2]

MERGER[2]

MERGER[4]

Recursion for D(n):

D(n) =

{0 if n = 1,D(n/2) + log n if n = 2k .

Solution: D(n) = Θ(log2 n).

SORTER[n] has depth Θ(log2 n) and sorts any input.

10101000

00000111

01010100

00110001

1 2 2 3 4 4 4 4 5 5 6depth(c)

(a) (b)

SORTER[n/2]

MERGER[n] MERGER[8]

MERGER[2]

SORTER[n/2]

MERGER[2]

MERGER[4]

10101000

00000111

01010100

00110001

1 2 2 3 4 4 4 4 5 5 6depth(c)

(a) (b)

SORTER[n/2]

MERGER[n] MERGER[8]

MERGER[2]

SORTER[n/2]

MERGER[2]

MERGER[4]

10101000

00000111

01010100

00110001

1 2 2 3 4 4 4 4 5 5 6depth(c)

(a) (b)

SORTER[n/2]

MERGER[n] MERGER[8]

MERGER[2]

SORTER[n/2]

MERGER[2]

MERGER[4]

Recursion for D(n):

D(n) =

{0 if n = 1,D(n/2) + log n if n = 2k .

10101000

00000111

01010100

00110001

1 2 2 3 4 4 4 4 5 5 6depth(c)

(a) (b)

SORTER[n/2]

MERGER[n] MERGER[8]

MERGER[2]

SORTER[n/2]

MERGER[2]

MERGER[4]

10101000

00000111

01010100

00110001

1 2 2 3 4 4 4 4 5 5 6depth(c)

(a) (b)

SORTER[n/2]

MERGER[n] MERGER[8]

MERGER[2]

SORTER[n/2]

MERGER[2]

MERGER[4]

10101000

00000111

01010100

00110001

1 2 2 3 4 4 4 4 5 5 6depth(c)

(a) (b)

SORTER[n/2]

MERGER[n] MERGER[8]

MERGER[2]

SORTER[n/2]

MERGER[2]

MERGER[4]

Recursion for D(n):

D(n) =

{0 if n = 1,D(n/2) + log n if n = 2k .

10101000

00000111

01010100

00110001

1 2 2 3 4 4 4 4 5 5 6depth(c)

(a) (b)

SORTER[n/2]

MERGER[n] MERGER[8]

MERGER[2]

SORTER[n/2]

MERGER[2]

MERGER[4]

10101000

00000111

01010100

00110001

1 2 2 3 4 4 4 4 5 5 6depth(c)

(a) (b)

SORTER[n/2]

MERGER[n] MERGER[8]

MERGER[2]

SORTER[n/2]

MERGER[2]

MERGER[4]

10101000

00000111

01010100

00110001

1 2 2 3 4 4 4 4 5 5 6depth(c)

(a) (b)

SORTER[n/2]

MERGER[n] MERGER[8]

MERGER[2]

SORTER[n/2]

MERGER[2]

MERGER[4]

Recursion for D(n):

D(n) =

{0 if n = 1,D(n/2) + log n if n = 2k .

10101000

00000111

01010100

00110001

1 2 2 3 4 4 4 4 5 5 6depth(c)

(a) (b)

SORTER[n/2]

MERGER[n] MERGER[8]

MERGER[2]

SORTER[n/2]

MERGER[2]

MERGER[4]

10101000

00000111

01010100

00110001

1 2 2 3 4 4 4 4 5 5 6depth(c)

(a) (b)

SORTER[n/2]

MERGER[n] MERGER[8]

MERGER[2]

SORTER[n/2]

MERGER[2]

MERGER[4]

10101000

00000111

01010100

00110001

1 2 2 3 4 4 4 4 5 5 6depth(c)

(a) (b)

SORTER[n/2]

MERGER[n] MERGER[8]

MERGER[2]

SORTER[n/2]

MERGER[2]

MERGER[4]

Recursion for D(n):

D(n) =

{0 if n = 1,D(n/2) + log n if n = 2k .

10101000

00000111

01010100

00110001

1 2 2 3 4 4 4 4 5 5 6depth(c)

(a) (b)

SORTER[n/2]

MERGER[n] MERGER[8]

MERGER[2]

SORTER[n/2]

MERGER[2]

MERGER[4]

10101000

00000111

01010100

00110001

1 2 2 3 4 4 4 4 5 5 6depth(c)

(a) (b)

SORTER[n/2]

MERGER[n] MERGER[8]

MERGER[2]

SORTER[n/2]

MERGER[2]

MERGER[4]

10101000

00000111

01010100

00110001

1 2 2 3 4 4 4 4 5 5 6depth(c)

(a) (b)

SORTER[n/2]

MERGER[n] MERGER[8]

MERGER[2]

SORTER[n/2]

MERGER[2]

MERGER[4]

Recursion for D(n):

D(n) =

{0 if n = 1,D(n/2) + log n if n = 2k .

A Glimpse at the AKS Network

There exists a sorting network with depth O(log n).Ajtai, Komlós, Szemerédi (1983)

Quite elaborate construction, and involves huges constants.

A perfect halver is a comparator network that, given any input, places then/2 smaller keys in b1, . . . , bn/2 and the n/2 larger keys in bn/2+1, . . . , bn.

Perfect Halver

Perfect halver of depth log2 n exist yields sorting networks of depth Θ((log n)2).

An (n, ε)-approximate halver, ε < 1, is a comparator network that forevery k = 1, 2, . . . , n/2 places at most εk of its k smallest keys inbn/2+1, . . . , bn and at most εk of its k largest keys in b1, . . . , bn/2.

Approximate Halver

We will prove that such networks can be constructed in constant depth!

Perfect Halver

Approximate Halver

Perfect Halver

Approximate Halver

Perfect Halver

Approximate Halver

Perfect Halver

Approximate Halver

Perfect Halver

Approximate Halver

i. sorting networks · batcher’s sorting network counting networks i. sorting networks...

Documents

journal of international academic research for ...the...

sorting networks on fpgas - jens...

characterizing biological networks using subgraph counting

sorting in linear time lower bound for comparison-based...

tel aviv...

sorting networks on fpgas -...

outline introduction sorting networks bubble sort and its...

fcn-rlstm: deep spatio-temporal neural networks for...

radix sort counting sort overview of sorts asymptotic ...1...

counting triangles in real-world networks

searching, sampling and counting in rna and gene networks

sorting algorithmsananth/cpts223/fall2010... · e.g., quick...

enhanced 3d convolutional networks for crowd counting

a self-contained microfluidic cell sorting system for...

sorting algorithms · sorting networks: bitonic sort • a...

futurail · 2 • soil sorting systems • post sorting...

vlsi design of sorting networks in cmos...

hlz - counting and sorting machines for ubcs presentation...

randomized routing and sorting on fixed-connection...

convolutional neural networks for counting fish in ... ·...