building a functional stream in scala

BuildingA

Functional Streamin Scala

Derek WyattTwitter: @derekwyatt

Email: derek@derekwyatt.org

A Functional Data Structure

A Functional Data StructureStreams are andinfinite lazy

A Functional Data StructureStreams are andinfinite lazy

They can be built using functions

A Functional Data StructureStreams are andinfinite lazy

They can be built using functionsDesigning one is fun and instructive

A Functional Data StructureStreams are andinfinite lazy

They can be built using functionsDesigning one is fun and instructive

I am not an FP expert (but I do play one in presentations).Much of what you see has been learned from

❊

Functional Programming in Scalaby Paul Chiusano and Rúnar Bjarnason

vs.Strictness Laziness

vs.Strictness Lazinessdef strict(row: DBRow): Unit = { if (somecondition) // do something with row}

vs.Strictness Lazinessdef strict(row: DBRow): Unit = { if (somecondition) // do something with row}

fetchRow(42)

vs.Strictness Lazinessdef strict(row: DBRow): Unit = { if (somecondition) // do something with row}

fetchRow(42)

def lazy(row: => DBRow): Unit = { if (somecondition) // do something with row}

vs.Strictness Lazinessdef strict(row: DBRow): Unit = { if (somecondition) // do something with row}

fetchRow(42)

def lazy(row: => DBRow): Unit = { if (somecondition) // do something with row}

somecond

ition =

false

vs.Strictness Lazinessdef strict(row: DBRow): Unit = { if (somecondition) // do something with row}

fetchRow(42)

def lazy(row: => DBRow): Unit = { if (somecondition) // do something with row}

somecond

ition =

false

evaluated

Not Evaluated

List: A “Strict” Data Type

List: A “Strict” Data Type(1 to 10).toList map { i => println(s”list -> map $i”) i + 10 } filter { i => println(s”list -> filter $i”) i % 2 == 0 }

[1, 2, ... 10] toList

List: A “Strict” Data Type(1 to 10).toList map { i => println(s”list -> map $i”) i + 10 } filter { i => println(s”list -> filter $i”) i % 2 == 0 }

[1, 2, ... 10] toList

[11, 12, ... 20]Map

// list -> map 1// list -> map 2// ...// list -> map 9// list -> map 10

List: A “Strict” Data Type(1 to 10).toList map { i => println(s”list -> map $i”) i + 10 } filter { i => println(s”list -> filter $i”) i % 2 == 0 }

[1, 2, ... 10] toList

[11, 12, ... 20]Map

[12, 14, ... 20]Filter

// list -> map 1// list -> map 2// ...// list -> map 9// list -> map 10

// list -> map 1// list -> map 2// ...// list -> map 9// list -> map 10// list -> filter 11// list -> filter 12// ...// list -> filter 19// list -> filter 20

Streams: Lazy Ass Seqs

Streams: Lazy Ass SeqsStream(1 to 10) map { i => println(s”stream -> map $i”) i + 10 } filter { i => println(s”stream -> filter $i”) i % 2 == 0 }

Streams: Lazy Ass SeqsStream(1 to 10) map { i => println(s”stream -> map $i”) i + 10 } filter { i => println(s”stream -> filter $i”) i % 2 == 0 }

[1, ?] Stream(1 to 10)

Streams: Lazy Ass SeqsStream(1 to 10) map { i => println(s”stream -> map $i”) i + 10 } filter { i => println(s”stream -> filter $i”) i % 2 == 0 }

[1, ?] Stream(1 to 10)

Map[11, ?]

Streams: Lazy Ass SeqsStream(1 to 10) map { i => println(s”stream -> map $i”) i + 10 } filter { i => println(s”stream -> filter $i”) i % 2 == 0 }

[1, ?] Stream(1 to 10)

The ?‘s are, essentiallyfunctions

Map[11, ?]Filter[12, ?]

Streams: Lazy Ass SeqsStream(1 to 10) map { i => println(s”stream -> map $i”) i + 10 } filter { i => println(s”stream -> filter $i”) i % 2 == 0 }

[1, ?] Stream(1 to 10)

The ?‘s are, essentiallyfunctions

Map[11, ?]Filter[12, ?]

For illustrative purposes only

Streams: Lazy Ass SeqsStream(1 to 10) map { i => println(s”stream -> map $i”) i + 10 } filter { i => println(s”stream -> filter $i”) i % 2 == 0 }

[1, ?] Stream(1 to 10)

The ?‘s are, essentiallyfunctions

Map[11, ?]Filter[12, ?]

Stream transformation does not require traversalFor illustrative purposes only

Streams: Lazy Ass SeqsStream(1 to 10) map { i => println(s”stream -> map $i”) i + 10 } filter { i => println(s”stream -> filter $i”) i % 2 == 0 }

[1, ?] Stream(1 to 10)

The ?‘s are, essentiallyfunctions

Map[11, ?]Filter[12, ?]

Stream transformation does not require traversalTransformations are applied on-demand as traversal happens

For illustrative purposes only

Stream Definition

Stream Definitiontrait Stream[+A] { def uncons: Option[(A, Stream[A])] }

Stream Definitiontrait Stream[+A] { def uncons: Option[(A, Stream[A])] }

The “data” structure

Stream Definitiontrait Stream[+A] { def uncons: Option[(A, Stream[A])] }

object Stream { def empty[A]: Stream[A] = new Stream[A] { def uncons = None } def cons[A](hd: => A, tl: => Stream[A]): Stream[A] = new Stream[A] { def uncons = Some((hd, tl)) } }

The “data” structure

Constructors

Stream Definitiontrait Stream[+A] { def uncons: Option[(A, Stream[A])] }

object Stream { def empty[A]: Stream[A] = new Stream[A] { def uncons = None } def cons[A](hd: => A, tl: => Stream[A]): Stream[A] = new Stream[A] { def uncons = Some((hd, tl)) } }

The “data” structure

Constructors

Constructing Streams

Constructing Streamsval streamOfOne = cons(1, empty[Int]) val streamOfThree = cons(1, cons(2, cons(3, empty[Int])))

Constructing Streamsval streamOfOne = cons(1, empty[Int]) val streamOfThree = cons(1, cons(2, cons(3, empty[Int])))

Well, that SUCKS...

Constructing Streamsval streamOfOne = cons(1, empty[Int]) val streamOfThree = cons(1, cons(2, cons(3, empty[Int])))

Well, that SUCKS...object Stream { def apply[A](as: A*): Stream[A] = { if (as.isEmpty) empty[A] else cons(as.head, apply(as.tail:_*)) } }

Constructing Streamsval streamOfOne = cons(1, empty[Int]) val streamOfThree = cons(1, cons(2, cons(3, empty[Int])))

Well, that SUCKS...object Stream { def apply[A](as: A*): Stream[A] = { if (as.isEmpty) empty[A] else cons(as.head, apply(as.tail:_*)) } }

val three = Stream(1, 2, 3)

The Stream... so far

The Stream... so fartrait Stream[+A] { def uncons: Option[(A, Stream[A])] } !object Stream { def empty[A]: Stream[A] = new Stream[A] { def uncons = None } def cons[A](hd: => A, tl: => Stream[A]): Stream[A] = new Stream[A] { def uncons = Some((hd, tl)) } def apply[A](as: A*): Stream[A] = { if (as.isEmpty) empty[A] else cons(as.head, apply(as.tail:_*)) } }

foldRight (does it all)(1 to 4).foldRight(z)(f)

foldRight (does it all)

f(1, f(2, f(3, f(4, z))))

(1 to 4).foldRight(z)(f)

makes

foldRight (does it all)

f(1, f(2, f(3, f(4, z))))

(1 to 4).foldRight(z)(f)

makes

f(1, ... f(1, f(2, ... f(1, f(2, f(3, ... f(1, f(2, f(3, f(4, z)))) f(1, f(2, f(3, res1))) f(1, f(2, res2)) f(1, res3) res4

or

foldRight (does it all)

f(1, f(2, f(3, f(4, z))))

(1 to 4).foldRight(z)(f)

makes

f(1, ... f(1, f(2, ... f(1, f(2, f(3, ... f(1, f(2, f(3, f(4, z)))) f(1, f(2, f(3, res1))) f(1, f(2, res2)) f(1, res3) res4

or

Builds a functional structure “to the right”, pushing successive evaluation to the second parameter.

Each function’s second parameter must be evaluated before its predecessor can be evaluated.

foldRight for Lists

foldRight for Listscase class List[+A](head: A, tail: List[A]) { def foldRight[B](z: B)(f: (A, B) => B): B = if (this.isEmpty) z else f(head, tail.foldRight(z)(f)) }

foldRight for Listscase class List[+A](head: A, tail: List[A]) { def foldRight[B](z: B)(f: (A, B) => B): B = if (this.isEmpty) z else f(head, tail.foldRight(z)(f)) }

val sum = (1 to 4).foldRight(0)(_ + _) // f(1, tail.foldRight... // f(1, f(2, tail.foldRight... // f(1, f(2, f(3, tail.foldRight... // f(1, f(2, f(3, f(4, 0)))) // f(1, f(2, f(3, 4))) // f(1, f(2, 7)) // f(1, 9) // 10

foldRight for Listscase class List[+A](head: A, tail: List[A]) { def foldRight[B](z: B)(f: (A, B) => B): B = if (this.isEmpty) z else f(head, tail.foldRight(z)(f)) }

val sum = (1 to 4).foldRight(0)(_ + _) // f(1, tail.foldRight... // f(1, f(2, tail.foldRight... // f(1, f(2, f(3, tail.foldRight... // f(1, f(2, f(3, f(4, 0)))) // f(1, f(2, f(3, 4))) // f(1, f(2, 7)) // f(1, 9) // 10

Here A and B are both Ints but they need not be. Note that full recursive expansion takes place at the call site.

Strictly Folding Right

Strictly Folding Right℥ No application of ‘f’ can complete until all of its parameters

have been applied

Strictly Folding Right℥ No application of ‘f’ can complete until all of its parameters

have been applied

℥ Strictly speaking, this means that foldRight must fully expand into a deeply nested function application

Strictly Folding Right℥ No application of ‘f’ can complete until all of its parameters

have been applied

℥ Strictly speaking, this means that foldRight must fully expand into a deeply nested function application

℥ If the collection is infinite, or even significantly large, your application is doomed

Strictly Folding Right℥ No application of ‘f’ can complete until all of its parameters

have been applied

℥ Strictly speaking, this means that foldRight must fully expand into a deeply nested function application

℥ If the collection is infinite, or even significantly large, your application is doomed

Building foldRight

trait Stream[+A] { def foldRight[B](z: ? B)(f: (A, ? B) => B): B }

Building foldRight

trait Stream[+A] { def foldRight[B](z: ? B)(f: (A, ? B) => B): B }

trait Stream[+A] { def uncons: Option[(A, Stream[A])] ! def foldRight[B](z: => B)(f: (A, => B) => B): B = uncons match { case None => z case Some((hd, tl)) => f(hd, tl.foldRight(z)(f)) } }

Building foldRight

trait Stream[+A] { def foldRight[B](z: ? B)(f: (A, ? B) => B): B }

trait Stream[+A] { def uncons: Option[(A, Stream[A])] ! def foldRight[B](z: => B)(f: (A, => B) => B): B = uncons match { case None => z case Some((hd, tl)) => f(hd, tl.foldRight(z)(f)) } }

Building foldRight

A profound change!A profound change!A profound change!A profound change!

trait Stream[+A] { def foldRight[B](z: ? B)(f: (A, ? B) => B): B }

trait Stream[+A] { def uncons: Option[(A, Stream[A])] ! def foldRight[B](z: => B)(f: (A, => B) => B): B = uncons match { case None => z case Some((hd, tl)) => f(hd, tl.foldRight(z)(f)) } }

Building foldRight

The recursive call is no longer evaluated at the call site because ‘f’ receives it by name

Learning to Relax

Learning to Relax

def foldRight[B](z: => B)(f: (A, => B) => B): B

Learning to Relax

def foldRight[B](z: => B)(f: (A, => B) => B): B

At this point, you’re potentially confused

Learning to Relax

def foldRight[B](z: => B)(f: (A, => B) => B): B

At this point, you’re potentially confused

What good is it to have a lazy => B when foldRight must return a strict B?

Learning to Relax

def foldRight[B](z: => B)(f: (A, => B) => B): B

At this point, you’re potentially confused

What good is it to have a lazy => B when foldRight must return a strict B?

def sum(ints: Stream[Int]): Int = ints.foldRight(0)(_ + _)

Learning to Relax

def foldRight[B](z: => B)(f: (A, => B) => B): B

At this point, you’re potentially confused

What good is it to have a lazy => B when foldRight must return a strict B?

def sum(ints: Stream[Int]): Int = ints.foldRight(0)(_ + _)

val neverGetsHere = sum(streamOfNaturalNumbers)

Learning to Relax

def foldRight[B](z: => B)(f: (A, => B) => B): B

At this point, you’re potentially confused

What good is it to have a lazy => B when foldRight must return a strict B?

def sum(ints: Stream[Int]): Int = ints.foldRight(0)(_ + _)

val neverGetsHere = sum(streamOfNaturalNumbers)

Int is a “STRICT” Type!

Learning to Relax

Learning to RelaxOk, I kinda lied… it’s not that Int is a “strict type”(You can, of course, do something useful with an Int… such as)

Learning to RelaxOk, I kinda lied… it’s not that Int is a “strict type”(You can, of course, do something useful with an Int… such as)

def sumToN(ints: Stream[Int], to: Int): Int = ints.take(to).foldRight(0)(_ + _)

Learning to RelaxOk, I kinda lied… it’s not that Int is a “strict type”(You can, of course, do something useful with an Int… such as)

It’s just that the strict type can cause a bit of confusion because of its non-lazy nature.

def sumToN(ints: Stream[Int], to: Int): Int = ints.take(to).foldRight(0)(_ + _)

Learning to RelaxOk, I kinda lied… it’s not that Int is a “strict type”(You can, of course, do something useful with an Int… such as)

It’s just that the strict type can cause a bit of confusion because of its non-lazy nature.

def sumToN(ints: Stream[Int], to: Int): Int = ints.take(to).foldRight(0)(_ + _)

The more “interesting” stuff, though happens when you can continue beings lazy and stay within the realm of the infinite

Learning to Relax

Learning to Relax

def foldRight[B](z: => B)(f: (A, => B) => B): B

Learning to Relax

def foldRight[B](z: => B)(f: (A, => B) => B): B

Switching to a lazy (by name) parameter gives ‘f’ control over the recursion

Learning to Relax

def foldRight[B](z: => B)(f: (A, => B) => B): B

Switching to a lazy (by name) parameter gives ‘f’ control over the recursion

‘f’ can decide to delay the execution of its second parameter, eliminating the recursive call, returning control to the caller

Learning to Relax

def foldRight[B](z: => B)(f: (A, => B) => B): B

Switching to a lazy (by name) parameter gives ‘f’ control over the recursion

However, you can’t just delay a computation without shoving it into some sort of context...

‘f’ can decide to delay the execution of its second parameter, eliminating the recursive call, returning control to the caller

Learning to Relax

def foldRight[B](z: => B)(f: (A, => B) => B): B

Switching to a lazy (by name) parameter gives ‘f’ control over the recursion

However, you can’t just delay a computation without shoving it into some sort of context...

‘f’ can decide to delay the execution of its second parameter, eliminating the recursive call, returning control to the caller

The STREAM

is that context!

A Match Made in Laziness

=> + Non-Strict Result = Lazy

Stream

Enter... Map

Enter... Mapdef map[T](f: A => T): Stream[T] = foldRight(empty[T]) { (head, tail) => cons(f(head), tail) }

Enter... Mapdef map[T](f: A => T): Stream[T] = foldRight(empty[T]) { (head, tail) => cons(f(head), tail) }

def foldRight[B](z: => B)(f: (A, => B) => B): B

Enter... Mapdef map[T](f: A => T): Stream[T] = foldRight(empty[T]) { (head, tail) => cons(f(head), tail) }

def foldRight[B](z: => B)(f: (A, => B) => B): B

Stream!Stream!

Stream!

Stream!

Enter... Filter

Enter... Filterdef filter(p: A => Boolean): Stream[A] = foldRight(empty[A]) { (head, tail) => if (p(head)) cons(head, tail) else tail }

Enter... Filterdef filter(p: A => Boolean): Stream[A] = foldRight(empty[A]) { (head, tail) => if (p(head)) cons(head, tail) else tail }

def foldRight[B](z: => B)(f: (A, => B) => B): B

Enter... Filterdef filter(p: A => Boolean): Stream[A] = foldRight(empty[A]) { (head, tail) => if (p(head)) cons(head, tail) else tail }

def foldRight[B](z: => B)(f: (A, => B) => B): B

Stream!

Stream!

Stream!Stream!

Returning Streams

Returning StreamsBoth map and filter return Streams

Returning StreamsBoth map and filter return Streams

Stream’s constructors eval neither head nor tail

Returning StreamsBoth map and filter return Streams

Stream’s constructors eval neither head nor tail

This allows for the chaining of laziness from the by name parameter of foldRight, into the return value

Returning Streams

Returning Streamsdef map[T](f: A => T): Stream[T] = foldRight(empty[T]) { (head, tail) =>

Returning Streamsdef map[T](f: A => T): Stream[T] = foldRight(empty[T]) { (head, tail) =>

Not Eval’d

Returning Streamsdef map[T](f: A => T): Stream[T] = foldRight(empty[T]) { (head, tail) => def map[T](f: A => T): Stream[T] = foldRight(empty[T]) { (head, tail) => !!!!! cons(f(head), tail) }

Not Eval’d

Not Eval’d

Returning Streamsdef map[T](f: A => T): Stream[T] = foldRight(empty[T]) { (head, tail) => def map[T](f: A => T): Stream[T] = foldRight(empty[T]) { (head, tail) => !!!!! cons(f(head), tail) }

Not Eval’d

Not Eval’d

Not Eval’d

Returning Streamsdef map[T](f: A => T): Stream[T] = foldRight(empty[T]) { (head, tail) => def map[T](f: A => T): Stream[T] = foldRight(empty[T]) { (head, tail) => !!!!! cons(f(head), tail) }

Not Eval’d

Not Eval’d

Not Eval’d

Jeez, does this code even do anything!?

Let’s find Out...

Let’s find Out...Stream(1, 2, 3, 4, 5) map { i => println(s"map -> $i") i * i }

Let’s find Out...Stream(1, 2, 3, 4, 5) map { i => println(s"map -> $i") i * i }

def map[T](f: A => T): Stream[T] = foldRight(empty[T]) { (head, tail) => cons(f(head), tail) }

Remember that...

def foldRight[B](z: => B)(f: (A, => B) => B): Band...

def cons[A](hd: => A, tl: => Stream[A]): Stream[A]and...

Which prints...

This page intentionally left blank

Putting it All Together

Putting it All Togetherval s = Stream(1 to 10) map { i => println(s”stream -> map $i”) i + 10 } filter { i => println(s”stream -> filter $i”) i % 2 == 0 }

Putting it All Togetherval s = Stream(1 to 10) map { i => println(s”stream -> map $i”) i + 10 } filter { i => println(s”stream -> filter $i”) i % 2 == 0 }

Prints nothing. OK

Putting it All Togetherval s = Stream(1 to 10) map { i => println(s”stream -> map $i”) i + 10 } filter { i => println(s”stream -> filter $i”) i % 2 == 0 }

def toList: List[A] = uncons match { case None => Nil case Some((h, t)) => h :: t.toList }

Prints nothing. OK

add toList()

Evaluating

Evaluatingval numList = s.toList

Evaluatingval numList = s.toList

// stream -> map 1// stream -> filter 11// stream -> map 2// stream -> filter 12// ...// stream -> map 10// stream -> filter 20

Evaluatingval numList = s.toList

def take(n: Int): Stream[A] = uncons match { case Some((h, t)) if n > 0 => cons(h, t.take(n - 1)) case _ => empty }

// stream -> map 1// stream -> filter 11// stream -> map 2// stream -> filter 12// ...// stream -> map 10// stream -> filter 20

Evaluatingval numList = s.toList

def take(n: Int): Stream[A] = uncons match { case Some((h, t)) if n > 0 => cons(h, t.take(n - 1)) case _ => empty }

// stream -> map 1// stream -> filter 11// stream -> map 2// stream -> filter 12// ...// stream -> map 10// stream -> filter 20

val numList = s.take(1).toList

Evaluatingval numList = s.toList

def take(n: Int): Stream[A] = uncons match { case Some((h, t)) if n > 0 => cons(h, t.take(n - 1)) case _ => empty }

// stream -> map 1// stream -> filter 11// stream -> map 2// stream -> filter 12// ...// stream -> map 10// stream -> filter 20

val numList = s.take(1).toList

// stream -> map 1// stream -> filter 11// stream -> map 2// stream -> filter 12

It’s All About Functions

It’s All About Functions

Functions hold the values

It’s All About Functions

Functions hold the values

Higher order functions pile functions on functions

It’s All About Functions

Functions hold the values

Higher order functions pile functions on functions

Even functions like take() merely store functions

It’s All About Functions

Functions hold the values

Higher order functions pile functions on functions

Even functions like take() merely store functions

Nothing “real” happens until you need it to happen

It’s All About Functions

Functions hold the values

Higher order functions pile functions on functions

Even functions like take() merely store functions

Nothing “real” happens until you need it to happen

Shiny!Shiny!Shiny!Shiny!

Derek WyattTwitter: @derekwyatt

Email: derek@derekwyatt.org

Lazy Ass Streams

Brought to you Sincere Couch Potato

semi