building a functional stream in scala
DESCRIPTION
I gave this presentation to my local Scala Meetup on Feb 12th 2014. It presents an aspect of functional programming by implementing a lazy data structure for storing an infinite collection of data. The act of building the stream is the point - you wouldn't use this in real life. The design for the stream is largely taken from Manning's "Functional Programming in Scala" by Paul Chiusano and Rúnar Bjarnason. Have a look at the book at http://www.manning.com/bjarnason/.TRANSCRIPT
A Functional Data Structure
A Functional Data StructureStreams are andinfinite lazy
A Functional Data StructureStreams are andinfinite lazy
They can be built using functions
A Functional Data StructureStreams are andinfinite lazy
They can be built using functionsDesigning one is fun and instructive
A Functional Data StructureStreams are andinfinite lazy
They can be built using functionsDesigning one is fun and instructive
I am not an FP expert (but I do play one in presentations).Much of what you see has been learned from
❊
Functional Programming in Scalaby Paul Chiusano and Rúnar Bjarnason
vs.Strictness Laziness
vs.Strictness Lazinessdef strict(row: DBRow): Unit = { if (somecondition) // do something with row}
vs.Strictness Lazinessdef strict(row: DBRow): Unit = { if (somecondition) // do something with row}
fetchRow(42)
vs.Strictness Lazinessdef strict(row: DBRow): Unit = { if (somecondition) // do something with row}
fetchRow(42)
def lazy(row: => DBRow): Unit = { if (somecondition) // do something with row}
vs.Strictness Lazinessdef strict(row: DBRow): Unit = { if (somecondition) // do something with row}
fetchRow(42)
def lazy(row: => DBRow): Unit = { if (somecondition) // do something with row}
somecond
ition =
false
vs.Strictness Lazinessdef strict(row: DBRow): Unit = { if (somecondition) // do something with row}
fetchRow(42)
def lazy(row: => DBRow): Unit = { if (somecondition) // do something with row}
somecond
ition =
false
evaluated
Not Evaluated
List: A “Strict” Data Type
List: A “Strict” Data Type(1 to 10).toList map { i => println(s”list -> map $i”) i + 10 } filter { i => println(s”list -> filter $i”) i % 2 == 0 }
[1, 2, ... 10] toList
List: A “Strict” Data Type(1 to 10).toList map { i => println(s”list -> map $i”) i + 10 } filter { i => println(s”list -> filter $i”) i % 2 == 0 }
[1, 2, ... 10] toList
[11, 12, ... 20]Map
// list -> map 1// list -> map 2// ...// list -> map 9// list -> map 10
List: A “Strict” Data Type(1 to 10).toList map { i => println(s”list -> map $i”) i + 10 } filter { i => println(s”list -> filter $i”) i % 2 == 0 }
[1, 2, ... 10] toList
[11, 12, ... 20]Map
[12, 14, ... 20]Filter
// list -> map 1// list -> map 2// ...// list -> map 9// list -> map 10
// list -> map 1// list -> map 2// ...// list -> map 9// list -> map 10// list -> filter 11// list -> filter 12// ...// list -> filter 19// list -> filter 20
Streams: Lazy Ass Seqs
Streams: Lazy Ass SeqsStream(1 to 10) map { i => println(s”stream -> map $i”) i + 10 } filter { i => println(s”stream -> filter $i”) i % 2 == 0 }
Streams: Lazy Ass SeqsStream(1 to 10) map { i => println(s”stream -> map $i”) i + 10 } filter { i => println(s”stream -> filter $i”) i % 2 == 0 }
[1, ?] Stream(1 to 10)
Streams: Lazy Ass SeqsStream(1 to 10) map { i => println(s”stream -> map $i”) i + 10 } filter { i => println(s”stream -> filter $i”) i % 2 == 0 }
[1, ?] Stream(1 to 10)
Map[11, ?]
Streams: Lazy Ass SeqsStream(1 to 10) map { i => println(s”stream -> map $i”) i + 10 } filter { i => println(s”stream -> filter $i”) i % 2 == 0 }
[1, ?] Stream(1 to 10)
The ?‘s are, essentiallyfunctions
Map[11, ?]Filter[12, ?]
Streams: Lazy Ass SeqsStream(1 to 10) map { i => println(s”stream -> map $i”) i + 10 } filter { i => println(s”stream -> filter $i”) i % 2 == 0 }
[1, ?] Stream(1 to 10)
The ?‘s are, essentiallyfunctions
Map[11, ?]Filter[12, ?]
For illustrative purposes only
Streams: Lazy Ass SeqsStream(1 to 10) map { i => println(s”stream -> map $i”) i + 10 } filter { i => println(s”stream -> filter $i”) i % 2 == 0 }
[1, ?] Stream(1 to 10)
The ?‘s are, essentiallyfunctions
Map[11, ?]Filter[12, ?]
Stream transformation does not require traversalFor illustrative purposes only
Streams: Lazy Ass SeqsStream(1 to 10) map { i => println(s”stream -> map $i”) i + 10 } filter { i => println(s”stream -> filter $i”) i % 2 == 0 }
[1, ?] Stream(1 to 10)
The ?‘s are, essentiallyfunctions
Map[11, ?]Filter[12, ?]
Stream transformation does not require traversalTransformations are applied on-demand as traversal happens
For illustrative purposes only
Stream Definition
Stream Definitiontrait Stream[+A] { def uncons: Option[(A, Stream[A])] }
Stream Definitiontrait Stream[+A] { def uncons: Option[(A, Stream[A])] }
The “data” structure
Stream Definitiontrait Stream[+A] { def uncons: Option[(A, Stream[A])] }
object Stream { def empty[A]: Stream[A] = new Stream[A] { def uncons = None } def cons[A](hd: => A, tl: => Stream[A]): Stream[A] = new Stream[A] { def uncons = Some((hd, tl)) } }
The “data” structure
Constructors
Stream Definitiontrait Stream[+A] { def uncons: Option[(A, Stream[A])] }
object Stream { def empty[A]: Stream[A] = new Stream[A] { def uncons = None } def cons[A](hd: => A, tl: => Stream[A]): Stream[A] = new Stream[A] { def uncons = Some((hd, tl)) } }
The “data” structure
Constructors
Constructing Streams
Constructing Streamsval streamOfOne = cons(1, empty[Int]) val streamOfThree = cons(1, cons(2, cons(3, empty[Int])))
Constructing Streamsval streamOfOne = cons(1, empty[Int]) val streamOfThree = cons(1, cons(2, cons(3, empty[Int])))
Well, that SUCKS...
Constructing Streamsval streamOfOne = cons(1, empty[Int]) val streamOfThree = cons(1, cons(2, cons(3, empty[Int])))
Well, that SUCKS...object Stream { def apply[A](as: A*): Stream[A] = { if (as.isEmpty) empty[A] else cons(as.head, apply(as.tail:_*)) } }
Constructing Streamsval streamOfOne = cons(1, empty[Int]) val streamOfThree = cons(1, cons(2, cons(3, empty[Int])))
Well, that SUCKS...object Stream { def apply[A](as: A*): Stream[A] = { if (as.isEmpty) empty[A] else cons(as.head, apply(as.tail:_*)) } }
val three = Stream(1, 2, 3)
The Stream... so far
The Stream... so fartrait Stream[+A] { def uncons: Option[(A, Stream[A])] } !object Stream { def empty[A]: Stream[A] = new Stream[A] { def uncons = None } def cons[A](hd: => A, tl: => Stream[A]): Stream[A] = new Stream[A] { def uncons = Some((hd, tl)) } def apply[A](as: A*): Stream[A] = { if (as.isEmpty) empty[A] else cons(as.head, apply(as.tail:_*)) } }
foldRight (does it all)(1 to 4).foldRight(z)(f)
foldRight (does it all)
f(1, f(2, f(3, f(4, z))))
(1 to 4).foldRight(z)(f)
makes
foldRight (does it all)
f(1, f(2, f(3, f(4, z))))
(1 to 4).foldRight(z)(f)
makes
f(1, ... f(1, f(2, ... f(1, f(2, f(3, ... f(1, f(2, f(3, f(4, z)))) f(1, f(2, f(3, res1))) f(1, f(2, res2)) f(1, res3) res4
or
foldRight (does it all)
f(1, f(2, f(3, f(4, z))))
(1 to 4).foldRight(z)(f)
makes
f(1, ... f(1, f(2, ... f(1, f(2, f(3, ... f(1, f(2, f(3, f(4, z)))) f(1, f(2, f(3, res1))) f(1, f(2, res2)) f(1, res3) res4
or
Builds a functional structure “to the right”, pushing successive evaluation to the second parameter.
Each function’s second parameter must be evaluated before its predecessor can be evaluated.
foldRight for Lists
foldRight for Listscase class List[+A](head: A, tail: List[A]) { def foldRight[B](z: B)(f: (A, B) => B): B = if (this.isEmpty) z else f(head, tail.foldRight(z)(f)) }
foldRight for Listscase class List[+A](head: A, tail: List[A]) { def foldRight[B](z: B)(f: (A, B) => B): B = if (this.isEmpty) z else f(head, tail.foldRight(z)(f)) }
val sum = (1 to 4).foldRight(0)(_ + _) // f(1, tail.foldRight... // f(1, f(2, tail.foldRight... // f(1, f(2, f(3, tail.foldRight... // f(1, f(2, f(3, f(4, 0)))) // f(1, f(2, f(3, 4))) // f(1, f(2, 7)) // f(1, 9) // 10
foldRight for Listscase class List[+A](head: A, tail: List[A]) { def foldRight[B](z: B)(f: (A, B) => B): B = if (this.isEmpty) z else f(head, tail.foldRight(z)(f)) }
val sum = (1 to 4).foldRight(0)(_ + _) // f(1, tail.foldRight... // f(1, f(2, tail.foldRight... // f(1, f(2, f(3, tail.foldRight... // f(1, f(2, f(3, f(4, 0)))) // f(1, f(2, f(3, 4))) // f(1, f(2, 7)) // f(1, 9) // 10
Here A and B are both Ints but they need not be. Note that full recursive expansion takes place at the call site.
Strictly Folding Right
Strictly Folding Right℥ No application of ‘f’ can complete until all of its parameters
have been applied
Strictly Folding Right℥ No application of ‘f’ can complete until all of its parameters
have been applied
℥ Strictly speaking, this means that foldRight must fully expand into a deeply nested function application
Strictly Folding Right℥ No application of ‘f’ can complete until all of its parameters
have been applied
℥ Strictly speaking, this means that foldRight must fully expand into a deeply nested function application
℥ If the collection is infinite, or even significantly large, your application is doomed
Strictly Folding Right℥ No application of ‘f’ can complete until all of its parameters
have been applied
℥ Strictly speaking, this means that foldRight must fully expand into a deeply nested function application
℥ If the collection is infinite, or even significantly large, your application is doomed
Building foldRight
trait Stream[+A] { def foldRight[B](z: ? B)(f: (A, ? B) => B): B }
Building foldRight
trait Stream[+A] { def foldRight[B](z: ? B)(f: (A, ? B) => B): B }
trait Stream[+A] { def uncons: Option[(A, Stream[A])] ! def foldRight[B](z: => B)(f: (A, => B) => B): B = uncons match { case None => z case Some((hd, tl)) => f(hd, tl.foldRight(z)(f)) } }
Building foldRight
trait Stream[+A] { def foldRight[B](z: ? B)(f: (A, ? B) => B): B }
trait Stream[+A] { def uncons: Option[(A, Stream[A])] ! def foldRight[B](z: => B)(f: (A, => B) => B): B = uncons match { case None => z case Some((hd, tl)) => f(hd, tl.foldRight(z)(f)) } }
Building foldRight
A profound change!A profound change!A profound change!A profound change!
trait Stream[+A] { def foldRight[B](z: ? B)(f: (A, ? B) => B): B }
trait Stream[+A] { def uncons: Option[(A, Stream[A])] ! def foldRight[B](z: => B)(f: (A, => B) => B): B = uncons match { case None => z case Some((hd, tl)) => f(hd, tl.foldRight(z)(f)) } }
Building foldRight
The recursive call is no longer evaluated at the call site because ‘f’ receives it by name
Learning to Relax
Learning to Relax
def foldRight[B](z: => B)(f: (A, => B) => B): B
Learning to Relax
def foldRight[B](z: => B)(f: (A, => B) => B): B
At this point, you’re potentially confused
Learning to Relax
def foldRight[B](z: => B)(f: (A, => B) => B): B
At this point, you’re potentially confused
What good is it to have a lazy => B when foldRight must return a strict B?
Learning to Relax
def foldRight[B](z: => B)(f: (A, => B) => B): B
At this point, you’re potentially confused
What good is it to have a lazy => B when foldRight must return a strict B?
def sum(ints: Stream[Int]): Int = ints.foldRight(0)(_ + _)
Learning to Relax
def foldRight[B](z: => B)(f: (A, => B) => B): B
At this point, you’re potentially confused
What good is it to have a lazy => B when foldRight must return a strict B?
def sum(ints: Stream[Int]): Int = ints.foldRight(0)(_ + _)
val neverGetsHere = sum(streamOfNaturalNumbers)
Learning to Relax
def foldRight[B](z: => B)(f: (A, => B) => B): B
At this point, you’re potentially confused
What good is it to have a lazy => B when foldRight must return a strict B?
def sum(ints: Stream[Int]): Int = ints.foldRight(0)(_ + _)
val neverGetsHere = sum(streamOfNaturalNumbers)
Int is a “STRICT” Type!
Learning to Relax
Learning to RelaxOk, I kinda lied… it’s not that Int is a “strict type”(You can, of course, do something useful with an Int… such as)
Learning to RelaxOk, I kinda lied… it’s not that Int is a “strict type”(You can, of course, do something useful with an Int… such as)
def sumToN(ints: Stream[Int], to: Int): Int = ints.take(to).foldRight(0)(_ + _)
Learning to RelaxOk, I kinda lied… it’s not that Int is a “strict type”(You can, of course, do something useful with an Int… such as)
It’s just that the strict type can cause a bit of confusion because of its non-lazy nature.
def sumToN(ints: Stream[Int], to: Int): Int = ints.take(to).foldRight(0)(_ + _)
Learning to RelaxOk, I kinda lied… it’s not that Int is a “strict type”(You can, of course, do something useful with an Int… such as)
It’s just that the strict type can cause a bit of confusion because of its non-lazy nature.
def sumToN(ints: Stream[Int], to: Int): Int = ints.take(to).foldRight(0)(_ + _)
The more “interesting” stuff, though happens when you can continue beings lazy and stay within the realm of the infinite
Learning to Relax
Learning to Relax
def foldRight[B](z: => B)(f: (A, => B) => B): B
Learning to Relax
def foldRight[B](z: => B)(f: (A, => B) => B): B
Switching to a lazy (by name) parameter gives ‘f’ control over the recursion
Learning to Relax
def foldRight[B](z: => B)(f: (A, => B) => B): B
Switching to a lazy (by name) parameter gives ‘f’ control over the recursion
‘f’ can decide to delay the execution of its second parameter, eliminating the recursive call, returning control to the caller
Learning to Relax
def foldRight[B](z: => B)(f: (A, => B) => B): B
Switching to a lazy (by name) parameter gives ‘f’ control over the recursion
However, you can’t just delay a computation without shoving it into some sort of context...
‘f’ can decide to delay the execution of its second parameter, eliminating the recursive call, returning control to the caller
Learning to Relax
def foldRight[B](z: => B)(f: (A, => B) => B): B
Switching to a lazy (by name) parameter gives ‘f’ control over the recursion
However, you can’t just delay a computation without shoving it into some sort of context...
‘f’ can decide to delay the execution of its second parameter, eliminating the recursive call, returning control to the caller
The STREAM
is that context!
A Match Made in Laziness
=> + Non-Strict Result = Lazy
Stream
Enter... Map
Enter... Mapdef map[T](f: A => T): Stream[T] = foldRight(empty[T]) { (head, tail) => cons(f(head), tail) }
Enter... Mapdef map[T](f: A => T): Stream[T] = foldRight(empty[T]) { (head, tail) => cons(f(head), tail) }
def foldRight[B](z: => B)(f: (A, => B) => B): B
Enter... Mapdef map[T](f: A => T): Stream[T] = foldRight(empty[T]) { (head, tail) => cons(f(head), tail) }
def foldRight[B](z: => B)(f: (A, => B) => B): B
Stream!Stream!
Stream!
Stream!
Enter... Filter
Enter... Filterdef filter(p: A => Boolean): Stream[A] = foldRight(empty[A]) { (head, tail) => if (p(head)) cons(head, tail) else tail }
Enter... Filterdef filter(p: A => Boolean): Stream[A] = foldRight(empty[A]) { (head, tail) => if (p(head)) cons(head, tail) else tail }
def foldRight[B](z: => B)(f: (A, => B) => B): B
Enter... Filterdef filter(p: A => Boolean): Stream[A] = foldRight(empty[A]) { (head, tail) => if (p(head)) cons(head, tail) else tail }
def foldRight[B](z: => B)(f: (A, => B) => B): B
Stream!
Stream!
Stream!Stream!
Returning Streams
Returning StreamsBoth map and filter return Streams
Returning StreamsBoth map and filter return Streams
Stream’s constructors eval neither head nor tail
Returning StreamsBoth map and filter return Streams
Stream’s constructors eval neither head nor tail
This allows for the chaining of laziness from the by name parameter of foldRight, into the return value
Returning Streams
Returning Streamsdef map[T](f: A => T): Stream[T] = foldRight(empty[T]) { (head, tail) =>
Returning Streamsdef map[T](f: A => T): Stream[T] = foldRight(empty[T]) { (head, tail) =>
Not Eval’d
Returning Streamsdef map[T](f: A => T): Stream[T] = foldRight(empty[T]) { (head, tail) => def map[T](f: A => T): Stream[T] = foldRight(empty[T]) { (head, tail) => !!!!! cons(f(head), tail) }
Not Eval’d
Not Eval’d
Returning Streamsdef map[T](f: A => T): Stream[T] = foldRight(empty[T]) { (head, tail) => def map[T](f: A => T): Stream[T] = foldRight(empty[T]) { (head, tail) => !!!!! cons(f(head), tail) }
Not Eval’d
Not Eval’d
Not Eval’d
Returning Streamsdef map[T](f: A => T): Stream[T] = foldRight(empty[T]) { (head, tail) => def map[T](f: A => T): Stream[T] = foldRight(empty[T]) { (head, tail) => !!!!! cons(f(head), tail) }
Not Eval’d
Not Eval’d
Not Eval’d
Jeez, does this code even do anything!?
Let’s find Out...
Let’s find Out...Stream(1, 2, 3, 4, 5) map { i => println(s"map -> $i") i * i }
Let’s find Out...Stream(1, 2, 3, 4, 5) map { i => println(s"map -> $i") i * i }
def map[T](f: A => T): Stream[T] = foldRight(empty[T]) { (head, tail) => cons(f(head), tail) }
Remember that...
def foldRight[B](z: => B)(f: (A, => B) => B): Band...
def cons[A](hd: => A, tl: => Stream[A]): Stream[A]and...
Which prints...
This page intentionally left blank
Putting it All Together
Putting it All Togetherval s = Stream(1 to 10) map { i => println(s”stream -> map $i”) i + 10 } filter { i => println(s”stream -> filter $i”) i % 2 == 0 }
Putting it All Togetherval s = Stream(1 to 10) map { i => println(s”stream -> map $i”) i + 10 } filter { i => println(s”stream -> filter $i”) i % 2 == 0 }
Prints nothing. OK
Putting it All Togetherval s = Stream(1 to 10) map { i => println(s”stream -> map $i”) i + 10 } filter { i => println(s”stream -> filter $i”) i % 2 == 0 }
def toList: List[A] = uncons match { case None => Nil case Some((h, t)) => h :: t.toList }
Prints nothing. OK
add toList()
Evaluating
Evaluatingval numList = s.toList
Evaluatingval numList = s.toList
// stream -> map 1// stream -> filter 11// stream -> map 2// stream -> filter 12// ...// stream -> map 10// stream -> filter 20
Evaluatingval numList = s.toList
def take(n: Int): Stream[A] = uncons match { case Some((h, t)) if n > 0 => cons(h, t.take(n - 1)) case _ => empty }
// stream -> map 1// stream -> filter 11// stream -> map 2// stream -> filter 12// ...// stream -> map 10// stream -> filter 20
Evaluatingval numList = s.toList
def take(n: Int): Stream[A] = uncons match { case Some((h, t)) if n > 0 => cons(h, t.take(n - 1)) case _ => empty }
// stream -> map 1// stream -> filter 11// stream -> map 2// stream -> filter 12// ...// stream -> map 10// stream -> filter 20
val numList = s.take(1).toList
Evaluatingval numList = s.toList
def take(n: Int): Stream[A] = uncons match { case Some((h, t)) if n > 0 => cons(h, t.take(n - 1)) case _ => empty }
// stream -> map 1// stream -> filter 11// stream -> map 2// stream -> filter 12// ...// stream -> map 10// stream -> filter 20
val numList = s.take(1).toList
// stream -> map 1// stream -> filter 11// stream -> map 2// stream -> filter 12
It’s All About Functions
It’s All About Functions
Functions hold the values
It’s All About Functions
Functions hold the values
Higher order functions pile functions on functions
It’s All About Functions
Functions hold the values
Higher order functions pile functions on functions
Even functions like take() merely store functions
It’s All About Functions
Functions hold the values
Higher order functions pile functions on functions
Even functions like take() merely store functions
Nothing “real” happens until you need it to happen
It’s All About Functions
Functions hold the values
Higher order functions pile functions on functions
Even functions like take() merely store functions
Nothing “real” happens until you need it to happen
Shiny!Shiny!Shiny!Shiny!
Derek WyattTwitter: @derekwyatt
Email: [email protected]
Lazy Ass Streams
Brought to you Sincere Couch Potato
semi