scala-gopher: async implementation of csp for scala

Download scala-gopher:  async implementation of CSP  for  scala

Post on 13-Jul-2015

757 views

Category:

Software

0 download

Embed Size (px)

TRANSCRIPT

  • https://github.com/rssh/scala-gopher

    .Ruslan Shevchenko @rssh1

  • https://github.com/rssh/scala-gopher

    CSP = Communicating Sequential Processes. Tony Hoar.

    classical paper from 1978 book: http://www.usingcsp.com/ (1985)

    Languages Occam (1983) for specialized hardware [transputers] Go (2007) - got popular:

    Rob Pike (C, Plan9) in Google

  • https://github.com/rssh/scala-gopher

    Channel (buffered)

    Channel (unbuffered)

    Write (blocked) Read (blocked)

    Computation blocks are connectedby channels

    // in INMOS transputers - physically

  • Flow A Flow B

  • Flow A Flow B

  • Flow A Flow B

  • Flow A Flow B

  • Flow A Flow B

  • Flow A Flow B

  • Flow A Flow B

  • Flow A Flow B

  • https://github.com/rssh/scala-gopher

    Simple mental model Ability to coordinate between input/output sinks

    go func(){ for { select { case x

  • https://github.com/rssh/scala-gopher

    Simple mental model Ability to coordinate between import

    go func(){ for { select { case x

  • https://github.com/rssh/scala-gopher

  • https://github.com/rssh/scala-gopher

    Not emulation of go in scala but CSP constructs in scala-way'

    Asynchronous build on top of Akka and SIP 22 - async

    scala-gopher:

    // block is not block // wait is not wait

  • https://github.com/rssh/scala-gopher

    go func(){ for { select { case x

  • https://github.com/rssh/scala-gopher

    go func(){ for { select { case x

  • https://github.com/rssh/scala-gopher

    select.forever { case x: inX.read => val y = inY.read if (x == y) { Console.println(Bingo!) } case q: inQuit.read => currentFlow.exit(())}

    Scala

    // select.forever: basic construct

  • https://github.com/rssh/scala-gopher

    select.forever { case x: String if (x==(inX|1inX2).read) => val y = inY.read if (x == y) { Console.println(Bingo!) } case q: Boolean if (q==inQuit.read) => currentFlow.exit(())}

    Scala

  • https://github.com/rssh/scala-gopher

    forever: apply(choice: PartialFunction[Any,Unit]): Future[Unit]

    once: apply[T](choice: PartialFunction[Any,T]): Future[T]

    Unblocked:

    go[T]: Future[T]

    selector:

    Blocked: (must be inside go or async ):

    forever: foreach(choice: PartialFunction[Any,Unit]): Unit

    once: foreach[T](choice: PartialFunction[Any,T]): T

  • https://github.com/rssh/scala-gopher

    Inside Partial Function:

    reading from Input[A] and do something Channel[A], Future[A], user input

    writing to Output[A] and do something Channel[A], user output

    do something on idle.

    do something wrapped in async block,so we can block there

  • https://github.com/rssh/scala-gopher

    Input[A]: unblocked:

    aread, atake, aforeach blocked:

    read, take, foreach . composition

    map, filter, zip, or, dup construction

    channel, future, collection, own implementation

    open: waitclosed: throw exception on reading after last element

  • https://github.com/rssh/scala-gopher

    Output[A]: unblocked:

    awrite, awriteAll, blocked:

    write, writeAll . composition

    (?) construction

    channel, actor, own implementation

    open: wait closed: throw exception on writing element

  • https://github.com/rssh/scala-gopher

    Channel[A]:

    Input[A] + Output[A] garbage-collected. gopherApi.makeChannel

    can be buffered

  • https://github.com/rssh/scala-gopher

    Timeouts:

    val (inReady, inTimeouts) = in.withInputTimeouts(30 seconds) for(s Console.println(sreceived ${a}) case t:inTimeouts.read => Console.println(timeout occurred) }

    Input => withInputTimeoutsOutput => withOutputTimeouts

  • https://github.com/rssh/scala-gopher

    go[T](T :=> Future[T]) async + defer/recover defer(callback: =>Unit): Unit recover(handler: PartialFunction[Throwable,T]):Boolean

    go { val source = fetchSource(url) val csv = parseCsv(source) val svFile = new FileOutputStream(csv.name, append=true) defer{ val r = recover{ case FileNotFoundException => signal(file not found) } if (!r) svFile.close() } for(s

  • https://github.com/rssh/scala-gopher

    go[T](T :=> Future[T])go { val source = fetchSource(url) val csv = parseCsv(source) val svFile = new FileOutputStream(csv.name, append=true) defer{ val r = recover{ case FileNotFoundException => signal(file not found) } if (!r) svFile.close() } for(s

  • https://github.com/rssh/scala-gopher

    goScope[T](T :=> T)

    goScope { val in = new FileInputStream(inf) defer { in.close() } val out = new FileOutputStream(outf) defer{ out.close() } out.getChannel().transferFrom(in.getChannel, 0,Long.MaxValue)}

  • https://github.com/rssh/scala-gopher

    All essential functionality of CSP model Fully asynchronous implementation In normal language, with generic, typing, .. . more: scala is object-oriented.

  • https://github.com/rssh/scala-gopher

    Set of input and output ports. Some internal state Functionality for signal transformations

    Reusable block:

    Transputer:

  • https://github.com/rssh/scala-gopher

    trait Bingo extends SelectTransputer {

    val inX = InPort[Int]()

    val inY = InPort[Int]()

    val out = OutPort[Boolean]()

    loop {

    case x: inX.read =>

    val y = inY.read

    Console.println(s"Bingo checker, received ${x}, ${y})

    out.write(x==y)

    }

    }

  • https://github.com/rssh/scala-gopher

    trait Acceptor extends SelectTransputer {

    val inA = InPort[Boolean]()

    @volatile var (nBingos, nPairs) = (0,0)

    loop {

    case x: inA.read =>

    Console.println(s"acceptor: ${nPairs} ${nBingos} ${x}")

    if (x) {

    nBingos += 1

    }

    nPairs += 1

    }

    }

  • https://github.com/rssh/scala-gopher

    val inX = gopherApi.makeChannel[Int]()

    val inY = gopherApi.makeChannel[Int]()

    val bingo = gopherApi.makeTransputer[Bingo]

    val acceptor = gopherApi.makeTransputer[Acceptor]

    bingo.inX connect inX

    bingo.inY connect inY

    bingo.out connect acceptor.inA

    (bingo + acceptor).start()

  • https://github.com/rssh/scala-gopher

    Like actors but works with ports We can wait inside Connected via channels. Restartable

    Transputers

    // in remind of INMOS transputers

  • Select

    select.foreach {

    }

  • Par

    P1

    P2 P3

    P1 + P2 + P3

  • Replicate

    Mapping

    gopherApi.replicate[X](5)

  • Replicate

    Mapping

    gopherApi.replicate[X](5).in.distribute(_ % 10)

    Element e from in will be directed to (e%10) instance of X

  • https://github.com/rssh/scala-gopher

    Select: [handle one transformation] Parallel[ transputers are run in parallel] Replicated

    [ run in parallel N instances.] [ custom logic for port shuffling ]

    Transputer Supervisor => ActorSystem

    Transputers

  • Implementation

  • Implementation

    sealed trait Continuated[T]

    case class ContRead[A,T]( callback , input, flow) extends Continuated[T]case class ContWrite[A,T]( callback , output, flow) ..case class ContSkip[T](callback, flow) case class Done[T]( value , flow) ..case object Never .

    // simular to Iteratee

  • Implementation

    sealed trait Continuated[T]

    case class ContRead[A,T]( callback , input, flow) extends Continuated[T]

  • Implementationsealed trait Continuated[T]

    case class ContRead[A,T]( function: ContRead[A,B] => Option[ ContRead.In[A] => Future[Continuated[T]] ] , input, flow) extends Continuated[T]

  • Implementationsealed trait Continuated[T]

    case class ContRead[A,T]( function: ContRead[A,B] => Option[ ContRead.In[A] => Future[Continuated[T]] ] , input, flow) extends Continuated[T]

    Value, Skip, Failure

    Read as Protocol: (check, if available, read, return next step)

  • Implementation

    Dispatch Actor

    wait in channel read/write

    FlowFlow

    wait in select

  • scala-gopher

    CSP within scala ecosystem Channels complementary to RxStreams

    async rw/async callback Transducers complementary to Actors

    transform streams/event reactions Can be used together.

  • scala-gopher

    https://github.com/rssh/scala-gopher Ruslan Shevchenko @rssh1

    Questions ?