WHY SCALA IS NOT MY IDEAL LANGUAGE

Ruslan Shevchenko

https://github.com/rssh@rssh1

AND WHAT I CAN DO WITH THIS

ANY SUFFICIENTLY ADVANCED DEVELOPMENT CONSIDERED HARMFUL

OVERVIEW

WHY SCALA IS NOT MY IDEAL LANGUAGE AND WHAT I CAN DO WITH THIS

• Implicit mismatch.

• inexistent ‘ideal’ solution

• ‘Cooperative’ solutions

• (ways to tell something to users of you code)

• Asynchronous programming

• Errors handling

• Async - the ideal way in ideal world and workarounds in real ;)

• Actors / Transputers

• Compiler

• Effects

• Extensibility

IMPLICIT MISMATCH

• Configuring runtime behaviour via implicit.

• Wildly used anti-pattern

• Mongo reactive-db driver

• Scala standard library.

MONGO SALACT CONTEXT

CONFIGURE RUNTIME BEHAVIOUR VIA IMPLICIT-S

import com.mongodb.casbah.Imports._ import com.novus.salat._ // import our context import com.mycompany.salatcontext._ ……

import com.mongodb.casbah.Imports._ import com.novus.salat._ // import standard context import com.novus.salat.global._ ……

File A:

File B:

SCALA STANDARD LIBRARY

CONFIGURE RUNTIME BEHAVIOUR VIA IMPLICIT-S

val zf = for( x <- XService.retrieve(); y <- YService.retrieve(x) ) yield combine(x, y) val z = Await(zf, 1 minute)

Q: Where it’s block ?

//note: slide changed, version shown during first presentation was without last line. [sorry]. and reference to full example.

//full example: http://bit.ly/1oPF5zM

WHAT TO DO ?

CONFIGURING RUNTIME BEHAVIOUR

• Do not use such libraries/frameworks (?)

• Block of imports as entity which possible to compose.

• require language changes

• motivation for @annotated imports pre-sip (2012)

• http://bit.ly/1NfVIej (proposal itself)

• patch against scalac & scaladoc 2.10

WHAT TO DO ?

CONFIGURING RUNTIME BEHAVIOUR

• Do not use such libraries/frameworks (?)

• Block of imports as entity which possible to compose.

• require language changes

• motivation for @annotated imports pre-sip (2012)

• http://bit.ly/1NfVIej (proposal itself)

• patch against scalac & scaladoc 2.10

EXAMPLE OF USAGE

ANNOTATED IMPORTS

object ConfiguredSalat { @exported import com.mongodb.casbah.Imports._ @exported import com.novus.salat._ // our context @exported import com.mycompany.salatcontext._ }

File ConfiguredSalat.scala

File A:import com.mycompany.ConfiguredSalat._ ……

File B:import com.mycompany.ConfiguredSalat._ ……

WHAT TO DO ?

CONFIGURING RUNTIME BEHAVIOUR

• Do not use such libraries/frameworks (?)

• Patch compiler. // submitted pre-SIP was closed as ‘too big change, maybe later’

WHAT TO DO ?

CONFIGURING RUNTIME BEHAVIOUR

• Do not use such libraries/frameworks (?)

• Patch compiler.

• Try not to write big programs.

• Provide wrapper which will force developers to read docs ;))))

• FORCE USERS OF YOU WRAPPER TO UNDERSTAND PROBLEM • ALLOW TO DO EVERYTHING

COOPERATIVE FIX

object ThinkAboutFuture { implicit val `[implicit execution context is disabled]`: ExecutionContext = ??? implicit val `(implicit execution context is disbled)`: ExecutionContext = ??? implicit object knowledge }

Config:

Service:class MyCoreContext { def baseService() (implicit ThinkAboutFuture.knowledge.type): Future[MyService] = ……… }

• FORCE USERS OF YOU WRAPPER TO UNDERSTAND PROBLEM • ALLOW TO DO EVERYTHING

COOPERATIVE FIX

import ThinkAboutFuture._

object ReenableGlobalContext { implicit def ec = ExecutionContext.Implicits.global._ ……… }

not

• MORE ABOUT FUTURE

ASYNC PROGRAMMING

Future{

throw new RuntimeException(“Be-Be-Be!!!”)

}

- community accepted way to spawn task without return.object FuturePlus {

def apply[T](body: =>T): Future[T] = Future(body)(myEc)

def exec[T](body: =>T): Unit = FuturePlus(body).onFailure{ case ex => handleException(_) }(myEc)

}

// write own spawn methods// use some lint tools, like wartremover

• MORE ABOUT FUTURE

ASYNC PROGRAMMING

Future{

throw new RuntimeException(“Be-Be-Be!!!”)

}

java.lang.RuntimeException: Be-Be-Be !!!at x.Test$$anonfun$main$2.apply(X.scala:49)at x.Test$$anonfun$main$2.apply(X.scala:47)at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)at scala.concurrent.impl.ExecutionContextImpl

$AdaptedForkJoinTask.exec(ExecutionContextImpl.scala:121)at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

• MORE ABOUT FUTURE

ASYNC PROGRAMMING

- From where it was spawn ?

java.lang.RuntimeException: Be-Be-Be !!!at x.Test$$anonfun$main$2.apply(X.scala:49)at x.Test$$anonfun$main$2.apply(X.scala:47)at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)at scala.concurrent.impl.ExecutionContextImpl

$AdaptedForkJoinTask.exec(ExecutionContextImpl.scala:121)at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

- Can I see full stack-trace ? // problem - that getting stack trace is expensive

TRACK FUTURE APPLY

• Let’s patch byte-code.

InvokeVirtual s/c/Future$ apply (Ls/Function0;Ls/c/ExecutionContext;)Ls/c/Future

InvokeStatic t/TrackedFuture rapply (Ls/c/Future;Ls/Function0;Ls/c/ExecutionContext;)Ls/c/Future

TRACK FUTURE APPLY

libraryDependencies += "com.github.rssh" %% "trackedfuture" % "0.1"

fork := true javaOptions += s"""-javaagent:${System.getProperty("user.home")}/.ivy2/local/com.github.rssh/trackedfuture_2.11/0.1/jars/trackedfuture_2.11.jar"""

small agent: https://github.com/rssh/trackedfuture

[error] java.lang.RuntimeException: Be-Be-Be !!![error] at x.Test$$anonfun$main$2.apply(X.scala:49)[error] at x.Test$$anonfun$main$2.apply(X.scala:47)[error] at trackedfuture.runtime.TrackedFuture$$anon$1.run(TrackedFuture.scala:21)[error] at scala.concurrent.impl.ExecutionContextImpl$AdaptedForkJoinTask.exec(ExecutionContextImpl.scala:121)[error] at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)[error] at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)[error] at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)[error] at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)[error] at java.lang.Thread.getStackTrace(Thread.java:1552)[error] at trackedfuture.runtime.TrackedFuture$.apply(TrackedFuture.scala:13)[error] at trackedfuture.runtime.TrackedFuture$.rapply(TrackedFuture.scala:39)

TRACK FUTURE APPLY[error] java.lang.RuntimeException: Be-Be-Be !!![error] at x.Test$$anonfun$main$2.apply(X.scala:49)[error] at x.Test$$anonfun$main$2.apply(X.scala:47)[error] at trackedfuture.runtime.TrackedFuture$$anon$1.run(TrackedFuture.scala:21)[error] at scala.concurrent.impl.ExecutionContextImpl$AdaptedForkJoinTask.exec(ExecutionContextImpl.scala:121)[error] at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)[error] at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)[error] at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)[error] at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)[error] at java.lang.Thread.getStackTrace(Thread.java:1552)[error] at trackedfuture.runtime.TrackedFuture$.apply(TrackedFuture.scala:13)[error] at trackedfuture.runtime.TrackedFuture$.rapply(TrackedFuture.scala:39)[error] at trackedfuture.runtime.TrackedFuture.rapply(TrackedFuture.scala)[error] at x.InDBFuture$.apply(X.scala:26)[error] at x.InDBFuture$.exec(X.scala:29)[error] at x.Test$.main(X.scala:47)[error] at x.Test.main(X.scala)[success] Total time: 1 s, completed Apr 9, 2016 2

ASYNC PROGRAMMING

Future is low-level interface, like manual memory management in C

— configure execution context for db connections [1] — spawn from Actor future with this db context [2] — actually run query and retrieve the data [3] — send retrieved data to place, where it can be used (in map or to the same mailbox) [4]

How to retrieve data few times via SQL Query in Actor ?

4 steps, why not 1 ?

ASYNC PROGRAMMING

Future is low-level interface, like manual memory management in C

Management execution flow: Async

• SIP-22 • transform code with async to state machine

val respFut = async { val dayOfYear = await(futureDOY).body val daysLeft = await(futureDaysLeft).body Ok(s"$dayOfYear: $daysLeft days left!") }

ASYNC

In ideal world, we should forget about Future and use Async in business logic.

Async is a second-size citizen in scala.

• Luck of exception support // no technical reason for this. • Luck of hight-level functions support • still SIP, no 1.0 version • expose set of macro/compiler bugs.

ASYNC => GO

ASYNC PROGRAMMING

def copy(inf: File, outf: File): Long = goScope { val in = new FileInputStream(inf) defer{ in.close() } val out = new FileOutputStream(outf); defer{ out.close() } out.getChannel() transferFrom(in.getChannel(), 0, Long.MaxValue) }

• Part of scala-gopher:

• https://github.com/rssh/scala-gopher

• wrapper around async:

• defer instead try/catch/finally

• Hight-level function support

NOT IMPLEMENTED IDEAS ABOUT HIGHT-LEVEL FUNCTION SUPPORT

ASYNC PROGRAMMING

async{ val l = for(i <- 1 to N) yield { await(retrieve something i) } } IMPOSSIBLE

async{ var l = IndexedSeq[X]() var i = 0 while(i < N) { l += await(retrieve something i) } }

• Hight-level function support

• before generation of async, wrap type classes for well-known hight-order functions:

• map( A (with awaits) ) => mapAsync(Future[A])

• allows to write own type classes

• when TASTY will be available: try to generate async variants in simple cases.

NOT IMPLEMENTED IDEAS ABOUT HIGHT-LEVEL FUNCTION SUPPORT

ASYNC PROGRAMMING

ASYNC

COMPILER SUPPORT

• can async be better integrated with compiler ?

• async is very low-level interface,

• like automatic memory management in C++

• Hight level way:

• Effects in type system.

• (X@Ex; Y@Ey) => Y @ (Ex+Ey) [in ideal world]

• (X; Y) => Y [ now ]

• ability to specialise execution context by effects.

// SCALA [Q PART]

• lim <?> Hight-level DSL

Specialised DSL (Q, SQL)

Scala

Java/NPJava

C

ASM

// SCALA [Q PART]

• lim <?> Hight-level DSL

Q

Scala

Java/NPJava

C

ASM

code <=> run time complexity possible to track behaviour looking at code

code =/= run time complexity syntax sugar, implicits, std

// SCALA [Q PART]

• lim <?> Hight-level DSL

Q

Scala

Java/NPJava

C

ASM

verbose API, verbose imports

leaked low-level issues

Hight-level fluent API; carefully optimised;

controlled low-level platform

// SCALA [Q PART]

• lim <?> Hight-level DSL

Q

Scala

Java/NPJava

C

ASM

s.split(‘|’).toSeq flatMap (_.split(‘=‘)). … verify that it-s int …

(!/) “I=|”0:fix

// SCALA [Q PART]

• DSL construction: boilerplate factory - Internal

- own set of types and globals; - — no way to set precedence - — manual construction - — import needed postfix,

- macroses (limited). - — awkward reflection API - — traversing instead construction. - — run after typing phase.

// SCALA [Q PART]

• Scala is one of my favourite languages.

• It can be sufficiently better.

• Maybe[support for compiler plugins] • Any[sufficiently advanced development]

considered harmful

• Questions [?]; Ideas[?] ; — call me