an excuse to try, either, folding, and future. sequence

An excuse to Try, Either, folding, and Future.

sequence or parsing command line options with

crappy-optional

@gerferra

crappy-optional?

“Don’t worry be crappy” mantra

optional library from DRMacIverand paulp

optional

optional is a command line option parser and library.

optional

YOU WRITE:

object MyAwesomeCommandLineTool extends optional.Application {

// for instance...

def main(count: Option[Int], file: Option[java.io.File],

arg1: String) {

[...]

}

}

optional

THEN YOU DO:

scala MyAwesomeCommandLineTool --count 5 quux

optional

AND YOUR MAIN METHOD WILL BE INVOKED SUCH THAT:

count = Some(5)file = Nonearg1 = quux

optional




// for instance...


arg1: String) {

[...]

}

}


Ref by name

optional




// for instance...


arg1: String) {

[...]

}

}


Ref by name

Optional parameter

optional




// for instance...


arg1: String) {

[...]

}

}


Ref by name

Optional parameter

Ref by position

optional

HOW IT WORKS:

optional

HOW IT WORKS:

Reflection, man.

optional

HOW IT WORKS:

(Java) Reflection, man.

optional

HOW IT WORKS:

(Java) Reflection, man.

doesn’t work OK with Scala since 2.9 (I guess) ...

used to work

So...

… copy the idea, but use Scala reflection instead …

… and don’t worry and be crappy :) ...

Show me the code!trait Application {

def main(args: Array[String]) {

runMain(this, args.toList)

}

private def runMain[T: ClassTag](

instance: T, args: List[String]) { // ... something happens here ...

}

}

Show me the code!private def runMain[T: ClassTag](instance: T, args: List[String]) {

val instanceMirror = ru.runtimeMirror(getClass.getClassLoader).reflect(instance)

val theType = instanceMirror.symbol.typeSignature

val methods = theType.members.collect {

case m if m.isMethod => m.asMethod

}

val mainMethods = methods.filter(m => m.name == ru.TermName("main") &&

m.fullName != classOf[Application].getName + ".main")

val methodOrError = mainMethods.headOption.toRight("No main method defined")

// Scala support multiple parameter lists, I care only about the first (if exists ...)

val = methodOrError.right.map(m => m.paramLists.headOption.getOrElse(Nil))

paramsOrError

val invocationOrError =

for {

method <- methodOrError.right

params <- paramsOrError.right

values <- instantiate(params, args).right // mysterious instantiation of things ...

} yield {

val refMethod = instanceMirror.reflectMethod(method)

refMethod(values.unzip._2: _*) // here is the execution of the main method happening

}

invocationOrError.fold(println, identity /* nothing to do because it already executed as a side effect */)

}

Step by step: method definition

Scala method:


instance: T, args: List[String]) {

private, so it’s not inherited

with a type parameter T

with a context bound ClassTag


… with a context bound ClassTag?


instance: T, args: List[String]) {

≡ private def runMain[T](

instance: T, args: List[String])(

implicit ev: ClassTag[T]) {

both expressions are analogous


we ask for an evidence that a ClassTag can be constructed for the type T ...

… so we can use runtime reflection on it ...

Step by step: reflecting on instance

val instanceMirror =

ru

.runtimeMirror(getClass.getClassLoader)

.reflect(instance)



ru


.reflect(instance)

(mystic voice) …You need a mirror to access the reflection…



ru


.reflect(instance)

`ru` here is just an alias to packagescala.reflect.runtime.universe

import scala.reflect.runtime.{ universe => ru }



ru


.reflect(instance)

A ClassTag is needed here to overcome the erasure of the type at runtime T

Step by step: obtaining the methodI want the methods ...

val theType =

instanceMirror.symbol.typeSignature

val methods = theType.members.collect {

case m if m.isMethod => m.asMethod

}

Step by step: obtaining the method… to get the right main methods ...

val mainMethods =

methods.filter(m =>

m.name == ru.TermName("main") &&

m.fullName !=

classOf[Application].getName + ".main")

Step by step: obtaining the method… to get the right main methods ...

val methodOrError =

mainMethods

.headOption

.toRight("No main method defined")

Give me either the first of the methods, if there is one, or an error message

Step by step: obtaining the method… to get the right main methods …

val methodOrError =

mainMethods

.headOption


Give me either the first of the methods, if there is one, or an error message


val methodOrError =

mainMethods

.headOption


headOption ~ List[A] => Option[A]


val methodOrError =

mainMethods

.headOption



Option[A] = Some(a: A) | None


val methodOrError =

mainMethods

.headOption



not scala (!)



val methodOrError =

mainMethods

.headOption



def headOption: Option[A] =

if (isEmpty) None else Some(head)



val methodOrError =

mainMethods

.headOption


toRight[X] ~ Option[A] => (=> X) => Either[X,A]


val methodOrError =

mainMethods

.headOption



Either[A, B] = Left(a: A) | Right(b: B)


val methodOrError =

mainMethods

.headOption



def toRight[X](left: => X): Either[X,A] =

if (isEmpty) Left(left) else Right(this.get)



val methodOrError =

mainMethods

.headOption



def toRight[X](left: => X): Either[X,A] =

if (isEmpty) Left(left) else Right(this.get)


Either[String, ru.MethodSymbol]

Step by step: obtaining the params

Ok. Now I need the parameters:

val paramsOrError =

methodOrError.right.map(m =>

m.paramLists

.headOption.getOrElse(Nil))

Scala support multiple parameter lists. I care only about the first (if exists …)



val paramsOrError =


m.paramLists

.headOption.getOrElse(Nil)) Either[String, ru.MethodSymbol]



val paramsOrError =


m.paramLists

.headOption.getOrElse(Nil)) Either[String, ru.MethodSymbol]

only the right part please



val paramsOrError =


m.paramLists


map[X] ~ Either[A,B] => (B => X) => Either[A,X]





val paramsOrError =


m.paramLists


map[X] ~ Either[A,B] => (B => X) => Either[A,X]

def map[X](f: B => X) = e match {

case Left(a) => Left(a)

case Right(b) => Right(f(b))

}





val paramsOrError =


m.paramLists


List[List[ru.Symbol]]



val paramsOrError =


m.paramLists


Option[List[ru.Symbol]]

List[List[ru.Symbol]]



val paramsOrError =


m.paramLists


f: ru.MethodSymbol => List[ru.Symbol]



val paramsOrError =


m.paramLists


f: ru.MethodSymbol => List[ru.Symbol]paramsOrError: Either[String, List[ru.Symbol]]



val paramsOrError =


m.paramLists


f: ru.MethodSymbol => List[ru.Symbol]paramsOrError: Either[String, List[ru.Symbol]]

paramsOrError have the parameters OR the original error "No main method defined"

Step by step: invocation


for {



values <- instantiate(params, args).right

} yield {

val refMethod =

instanceMirror.reflectMethod(method)

refMethod(values.unzip._2: _*)

}



for {




} yield {

val refMethod =



}

Same as before



for {




} yield {

val refMethod =



}

Same as before

The main method if all is right



for {




} yield {

val refMethod =



}

Same as before

The parameters if all is right



for {




} yield {

val refMethod =



}

Same as before

The values to use to invoke the method



for {




} yield {

val refMethod =



}

Same as before

The values to use to invoke the method if all is right :-)



for {




} yield {

val refMethod =



}

Will compute only if all OK



for {




} yield {

val refMethod =



}

Either[String, Any]



for {




} yield {

val refMethod =



}

Either[String, Any]

The first error that occurs (if any)



for {




} yield {

val refMethod =



}

Either[String, Any]

The first error that occurs (if any)

The value returned by the method (if all OK)

Step by step: handling ending cases

invocationOrError.fold(println, identity)



fold[X] ~

Either[A, B] => (A => X) => (B => X) => X



fold[X] ~

Either[A, B] => (A => X) => (B => X) => X

If you can convert the value at the left to X, and the value at the right to X, then you can convert an Either[A,B] to X



fold[X] ~

Either[A, B] => (A => X) => (B => X) => X

def fold[X](fa: A=>X, fb: B=>X) = this match {

case Left(a) => fa(a)

case Right(b) => fb(b)

}



fold[X] ~

Either[A, B] => (A => X) => (B => X) => X

In this case, if there is an error (a Left value) it prints it to the console. If all went OK (a Right value) it does nothing, because the method was already executed as a side-effect

instantiate?def instantiate(paramList: List[ru.Symbol], argList: List[String]): Either[Error, List[(Name, Any)]] = {

val params = paramList.map(p => p.name.encodedName.toString -> p.typeSignature)

val paramNames = params.unzip._1

val argsOptName = assignNames(argList)

val argsPartition = argsOptName.partition(_._1.isEmpty)

val unnamedArgs = argsPartition._1.unzip._2

val namedArgsTmp = argsPartition._2.collect { case (Some(n), value) => n -> value }

def sortFunc(s: String): Int = { val i = paramNames.indexOf(s); if (i == -1) Int.MaxValue else i }

val namedArgs = namedArgsTmp.sortBy(p => sortFunc(p._1))

val argsNames = namedArgs.unzip._1

val (unreferencedParams, referencedParams) = params.partition(p => !argsNames.contains(p._1))

val unreferenced = instantiateUnreferencedParams(unreferencedParams, unnamedArgs)

val referenced = instantiateParams(referencedParams, namedArgs)

for {

unref <- unreferenced.right

ref <- referenced.right

} yield {

(unref ++ ref).sortBy(p => sortFunc(p._1))

}

}

More ...

assignNames

instantiateUnreferencedParams

instantiateParams

coerce

Example:

Suppose we want to query user information, but it is scattered in multiple servers (ldap and database).

Most of the time we will be searching by name, but sometimes we will want to search by uid or by the user login.

And we want it now! :)

Example: usr

So we create a command `usr` with the following arguments:

name: Option[String]

limit: Option[Int]

uid: Option[String]

login: Option[String]

debug: Option[Boolean]

Example: usr



limit: Option[Int]

uid: Option[String]



limit the number of results

Example: usr



limit: Option[Int]

uid: Option[String]


debug: Option[Boolean] extra debug info?

Example: usr



limit: Option[Int]

uid: Option[String]



First so they can be used without name

Example: usr



limit: Option[Int]

uid: Option[String]



Any individual argument is optional

Example: usr



limit: Option[Int]

uid: Option[String]



String, Int, Boolean ...

Example: usr

Some usage examples:

usr "juan perez"

usr "juan perez" 10

usr --uid jperez

usr --login jperez --limit 10 --debug

Example: usr

object usr extends optional.Application {

def main(name: Option[String], limit: Option[Int], uid: Option[String], login: Option[String], debug: Option[Boolean]) {

// THE LOGIC ...

}}

Example: usrobject usr extends optional.Application {

def main(name: Option[String], limit: Option[Int], uid: Option[String],

login: Option[String], debug: Option[Boolean]) {

if (Seq(name, uid, login).flatten.isEmpty) {

println("You must specify at least one search criteria")

System.exit(1)

}

val effDebug = debug.getOrElse(false)

val futUsrLDAPA = Future { ldapA.findUsr(name, limit, uid, login) }

val futUsrLDAPB = Future { ldapB.findUsr(name, limit, uid, login) }

val futUsrDB = Future { dbase.findUsr(name, limit, uid, login) }

futUsrLDAPA.onComplete(printResult("--- LDAP A ---", debug = effDebug))

futUsrLDAPB.onComplete(printResult("--- LDAP B ---", debug = effDebug))

futUsrDB.onComplete(printResult("--- Data Base ---", debug = effDebug))

val res = Await.ready(

Future.sequence(Seq(futUsrLDAPA, futUsrLDAPB, futUsrDB)), Duration.Inf)

}

}






System.exit(1)

}










}

}

One of them must be specified






System.exit(1)

}










}

}

No debug by default






System.exit(1)

}










}

}

Query the three sources in parallel






System.exit(1)

}










}

}

After completion, print the result as a side effect






System.exit(1)

}










}

}

Wait until the three queries finish






System.exit(1)

}










}

}

Has a bug!

Example: completing futures

futRes.onComplete(printResult)

def onComplete[U](f: Try[T] => U): Unit

Try[T] = Success(t: T) | Failure(e: Throwable)




Either[Throwable, T]~~




When this Future finishes, either correctly (Success) or with an exception (Failure), apply the function f which knows how to handle both cases to compute some side effect


Either[Throwable, T]Try[T] = Success(t: T) | Failure(e: Throwable)

~~



In this case the side effect is just printing the result to the console.


Either[Throwable, T]~~


Future.sequence(

Seq(futUsrLDAPA, futUsrLDAPB, futUsrDB))

def sequence[A, M[_]](

in: M[Future[A]]): Future[M[A]]

Example: sequencing futures

Future.sequence(




Convert some container M of Future[A] in a Future M[A].


Future.sequence(




Convert some container M of Future[A] in a Future M[A].

Example: sequencing futuresM[X] <: TraversableOnce[X]

Future.sequence(




Convert some container M of Future[A] in a Future M[A].Useful to reduce many Futures into a single Future.


Future.sequence(




Convert some container M of Future[A] in a Future M[A].Useful to reduce many Futures into a single Future.In this case we reduce three independent future results to a single future result with a Seq of the individual results.


implicit class TryHarder[T](val t: Try[T]) extends AnyVal {

def toEither: Either[Throwable, T] = t match { case Success(s) => Right(s) case Failure(f) => Left(f) }

def fold[U](success: T => U, failure: Throwable => U): U = toEither.fold(failure, success)}

Example: folding on Try

implicit class TryHarder[T](val t: Try[T]) extends AnyVal {

def toEither: Either[Throwable, T] = t match { case Success(s) => Right(s) case Failure(f) => Left(f) }

def fold[U](success: T => U, failure: Throwable => U): U = toEither.fold(failure, success)}

Example: folding on Try

Try is pretty much a specialized Either

Example: fixing the bugobject usr extends optional.Application {





System.exit(1)

}










}

}

The problem is that onComplete executes after the Future is completed. Await only awaits to the completion, not the execution of onComplete and the program can terminate “too soon”.

implicit class FutureOps[T](val f: Future[T])

extends AnyVal {

def continue[S]( cont: Try[T] => S): Future[S] = {

val p = Promise[S]() f.onComplete { tt => p.complete(Try(cont(tt))) } p.future

}}

Example: fixing the bug

implicit class FutureOps[T](val f: Future[T])

extends AnyVal {

def continue[S]( cont: Try[T] => S): Future[S] = {

val p = Promise[S]() f.onComplete { tt => p.complete(Try(cont(tt))) } p.future

}}

Example: fixing the bug

Like onComplete but returning another Future

Example: now works :)object usr extends optional.Application {





System.exit(1)

}





val endLDAPA = futUsrLDAPA.continue(printResult("--- LDAP A ---"))

val endLDAPB = futUsrLDAPB.continue(printResult("--- LDAP B ---"))

val endDB = futUsrDB.continue(printResult("--- Data Base ---"))


Future.sequence(Seq(endLDAPA, endLDAPB, endDB)), Duration.Inf)

}

}

github.com/gerferra/crappy-optional

FIN

sub

mini-demo?

FIN

an excuse to try, either, folding, and future. sequence

Software

torightno main method

main val methodorerror

val mainmethods

typesignature val methods

optiona optiona

val instancemirror

optional parameter ref

some5 file