embedding a language into string interpolator

N

Embedding a language into stringinterpolator

Mikhail LimanskiyJune 10, 2015

String interpolation is easy

val language = "English"

val embedded = s"Embedded $language"

String interpolation is easy

val language = "English"

val embedded = s"Embedded $language"

MongoDB

MongoDB is a document oriented database, storing BSON documents.db.people.insert({ name: "John Doe", age: 42 })

db.people.insert({name: "William Smith",age: 28,phone: [ "1234567", "7654321" ]})

db.people.insert({name: "Alice White",age: 29,address: {

country: "UK",city: "London"

}})db.people.insert({ name : "Ivan Petrov", age : 28 })

MongoDB

MongoDB is a document oriented database, storing BSON documents.db.people.insert({ name: "John Doe", age: 42 })db.people.insert({

name: "William Smith",age: 28,phone: [ "1234567", "7654321" ]})

db.people.insert({name: "Alice White",age: 29,address: {

country: "UK",city: "London"

}})db.people.insert({ name : "Ivan Petrov", age : 28 })

MongoDB

MongoDB is a document oriented database, storing BSON documents.db.people.insert({ name: "John Doe", age: 42 })db.people.insert({

name: "William Smith",age: 28,phone: [ "1234567", "7654321" ]})

db.people.insert({name: "Alice White",age: 29,address: {

country: "UK",city: "London"

}})

db.people.insert({ name : "Ivan Petrov", age : 28 })

MongoDB

MongoDB is a document oriented database, storing BSON documents.db.people.insert({ name: "John Doe", age: 42 })db.people.insert({

name: "William Smith",age: 28,phone: [ "1234567", "7654321" ]})

db.people.insert({name: "Alice White",age: 29,address: {

country: "UK",city: "London"

}})db.people.insert({ name : "Ivan Petrov", age : 28 })

Search and update

Quering:db.people.find({ name: "John Doe"})db.people.find({ age: { $lt : 30 }})db.people.find({ phone: { $not: { $size : 0 }}})

db.people.update({ age : 42},{ $set : { name : "Ford Prefect" } })

db.people.aggregate([ { $group : { _id : "$age", count : {$sum : 1} } },

{ $sort : { count : -1 } },{ $limit : 5 }

])

Search and update

Quering:db.people.find({ name: "John Doe"})db.people.find({ age: { $lt : 30 }})db.people.find({ phone: { $not: { $size : 0 }}})

db.people.update({ age : 42},{ $set : { name : "Ford Prefect" } })

db.people.aggregate([ { $group : { _id : "$age", count : {$sum : 1} } },

{ $sort : { count : -1 } },{ $limit : 5 }

])

Search and update

Quering:db.people.find({ name: "John Doe"})db.people.find({ age: { $lt : 30 }})db.people.find({ phone: { $not: { $size : 0 }}})

db.people.update({ age : 42},{ $set : { name : "Ford Prefect" } })

db.people.aggregate([ { $group : { _id : "$age", count : {$sum : 1} } },

{ $sort : { count : -1 } },{ $limit : 5 }

])

MongoDB in Scala

There are three main drivers for MongoDB:

I Casbah – synchronous, on top of Java driver.

I ReactiveMongo – asynchronous, built on Akka actors.I Tepkin – reactive, on top of Akka IO and Akka Streams.

MongoDB in Scala

There are three main drivers for MongoDB:

I Casbah – synchronous, on top of Java driver.I ReactiveMongo – asynchronous, built on Akka actors.

I Tepkin – reactive, on top of Akka IO and Akka Streams.

MongoDB in Scala

There are three main drivers for MongoDB:

I Casbah – synchronous, on top of Java driver.I ReactiveMongo – asynchronous, built on Akka actors.I Tepkin – reactive, on top of Akka IO and Akka Streams.

How Casbah API looks like

val name = "John Doe"

people.insert(MongoDBObject("name" -> "James Bond","age" -> 80,"phone" -> List("007007"),"address" -> MongoDBObject("country" -> "UK")))

val a = people.findOne(MongoDBObject("name" -> name))val b = people.find(MongoDBObject("age" ->

MongoDBObject("$lt" -> 30)))

// Using Casbah DSLval c = people.find("age" $lt 30)val d = people.find("phone" -> $not(_ $size 0))

people.update(MongoDBObject("age" -> 42),$set("name" -> "Ford Prefect"))

val e = people.aggregate(List(MongoDBObject("$group" ->

MongoDBObject("_id" -> "$age", "count" ->MongoDBObject("$sum" -> 1))),

MongoDBObject("$sort" -> MongoDBObject("count" -> -1)),MongoDBObject("$limit" -> 5)))

How Casbah API looks like

val name = "John Doe"

people.insert(MongoDBObject("name" -> "James Bond","age" -> 80,"phone" -> List("007007"),"address" -> MongoDBObject("country" -> "UK")))

val a = people.findOne(MongoDBObject("name" -> name))val b = people.find(MongoDBObject("age" ->

MongoDBObject("$lt" -> 30)))

// Using Casbah DSLval c = people.find("age" $lt 30)val d = people.find("phone" -> $not(_ $size 0))

people.update(MongoDBObject("age" -> 42),$set("name" -> "Ford Prefect"))

val e = people.aggregate(List(MongoDBObject("$group" ->

MongoDBObject("_id" -> "$age", "count" ->MongoDBObject("$sum" -> 1))),

MongoDBObject("$sort" -> MongoDBObject("count" -> -1)),MongoDBObject("$limit" -> 5)))

How Casbah API looks like

val name = "John Doe"

people.insert(MongoDBObject("name" -> "James Bond","age" -> 80,"phone" -> List("007007"),"address" -> MongoDBObject("country" -> "UK")))

val a = people.findOne(MongoDBObject("name" -> name))val b = people.find(MongoDBObject("age" ->

MongoDBObject("$lt" -> 30)))

// Using Casbah DSLval c = people.find("age" $lt 30)val d = people.find("phone" -> $not(_ $size 0))

people.update(MongoDBObject("age" -> 42),$set("name" -> "Ford Prefect"))

val e = people.aggregate(List(MongoDBObject("$group" ->

MongoDBObject("_id" -> "$age", "count" ->MongoDBObject("$sum" -> 1))),

MongoDBObject("$sort" -> MongoDBObject("count" -> -1)),MongoDBObject("$limit" -> 5)))

How Casbah API looks like

val name = "John Doe"

people.insert(MongoDBObject("name" -> "James Bond","age" -> 80,"phone" -> List("007007"),"address" -> MongoDBObject("country" -> "UK")))

val a = people.findOne(MongoDBObject("name" -> name))val b = people.find(MongoDBObject("age" ->

MongoDBObject("$lt" -> 30)))

// Using Casbah DSLval c = people.find("age" $lt 30)val d = people.find("phone" -> $not(_ $size 0))

people.update(MongoDBObject("age" -> 42),$set("name" -> "Ford Prefect"))

val e = people.aggregate(List(MongoDBObject("$group" ->

MongoDBObject("_id" -> "$age", "count" ->MongoDBObject("$sum" -> 1))),

MongoDBObject("$sort" -> MongoDBObject("count" -> -1)),MongoDBObject("$limit" -> 5)))

How Casbah API looks like

val name = "John Doe"

people.insert(MongoDBObject("name" -> "James Bond","age" -> 80,"phone" -> List("007007"),"address" -> MongoDBObject("country" -> "UK")))

val a = people.findOne(MongoDBObject("name" -> name))val b = people.find(MongoDBObject("age" ->

MongoDBObject("$lt" -> 30)))

// Using Casbah DSLval c = people.find("age" $lt 30)val d = people.find("phone" -> $not(_ $size 0))

people.update(MongoDBObject("age" -> 42),$set("name" -> "Ford Prefect"))

val e = people.aggregate(List(MongoDBObject("$group" ->

MongoDBObject("_id" -> "$age", "count" ->MongoDBObject("$sum" -> 1))),

MongoDBObject("$sort" -> MongoDBObject("count" -> -1)),MongoDBObject("$limit" -> 5)))

How Casbah API looks like

ReactiveMongo

// Future[BSONDocument]val a = people.find(BSONDocument("name" -> "John Doe"))

.one[BSONDocument]

// Future[List[Person]]val b = people.find(BSONDocument("age" ->

BSONDocument("$lt" -> 30))).cursor[Person].collect[List]()

val futureUpdate = people.update(BSONDocument("age" -> 42),BSONDocument("$set" -> BSONDocument("name" -> "Ford Prefect")))

// Futureval e = db.command(RawCommand(BSONDocument(

"aggregate" -> "people","pipeline" -> BSONArray(

BSONDocument("$group" ->BSONDocument("_id" -> "$age",

"count" -> BSONDocument("$sum" -> 1))),BSONDocument("$sort" -> BSONDocument("count" -> -1)),BSONDocument("$limit" -> 5)))))

Why?

Meet MongoQuery

Using MongoQuery with Casbah:import com.github.limansky.mongoquery.casbah._

val name = "John Doe"

val a = people.findOne(mq"{ name : $name }")

val b = people.find(mq"{age : { $$lt : 30 }}")

val d = people.find(mq"{ phone : { $$not : { $$size : 0 }}}")

people.update(mq"{ age : 42 }",mq"{ $$set { name : 'Ford Prefect' }}")

val e = people.aggregate(List(mq"""{ $$group :

{ _id : "$$age", count : { $$sum : 1 }}}""",mq"{ $$sort : { count : -1 }}",mq"{ $$limit : 5}"))

Meet MongoQuery

Using MongoQuery with Casbah:import com.github.limansky.mongoquery.casbah._

val name = "John Doe"

val a = people.findOne(mq"{ name : $name }")val b = people.find(mq"{age : { $$lt : 30 }}")

val d = people.find(mq"{ phone : { $$not : { $$size : 0 }}}")

people.update(mq"{ age : 42 }",mq"{ $$set { name : 'Ford Prefect' }}")

val e = people.aggregate(List(mq"""{ $$group :

{ _id : "$$age", count : { $$sum : 1 }}}""",mq"{ $$sort : { count : -1 }}",mq"{ $$limit : 5}"))

Meet MongoQuery

Using MongoQuery with Casbah:import com.github.limansky.mongoquery.casbah._

val name = "John Doe"

val a = people.findOne(mq"{ name : $name }")val b = people.find(mq"{age : { $$lt : 30 }}")

val d = people.find(mq"{ phone : { $$not : { $$size : 0 }}}")

people.update(mq"{ age : 42 }",mq"{ $$set { name : 'Ford Prefect' }}")

val e = people.aggregate(List(mq"""{ $$group :

{ _id : "$$age", count : { $$sum : 1 }}}""",mq"{ $$sort : { count : -1 }}",mq"{ $$limit : 5}"))

String interpolation

implicit class MongoHelper(val sc: StringContext)extends AnyVal {

def mq(args: Any*): DBObject = {Parser.parseQuery(sc.parts, args) match {

case Success(v, _) =>createObject(v)

case NoSuccess(msg, _) =>throw new MqException(s"Invalid object: $msg")

}}

}

mq"{ name : $name }"

sc.parts == List("{ name: ", " }")args = List(name)

String interpolation

implicit class MongoHelper(val sc: StringContext)extends AnyVal {

def mq(args: Any*): DBObject = {Parser.parseQuery(sc.parts, args) match {

case Success(v, _) =>createObject(v)

case NoSuccess(msg, _) =>throw new MqException(s"Invalid object: $msg")

}}

}

mq"{ name : $name }"

sc.parts == List("{ name: ", " }")args = List(name)

String interpolation

Wrapping it into macro

implicit class MongoHelper(val sc: StringContext) extends AnyVal {def mq(args: Any*): DBObject = macro MongoHelper.mq_impl

}

object MongoHelper {def mq_impl(c: Context)(args: c.Expr[Any]*):

c.Expr[DBObject] = ???}

Wrapping it into macro

implicit class MongoHelper(val sc: StringContext) extends AnyVal {def mq(args: Any*): DBObject = macro MongoHelper.mq_impl

}

object MongoHelper {def mq_impl(c: Context)(args: c.Expr[Any]*):

c.Expr[DBObject] = {import c.universe._

val q"$cn(scala.StringContext.apply(..$pTrees))"= c.prefix.tree

val parsed = parse(c)(pTrees)wrapObject(c)(parsed, args.map(_.tree).iterator)

}}

Wrapping it into macro

object MongoHelper {

def parse(c: Context)(pTrees: List[c.Tree]) = {import c.universe._

val parts = pTrees map {case Literal(Constant(s: String)) => s

}

parser.parse(parts) match {case Success(v, _) => vcase NoSuccess(msg, reader) =>

val partIndex = reader.asInstanceOf[PartReader].partval pos = pTrees(partIndex).posc.abort(pos.withPoint(pos.point + reader.offset)),

s"Invalid BSON object: $msg")}

}}

Parsing BSON

mq"{ name : $name, age : { $$gte : 18, $$lte : $max }}"

Lexical

List("{", Field("name"), ":", Placeholder , ",", Field("age"),":", "{", Keyword("$gte"), ":", NumericLit(18), ",",Keyword("$lte"), ":", Placeholder , ",", "}", "}")

Syntactical

Object(List((Member("name"), Placeholder),(Member("age"), Object(List(

(Keyword("$gte"), 18),(Keyword("$lte"), Placeholder))

))))

Parsing BSON

mq"{ name : $name, age : { $$gte : 18, $$lte : $max }}"

Lexical

List("{", Field("name"), ":", Placeholder , ",", Field("age"),":", "{", Keyword("$gte"), ":", NumericLit(18), ",",Keyword("$lte"), ":", Placeholder , ",", "}", "}")

Syntactical

Object(List((Member("name"), Placeholder),(Member("age"), Object(List(

(Keyword("$gte"), 18),(Keyword("$lte"), Placeholder))

))))

Parsing BSON

mq"{ name : $name, age : { $$gte : 18, $$lte : $max }}"

Lexical

List("{", Field("name"), ":", Placeholder , ",", Field("age"),":", "{", Keyword("$gte"), ":", NumericLit(18), ",",Keyword("$lte"), ":", Placeholder , ",", "}", "}")

Syntactical

Object(List((Member("name"), Placeholder),(Member("age"), Object(List(

(Keyword("$gte"), 18),(Keyword("$lte"), Placeholder))

))))

Create objects

protected def wrapObject(c: Context)(obj: Object,args: Iterator[c.Tree]): c.Expr[DBType] = {

val dbparts = obj.members.map {case (lv, v) => (lv.asString , wrapValue(c)(v, args))

}

c.Expr(q"com.mongodb.casbah.commons.MongoDBObject(..$dbparts)")}

Wrapping values

protected def wrapValue(c: Context) (value: Any,args: Iterator[c.Tree]): c.Expr[Any] = {

import c.universe._value match {

case BSON.Placeholder =>c.Expr(args.next())

case o: BSON.Object =>wrapObject(c)(o, args)

case BSON.Id(id) =>c.Expr(q"new org.bson.types.ObjectId($id)")

case a: List[_] =>val wrapped = a.map(i => wrapValue(c)(i, args))c.Expr[List[Any]](q"List(..$wrapped)")

case v =>c.Expr[Any](Literal(Constant(v)))

}}

Wrapping values

protected def wrapValue(c: Context) (value: Any,args: Iterator[c.Tree]): c.Expr[Any] = {

import c.universe._value match {

case BSON.Placeholder =>c.Expr(args.next())

case o: BSON.Object =>wrapObject(c)(o, args)

case BSON.Id(id) =>c.Expr(q"new org.bson.types.ObjectId($id)")

case a: List[_] =>val wrapped = a.map(i => wrapValue(c)(i, args))c.Expr[List[Any]](q"List(..$wrapped)")

case v =>c.Expr[Any](Literal(Constant(v)))

}}

Wrapping values

protected def wrapValue(c: Context) (value: Any,args: Iterator[c.Tree]): c.Expr[Any] = {

import c.universe._value match {

case BSON.Placeholder =>c.Expr(args.next())

case o: BSON.Object =>wrapObject(c)(o, args)

case BSON.Id(id) =>c.Expr(q"new org.bson.types.ObjectId($id)")

case a: List[_] =>val wrapped = a.map(i => wrapValue(c)(i, args))c.Expr[List[Any]](q"List(..$wrapped)")

case v =>c.Expr[Any](Literal(Constant(v)))

}}

Wrapping values

protected def wrapValue(c: Context) (value: Any,args: Iterator[c.Tree]): c.Expr[Any] = {

import c.universe._value match {

case BSON.Placeholder =>c.Expr(args.next())

case o: BSON.Object =>wrapObject(c)(o, args)

case BSON.Id(id) =>c.Expr(q"new org.bson.types.ObjectId($id)")

case a: List[_] =>val wrapped = a.map(i => wrapValue(c)(i, args))c.Expr[List[Any]](q"List(..$wrapped)")

case v =>c.Expr[Any](Literal(Constant(v)))

}}

Wrapping values

protected def wrapValue(c: Context) (value: Any,args: Iterator[c.Tree]): c.Expr[Any] = {

import c.universe._value match {

case BSON.Placeholder =>c.Expr(args.next())

case o: BSON.Object =>wrapObject(c)(o, args)

case BSON.Id(id) =>c.Expr(q"new org.bson.types.ObjectId($id)")

case a: List[_] =>val wrapped = a.map(i => wrapValue(c)(i, args))c.Expr[List[Any]](q"List(..$wrapped)")

case v =>c.Expr[Any](Literal(Constant(v)))

}}

Type safety

mqt – typechecking interpolator

case class Phone(kind: String, number: String)case class Person(name: String, age: Int, phone: List[Phone])

// OKpersons.update(mq"{}", mqt"{ $$inc : { age : 1 } }"[Person])persons.find(mqt"{ phone.number : '223322' }"[Person])

// COMPILE ERRORpersons.update(mq"{}", mqt"""{$$set : { nme : "Joe" }}"""[Person])persons.find(mqt"{ name.1 : 'Joe' }"[Person])persons.find(mqt"{ phone.num : '223322' }"[Person])

mqt – typechecking interpolator

case class Phone(kind: String, number: String)case class Person(name: String, age: Int, phone: List[Phone])

// OKpersons.update(mq"{}", mqt"{ $$inc : { age : 1 } }"[Person])persons.find(mqt"{ phone.number : '223322' }"[Person])

// COMPILE ERRORpersons.update(mq"{}", mqt"""{$$set : { nme : "Joe" }}"""[Person])persons.find(mqt"{ name.1 : 'Joe' }"[Person])persons.find(mqt"{ phone.num : '223322' }"[Person])

mqt – typechecking interpolator

case class Phone(kind: String, number: String)case class Person(name: String, age: Int, phone: List[Phone])

// OKpersons.update(mq"{}", mqt"{ $$inc : { age : 1 } }"[Person])persons.find(mqt"{ phone.number : '223322' }"[Person])

// COMPILE ERRORpersons.update(mq"{}", mqt"""{$$set : { nme : "Joe" }}"""[Person])persons.find(mqt"{ name.1 : 'Joe' }"[Person])persons.find(mqt"{ phone.num : '223322' }"[Person])

Passing type into intepolator

implicit class MongoHelper(val sc: StringContext) extends AnyVal {def mq(args: Any*): DBObject = macro MongoHelper.mq_impl

def mqt(args: Any*) = new QueryWrapper}

class QueryWrapper {def apply[T]: DBObject = macro MongoHelper.mqt_impl[T]

}

object MongoHelper {

def mqt_impl[T: c.WeakTypeTag](c: Context):c.Expr[DBObject] = ???

}

Inside mqt_impl

def mqt_impl[T: c.WeakTypeTag](c: Context): c.Expr[DBObject] = {val q"$cn(scala.StringContext.apply(..$pTrees)).mqt(..$aTrees)"

= c.prefix.treeval args = aTrees.map(c.Expr(_))val parsed = parse(c)(pTrees)

checkObject(c)(c.weakTypeOf[T], parsed)

wrapObject(c)(parsed, args.iterator)}

Verifing the type

def checkType(c: Context)(tpe: c.Type, obj: Object) = {import c.universe._

val ctor = tpe.decl(termNames.CONSTRUCTOR).asMethodval params = ctor.paramLists.headval className = t.typeSymbol.name.toString

val fields = params.map(s => s.name.toString -> s).toMap

obj.members.foreach { case (m, _) =>if (!fields.contains(m.name)) {

c.abort(c.enclosingPosition ,s"Class $className doesn't contain field '${m.name}'")

}}

}

Testing interpolator

it should "support nested objects" in {val q = mq"""{ user : "Joe", age : {$$gt : 25}}"""q should equal(MongoDBObject("user" -> "Joe",

"age" -> MongoDBObject("$gt" -> 25)))}

Testing error scenarios

import scala.reflect.runtime.{ universe => ru }

class CompileTest extends FlatSpec {

val cl = getClass.getClassLoader.asInstanceOf[URLClassLoader]val cp = cl.getURLs.map(_.getFile).mkString(File.pathSeparator)val mirror = ru.runtimeMirror(cl)val tb = mirror.mkToolBox(options = s"-cp $cp")

def getError(q: String): String = {val e = intercept[ToolBoxError] {

tb.eval(tb.parse(q))}e.message

}

it should "fail on malformed BSON objects" in {val e = getError("""mq"{ test 5 }" """)e should include("`:' expected, but 5 found")

}}

Testing error scenarios

import scala.reflect.runtime.{ universe => ru }

class CompileTest extends FlatSpec {

val cl = getClass.getClassLoader.asInstanceOf[URLClassLoader]val cp = cl.getURLs.map(_.getFile).mkString(File.pathSeparator)val mirror = ru.runtimeMirror(cl)val tb = mirror.mkToolBox(options = s"-cp $cp")

def getError(q: String): String = {val e = intercept[ToolBoxError] {

tb.eval(tb.parse(q))}e.message

}

it should "fail on malformed BSON objects" in {val e = getError("""mq"{ test 5 }" """)e should include("`:' expected, but 5 found")

}}

Testing error scenarios

import scala.reflect.runtime.{ universe => ru }

class CompileTest extends FlatSpec {

val cl = getClass.getClassLoader.asInstanceOf[URLClassLoader]val cp = cl.getURLs.map(_.getFile).mkString(File.pathSeparator)val mirror = ru.runtimeMirror(cl)val tb = mirror.mkToolBox(options = s"-cp $cp")

def getError(q: String): String = {val e = intercept[ToolBoxError] {

tb.eval(tb.parse(q))}e.message

}

it should "fail on malformed BSON objects" in {val e = getError("""mq"{ test 5 }" """)e should include("`:' expected, but 5 found")

}}

Summary

ConsI Not easy to implement

I Not highlighted in IDE

Pros

I Less limitations on language structureI Can preserve existing languageI Martin said that string interpolation is cool

Summary

ConsI Not easy to implementI Not highlighted in IDE

Pros

I Less limitations on language structureI Can preserve existing languageI Martin said that string interpolation is cool

Summary

ConsI Not easy to implementI Not highlighted in IDE

Pros

I Less limitations on language structureI Can preserve existing languageI Martin said that string interpolation is cool

Summary

ConsI Not easy to implementI Not highlighted in IDE

ProsI Less limitations on language structure

I Can preserve existing languageI Martin said that string interpolation is cool

Summary

ConsI Not easy to implementI Not highlighted in IDE

ProsI Less limitations on language structureI Can preserve existing language

I Martin said that string interpolation is cool

Summary

ConsI Not easy to implementI Not highlighted in IDE

ProsI Less limitations on language structureI Can preserve existing languageI Martin said that string interpolation is cool

k

Thanks. Questions?