gore: go repl
TRANSCRIPT
A Tale of Go REPLmotemen Go Conference 2015 Summer @ Tokyo
About Me
- @motemen
- Web engineer
- Chief engineer at Hatena, Kyoto
Mackerel mackerel.io
Agenda
- Introduction of ! motemen/gore
- Gore’s architecture
- Implementation details
Gore /gɔː/
Case: Learning New Languages
- Playing with language features
- Trying libraries
My Case — Perl
- perl ~/sketch.pl
- Edit and Run
My Case — Ruby- irb
- pry
- A “REPL”
(Read-Eval-Print-Loop)
My Case — Go?- tmp.go
- “packge main” “func main()”
- Removing unused variables by hand
- Question: Can gophers haz a REPL?
Go "REPL" impls (AFAIK)- ! sbinet/igo — built upon ! sbinet/go-eval, with liner
- ! emicklei/rango — importing, statements, listing variables
- ! vito/go-repl — importing, statements, useful commands
- ! d4l3k/go-pry — attaches to running program, uses reflection
- (play.golang.org)
Needed a REPL that:- Evaluates statements
- Imports any module
- Pretty prints values automatically
- Completes code
The Difficulties
- How can we achieve Go semantics?
- No APIs provided
- Re-implement Go compiler? 😩
Solution: “go run”
Solution: “go run”- An “eval” to Go
- Perfect implementation & Always up-to-date
- Dead simple
- In: a program Out: the result 🎉
Caveats 🙏
- Gore runs all code input for each run
- Code with side effects will be repeated
- eg. Printing output / Sleeping
Stories Inside Gore
Overview: not a “REPL” exactly- REPL: Read-Eval-Print-Loop
- No “Eval” and “Print” phase
- As we do them by “go run”
- Much like: Read-Generate-Run-Loop
Read-Gen-Run-Loop1. Read user input to evaluate
2. Generate a .go file that prints result
3. “go run” it
4. Back to 1.
Step: Read- Codes go under AST (abstract syntax tree) form
- Syntax checking
- Easier to manipulate
- Input String → AST → .go → “go run”
Read: Input- ! peterh/liner
- Line editing
- History
- Supports Windows
Read: Input to AST
- package go/parser
- Go has rich support for parsing/typing its code
- Easy to generate & manipulate
Read: Input to AST
- Parse input as an expression
- If failed: a statement
- Otherwise: continue input (multi-line)
Read: Expression
- Easy
- Dedicated API
- parser.ParseExpr(in)
Read: Statements - $
func (s *Session) evalStmt(in string) error { src := fmt.Sprintf("package P; func F() { %s }", in) f, err := parser.ParseFile(s.Fset, "stmt.go", src, 0) if err != nil { return err }
enclosingFunc := f.Scope.Lookup("F").Decl.(*ast.FuncDecl) stmts := enclosingFunc.Body.List }
Generate: The Delicious Part
- Append the input AST nodes to our main()
- Add the code to print the value
Printing Values- The generated program prints values (not gore)
- “New Values” should be printed
- Expression: Its resulting value
- Statement: Values assigned
Printing Values: Expression
- User: foo.Bar(a+1)
- Gore: __gore_p(foo.Bar(a+1))
Printing Values: Statements
- ast.AssignStmt stands for assign/define
- User: a, b := foo()
- Gore: a, b := foo(); __gore_p(a, b)
Printing Values: __gore_p()- “Pretty prints” values
- Defined along with the main()
- Depends on installed packages
- ! k0kubun/pp, ! davecgh/go-spew or plain old %#v
The Initial .go File (pp)package main
import "github.com/k0kubun/pp"
func __gore_p(xx ...interface{}) { for _, x := range xx { pp.Println(x) } } func main() { // User input goes here }
The Initial .go File (go-spew)package main
import "github.com/davecgh/go-spew/spew"
func __gore_p(xx ...interface{}) { for _, x := range xx { spew.Printf("%#v\n", x) } } func main() { // User input goes here }
The Initial .go File (%#v)package main
import "fmt"
func __gore_p(xx ...interface{}) { for _, x := range xx { fmt.Printf("%#v\n", x) } } func main() { // User input goes here }
Generate: Append to main()
- Just append those generated statements
s.mainBody.List = append(s.mainBody.List, stmts…)
- Now we’ve got the working code!
- Really? No! %
Go compiler complaints (you know)
- “x declared but not used”
- “p imported but not used”
- “no new variables of left side of :=“
- & Should remove these to a successful go run
Quick Fixing Erroneous Program
- “x declared but not used”
- “p imported but not used”
- “no new variables on left side of :=“
Use it!!
Anonymize it!!
Make it `=‘ !!
“declared but not used”
package P
func main() { s := "hello" }
package P
func main() { s := "hello" _ = s }
”imported but not used”package P
import "fmt"
func main() { }
package P
import _ "fmt"
func main() { }
“no new variables on left side of :=“
package P
func main() { var a int a := 1 }
package P
func main() { var a int a = 1 }
! motemen/go-quickfix to do the work
- Type check source code using go/types.Check()
- Catch errors and modify AST
- Packed with a bin
- goquickfix -w main.go
Gore specific quickfix
- __gore_p() for non-value expressions
Running
- Output the AST to file — go/printer
- Then go run
- If failed to run, revert the last input
Recap: Read-Gen-Quickfix-Run-Loop1. Read and parse the input to obtain AST
2. Add printing function call __gore_p()
3. Append the code to main()
4. Quickfix it so that it compiles well
5. go run
PRs welcome! 🍻
- ! motemen/gore
- ! motemen/go-quickfix
A Tale Goremotemen
Go Conference 2015 Summer @ Tokyo
Appendix:
More Features
Code Completion- liner has support for completion
- Great tool for editors: ! nsf/gocode
- Server/client model
- And an interface to it: motemen/gore/gocode
- If gocode binary can be located, use it
Commands- :print
- :write <file>
- :import <pkg>
- :doc <expr>
:import <pkg>
- Imports arbitrary packages
- Just append import statement to the file
- Unused imports are handled by quickfix
:doc <expr>- Invokes go doc for given expression
- Package: :doc json
- Function: :doc json.Marshal
- Method: :doc json.NewEncoder().Encode