polyglot architecture: a rational approach to software design
TRANSCRIPT
Polyglot Architecture: A Rational Approach to Software Design
Richard Minerich Senior Researcher at Bayard Rock @Rickasaurus
Research and Development Perspective
▪ A focus on long term solutions to complex problems
▪ A lot of room for experimentation, but decisions must be justified
▪ Very small teams, and so few resources for maintenance, and little time for production issues
▪ Talented people who are generally excited about learning new things
Why do we use the languages we use?
▪ I always use the same one or two languages.
▪ Pseudo-Technical management picks all of our technologies.
▪ I need to Maximize Synergy by Hadooping the Big Data Analytics.
▪ I am fanatically obsessed with a particular language.
▪ Weighted random sampling fed by the benchmark game.
▪ We came up with a set of criteria and judged several candidates empirically in context.
Considerations (oh god)
Safety, Finding Talent, CPU Performance, Training, Platform Limitations, Compiler Support, Community, External Libraries, Tooling, Debugging, Primary Community Focus, Unique Features, Simplicity, Development Environments, Familiarity, Cross Platform Support, Asynchrony, Hype, Industry Standards, Algorithms, Tutorials, Data Structures, Stability, Memory Usage, Garbage Collection, Syntactic Flexibility, Compilation Targets, Language/Platform Interop, Taste, Core Libs, Code Clarity, Licensing, Error Messages, Build Tools, Testability, Requirements, Type Systems, Embeddability, Scalability, Fault Tolerance, Parallelization, Binary Package Size, Permissions, Algebraic Data Types, Obfuscation, Peer Pressure, Technical Debt, Does it have what plants crave?
Considerations (fixed)
Safety – “In the limit”, cost of unexpected changes, testing burden, sanity of core libs, dependency handling
Community – Support options, library quantity and quality, training, hiring, enthusiasm, culture of craftsmanship
Performance – Relative dimensions include cpu, memory and network
Tooling – Debugging, static analysis, linting, test frameworks
And they’re all tradeoffs based on Context.
i.e. Requirements, Platform(s), Team Skills, Available Funds
Case Study: 0install (by Thomas Leonard) Choosing a Replacement for Python
Language OCaml Python Haskell Rust ATS C# Go
Speed 4 2 4 3 5 1 3
Dependencies 4 5 3 2 5 1 3
Bin. compatibility 4 5 2 2 4 5 3
Bad stdout 1 1 1 5 1 1 1
Missing env 5 5 5 5 5 3 1
Memory safety 5 5 5 4 3 5 5
Diagnostics 1 5 3 3 2 1 1
Ease of coding 4 5 4 3 1 4 3
Shared libraries 1 5 2 2 5 5 1
Static types 4 1 4 5 5 3 2
Privilege bounds 5 1 5 1 2 5 1
Mutability 4 3 2 5 4 4 4
C interoperability 3 3 4 4 5 4 4
Asynchronous 5 4 5 5 1 5 5
Total 50 50 49 49 48 47 37
http://is.gd/0install_language_shootout
What is Safety? (My favorite topic)
Imagine your goal is perfect bug-free software (in the limit)
▪ Is the compiler your friend?
▪ Is it easy to write correct code?
▪ Will it “fail fast” when
in an unexpected state?
▪ When things fail will it
get up and keep going?
▪ Can you trust external libraries?
Context is everything
▪ Projected lifetime, time to market, burn rate
▪ Explicit requirements, technology interop, customer systems
▪ Current talent, willingness to learn, and training expenditure
▪ Previous purchases and existing platform limitations
▪ Domain problems that are helped greatly certain unique features
It Usually Comes Down To Unique Features
▪ JavaScript – Do we really have any other choice?
▪ C# – Great GUI builders, code generation tooling
▪ F# – Type providers, data modeling and transformation
▪ Matlab – Fast linear algebra, tons of high quality algorithms
▪ C++ (/CLI) – Library selection, managed-unmanaged interop
Currently Under Consideration
▪ R – table structured data analysis and visualization, type provider
▪ Haskell – Unique libraries, fantastically safe
▪ Scala – Beautiful and concise leverage of Spark/Hadoop, quite safe
Isn’t writing your own language generally a bad idea?
What I found for scripting options on .NET:
IronPython, IronRuby, IronJS, DynamicLinq…
DynamicLinq was almost exactly what I wanted. Almost…
…If it weren’t for the darned manual conversions…
… So after much consideration, I wrote Barb.
Barb?! (github.com/Rickasaurus/Barb) It’s a simple .net scripting language
Name.Contains "John“ and (Age > 20 or Weight > 200)
Barb in Action: Safe Alert Manager
Slots to fill (Safe Alert Manager)
▪ User Interface
▪ User Behavior Model
▪ Data Access
▪ Data Transform
▪ Algorithms / Data Structures
▪ Ad-Hoc Behaviors
▪ Glue
UI (C#) & Analysis (C#)
Glue (F# and Barb)
Data & Config
In
Data Out
Algorithms (F#)
Key Insight: Reverential Transparency
Matlab C++ Haskell R Python
F# Type Providers Typed Access to Data and Languages
MAP: Secure Distributed ML
UI (JS), Analysis (JS & SQL) & Computation (F#)
Glue (F#)
Data
Computation in F# or via Type
Provider in: Matlab, R,
Python
Goal: Experts and users coexist in the same system with different tools.
Typed Dataset Representation
MAP Language Choices
▪ User Interface / Model – Javascript/Type Script
▪ Analysis Data Access – BRSQL in the UI
▪ Analysis Data Transform – Javascript in the UI
▪ Glue – Almost entirely F#
▪ Algorithms / Data Structures – F#, Type Provider, C++/CLI
On the Horizon: Asm.js (with emscripten) is.gd/cool_asmjs_demos
On the Horizon: Rust www.rust-lang.org
Thanks for Coming!
@Rickasaurus
RichardMinerich.com
Come visit the NYC Haskell and F# User Groups!