what's next in julia

What’s next in Jul_

Jiahao Chen MIT Computer Science and Artificial Intelligence Laboratory

julialang.org

Alan EdelmanJeff Bezanson Stefan Karpinski Viral B. Shah

http://julialang.org

What’s the big deal about Julia ?

julialang.org/benchmarks

http://julialang.org/benchmarks

It bridges the divide between computer science and computational science




data abstraction

performance



data abstraction

performance

What if you didn’t have to choose between data abstraction and performance?

It’s a programming language designed for technical computing


It’s a programming language designed for technical computing


The key ingredients

Multi-methods (multiple dispatch)

Dataflow type inference

together allow for cost-efficient data abstraction

Object-oriented programming with classes

What can I do with/to a thing?



top uppay fare

lose

buy



top uppay fare

lose

buy

top uppay fare

lose

buy



top uppay fare

lose

buy

top uppay fare

lose

buy

pay farelose

buy

methods objects



top uppay fare

lose

buy

top uppay fare

lose

buy

pay farelose

buy

methods objects



top uppay fare

lose

buy

top uppay fare

lose

buy

pay farelose

buy

class-based OO !classes are more fundamental than methods

Object-oriented programming with multi-methods


top up

pay fare

lose

buy

generic function

objectsmethods

Object-oriented programming with multi-methods


top up

pay fare

lose

buy

generic function

objectsmethods

multimethods !relationships between objects and functions

Multi-methods with type hierarchy

top up

pay fare

lose

buy

generic function

objectsmethods

rechargeable subway pass

single-use subway ticket

is a subtype of

subway ticket

abstract object

generic function

objectsmethods



is a subtype of

subway ticket

top up

pay fare

lose

buy

abstract object


generic function

objectsmethods



is a subtype of

subway ticket

top up

pay farelose

buy

abstract object


The Julia codebase is compact

5,774 lines of Scheme 32,707 lines of C 59727 lines of Julia !+LLVM, BLAS, LAPACK, SuiteSparse, ARPACK, Rmath, GMP, MPFR, FFTW,…

Data types as a lattice

Dana Scott, Data types as lattices, SIAM J. Comput. 5: 522-87. 1976

Real

Number

FloatingPoint Rational

Complex

Float64 BigFloat…

…

Integer

is a parameter ofis a subtype of

Signed BigInt

Any

Int64…

…

Signed

Multiple dispatch with type lattice traversal

Real

Number


Float64 BigFloat…

Integer


BigInt

Any

Int64…

…

Signed


Real

Number


Float64 BigFloat…

Integer


BigInt

Any

Int64…

…

no method here try supertype

super(Int64) = Signed

Signed


Real

Number


Float64 BigFloat…

Integer


BigInt

Any

Int64…

…



no method here either super(Signed) = Integer

Signed


Real

Number


Float64 BigFloat…

Integer


BigInt

Any

Int64…

…



no method here either super(Signed) = Integer

found a method

Signed


Real

Number


Float64 BigFloat…

Integer


BigInt

Any

Int64…

…

Julia

LLVM IR

Machine assembly

aggressive type specialization !compiler generates specialized methods for different input types to the same generic function

unitful computations with essentially no runtime overhead

Keno Fischer Harvard Physics and Mathematics

Type lattice of arrays

DenseArray{T, N}

AbstractArray{T, N}

Array{T, N}

…

is a subtype of

Any

Array{T,1}===Vector{T}Array{T,2}===Matrix{T}

A single parametric type defines many types of matrices Matrix{Float64}, Matrix{Int64},Matrix{BigFloat}, Matrix{Complex128},Matrix{Rational}, Matrix{Quaternion{Float64}},…

Type lattice of arrays

DenseArray{T, N}

AbstractArray{T, N}

Array{T, N}

…

is a subtype of

Any

Array{T,1}===Vector{T}Array{T,2}===Matrix{T}

rotation matrices structured matrices

distributed arrays

A single parametric type defines many types of matrices Matrix{Float64}, Matrix{Int64},Matrix{BigFloat}, Matrix{Complex128},Matrix{Rational}, Matrix{Quaternion{Float64}},…

Multi-methods for linear algebra


find eigenvalues and eigenvectors

find singular values

find singular values and vectors

find eigenvalues

generic function

objectsmethods

general matrix

symmetric tridiagonal matrix

bidiagonal matrix

Methods can take advantage of special matrix structures

eigvals

eigfact

svdvals

svdfact

Matrix

SymTridiagonal

Bidiagonal

So how does this help us with linear algebra?


Multi-method dispatch on special matrix types


Multi-method dispatch on special matrix types

stev!{T<:BlasFloat} calls sgestvdgestvcgestvzgestv

and handles workspace memory allocation


Multi-method dispatch with generic fallbacks

Matrix operations on general rings



Matrix operations on general rings textbook algorithm



Matrix operations on general rings

Matrix factorization types

Iterative algorithms as iterators

for item in iterable #bodyend!

#is equivalent to!

state = start(iterable) while !done(iterable, state) item, state = next(iterable, state) # bodyend

Conjugate gradients (Hestenes-Stiefel)

http://en.wikipedia.org/wiki/Conjugate_gradient_method




Can we write this as an iterator?




Can we write this as an iterator?

state = start(iterable)

done(iterable, state)




state = start(iterable)

done(iterable, state)

state, dx = next(iterable, state)


immutable cg_hs #iterative solver K :: KrylovSpace #wraps A, v0, k t :: Terminator #termination criteriaend immutable cg_hs_state r :: Vector #residual p :: Vector #search direction rnormsq :: Float64 #Squared norm of previous residual iter :: Int #iteration countend start(a::cg_hs) = cg_hs_state(a.K.v0, zeros(size(a.K.v0,1)), Inf, 0) function next(a::cg_hs, s::cg_hs_state) rnormsq = dot(s.r, s.r) p = s.r + (rnormsq/s.rnormsq)*s.p Ap = a.K.A*p α = rnormsq / dot(p, Ap) α*p, cg_hs_state(s.r-α*Ap, p, rnormsq, s.iter+1)end done(a::cg_hs, s::cg_hs_state) = done(a.t, s)!K = method(KrylovSpace(A, b, k), Terminator())x += reduce(+, K) # for dx in K; x += dx; end

Hestenes-Stiefel CG

immutable cg_hs #iterative solver K :: KrylovSpace #wraps A, v0, k t :: Terminator #termination criteriaend immutable cg_hs_state r :: Vector #residual p :: Vector #search direction rnormsq :: Float64 #Squared norm of previous residual iter :: Int #iteration countend start(a::cg_hs) = cg_hs_state(a.K.v0, zeros(size(a.K.v0,1)), Inf, 0) function next(a::cg_hs, s::cg_hs_state) rnormsq = dot(s.r, s.r) p = s.r + (rnormsq/s.rnormsq)*s.p Ap = a.K.A*p α = rnormsq / dot(p, Ap) α*p, cg_hs_state(s.r-α*Ap, p, rnormsq, s.iter+1)end done(a::cg_hs, s::cg_hs_state) = done(a.t, s)!K = method(KrylovSpace(A, b, k), Terminator())x += reduce(+, K) # for dx in K; x += dx; end

Hestenes-Stiefel CG

Abstracts out termination check and solution update steps

Native parallelism constructs

Distributed arrays

IJulia: Julia in IPython Notebook

JuMP: writing simple DSLs in Julia

Iain Dunning Miles LubinMIT

Operations Research

Andreas N. Jensen U. Copenhagen

Economics

Alan EdelmanJeff Bezanson Stefan Karpinski Viral B. Shah

Carlo Baldassi Poly. Torino

Neuroscience

Tim E. Holy WUSTL

Anatomy

Douglas M. Bates Wisconsin-Madison

Statistics

Steven G. Johnson MIT

Mathematics

what's next in julia

Data & Analytics