Torch InternalsRonan Collobert locronan@fb.com torch.ch

Introduction

Machine Learning Library

• Around since 2000

• 5 versions, implemented in C, C++, Objective C…

• Implements main machine learning algorithms:

• SVMs, Neural Networks, HMMs, GMMs…

• Goal: research on large-scale machine learning

It is for researchers…• It has to be modular!

organized as packages

• It must be easy to extend!

• It must be simple to understand!

a high-level language is essential

• It must be fast (large-scale machine learning)

JIT-compile or

efficient C interface for core routines

Overview

Overview

TH (C)

torch (lua)

optim

nn

svm

sndfile

image

nngraph

Overview

TH (C)

torch (lua)

optim

nn

svm

sndfile

image

nngraph

core

Overview

TH (C)

torch (lua)

optim

nn

svm

sndfile

image

nngraph

core

packages

Overview

TH (C)

torch (lua)

optim

nn

svm

sndfile

image

nngraph

core

packages

Overview

TH (C)

torch (lua)

optim

nn

svm

sndfile

image

nngraph

core

packages

Overview

TH (C)

torch (lua)

optim

nn

svm

sndfile

image

nngraph

core

packages

Overview

TH (C)

torch (lua)

optim

nn

svm

sndfile

image

nngraph

core

packages

luaT (C)

cwrap

glue

Which Programming Language?

• LISP? (Lush http://lush.sourceforge.net)

• A subset could be compiled

• Could inline C code

• JIT-compiled Languages? (e.g. LuaJIT)

• Only a subset is actually compiled

• Python?

• err… Lua?

• Looks like pseudo-code (yet very powerful)

• Simple C API

Which Programming Language?

• C++? (Torch3, PLearn, EBLearn…)

• Objective C? (Torch4)

• Avoid too many abstractions

• Avoid complicated syntax

• We chose C for Torch7

The CoreTH and torch

The core library (TH+torch)

• ML algorithms manipulate all kind of data

• Represent data as a n-dimensional Tensor

• 1D: a bunch of features

• 2D: a gray image, some audio features

• 3D: RGB images

• 4D: Videos…

• The core lib provides many tensor operations

• Available from C (TH lib) and Lua (torch package)

The Core Library (TH+torch)• Avoid memory copies

• A Tensor is a view of a Storage (memory chunk)

• Storages might have different views (Tensors)

• Storages can be in-memory, or on-disk (mmap!)

The Core Library (TH+torch)• Avoid memory copies

• A Tensor is a view of a Storage (memory chunk)

• Storages might have different views (Tensors)

• Storages can be in-memory, or on-disk (mmap!)

• Easy to get different views from another tensor x:

• x:narrow(dim, idx, size)

• x:select(dim, idx)

• x[idx]

• x:unfold(dim, kw, dw)

• x:transpose(dim1, dim2)

The Core Library (TH+torch)• Avoid memory copies

• A Tensor is a view of a Storage (memory chunk)

• Storages might have different views (Tensors)

• Storages can be in-memory, or on-disk (mmap!)

• Easy to get different views from another tensor x:

• x:narrow(dim, idx, size)

• x:select(dim, idx)

• x[idx]

• x:unfold(dim, kw, dw)

• x:transpose(dim1, dim2)

new tensors sharesame storage

The Core Library (TH+torch)

• Avoid memory copies

• All C functions are of the form

• THTensor_foobar(THTensor *dst, THTensor *src1, THTensor *src2)

• The Lua interface supports the destination as an optional argument:

torch.foobar([dst], src1, src2)

or

dst:foobar(src1, src2)

The GlueluaT and cwrap

C/Lua Glue• Lua is a stack-based language

• In Lua: a = f("how", t.x, 14)

• In C:

• How to extend it? (luaT or FFI)

• Lua provides the “userdata” type (a GCed C pointer)

• Need to do proper type checking both in C and Lua

• How to limit the pain? (cwrap or FFI)

C/Lua Glue (luaT)

C/Lua Glue (luaT)

check argumentson the stack

C/Lua Glue (luaT)

check argumentson the stack

push the result

C/Lua Glue (luaT)

check argumentson the stack

push the resultcall TH

C/Lua Glue (cwrap)

C/Lua Glue (cwrap)

for all tensor types

C/Lua Glue (cwrap)

for all tensor typestorch.cmul(dst, src)

ordst:cmul(src)

C/Lua Glue (cwrap)

C/Lua Glue (cwrap)

another default value

C/Lua Glue (cwrap)

two different C functions

another default value

C/Lua Glue (FFI)

• Cut-and-paste the C header

• Caveats:

• Need to do proper argument checking (sub-classes…)

• Need to overload methods/functions properly

• Less robust across different systems (#define…)

• Need also to handle different types (templates)

Tensor Types

Dynamic Typing (lua)• Write generic code with torch.Tensor()

• Specify alias with (e.g.) torch.setdefaulttensortype(‘torch.FloatTensor’)

Dynamic Typing (lua)• Write generic code with torch.Tensor()

• Specify alias with (e.g.) torch.setdefaulttensortype(‘torch.FloatTensor’)

return atorch.Tensor()

CUDA

• cutorch package defines CudaTensor

• Most CPU Tensor methods are available

• Relies on few custom-made iterators and thrust

• x:cuda() or x:float() for GPU <-> CPU copies

• cunn package provides a GPU backend to nn

• Most popular nn layers have a GPU backend

Deep Learningnn

nn package

nn package

nn package

• Two main classes: Module and Criterion

• Three main methods:

• updateOutput(input)

• updateGradInput(input, gradOutput)

• accGradParameters(input, gradOutput)

nn package

Tutorial• Torch7

• https://github.com/torch/torch7

• nn package

• https://github.com/torch/nn

• itorch

• https://github.com/facebook/iTorch

• Installation instructions

• http://torch.ch/docs/getting-started.html

• Get the tutorial

• http://ronan.collobert.com/torch