Download - Seamless R and C++ Integration with Rcpp: Part 2 ...dirk.eddelbuettel.com/papers/rcpp_ku_nov2013-part2.pdf · IntroExFastLMKalmanSparseXPtr Seamless R and C++ Integration with Rcpp:

Intro Ex FastLM Kalman Sparse XPtr

Seamless R and C++Integration with Rcpp:

Part 2 – RcppArmadillo Examples

Dirk [email protected]

Center for Research Methods and Data AnalysisUniversity of KansasNovember 16, 2013

Dirk Eddelbuettel Rcpp Intro & Examples Part 2

[email protected]

Intro Ex FastLM Kalman Sparse XPtr Armadillo Users

Outline

1 IntroArmadilloUsers



Armadillo



What is Armadillo?From arma.sf.net and slightly edited

Armadillo is a C++ linear algebra library (matrix maths)aiming towards a good balance between speed and easeof use.

The syntax is deliberately similar to Matlab.

Integer, floating point and complex numbers aresupported.

A delayed evaluation approach is employed (atcompile-time) to combine several operations into one andreduce (or eliminate) the need for temporaries.

Useful for conversion of research code into productionenvironments, or if C++ has been decided as the languageof choice, due to speed and/or integration capabilities.



What is Armadillo?From arma.sf.net and slightly edited

Armadillo is a C++ linear algebra library (matrix maths)aiming towards a good balance between speed and easeof use.

The syntax is deliberately similar to Matlab.

Integer, floating point and complex numbers aresupported.

A delayed evaluation approach is employed (atcompile-time) to combine several operations into one andreduce (or eliminate) the need for temporaries.

Useful for conversion of research code into productionenvironments, or if C++ has been decided as thelanguage of choice, due to speed and/or integrationcapabilities.



Armadillo highlights

Provides integer, floating point and complex vectors,matrices and fields (3d) with all the commonoperations.Very good documentation and examples at websitehttp://arma.sf.net, a technical report(Sanderson, 2010)Modern code, building upon and extending fromearlier matrix libraries.Responsive and active maintainer, frequent updates.Used by MLPACK; cf Curtin et al (JMLR, 2013)


http://arma.sf.net

http://elec.uq.edu.au/~conrad/code.html

http://elec.uq.edu.au/~conrad/code.html

http://www.mlpack.org


RcppArmadillo highlights

Template-only builds—no linking, and availablewhereever R and a compiler work (but Rcpp isneeded)!Easy with R packages: just add LinkingTo:RcppArmadillo, Rcpp to DESCRIPTION (i.e., noadded cost beyond Rcpp)Data exchange really seamless from R via RcppFrequently updated; documentation includesEddelbuettel and Sanderson (CSDA, 2013/in press).


http://dx.doi.org/10.1016/j.csda.2013.02.005


Well-know packages using RcppArmadillo

Amelia by Gary King et al: Multiple Imputation fromcross-section, time-series or both;

forecast by Rob Hyndman et al: Time-seriesforecasting including state space andautomated ARIMA modeling;

rugarch by Alexios Ghalanos: Sophisticated financialtime series models;

gRbase by Søren Højsgaard: Graphical modeling


Intro Ex FastLM Kalman Sparse XPtr Eigenvalues Multiv. Normal RNGs

Outline

2 Simple ExamplesEigenvaluesMultivariate Nornal RNGs



Armadillo Eigenvalueshttp://gallery.rcpp.org/articles/armadillo-eigenvalues/

#include <RcppArmadillo.h>

// [[Rcpp::depends(RcppArmadillo)]]

// [[Rcpp::export]]arma::vec getEigenValues(arma::mat M) {

return arma::eig_sym(M);}



Armadillo Eigenvalueshttp://gallery.rcpp.org/articles/armadillo-eigenvalues/

set.seed(42); X <- matrix(rnorm(4*4), 4, 4)Z <- X %*% t(X); getEigenValues(Z)

## [,1]## [1,] 0.3319## [2,] 1.6856## [3,] 2.4099## [4,] 14.2100

# R gets the same results (in reverse)# and also returns the eigenvectors.



Multivariate Normal RNG Drawhttp://gallery.rcpp.org/articles/simulate-multivariate-normal/

#include <RcppArmadillo.h>// [[Rcpp::depends(RcppArmadillo)]]

// [[Rcpp::export]]arma::mat mvrnormArma(int n, arma::vec mu,

arma::mat sigma) {arma::mat Y = arma::randn(n, sigma.n_cols);return arma::repmat(mu, 1, n).t() +

Y * arma::chol(sigma);}



Outline

3 Case Study: FastLM



Faster Linear Model with FastLmBackground

Implementations of ‘fastLm()‘ have been a staple allalong the development of RcppThe very first version was in response to a questionby Ivo Welch on r-help.The request was for a fast function to estimateparameters – and their standard errors – from alinear model,It used GSL functions to estimate β̂ as well as itsstandard errors σ̂ – as lm.fit() in R only returnsthe former.It had since been reimplemented for RcppArmadilloand RcppEigen.



Faster Linear Model with FastLmInitial RcppArmadillo src/fastLm.cpp


extern "C" SEXP fastLm(SEXP Xs, SEXP ys) {

try {Rcpp::NumericVector yr(ys); // creates Rcpp vector from SEXPRcpp::NumericMatrix Xr(Xs); // creates Rcpp matrix from SEXPint n = Xr.nrow(), k = Xr.ncol();arma::mat X(Xr.begin(), n, k, false); // reuses memory and avoids extra copyarma::colvec y(yr.begin(), yr.size(), false);

arma::colvec coef = arma::solve(X, y); // fit model y∼ Xarma::colvec res = y - X*coef; // residualsdouble s2 = std::inner_product(res.begin(), res.end(), res.begin(), 0.0)/(n - k);arma::colvec std_err = // std.errors of coefficients

arma::sqrt(s2*arma::diagvec(arma::pinv(arma::trans(X)*X)));

return Rcpp::List::create(Rcpp::Named("coefficients") = coef,Rcpp::Named("stderr") = std_err,Rcpp::Named("df.residual") = n - k );

} catch( std::exception &ex ) {forward_exception_to_r( ex );

} catch(...) {::Rf_error( "c++ exception (unknown reason)" );

}return R_NilValue; // -Wall

}



Faster Linear Model with FastLmEdited version of RcppArmadillo’s src/fastLm.cpp

// [[Rcpp::depends(RcppArmadillo)]]#include <RcppArmadillo.h>using namespace Rcpp; using namespace arma;

// [[Rcpp::export]]List fastLm(NumericVector yr, NumericMatrix Xr) {

int n = Xr.nrow(), k = Xr.ncol();mat X(Xr.begin(), n, k, false);colvec y(yr.begin(), yr.size(), false);

colvec coef = solve(X, y);colvec resid = y - X*coef;

double sig2 = as_scalar(trans(resid)*resid/(n-k));colvec stderrest = sqrt(sig2 * diagvec( inv(trans(X)*X)) );

return List::create(Named("coefficients") = coef,Named("stderr") = stderrest,Named("df.residual") = n - k );

}



Faster Linear Model with FastLmNewer version of RcppArmadillo’s src/fastLm.cpp

// [[Rcpp::depends(RcppArmadillo)]]#include <RcppArmadillo.h>using namespace Rcpp;using namespace arma;

// [[Rcpp::export]]List fastLm2(const colvec& y, const mat& X) {

int n = X.n_rows, k = X.n_cols;

colvec coef = solve(X, y);colvec resid = y - X*coef;

double sig2 = as_scalar(trans(resid)*resid/(n-k));colvec stderrest = sqrt(sig2 * diagvec( inv(trans(X)*X)) );

return List::create(Named("coefficients") = coef,Named("stderr") = stderrest,Named("df.residual") = n - k );

}



Faster Linear Model with FastLmNote on as<>() casting with Armadillo

arma::colvec y = Rcpp::as<arma::colvec>(ys);

arma::mat X = Rcpp::as<arma::mat>(Xs);

Convenient, yet incurs an additional copy. Next variantuses two steps, but only a pointer to objects is copied:

Rcpp::NumericVector yr(ys);Rcpp::NumericMatrix Xr(Xs);int n = Xr.nrow(), k = Xr.ncol();arma::mat X(Xr.begin(), n, k, false);

arma::colvec y(yr.begin(), yr.size(), false);

Preferable if performance is a concern. NewestRcppArmadillo has efficient const references too.



Faster Linear Model with FastLmPerformance comparison

Running the script included in the RcppArmadillopackage:

edd@max:~/svn/rcpp/pkg/RcppArmadillo/inst/examples$ r fastLm.rLoading required package: Rcpp

test replications relative elapsed2 fLmTwoCasts(X, y) 5000 1.000 0.1883 fLmConstRef(X, y) 5000 1.000 0.1881 fLmOneCast(X, y) 5000 1.005 0.1895 fastLmPureDotCall(X, y) 5000 1.064 0.2004 fastLmPure(X, y) 5000 2.000 0.3767 lm.fit(X, y) 5000 2.691 0.5066 fastLm(frm, data = trees) 5000 35.596 6.6928 lm(frm, data = trees) 5000 44.883 8.438edd@max:~/svn/rcpp/pkg/RcppArmadillo/inst/examples$


Intro Ex FastLM Kalman Sparse XPtr Setup Matlab R C++ Performance

Outline

4 Case Study: Kalman FilterSetupMatlabRC++Performance



Kalman FilterSetup at Mathworks site

The position of an object is estimated based on pastvalues of 6 × 1 state vectors X and Y for position, VX andVY for speed, and AX and AY for acceleration.

Position updates as a function of the speed

X = X0 + VX dt and Y = Y0 + VY dt ,

which is updated as a function of the (unobserved)acceleration:

Vx = VX ,0 + AX dt and Vy = VY ,0 + AY dt .



Kalman FilterBasic Matlab Function

% Copyright 2010 The MathWorks, Inc.function y = kalmanfilter(z)% #codegen

dt=1;% Initialize state transition matrixA=[1 0 dt 0 0 0;... % [x ]

0 1 0 dt 0 0;... % [y ]0 0 1 0 dt 0;... % [Vx]0 0 0 1 0 dt;... % [Vy]0 0 0 0 1 0 ;... % [Ax]0 0 0 0 0 1 ]; % [Ay]

H = [ 1 0 0 0 0 0; 0 1 0 0 0 0 ];Q = eye(6);R = 1000 * eye(2);persistent x_est p_estif isempty(x_est)

x_est = zeros(6, 1);p_est = zeros(6, 6);

end

% Predicted state and covariancex_prd = A * x_est;p_prd = A * p_est * A’ + Q;% EstimationS = H * p_prd’ * H’ + R;B = H * p_prd’;klm_gain = (S \ B)’;% Estimated state and covariancex_est = x_prd+klm_gain*(z-H*x_prd);p_est = p_prd-klm_gain*H*p_prd;% Compute the estimated measurementsy = H * x_est;

end % of the function

Plus a simple wrapper function calling this function.



Kalman Filter: In REasy enough – first naive solution

FirstKalmanR <- function(pos) {

kf <- function(z) {dt <- 1

A <- matrix(c(1, 0, dt, 0, 0, 0, # x0, 1, 0, dt, 0, 0, # y0, 0, 1, 0, dt, 0, # Vx0, 0, 0, 1, 0, dt, # Vy0, 0, 0, 0, 1, 0, # Ax0, 0, 0, 0, 0, 1), # Ay

6, 6, byrow=TRUE)H <- matrix( c(1, 0, 0, 0, 0, 0,

0, 1, 0, 0, 0, 0),2, 6, byrow=TRUE)

Q <- diag(6)R <- 1000 * diag(2)

N <- nrow(pos)y <- matrix(NA, N, 2)

## predicted state and covriancexprd <- A %*% xestpprd <- A %*% pest %*% t(A) + Q

## estimationS <- H %*% t(pprd) %*% t(H) + RB <- H %*% t(pprd)## kalmangain <- (S \ B)’kg <- t(solve(S, B))

## est. state and cov, assign to vars in parent envxest <<- xprd + kg %*% (z-H%*%xprd)pest <<- pprd - kg %*% H %*% pprd

## compute the estimated measurementsy <- H %*% xest

}

xest <- matrix(0, 6, 1)pest <- matrix(0, 6, 6)

for (i in 1:N) {y[i,] <- kf(t(pos[i„drop=FALSE]))

}

invisible(y)}



Kalman Filter: In REasy enough – with some minor refactoring

KalmanR <- function(pos) {

kf <- function(z) {## predicted state and covriancexprd <- A %*% xestpprd <- A %*% pest %*% t(A) + Q

## estimationS <- H %*% t(pprd) %*% t(H) + RB <- H %*% t(pprd)## kg <- (S \ B)’kg <- t(solve(S, B))

## estimated state and covariance## assigned to vars in parent envxest <<- xprd + kg %*% (z-H%*%xprd)pest <<- pprd - kg %*% H %*% pprd

## compute the estimated measurementsy <- H %*% xest

}dt <- 1

A <- matrix(c(1, 0, dt, 0, 0, 0, # x0, 1, 0, dt, 0, 0, # y0, 0, 1, 0, dt, 0, # Vx0, 0, 0, 1, 0, dt, # Vy0, 0, 0, 0, 1, 0, # Ax0, 0, 0, 0, 0, 1),# Ay6, 6, byrow=TRUE)

H <- matrix(c(1, 0, 0, 0, 0, 0,0, 1, 0, 0, 0, 0),

2, 6, byrow=TRUE)Q <- diag(6)R <- 1000 * diag(2)

N <- nrow(pos)y <- matrix(NA, N, 2)

xest <- matrix(0, 6, 1)pest <- matrix(0, 6, 6)

for (i in 1:N) {y[i,] <- kf(t(pos[i„drop=FALSE]))

}invisible(y)

}



Kalman Filter: In C++Using a simple class

// [[Rcpp::depends(RcppArmadillo)]]#include <RcppArmadillo.h>

using namespace arma;

class Kalman {private:

mat A, H, Q, R, xest, pest;double dt;

public:// constructor, sets up data structuresKalman() : dt(1.0) {A.eye(6,6);A(0,2) = A(1,3) = dt;A(2,4) = A(3,5) = dt;H.zeros(2,6);H(0,0) = H(1,1) = 1.0;Q.eye(6,6);R = 1000 * eye(2,2);xest.zeros(6,1);pest.zeros(6,6);

}

// sole member func.: estimate modelmat estimate(const mat & Z) {unsigned int n = Z.n_rows,

k = Z.n_cols;mat Y = zeros(n, k);mat xprd, pprd, S, B, kg;colvec z, y;

for (unsigned int i = 0; i<n; i++) {z = Z.row(i).t();// predicted state and covariancexprd = A * xest;pprd = A * pest * A.t() + Q;// estimationS = H * pprd.t() * H.t() + R;B = H * pprd.t();kg = (solve(S, B)).t();// estimated state and covariancexest = xprd + kg * (z - H * xprd);pest = pprd - kg * H * pprd;// compute estimated measurementsy = H * xest;Y.row(i) = y.t();

}return Y;

}

};Dirk Eddelbuettel Rcpp Intro & Examples Part 2


Kalman Filter in C++Trivial to use from R

Given the code from the previous slide, we just add

// [[Rcpp::export]]mat KalmanCpp(mat Z) {Kalman K;mat Y = K.estimate(Z);return Y;

}



Kalman Filter: PerformanceQuite satisfactory relative to R

Even byte-compiled ’better’ R version is 66 times slower:R> FirstKalmanRC <- cmpfun(FirstKalmanR)R> KalmanRC <- cmpfun(KalmanR)R>R> stopifnot(identical(KalmanR(pos), KalmanRC(pos)),+ all.equal(KalmanR(pos), KalmanCpp(pos)),+ identical(FirstKalmanR(pos), FirstKalmanRC(pos)),+ all.equal(KalmanR(pos), FirstKalmanR(pos)))R>R> res <- benchmark(KalmanR(pos), KalmanRC(pos),+ FirstKalmanR(pos), FirstKalmanRC(pos),+ KalmanCpp(pos),+ columns = c("test", "replications",+ "elapsed", "relative"),+ order="relative",+ replications=100)R>R> print(res)

test replications elapsed relative5 KalmanCpp(pos) 100 0.087 1.00002 KalmanRC(pos) 100 5.774 66.36781 KalmanR(pos) 100 6.448 74.11494 FirstKalmanRC(pos) 100 8.153 93.71263 FirstKalmanR(pos) 100 8.901 102.3103



Kalman Filter: FigureLast but not least we can redo the plot as well

−0.5 0.0 0.5 1.0

−1.

0−

0.5

0.0

0.5

1.0

Object Trajectory and Kalman Filter Estimate

x

y

TrajectoryEstimate


Intro Ex FastLM Kalman Sparse XPtr R C++ Example

Outline

5 Case Study: Sparse MatricesRC++Example



Sparse MatricesGrowing (but incomplete) support in Armadillo

A nice example for work on R objects.

i <- c(1,3:8)j <- c(2,9,6:10)x <- 7 * (1:7)A <- sparseMatrix(i, j, x = x)A

## 8 x 10 sparse Matrix of class "dgCMatrix"#### [1,] . 7 . . . . . . . .## [2,] . . . . . . . . . .## [3,] . . . . . . . . 14 .## [4,] . . . . . 21 . . . .## [5,] . . . . . . 28 . . .## [6,] . . . . . . . 35 . .## [7,] . . . . . . . . 42 .## [8,] . . . . . . . . . 49



Sparse MatricesRepresentation in R

str(A)

## Formal class 'dgCMatrix' [package "Matrix"] with 6 slots## ..@ i : int [1:7] 0 3 4 5 2 6 7## ..@ p : int [1:11] 0 0 1 1 1 1 2 3 4 6 ...## ..@ Dim : int [1:2] 8 10## ..@ Dimnames:List of 2## .. ..$ : NULL## .. ..$ : NULL## ..@ x : num [1:7] 7 21 28 35 14 42 49## ..@ factors : list()

Note how the construction was in terms of < i , j , x >, yetthe representation in in terms of < i ,p, x > – CSC format.



Sparse MatricesC++ access


using namespace Rcpp;using namespace arma;

// [[Rcpp::depends(RcppArmadillo)]]

// [[Rcpp::export]]sp_mat armaEx(S4 mat, bool show) {

IntegerVector dims = mat.slot("Dim");arma::urowvec i = Rcpp::as<arma::urowvec>(mat.slot("i"));arma::urowvec p = Rcpp::as<arma::urowvec>(mat.slot("p"));arma::vec x = Rcpp::as<arma::vec>(mat.slot("x"));

int nrow = dims[0], ncol = dims[1];arma::sp_mat res(i, p, x, nrow, ncol);if (show) Rcpp::Rcout << res << std::endl;return res;

}



Sparse MatricesC++ access

sourceCpp('code/sparseEx.cpp')i <- c(1,3:8)j <- c(2,9,6:10)x <- 7 * (1:7)A <- sparseMatrix(i, j, x = x)B <- armaEx(A, TRUE)

## [matrix size: 8x10; n_nonzero: 7; density: 8.75%]#### (0, 1) 7.0000## (3, 5) 21.0000## (4, 6) 28.0000## (5, 7) 35.0000## (2, 8) 14.0000## (6, 8) 42.0000## (7, 9) 49.0000



Outline

6 XPtr



Function Pointershttp://gallery.rcpp.org/articles/passing-cpp-function-pointers/

Consider two simple functions modifying a givenArmadillo vector:

// [[Rcpp::depends(RcppArmadillo)]]#include <RcppArmadillo.h>

using namespace arma;using namespace Rcpp;

vec fun1_cpp(const vec& x) { // a first functionvec y = x + x;return (y);

}

vec fun2_cpp(const vec& x) { // and a second functionvec y = 10*x;return (y);

}




Using a typedef to declare an interface to a functiontaking and returning a vector — and a function returning afunction pointer given a string argument

typedef vec (*funcPtr)(const vec& x);

// [[Rcpp::export]]XPtr<funcPtr> putFunPtrInXPtr(std::string fstr) {

if (fstr == "fun1")return(XPtr<funcPtr>(new funcPtr(&fun1_cpp)));

else if (fstr == "fun2")return(XPtr<funcPtr>(new funcPtr(&fun2_cpp)));

elsereturn XPtr<funcPtr>(R_NilValue); // runtime err.: NULL no XPtr

}




We then create a function calling the supplied function ona given vector by ’unpacking’ the function pointer:

// [[Rcpp::export]]vec callViaXPtr(const vec x, SEXP xpsexp) {

XPtr<funcPtr> xpfun(xpsexp);funcPtr fun = *xpfun;vec y = fun(x);return (y);

}




## get us a functionfun <- putFunPtrInXPtr("fun1")## and pass it down to C++ to## have it applied on given vectorcallViaXPtr(1:4, fun)

## [,1]## [1,] 2## [2,] 4## [3,] 6## [4,] 8

Could use same mechanism for user-supplied functions,gradients, or samplers, ...


Download - Seamless R and C++ Integration with Rcpp: Part 2 ...dirk.eddelbuettel.com/papers/rcpp_ku_nov2013-part2.pdf · IntroExFastLMKalmanSparseXPtr Seamless R and C++ Integration with Rcpp:

Top Related