Taichi: An Open-Source Computer Graphics Library

YUANMING HU, MIT CSAIL

Fig. 1. Results simulated and rendered using Taichi.

An ideal so�ware system in computer graphics should be a combination ofinnovative ideas, solid so�ware engineering and rapid development. How-ever, in reality these requirements are seldom met simultaneously. In thispaper, we present early results on an open-source library named Taichi(http://taichi.graphics) which alleviates this practical issue by providing anaccessible, portable, extensible, and high-performance infrastructure that isreusable and tailored for computer graphics. As a case study, we share ourexperience in building a novel physical simulation system using Taichi.

1 WHY NEW LIBRARY FOR COMPUTER GRAPHICS?

Computer graphics research has high standards on novelty andquality, which lead to a trade-o� between rapid prototyping andgood so�ware engineering. The former allows researchers to codequickly but inevitably leads to defects such as unsatisfactory per-formance, poor portability and maintainability. As a result, manyprojects are neither reused nor open-sourced, stopping other peoplefrom experimenting, reproducing, and developing based on priorart. The la�er results in reusable codes for future projects, but slowsdown research progress due to low-level engineering such as infras-tructure building. Admi�edly, for a researcher too much low-levelengineering may become an obstacle for high-level thinking.

Existing open-source projects are focused on certain functionality,like rendering (e.g. Mitsuba [Jakob 2010], PBRT [Pharr et al. 2016],Lightmetrica [Otsu 2015], POV-Ray [Buck and Collins 2004], etc.), ge-ometry processing (libIGL [Jacobson et al. 2013], MeshLab [Cignoniet al. 2008], CGAL [Fabri and Pion 2009], etc.), simulation (Bul-let [Coumans et al. 2013], ODE [Smith et al. 2005], ArcSim [Narainet al. 2004], VegaFEM [Sin et al. 2013], MantaFlow [Thuerey andPfa� 2017], Box2D [Ca�o 2011], PhysBAM [Dubey et al. 2011], etc.).

The aforementioned libraries have proven successful in their ownapplications. However, though some projects can more or less reuse

these frameworks, building prototypical systems from scratch iso�en necessary when modifying these libraries becomes even moreexpensive than starting over. Taichi is designed to be a reusableinfrastructure for the la�er situation, by providing abstractions atdi�erent levels from vector/matrix arithmetics to scene configura-tion and scripting. Compared with existing reusable componentssuch as Boost or Eigen, Taichi is tailored for computer graphics.The domain-specific design has many benefits over those generalframeworks, as illustrated in Fig. 2 with a concrete example.

Tight loop 4-way unrolled loop0.0

0.5

1.0

1.5

2.0

2.5

(Int

el i7

-770

0K@

4.2G

, 1-th

read

; dat

a fit

L1

$) G

iter

s/s

Single-precision 3D Matrix-Vector Multiplication Throughputs

CPU intrinsics (vfmaddps)taichi::(Vector|Matrix)3fEigen::(Vector|Matrix)3f

Fig. 2. Why do we need a tool tailored for graphics? Con-sider 3D matrix-vector multiplication as an example. In Eigen single-precision 3D vectors are stored compactly using 12 bytes. Comparedwith 16-byte-aligned storage, this scheme saves some space, but o�ersunsatisfactory performance since additional permutation in SIMDregisters is needed for vectorized computation. In graphics we caremore about throughput and latency instead of space, so Taichi’s vectorsystem is designed in the la�er way and is 1.8 − 2.3× faster.

arX

iv:1

804.

0929

3v1

[cs

.GR

] 2

4 A

pr 2

018

2 DESIGN GOALS

Accessibility. Taichi is open-source and easy-to-use by design. Weavoid unnecessary dependencies, especially hard-to-deploy onessuch as Boost. Installation can be done simply by executing a pythonscript that automatically installs all dependencies. For beginners,it contains many working demos that showcase how this so�wareshould be used at a high level.

Portability. Taichi is cross-platform and supports Linux, OS X andWindows. Our aim is to make every project built on Taichi automat-ically portable by providing an abstraction of the actual operatingsystems. This makes open-sourcing projects much easier.

Reusability. Taichi contains common utilities for developing graph-ics projects. Reusing them can significantly speed up developmentsince the time spent on (re)inventing the wheels is saved.

Extensibility. With a plugin-based design, Taichi allows users toquickly add their own implementations of existing or new interfaces.The implementations (“units") will be automatically registered atprogram load time and can be instantiated conveniently with aname identifier through the factory.

High-performance. We o�er not only e�icient (sometimes zero-cost)abstractions at di�erent levels of Taichi, but also performance mon-itoring tools such as scoped timers and profilers to make perfor-mance engineering convenient.

3 COMPONENTS

Here we present some representative components that can bereused for developing new computer graphics systems.

(De)serialization. The serialization system allows marshaling inter-nal data structures in Taichi into bit streams. A typical application issnapsho�ing. Many computer graphics programs take a long timeto execute and it is necessary to take snapshots of current states. Incase of run-time error, the program can continue running from thelatest snapshot a�er the bug causing crashing is repaired. Otheruse cases include inter-program communication when there is noshared memory. Our serialization library is header-only and canbe easily integrated into other projects for reading the serializedTaichi data.

Logging and Forma�ing. Appropriate logging is an e�ective way todiagnose a long-running program. Taichi internally uses spdlog andfmtlib. The former manages logging and the la�er is a modern for-ma�ing library which unifies string forma�ing in C++ and Python.It is safer, easier to use and more portable than its alternatives likestd::cout and printf.

Profiling. Though there are mature external profiling tools like gprofor Intel VTune, using them needs additional manual operations.Taichi has an integrated scoped profiling system that records the

time consumption of each part automatically, simplifying hotspotanalysis.

Debugging and Testing. C++ programs do not generate much usefuldebugging information when it crashes. This makes debugginghard and is the reason why Taichi captures run-time errors andprints stack back-trace. On Linux, it additionally triggers gdb fordebugging. Once a task finishes (or crashes), Taichi can send emailsto notify the user. Taichi uses Catch2 for testing and we aim to builda high-coverage test suite for the library.

Rendering. Though not designed to compete with existing render-ing frameworks, Taichi provides demonstrative implementationsof popular rendering algorithms, from basic path tracing to mod-ern ones such as vertex connection and merging [Georgiev et al.2012]/unified path sampling [Hachisuka et al. 2012] with adaptiveMarkov chain Monte Carlo [Šik et al. 2016].

Simulation. Taichi is shipped with several physical simulators in-cluding APIC [Jiang et al. 2015]/FLIP [Zhu and Bridson 2005] liq-uid/smoke simulation with a multigrid-preconditioned conjugategradient (MGPCG) [McAdams et al. 2010] Poisson solver for projec-tion. We will also release a high-performance material point methodcode, as detailed in section 4.

File IO support. Taichi supports reading and writing of popular fileformats for computer graphics, including obj (via tinyobjloader) andply, jpg, png, bmp, �f (via stb_image, stb_image_write, stb_truetype),In addition, Taichi wraps common utilities provided by �mpeg. Itcan generate mp4 or gif videos from an array of images, either inmemory or on disk.

Scripting. Taichi has a hybrid design with a kernel part in C++14and an easy-to-use interface in Python 3. Such approach decouplesinputs like demo set-up from the actual kernel computation code.pybind11 is used for binding C++ interface for Python.

In addition, we built Taichi based on Intel Thread Building Blocksfor multithreading. An e�iciently vectorized linear algebra libraryis also developed to suit computer graphics applications be�er,as mentioned in Figure 2. The Taichi developer team is in chargeof maintaining and simplifying the deployment process of all theaforementioned dependencies of Taichi.

4 CASE STUDY: A PHYSICAL SIMULATION PROJECT

So far, we have used Taichi in five research projects. Published onesinclude [Hu and Fang 2017; Hu et al. 2018]. Here we summarize ourexperience during the development of the SIGGRAPH 2018 paper“A Moving Least Squares Material Point Method with DisplacementDiscontinuity and Two-Way Rigid Body Coupling" [Hu et al. 2018],abbreviated as “MLSMPM-CPIC". Taichi is used as the backboneof the simulator development of this project. Visual results aredisplayed in Figure 3.

2

Fig. 3. Results in the MLSMPM-CPIC paper. Using Taichi as the code base significantly sped up the development of this project.

Productivity. All features mentioned in the previous section proveduseful. In fact, many of them are developed during research projectslike this one for productivity and future reusability. For example,the serialization system allows us to periodically save simulationstates to disk, and conveniently restart from these snapshots. It isespecially important for long-running simulations which can takehours or even days. Performance engineering is guided by the Taichiprofiler, which shows a breakdown of time consumption and hasled to significantly higher e�iciency compared with the previousstate-of-the-art [Tampubolon et al. 2017].

Team Scalability. The python scripting system makes our develop-ment especially suitable for team working, because the algorithm de-velopment (in C++) and experimental validation (via python scripts)are nicely decoupled: for most of the time only one or two coremembers need to develop the C++ simulation code, and all teammembers can help conducting experiments without ge�ing involvedinto low-level details.

In general, the team is satisfied with this infrastructure and hasdecided to build future projects on it. We believe the user experiencewill continue to improve as we ba�le-test it in more projects.

5 FUTURE WORK

Clearly, a lot more engineering e�orts are needed to improve Taichi.Detailed documentation and be�er test coverage are two urgenttasks. Improving and stabilizing the interface design, support forother open-source so�wares like Blender are also very meaningful.

ACKNOWLEDGEMENTS

The author would like to thank Toshiya Hachisuka and SeiichiKoshizuka for hosting his internship at the University of Tokyo,where he developed the initial version of Taichi. Chenfanfu Jiangprovided helpful suggestions for developing Taichi and supported itsadoption in several projects. Other developers, especially Yu Fang,also contributed to Taichi. Some demo scenes are from [Bi�erli2016]. Finally, thank all the open-source so�ware developers formaking their achievements freely available to everyone.

REFERENCES

Benedikt Bi�erli. 2016. Rendering resources. (2016). h�ps://benedikt-bi�erli.me/resources/.

David K Buck and Aaron A Collins. 2004. Pov-ray: the persistence of vision raytracer.(2004). h�p://www.povray.org/.

Erin Ca�o. 2011. Box2d: A 2d physics engine for games. (2011).Paolo Cignoni, Marco Callieri, Massimiliano Corsini, Ma�eo Dellepiane, Fabio Ganov-

elli, and Guido Ranzuglia. 2008. Meshlab: an open-source mesh processing tool.. InEurographics Italian Chapter Conference, Vol. 2008. 129–136.

Erwin Coumans et al. 2013. Bullet physics library. Open source: bulletphysics. org 15(2013), 49.

Pradeep Dubey, Pat Hanrahan, Ronald Fedkiw, Michael Lentine, and Craig Schroeder.2011. Physbam: Physically based simulation. In ACM SIGGRAPH 2011 Courses.ACM, 10.

Andreas Fabri and Sylvain Pion. 2009. CGAL: The computational geometry algorithmslibrary. In Proceedings of the 17th ACM SIGSPATIAL international conference onadvances in geographic information systems. ACM, 538–539.

Iliyan Georgiev, Jaroslav Krivánek, Tomas Davidovic, and Philipp Slusallek. 2012. Lighttransport simulation with vertex connection and merging. ACM Trans. Graph. 31, 6(2012), 192–1.

Toshiya Hachisuka, Jacopo Pantaleoni, and Henrik Wann Jensen. 2012. A path spaceextension for robust light transport simulation. ACM Transactions on Graphics(TOG) 31, 6 (2012), 191.

Yuanming Hu and Yu Fang. 2017. An asynchronous material point method. In ACMSIGGRAPH 2017 Posters. ACM, 60.

Yuanming Hu, Yu Fang, Ziheng Ge, Ziyin �, Yixin Zhu, Andre Pradhana Tampubolon,and Chenfanfu Jiang. 2018. A Moving Least Squares Material Point Method withDisplacement Discontinuity and Two-Way Rigid Body Coupling. ACM Transactionson Graphics (TOG), presented at SIGGRAPH 2018 37, 4 (2018).

Alec Jacobson, Daniele Panozzo, et al. 2013. libigl: A simple C++ geometry processinglibrary, 2016. (2013). h�ps://github.com/libigl/libigl.

Wenzel Jakob. 2010. Mitsuba renderer. (2010). h�p://www.mitsuba-renderer.org.Chenfanfu Jiang, Craig Schroeder, Andrew Selle, Joseph Teran, and Alexey Stomakhin.

2015. The a�ine particle-in-cell method. ACM Transactions on Graphics (TOG) 34, 4(2015), 51.

Aleka McAdams, E�ychios Sifakis, and Joseph Teran. 2010. A parallel multigrid Pois-son solver for fluids simulation on large grids. In Proceedings of the 2010 ACMSIGGRAPH/Eurographics Symposium on Computer Animation. Eurographics Associ-ation, 65–74.

Rahul Narain, Armin Samii, Tobias Pfa�, and James F. O’Brien. 2004. ARCSim: AdaptiveRefining and Coarsening Simulator. (2004). h�p://www.povray.org/.

Hisanari Otsu. 2015. Lightmetrica: A modern, research-oriented renderer. (2015).h�p://lightmetrica.org/.

Ma� Pharr, Wenzel Jakob, and Greg Humphreys. 2016. Physically based rendering:From theory to implementation. Morgan Kaufmann.

Martin Šik, Hisanari Otsu, Toshiya Hachisuka, and Jaroslav Křivánek. 2016. Robust lighttransport simulation via metropolised bidirectional estimators. ACM Transactionson Graphics (TOG) 35, 6 (2016), 245.

Fun Shing Sin, Daniel Schroeder, and Jernej Barbič. 2013. Vega: Non-Linear FEMDeformable Object Simulator. In Computer Graphics Forum, Vol. 32. Wiley OnlineLibrary, 36–48.

Russell Smith et al. 2005. Open dynamics engine. (2005).Andre Pradhana Tampubolon, Theodore Gast, Gergely Klár, Chuyuan Fu, Joseph Teran,

Chenfanfu Jiang, and Ken Museth. 2017. Multi-species simulation of porous sandand water mixtures. ACM Transactions on Graphics (TOG), presented at SIGGRAPH

3

2017 36, 4 (2017), 105.Nils Thuerey and Tobias Pfa�. 2017. MantaFlow. (2017). h�p://mantaflow.com.Yongning Zhu and Robert Bridson. 2005. Animating sand as a fluid. ACM Transactions

on Graphics (TOG) 24, 3 (2005), 965–972.

CHANGE LOG

April 24, 2018: initial version.

4