managed runtime technology: general...

37
Managed Runtime Technology: General Introduction Xiao-Feng Li ([email protected]) 2012-10-10

Upload: others

Post on 26-Mar-2020

35 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Managed Runtime Technology: General Introductionpeople.apache.org/~xli/presentations/managed-runtime-introduction.pdf · –Cons: runtime overhead to maintain counters •Reachability

Managed Runtime Technology:

General Introduction

Xiao-Feng Li ([email protected])

2012-10-10

Page 2: Managed Runtime Technology: General Introductionpeople.apache.org/~xli/presentations/managed-runtime-introduction.pdf · –Cons: runtime overhead to maintain counters •Reachability

Agenda

• Virtual machines

• Managed runtime systems

• EE and MM (JIT and GC)

• Summary

10/10/2012 Managed Runtime Technology Introduction - Xiao-Feng Li 2

Page 3: Managed Runtime Technology: General Introductionpeople.apache.org/~xli/presentations/managed-runtime-introduction.pdf · –Cons: runtime overhead to maintain counters •Reachability

What is Virtual Machine?

• A virtual machine is implemented by adding a layer of software to a real machine to support the desired virtual machine’s architecture. – Virtualization constructs an isomorphism that

maps a virtual guest system to a real host.

• Virtualization vs. abstraction – Virtualization does not necessarily hide details

– Abstraction uses interfaces to express the abstract model

10/10/2012 Managed Runtime Technology Introduction - Xiao-Feng Li 3

Page 4: Managed Runtime Technology: General Introductionpeople.apache.org/~xli/presentations/managed-runtime-introduction.pdf · –Cons: runtime overhead to maintain counters •Reachability

System (ISA) Virtual Machine

• Such as Vmware, Xen, VirtualBox, etc.

Picture from “Virtual Machines: Versatile Platforms for Systems and Processes”

10/10/2012 Managed Runtime Technology Introduction - Xiao-Feng Li 4

Page 5: Managed Runtime Technology: General Introductionpeople.apache.org/~xli/presentations/managed-runtime-introduction.pdf · –Cons: runtime overhead to maintain counters •Reachability

Process (ABI) Virtual Machine

• Different ISAs: Digital FX!32, IA32 EL, etc.

• Same ISAs: such as Dynamo, etc.

10/10/2012 Managed Runtime Technology Introduction - Xiao-Feng Li 5

Page 6: Managed Runtime Technology: General Introductionpeople.apache.org/~xli/presentations/managed-runtime-introduction.pdf · –Cons: runtime overhead to maintain counters •Reachability

Virtual-ISA Virtual Machine

• Such as JVM, CLI, etc.

• Virtual machine Runtime engine

10/10/2012 Managed Runtime Technology Introduction - Xiao-Feng Li 6

Page 7: Managed Runtime Technology: General Introductionpeople.apache.org/~xli/presentations/managed-runtime-introduction.pdf · –Cons: runtime overhead to maintain counters •Reachability

Language Runtime Engine

• Such as Javascript, Python, HTML, etc.

• Levels: ISA ABI Virtual-ISA -> Language

• Abstractions:

– From all architecture details to language interface

– Usually depend on other supportive functions

• Our focus: “Managed Runtime System”

– Virtual-ISA virtual machine, and language runtime engine

10/10/2012 Managed Runtime Technology Introduction - Xiao-Feng Li 7

Page 8: Managed Runtime Technology: General Introductionpeople.apache.org/~xli/presentations/managed-runtime-introduction.pdf · –Cons: runtime overhead to maintain counters •Reachability

Link Runtime to Services

• Language

– Programming language interface description language service description language

– Such as AIDL (Android), WSDL (web service)

• Service entity

– Process local service Inter-process service cross-machine service Internet cloud computing

10/10/2012 Managed Runtime Technology Introduction - Xiao-Feng Li 8

Page 9: Managed Runtime Technology: General Introductionpeople.apache.org/~xli/presentations/managed-runtime-introduction.pdf · –Cons: runtime overhead to maintain counters •Reachability

Agenda

• Virtual machines

• Managed runtime systems

• EE and MM (JIT and GC)

• Summary

10/10/2012 Managed Runtime Technology Introduction - Xiao-Feng Li 9

Page 10: Managed Runtime Technology: General Introductionpeople.apache.org/~xli/presentations/managed-runtime-introduction.pdf · –Cons: runtime overhead to maintain counters •Reachability

Managed Runtime Systems

• Why?

– Portability

– Security

– Productivity

10/10/2012 Managed Runtime Technology Introduction - Xiao-Feng Li 10

Page 11: Managed Runtime Technology: General Introductionpeople.apache.org/~xli/presentations/managed-runtime-introduction.pdf · –Cons: runtime overhead to maintain counters •Reachability

Modern Programming

• Programming in safe languages – No illegal pointer dereference

• a = 0; b = *a;

– No illegal control flow • jmp format_disk;

• Then what’s safe language? – “type-safe” language and plus

• If it does not allow operations or conversions which lead to erroneous conditions

– Memory safety and Control safety

– Can download-and-run safely

10/10/2012 Managed Runtime Technology Introduction - Xiao-Feng Li 11

Page 12: Managed Runtime Technology: General Introductionpeople.apache.org/~xli/presentations/managed-runtime-introduction.pdf · –Cons: runtime overhead to maintain counters •Reachability

Safe Languages

• Almost all the modern languages are safe – Commercial: Java, C#, SQL, etc.

– Academic: LISP, Haskell, ML, etc.

– Scripting: Ruby, Javascript, PHP, etc.

– Widely used today and probably tomorrow

• Unsafe languages – C, C++, asm, etc.

– Will only be used by system programmers

– Specifically, unsafe languages are commonly used to implement safe languages

10/10/2012 Managed Runtime Technology Introduction - Xiao-Feng Li 12

Page 13: Managed Runtime Technology: General Introductionpeople.apache.org/~xli/presentations/managed-runtime-introduction.pdf · –Cons: runtime overhead to maintain counters •Reachability

Modern Programming Is Good

• Programming is becoming higher-level – Programmers care only logics

• “What I want to achieve?”

– Leave the details to system • “How it is achieved?”

– E.g., details like memory allocation/release, thread scheduling, etc.

• The question: – How the system provides the supports?

– This is our work

10/10/2012 Managed Runtime Technology Introduction - Xiao-Feng Li 13

Page 14: Managed Runtime Technology: General Introductionpeople.apache.org/~xli/presentations/managed-runtime-introduction.pdf · –Cons: runtime overhead to maintain counters •Reachability

Supports to Modern Prog.

• Original system supports – Compiler:

• Syntax checking • Compilation into binary

– OS • Page allocation and management • Process scheduling

• What are missing? – Runtime safety checking – Runtime interpretation or compilation – Memory management at object granularity

10/10/2012 Managed Runtime Technology Introduction - Xiao-Feng Li 14

Page 15: Managed Runtime Technology: General Introductionpeople.apache.org/~xli/presentations/managed-runtime-introduction.pdf · –Cons: runtime overhead to maintain counters •Reachability

C code

binary

Compiler

OS

SpiderMonkey

OS

Javascript

10/10/2012 Managed Runtime Technology Introduction - Xiao-Feng Li 15

Page 16: Managed Runtime Technology: General Introductionpeople.apache.org/~xli/presentations/managed-runtime-introduction.pdf · –Cons: runtime overhead to maintain counters •Reachability

YARV Core

OS

Ruby code

wordcode

Compiler

JVM

OS

Java code

bytecode

Compiler

binary

10/10/2012 Managed Runtime Technology Introduction - Xiao-Feng Li 16

Page 17: Managed Runtime Technology: General Introductionpeople.apache.org/~xli/presentations/managed-runtime-introduction.pdf · –Cons: runtime overhead to maintain counters •Reachability

Example: Ruby Engine

• Ruby VM

– Parser: convert scripts into AST

– Compiler: compile AST to wordcode sequence

– Runtime environment: Yarv core

– Object/class manager

– Heap manager

Ruby Parser

InstructionSeq Compiler

Runtime Envirement

Heap Manager (GC)

Object/Class Manager

Built-in Classes

Ruby VM

OS/Hardware

Ruby scripts

10/10/2012 Managed Runtime Technology Introduction - Xiao-Feng Li 17

Page 18: Managed Runtime Technology: General Introductionpeople.apache.org/~xli/presentations/managed-runtime-introduction.pdf · –Cons: runtime overhead to maintain counters •Reachability

Example: Java Virtual Machine

Class loader class

libraries Interpreter

compiler

Memory manager

JVM

Linux Windows Solaris

Hardware

Java code (*.java)

Java bytecode (*.class)

Java Compiler

Thread manager

Execution manager

10/10/2012 Managed Runtime Technology Introduction - Xiao-Feng Li 18

Page 19: Managed Runtime Technology: General Introductionpeople.apache.org/~xli/presentations/managed-runtime-introduction.pdf · –Cons: runtime overhead to maintain counters •Reachability

JVM: a Computing Environment

• For any computing environment, you have: – Programming language: C/C++ – Compiler + OS + machine: gcc/g++ + Linux + X86 – Runtime library: glibc/glibc++

• Java provides the same things: – Programming language: defined by Gosling/Joy/Steele in

1995 – Virtual machine: JVM has execution engine +

threading/MM + ISA – Library: class libraries with standard API – All are specified into standards

10/10/2012 Managed Runtime Technology Introduction - Xiao-Feng Li 19

Page 20: Managed Runtime Technology: General Introductionpeople.apache.org/~xli/presentations/managed-runtime-introduction.pdf · –Cons: runtime overhead to maintain counters •Reachability

JVM Specification

• Mainly defines four things: 1. Instruction set for the VM (bytecodes) 2. JVM executable file format (Java class file) 3. Class loading and verification (ensure the program does

not compromise the JVM integrity) 4. Java threading and memory model (the system)

• Specification says nothing on implementation

method – Can be implemented as a process in OS, or – Part of OS kernel, or OS is part of JVM, or – In hardware, or anything

10/10/2012 Managed Runtime Technology Introduction - Xiao-Feng Li 20

Page 21: Managed Runtime Technology: General Introductionpeople.apache.org/~xli/presentations/managed-runtime-introduction.pdf · –Cons: runtime overhead to maintain counters •Reachability

Agenda

• Virtual machines

• Managed runtime systems

• EE and MM (JIT and GC)

• Summary

10/10/2012 Managed Runtime Technology Introduction - Xiao-Feng Li 21

Page 22: Managed Runtime Technology: General Introductionpeople.apache.org/~xli/presentations/managed-runtime-introduction.pdf · –Cons: runtime overhead to maintain counters •Reachability

JVM Execution Engine

• Executes Java bytecodes either using interpreter or Just-In-Time compiler

• Registers: – PC: Program Counter

– FP: Frame Pointer

– SP: Operand Stack Top Pointer

• Interpreter: directly interpret the bytecodes

• JIT compiler: compile the bytecode into machine code then execute

10/10/2012 Managed Runtime Technology Introduction - Xiao-Feng Li 22

Page 23: Managed Runtime Technology: General Introductionpeople.apache.org/~xli/presentations/managed-runtime-introduction.pdf · –Cons: runtime overhead to maintain counters •Reachability

Bytecode Interpreter

while( program not end ) { fetch next bytecode => b switch(b) { case ILOAD: load an integer from the local variable array and push on top of current operand stack; case ISTORE: pop an integer from the top of current operand stack and store it into the local variable array; case ALOAD: ... ... } // end of switch } // end of while

10/10/2012 Managed Runtime Technology Introduction - Xiao-Feng Li 23

Page 24: Managed Runtime Technology: General Introductionpeople.apache.org/~xli/presentations/managed-runtime-introduction.pdf · –Cons: runtime overhead to maintain counters •Reachability

Object and Vtable

object vtable

bar

foo

x

t = [x]

[t+64]

64

Data

fields

x.foo()

Methods

addresses

10/10/2012 Managed Runtime Technology Introduction - Xiao-Feng Li 24

Page 25: Managed Runtime Technology: General Introductionpeople.apache.org/~xli/presentations/managed-runtime-introduction.pdf · –Cons: runtime overhead to maintain counters •Reachability

JIT Compiler

compile(methA)

Object meta data (synchronization, etc.) Object info

Vtable Pointer

Field 0

Field n

Object reference

JIT’ed binary code

VTable

Method X

Class Pointer

Method A

Interface Table

JIT compiler

10/10/2012 Managed Runtime Technology Introduction - Xiao-Feng Li 25

Page 26: Managed Runtime Technology: General Introductionpeople.apache.org/~xli/presentations/managed-runtime-introduction.pdf · –Cons: runtime overhead to maintain counters •Reachability

Interpreter vs. Compiler

• Key difference – Whether the engine needs to generate native code during

execution

– Interpreter: Portable code execution • Easy to implement, no extra memory for compiled code

– JIT Compiler: Faster code execution

• Tradeoffs between execution time and compilation time – Also tradeoffs between optimization levels

10/10/2012 Managed Runtime Technology Introduction - Xiao-Feng Li 26

Page 27: Managed Runtime Technology: General Introductionpeople.apache.org/~xli/presentations/managed-runtime-introduction.pdf · –Cons: runtime overhead to maintain counters •Reachability

Adaptive Compilation

Unoptimized Native

Execution Manager

Counters

Bytecode

Interpreter or fast Code Generator

Optimized Native

Optimizing Compiler

10/10/2012 Managed Runtime Technology Introduction - Xiao-Feng Li 27

Page 28: Managed Runtime Technology: General Introductionpeople.apache.org/~xli/presentations/managed-runtime-introduction.pdf · –Cons: runtime overhead to maintain counters •Reachability

Method-based vs. Trace-based JIT

• Method-based JIT – The method to compile is dynamically identified

– Pros: large optimization scope

– Cons: not all the method code is executed • If execution is very dynamic, optimization is difficult

• Trace-based JIT – The code in the trace is dynamically identified

– Pros: only compile hot traces and no control flow

– Cons: hot trace has to be stable • If execution is very dynamic, potential trace explosion

10/10/2012 Managed Runtime Technology Introduction - Xiao-Feng Li 28

Page 29: Managed Runtime Technology: General Introductionpeople.apache.org/~xli/presentations/managed-runtime-introduction.pdf · –Cons: runtime overhead to maintain counters •Reachability

Garbage Collection

• Reference counting vs. Reachability analysis

• Reference counting: – Object dead when reference count is zero

– Pros: Real-time death identification

– Cons: runtime overhead to maintain counters

• Reachability analysis – Object alive when it is reachable from app

– Pros: no need to maintain counters

– Cons: need suspend app to find reachable objects Xiao-Feng Li, et al, Cycler: Improve Heap Management for Allocation-intensive Applications with On-the-fly Object Reuse, PDCS’2011

10/10/2012 Managed Runtime Technology Introduction - Xiao-Feng Li 29

Page 30: Managed Runtime Technology: General Introductionpeople.apache.org/~xli/presentations/managed-runtime-introduction.pdf · –Cons: runtime overhead to maintain counters •Reachability

Reachability Analysis

• Roots – object references in threads’ execution context

• Live object – The object that is reachable from root reference or other live object

• Dead object (garbage) – Objects that are not accessible to the application

A

C

A B

C D

E F

G

roots

10/10/2012 Managed Runtime Technology Introduction - Xiao-Feng Li 30

Page 31: Managed Runtime Technology: General Introductionpeople.apache.org/~xli/presentations/managed-runtime-introduction.pdf · –Cons: runtime overhead to maintain counters •Reachability

Mark-Sweep-Compaction

• Algorithm

– Trace heap from roots to mark all live objects

– Sweep garbage from the heap, reclaiming their space

– Compact the heap to reduce the fragmentation

• Pros

– Fast, non-moving

• Cons

– Space fragmentation

– Object access locality

10/10/2012 Managed Runtime Technology Introduction - Xiao-Feng Li 31

Page 32: Managed Runtime Technology: General Introductionpeople.apache.org/~xli/presentations/managed-runtime-introduction.pdf · –Cons: runtime overhead to maintain counters •Reachability

roots

A

B

C

D

E

F

G

A

B

C

D

E

F

G

B

E

G

roots

B

E

G

Before GC

Live Garbage Unknown Free

After Mark After Sweep After Compact

roots roots

10/10/2012 Managed Runtime Technology Introduction - Xiao-Feng Li 32

Page 33: Managed Runtime Technology: General Introductionpeople.apache.org/~xli/presentations/managed-runtime-introduction.pdf · –Cons: runtime overhead to maintain counters •Reachability

Generational Collection

• Hypothesis: most objects die young, small percentage long live

• Algorithm – Partition the heap into generations, collect only the

younger generation mostly

– When older generations are full, collect entire heap

• Pros – Short pause time

• Cons – Floating garbage

10/10/2012 Managed Runtime Technology Introduction - Xiao-Feng Li 33

Page 34: Managed Runtime Technology: General Introductionpeople.apache.org/~xli/presentations/managed-runtime-introduction.pdf · –Cons: runtime overhead to maintain counters •Reachability

roots

A

B

C

D

E

A

B

C

D

E E

Before GC After Mark After Collect

roots roots

C

B

Young generation

Old generation

Live Garbage Unknown Free

10/10/2012 Managed Runtime Technology Introduction - Xiao-Feng Li 34

Page 35: Managed Runtime Technology: General Introductionpeople.apache.org/~xli/presentations/managed-runtime-introduction.pdf · –Cons: runtime overhead to maintain counters •Reachability

Concurrent GC

10/10/2012 Managed Runtime Technology Introduction - Xiao-Feng Li 35

App Thread 1

App Thread 2

App Thread 1

App Thread 2

GC Thread

App execution (mutator)

GC execution (collector)

Stop-the-world

Concurrent

pauses

pauses

time

Sleeping

GC Thread

Xiao-Feng Li, et al, Tick: Concurrent GC in Apache Harmony, 2009

time

Page 36: Managed Runtime Technology: General Introductionpeople.apache.org/~xli/presentations/managed-runtime-introduction.pdf · –Cons: runtime overhead to maintain counters •Reachability

Agenda

• Virtual machines

• Managed runtime systems

• EE and MM (JIT and GC)

• Summary

10/10/2012 Managed Runtime Technology Introduction - Xiao-Feng Li 36

Page 37: Managed Runtime Technology: General Introductionpeople.apache.org/~xli/presentations/managed-runtime-introduction.pdf · –Cons: runtime overhead to maintain counters •Reachability

Summary

• Evolution of Virtual Machines and Runtime Systems

• Modern Programming and Managed Runtime Systems

• Differences between a managed runtime system and a traditional runtime system

• Execution Engine and Garbage Collection

10/10/2012 Managed Runtime Technology Introduction - Xiao-Feng Li 37