performance of programs written in different programming ... · pascal is a historically...

A is for AdaAda is a structured, statically typed, imperative, high-level computer

programming language, extended from Pascal and other languages. It has language support for design-by-contract, strong typing, explicit

concurrency, offering tasks, synchronous message passing, protected objects, and non-determinism.

C is for CC is a general-purpose, imperative computer programming language,

supporting structured programming, lexical variable scope and recursion. C provides constructs that map efficiently to typical machine

instructions.

C is for C++C++ is a middle-level programming language developed by Bjarne Stroustrup starting in 1979 at Bell Labs. C++ runs on a variety of

platforms, such as Windows, Mac OS, and the various versions of UNIX.

C is for C#C# is a general-purpose, object-oriented programming language. It was

developed by Microsoft within its .NET initiative.

F is for FortranFortran is an imperative programming language suited for numeric and

scientific computation that has gone through many decades of revisions.

F is for F#F# is developed by the F# Software Foundation, Microsoft and open

contributors. An open source, cross-platform compiler for F# is available. F# is also a fully supported language in Visual Studio and Xamarin Studio. Other

tools supporting F# development include Mono. F# originated as .NET implementation of a core of the OCaml programming language.

G is for GoGo is an open source programming language created at Google in 2007. It is a compiled, statically typed language in the tradition of Algol and C,

with garbage collection, memory safety features and CSP-style concurrent programming features added.

H is for HaskellHaskell is a standardized, general-purpose purely functional

programming language, with non-strict semantics and strong static typing. It is named after logician Haskell Curry.

J is for JavaJava is an imperative, object-oriented programming language designed

as an improvement over C++.

L is for LuaLua (LOO-ə, from Portuguese: meaning moon) is a lightweight multi-paradigm programming language designed primarily for embedded

systems and clients.

O is for OCamlOCaml, originally known as Objective Caml, is the main implementation of the Caml programming language. OCaml is a free open source project managed and principally maintained by INRIA. Many newer languages

have drawn elements from OCaml, most notably F# and Scala.

P is for PascalPascal is a historically influential imperative and procedural

programming language, designed in 1968–69 and published in 1970 by Niklaus Wirth as a small and efficient language intended to encourage good programming practices using structured programming and data

structuring.

P is for PythonPython is a widely used high-level, general-purpose, interpreted,

dynamic programming language. Its design philosophy emphasizes code readability, and its syntax allows programmers to express concepts in

fewer lines of code than would be possible in other.

R is for RustRust is a general-purpose, multi-paradigm, compiled programming

language sponsored by Mozilla Research. It is designed to be a "safe, concurrent, practical language", supporting pure-functional, imperative-

procedural, and object-oriented styles.

S is for ScalaScala is object-oriented, and uses a curly-brace syntax reminiscent of the C

programming language. Scala has many features of functional programming languages like Scheme, Standard ML and Haskell, including currying, type

inference, immutability, lazy evaluation, and pattern matching. It also has an advanced type system supporting algebraic data types, covariance and

contravariance, higher-order types, and anonymous types.

Hardware on hc210-059044

Processors 8

Model name Intel(R) Core(TM) i7-4790 CPU @ 3.60GHz

Cpu cores 4

Bogomips 7183.69

Hardware on ip-172-31-11-81Amazon Cloud

Processors 2

Model name Intel(R) Xeon(R) CPU E5-2666 v3 @ 2.90GHz

Cpu cores 1

Bogomips 5800.13

Versions on hc210-059044Language Version

Ada (Gnat) GNATMAKE GPL 2015 (20150428-49)

C/C++/Fortran (Ubuntu 4.8.4-2ubuntu1~14.04.1) 4.8.4

C# (Mono) Mono C# compiler version 3.2.8.0

Go go version go1.2.1 linux/amd64

Haskell (ghc) The Glorious Glasgow Haskell Compilation System, version 7.6.3

Java javac 1.8.0_91

Java(TM) SE Runtime Environment (build 1.8.0_91-b14)

Java HotSpot(TM) 64-Bit Server VM (build 25.91-b14, mixed mode)

Lua Lua 5.2.3 Copyright (C) 1994-2013 Lua.org, PUC-Rio

Pascal (fpc) 2014/01/22 2.6.2-8

Python 2 Python 2.7.6


Scala Scala compiler version 2.12.0-M2 -- Copyright 2002-2013, LAMP/EPFL

Scala code runner version 2.12.0-M2 -- Copyright 2002-2013, LAMP/EPFL

Versions on ip-172-31-11-81Language Version

Ada (Gnat) GNATMAKE 4.6

C/C++/Fortran (Ubuntu 4.8.4-2ubuntu1~14.04.1) 4.8.4

C# (mono) Mono C# compiler version 3.2.8.0

Go go version go1.2.1 linux/amd64

Haskell (ghc) The Glorious Glasgow Haskell Compilation System, version 7.6.3

Java javac 1.8.0_66

Java(TM) SE Runtime Environment (build 1.8.0_66-b17)

Java HotSpot(TM) 64-Bit Server VM (build 25.66-b17, mixed mode)

Lua Lua 5.2.3 Copyright (C) 1994-2013 Lua.org, PUC-Rio

Pascal (fpc) 2014/01/22 2.6.2-8



Scala Scala compiler version 2.11.7-M2 -- Copyright 2002-2013, LAMP/EPFL

Scala code runner version 2.11.7-M2 -- Copyright 2002-2013, LAMP/EPFL

Compiler Flags/Options

Ada (Gnat) gnatmake –f is –O2 {exec} {file}

C (clang) clang –o {exec} {file}

C (GNU) gcc –std=gnul11 –O2 –w –o {exec} {file}

C++ (GNU) g++ -O2 –w –o {exec} {file}

C# (Mono) mcs –out:{exec} {file}

Fortran (GNU) gfortran –O2 –o {exec} {file}

Go go build –o {exec} {file}

Haskell (ghc) ghc –fforce-recomp –O2 –outputdir ./build –o {exec} {file}

Java javac –d ./build {file}; jar cevf {base} {exec} –C ./build

java –jar {exec}

Pascal (fpc) fpc –O2 –Sa –o{exec} {file}

Scala scalac –deprecation –d {exec} {file}

scala {exec}

Methodology: Python wall-clock time

• Python script

start_time = time.time()

subprocess.call (<run command>,

stdin=<test case>, stdout=NULL, stderr=NULL,

timeout=100)

clocktime = time.time()-start_time

Tag Source Title Note

balancing (617,619,624)

USACO, February 2016 Load Balancing O(n^3) / scan / Fenwick Trees

heart ACM ICPC SER, 2008 Heart of the County Iterative fixed point

lawrence ACM ICPC SER, 2008 Lawrence of Arabia DP, precomputation, partial sums

rare ACM ICPC World Finals, 1990 Rare Order Topological sort

cheese ACM ICPC World Finals, 2015 Cutting Cheese Binary search, 3D geometry

sort O(n log n)

quadratic O(n^2)

cubic O(n^3)

The Problems

Results

Old ResultsGenerally consistent even though completely different version and platform

0.5

1

2

4

8

16

32

64

128

256

512

200 250 300 350 400 450 500

Nu

mb

er o

f ti

mes

co

mp

ared

to

C

N, the size of the problem instance "Lawrence of Arabia"

Time Compared with to C

Python3

Python2

Lua

Haskell

C#

Scala

Pascal

Java

Ada

C

C++

Fortran

0.5

1

2

4

8

16

32

64

128

0 100 200 300 400 500 600 700 800 900 1000

tim

es t

he

C p

rogr

am

"Heart of the Country"

Time Compared to C for "Heart of the Country"

Haskell

Python3

Lua

Java

Fortran

Python2

Scala

Ada

C++

C#

C

Pascal

Ratio Program Time (seconds)

1.00 C gcc 1.73

1.00 Pascal Free Pascal #3 1.73

1.00 Rust 1.76

1.10 Fortran Intel #3 1.92

1.10 Lua #5 1.94

1.30 Racket #2 2.18

1.30 Python 3 #5 2.20

1.30 C++ g++ #3 2.29

1.30 Ada 2005 GNAT #2 2.29

1.30 C gcc #4 2.30

1.60 Haskell GHC #4 2.83

1.70 Go 2.85

1.70 Scala #4 2.97

1.80 Java #2 3.11

6.50 F# Mono #3 11.19

The Computer Language Benchmarks Gamepidigits

The Computer Language Benchmarks Game:Python 3 versus C (Gnu)

Benchmark Python C Ratio

Pidigits 2.20 1.73 1.28

Regex-dna 10.58 2.46 4.30

Reverse-complement 3.11 0.62 5.02

K-nucleotide 76.50 12.53 6.11

Binary-trees 152.06 3.26 46.64

Mandlebrot 293.92 5.92 49.65

Fannkuch-redux 567.35 8.97 63.25

Fasta 111.44 1.36 81.94

N-body 923.74 9.56 96.63

Future Work

• F#, Rust, OCaml, …

• Java 8 – streams

• Haskell – lazy dynamic programming

• Lazy versus eager functional programming

• Clang

• Compiler options

On-line Repository

• A trusting person can repeat the experimentwget https://ser.cs.fit.edu/testing/time.sh | /bin/sh

• (This takes a very long time.)

• This also downloads all the source code and all the test cases

https://ser.cs.fit.edu/testing/time.sh

• John Clevenger, “Performance Analysis of Virtual vs. Physical Machines for Automated Judging in Programming Contests”, Competitive Learning Institute, 2015.

• The Computer Language Benchmarks Game, http://benchmarksgame.alioth.debian.org/

performance of programs written in different programming ... · pascal is a historically...

Documents