CS412/413
Introduction to Compilers
Spring 2001

Iota+ Compiler Benchmark Results

Group Letter A B C D E F G H C++ Java
Compiler size (tokens) 100,782 117,185 107,910 56,718 88,423 95,257 87,309 90,396 - -
Ackermann's function Compile  3.58 2.31 4.23 2.67 2.53 2.44 3.79 2.09 .69 0.93
Run  7.34 2.03 2.38 2.62 2.99 3.89 2.54 13.88 2.36 1.63
.obj size 2,359 1,362 1,661 1,913 1,790 1,539 1,691 2,094 951 815
File reversal Compile  ex 2.76 3.39 3.01 4.73 3.30 sem 2.87 2.02 1.01
Run  - 0.16 0.14 0.13 0.14 failed - failed 0.38 1.09
.obj size - 2,011 4,361 8,281 10,162 - - - 21,670 916
Constant folding Compile  ex 2.45 2.50 2.21 2.47 2.60 2.22 2.22 0.72 1.11
Run  - 0.67 0.75 0.84 0.88 failed 0.86 1.19 0.20 0.30
.obj size - 1,053 1,354 1,376 1,532 - 1,266 1,651 788 561
Heapsort Compile  2.59 2.93 3.28 2.93 15.75 4.82 3.01 2.78 0.78 0.94
Run  6.79 3.13 3.54 3.61 4.76 failed 3.72 14.31 1.55 2.60
.obj size 8,543 2,814 3,171 7,321 9,374 - 5,109 9,984 1,807 1,497
List operations Compile  ex 3.00 3.33 2.96 7.65 3.78 2.96 3.11 1.65 1.06
Run  - 1.28 1.28 1.30 1.31 failed time crash 1.32 1.49
.obj size - 4,322 5,341 10,174 12,424 - - - 11,100 1,710
Matrix computations Compile  ex 4.41 6.07 7.39 time time 7.05 ex 0.65 0.96
Run  - 8.54 8.05 8.41 - - 10.75 - 1.32 2.15
.obj size - 5,053 4,710 11,238 - - 10,894 - 1,716 1,259
Method calls Compile  2.37 2.58 2.74 2.43 4.16 2.89 2.43 2.36 0.71 0.96
Run  failed 1.30 1.73 2.30 2.39 2.43 failed 6.18 0.50 1.42
.obj size - 2,135 2,562 3,372 4,367 3,140  - 5,408 2,275 1,593
Nested loops Compile  2.19 2.38 2.40 2.31 3.40 2.52 2.37 2.10 0.69 0.93
Run  8.14 2.56 2.03 2.33 3.22 3.17 11.20 9.93 0.81 2.33
.obj size 1,822 1,157 1,450 1,442 1,737 1,390  1,402 1,965 820 614
Object instantiation Compile  2.39 3.12 2.81 2.49 5.45 3.15 2.54 2.47 0.75 0.97
Run  failed 1.81 1.78 1.84 1.88 1.90 failed 2.12 0.06 0.86
.obj size - 2,504 2,934 3,962 5,360 3,600  - 6,524 2,463 1,865
Sieve of Eratosthenes Compile  2.28 2.44 2.56 2.42 4.81 3.03 2.34 2.22 1.27 0.95
Run  2.79 1.01 1.18 1.20 1.70 failed 1.37 5.82 0.26 0.71
.obj size 2,827 1,334 1,755 2,471 2,945 - 1,921 3,332 7,421 868
Word frequency

(WordFreq, Collection)

 Compile  sem
sem		 3.63
5.86		 4.44
8.59		 4.09
6.88		 28.88
56.51		 7.97
sem		 asm
7.59		 4.11
12.49		 3.68
-		 1.12
Run  - 1.14 1.24 1.25 failed - - crash .23 0.80
.obj size - 6,117
27,647		 8,249
32,708		 15,374
71,264		 - - - - 42,043
-		 3,456
-
Cases Passed 4 11 11 11 9 4 6 7 -

Geo. Mean of Run Times

5.80 1.43 1.49 1.59 1.57 2.75 3.32 5.63 0.52 1.20
Go failed ok ok failed failed failed ok failed -
RSA failed failed ok ok failed failed failed failed
Game of Life failed ok ok ok ok failed failed failed
Mandelbrot failed ok ok failed failed failed ok failed

 

Legend
ex Exception during compile
sem semantic error during compile
cmp other compile-time error
link link error
asm incorrect assembly code
crash run-time crash
failed incorrect output (other than white space) or other run-time error
time killed after time limit (3 minutes)
ok demo succeeded
failed demo failed for any reason