The title of "Best Iota Compiler" is once again up for grabs!
The benchmark results are available.
During your PA6 demo (schedule), we will run a series of benchmarks on your compiler. We want this to go as smoothly as possible, so we're providing you with a skeleton benchmark suite. Instructions for using the suite are included in a readme file. You must test this suite before your demo -- it requires that you supply some files of your own that you'll need to have available at the time of the demo. Also, you'll want to make sure your compiler is compliant with the expectations built into the testing scripts. After all, it will be a very one-sided demo if we can't compile any of our programs! Any updates to the suite will be posted here.
[5/9] Apparently not all the CSUGLAB computers have Cygwin installed, so make sure to pick one that does for your demo!
[5/8] Here's a test case that hints at constructs that some of the more complex test cases in the suite might require, but that we didn't test very thoroughly for PA5: hint.im
[5/8] If you want to do additional optimizations, you can use the flag variable described below to enable them. We will allow your compiler two attempts at the suite -- one with the extra optimizations enabled, and if any test case fails in that attempt, a second with just the default optimizations.
[5/8] An extra environment variable IOTAC_FLAGS has been added to bench.sh and the sample setenv.bat. You can use this to pass additional flags to your compiler.
[5/7] The linker switch /nodefaultlib:libc has been removed from bench.sh. The skeleton suite has been updated. Make sure to use the release build of your iotaextra.lib, otherwise the removal of this switch could cause linker warnings.
[5/3] About half of the full benchmark suite will be Iota+ programs, while the other half will be nearly-Iota programs. "Nearly-Iota" means they might use some Iota+ operators such as ++ or <=, as well as constants, but nothing else, especially not classes or interface types. sieve.im (which is in the skeleton suite) is the most non-Iota-conformant of them, for example.
[5/3] In case this wasn't clear: during your demo, we expect to be able to copy the full suite into a new directory at the top level of your hierarchy, open a command shell, and run bench.bat. No extra actions on your part or ours should be necessary.
Your compiler will be evaluated using at least the following criteria:
Correctness
Performance of compiler
Performance of generated code
Quality of code (size), documentation
Extra features and optimizations added
User interface
Bragging rights, of course, plus an additional award which could be a plaque, a gift certificate, food, etc.
|
Thursday, May 10 |
|
| 1:30-2:15 | Group 3 |
| 2:15-3:00 | Group 1 |
| 3:00-3:45 | Group 7 |
| 3:45-4:30 | Group 9 |
|
Friday, May 11 |
|
| 1:30-2:15 | Group 8 |
| 2:15-3:00 | Group 4 |
| 3:00-3:45 | Group 2 |
| 3:45-4:30 | Group 5 |