[ Up ]

# ALE Benchmarks

Times listed are process user time; ALE is compiled with GCC 3.3.4 and linked with ImageMagick version 6.0.1; and all images are 3x8-bit RGB non-interlaced PNGs. Test names are constructed as follows:

Machine code | Description
--- | ---
PIII | Pentium III 700MHz 100MHz FSB 256K L2 cache
ATH | Athlon XP 3000+ 400MHz FSB 512K L2 cache
Test code | Description
A+R | 3 256x256 100% white images, --perturb-upper=32
R | 3 256x256 100% white images, --perturb-upper=0
Run Code | Description
1 | Run number 1
2 | Run number 2
3 | Run number 3

## Benchmark Table

Version | Configuration | PIII A+R 1 | PIII A+R 2 | PIII A+R 3 | PIII R 1 | PIII R 2 | PIII R 3 | ATH A+R 1 | ATH A+R 2 | ATH A+R 3 | ATH R 1 | ATH R 2 | ATH R 3
0.5.1-patch3 | IMAGEMAGICK=1 BITS=8 | 0m16.830s | 0m16.884s | 0m16.871s | 0m1.432s | 0m1.438s | 0m1.432s | 0m4.819s | 0m4.821s | 0m4.818s | 0m0.400s | 0m0.401s | 0m0.401s
0.5.1-patch3 | IMAGEMAGICK=1 BITS=16 | 0m17.024s | 0m17.062s | 0m17.060s | 0m1.493s | 0m1.500s | 0m1.493s | 0m4.904s | 0m4.902s | 0m4.904s | 0m0.414s | 0m0.415s | 0m0.415s
0.6.0 | IMAGEMAGICK=1 --8bpc | 0m13.917s | 0m13.881s | 0m13.888s | 0m2.632s | 0m2.642s | 0m2.640s | 0m4.251s | 0m4.257s | 0m4.254s | 0m0.771s | 0m0.764s | 0m0.750s
0.6.0 | IMAGEMAGICK=1 --16bpc | 0m14.007s | 0m14.015s | 0m14.020s | 0m2.754s | 0m2.751s | 0m2.753s | 0m4.307s | 0m4.299s | 0m4.302s | 0m0.794s | 0m0.803s | 0m0.796s
0.7.0 | IMAGEMAGICK=1 --q0 --8bpc | 0m13.328s | 0m13.366s | 0m13.352s | 0m8.245s | 0m8.255s | 0m8.253s | 0m3.739s | 0m3.742s | 0m3.731s | 0m2.325s | 0m2.328s | 0m2.322s
0.7.0 | IMAGEMAGICK=1 --q0 --16bpc | 0m13.439s | 0m13.466s | 0m13.462s | 0m8.378s | 0m8.337s | 0m8.342s | 0m3.765s | 0m3.762s | 0m3.768s | 0m2.356s | 0m2.354s | 0m2.354s
0.7.1 | IMAGEMAGICK=1 --q0 --8bpc | 0m13.133s | 0m13.153s | 0m13.143s | 0m8.100s | 0m8.109s | 0m8.108s | 0m3.774s | 0m3.781s | 0m3.776s | 0m2.327s | 0m2.332s | 0m2.322s
0.7.1 | IMAGEMAGICK=1 --q0 --16bpc | 0m13.243s | 0m13.249s | 0m13.269s | 0m8.177s | 0m8.196s | 0m8.204s | 0m3.808s | 0m3.810s | 0m3.799s | 0m2.371s | 0m2.368s | 0m2.365s

## Summary

Between versions 0.5.1 and 0.7.0, alignment times show a steady improvement, but this can be partly explained by a change in the default alignment settings in 0.7.0. Rendering times, on the other hand, are about 6 times longer for 0.7.0 than for 0.5.1, as a consequence of renderer changes. More detailed summaries for alignment and rendering tests follow.

### Alignment

Alignment times improve as the version number increases, through version 0.7.0, but note that version 0.7.0 with the --q0 option defaults to --mc 30, which offers less accurate alignment than is offered by other versions' configurations. (Instead of sampling the difference between images for each reference image pixel, --mc 30 specifies that differences be sampled at roughly 30% of reference image pixels.)

### Rendering

Rendering times increase significantly with version number, through version 0.7.0, the most significant increase being between 0.6.0 and 0.7.0, as the latter uses a computationally more expensive rendering technique. Whether the improved rendering quality is worth the extra rendering time likely depends on subjective criteria, but the differences seem to be most apparent when performing relatively complex rendering tasks.

Copyright 2004 David Hilvert

Verbatim copying and distribution of this entire article is permitted in any medium, provided this notice is preserved.