These are the old pages from the weblog as they were published at Cornell. Visit www.allthingsdistributed.com for up-to-date entries.

September 12, 2003

Method Size and CLR performance

In preparation for the trip to the SSCLI Workshop next week I am prepping the CLI port of the Java Grande Benchmarks, given I had promised to make them available before the conferences where the benchmarks are discussed. I will talk about them next week at the Rotor Conference, but also later in the year also at Supercomputing 2003 and at the MSR German Academic Conference.

One of the reasons why I took my time with putting out the code was that I am experimenting with small changes to the code organization to investigate the impact of those changes on the benchmark results. My goal has always been to try not to change the code layout of the benchmarks, to be as fair as possible when comparing the CLR results with the JVM results. But there are some changes that have a significant impact on the benchmark results even with only small changes to the code.

One of the conclusions of running the benchmarks was that the ability of the JIT to optimize code was central to getting decent performance. See my May posts for more details. There are many code organization structures that impact the way the JIT engine emits code, one of them being the size of a method. The CLR JIT has a limit on the variable it considers for enregistration (tracking local variables for storage in variables).

An example where this has a significant impact is in the arithmetic benchmark. In the original JGF version all the add, multiply and divide tests are in one single method. By moving each of the tests into their own method you can see performance improvements between 1.2 and 35 times when compared to executing the same test as part of a large method.

.
Improvements to the 'add' results when moving to small methods

The question is whether comparing these results is fair, given that for Java this reorganization has no real performance impact. I feel that it is fair, given that the basics of the test namely the loop of integers additions which is being measured remains identical.

In the release of the code there will a be Section1 which holds the 100% identical code and a Section1M with modified code that has simple modifications to the code organization. Code will be available by the middle of next week.

Posted by Werner Vogels at September 12, 2003 02:29 PM
TrackBacks

New and Notable 10
Excerpt:
Weblog: Sam Gentile's Blog
Tracked: September 16, 2003 08:23 AM
New and Notable 10
Excerpt:
Weblog: Sam Gentile's Blog
Tracked: September 16, 2003 08:30 AM
New and Notable 10
Excerpt:
Weblog: Sam Gentile's Blog
Tracked: September 16, 2003 08:52 AM
New and Notable 10
Excerpt:
Weblog: Sam Gentile's Blog
Tracked: September 16, 2003 08:53 AM
New and Notable 10
Excerpt:
Weblog: Sam Gentile's Blog
Tracked: September 16, 2003 12:14 PM
New and Notable 10
Excerpt:
Weblog: Sam Gentile's Blog
Tracked: September 16, 2003 12:38 PM
New and Notable 10
Excerpt:
Weblog: Sam Gentile's Blog
Tracked: December 31, 2003 12:00 AM
Comments