Comments on: How well do you know Java? http://labnotes.org/2007/02/10/how-well-do-you-know-java/ Sun, 15 Jan 2012 21:29:54 +0000 hourly 1 http://wordpress.org/?v=3.3.1 By: Refresh Your Mind - Hendy Irawan http://labnotes.org/2007/02/10/how-well-do-you-know-java/comment-page-1/#comment-137605 Refresh Your Mind - Hendy Irawan Tue, 13 Feb 2007 02:16:43 +0000 http://blog.labnotes.org/2007/02/10/how-well-do-you-know-java/#comment-137605 [...] from Assaf: … every once in a while, we need to step back, forget all that we believe is true, and look [...] [...] from Assaf: … every once in a while, we need to step back, forget all that we believe is true, and look [...]

]]> By: http://dvae.livejournal.com/ http://labnotes.org/2007/02/10/how-well-do-you-know-java/comment-page-1/#comment-137602 http://dvae.livejournal.com/ Mon, 12 Feb 2007 13:00:33 +0000 http://blog.labnotes.org/2007/02/10/how-well-do-you-know-java/#comment-137602 If you use a more recent version of java, then it's start up is faster than ruby. At least that is what Charles Nutter found with these numbers: $ time java -cp lib/jruby.jar:lib/asm-2.2.2.jar:. fib_recursive 5702887 real 0m8.126s user 0m7.632s sys 0m0.208s $ time ruby fib_recursive.rb 5702887 real 0m14.649s user 0m12.945s sys 0m1.480s Full (long) blog post on jruby JIT: http://headius.blogspot.com/2007/01/jruby-compiler-in-trunk-and-ready-to.html If you use a more recent version of java, then it’s start up is faster than ruby.

At least that is what Charles Nutter found with these numbers:

$ time java -cp lib/jruby.jar:lib/asm-2.2.2.jar:. fib_recursive
5702887
real 0m8.126s
user 0m7.632s
sys 0m0.208s

$ time ruby fib_recursive.rb
5702887
real 0m14.649s
user 0m12.945s
sys 0m1.480s

Full (long) blog post on jruby JIT: http://headius.blogspot.com/2007/01/jruby-compiler-in-trunk-and-ready-to.html

]]> By: Assaf http://labnotes.org/2007/02/10/how-well-do-you-know-java/comment-page-1/#comment-137599 Assaf Sun, 11 Feb 2007 12:49:36 +0000 http://blog.labnotes.org/2007/02/10/how-well-do-you-know-java/#comment-137599 Apotheon, The narrow focus on only measuring performance in the app server is the problem. Let's drop desktop apps and Applets, because Java is not even a contender there. Java's worst enemy is not Ruby or Perl or C#; Java's worst enemy is Java. If I can write a simple task in Ruby in five minutes, that will take 10 seconds to execute. And I can write the same task in Java in two hours, that will take 5 seconds to execute. Which one is faster? There's the holisitic view to measuring performance. And then there's the beauty contest view to measuring performance, where you only show the flattering profile. Now, someone may say apples vs oranges. But that's real life. Software is about automating things, at least the kind of software I write about, and what you gain from automation comes after the cost it takes to create it. If some tool allows me to automate more things, cheaper and better, I'd use it. The beauty contest approach to measuring performance made Java incredibly fast in a very narrow context, but incredibly slow in so many other tasks a language should perform. I don't think people would even bother to respond to my post if Java wasn't, right now, on the defensive from other languages that are eating its lunch, percisely because it optimized the wrong thing. Apotheon,

The narrow focus on only measuring performance in the app server is the problem. Let’s drop desktop apps and Applets, because Java is not even a contender there.

Java’s worst enemy is not Ruby or Perl or C#; Java’s worst enemy is Java.

If I can write a simple task in Ruby in five minutes, that will take 10 seconds to execute. And I can write the same task in Java in two hours, that will take 5 seconds to execute. Which one is faster?

There’s the holisitic view to measuring performance. And then there’s the beauty contest view to measuring performance, where you only show the flattering profile.

Now, someone may say apples vs oranges. But that’s real life. Software is about automating things, at least the kind of software I write about, and what you gain from automation comes after the cost it takes to create it.

If some tool allows me to automate more things, cheaper and better, I’d use it.

The beauty contest approach to measuring performance made Java incredibly fast in a very narrow context, but incredibly slow in so many other tasks a language should perform.

I don’t think people would even bother to respond to my post if Java wasn’t, right now, on the defensive from other languages that are eating its lunch, percisely because it optimized the wrong thing.

]]> By: apotheon http://labnotes.org/2007/02/10/how-well-do-you-know-java/comment-page-1/#comment-137598 apotheon Sun, 11 Feb 2007 10:03:26 +0000 http://blog.labnotes.org/2007/02/10/how-well-do-you-know-java/#comment-137598 The actual VM startup time isn't the only thing at issue here, though. Java suffers (or benefits, if you're a Java-head) from a number of phenomena in the way it's implemented that significantly affect performance over the lifetime of a process. The startup time of the VM is one factor: the optimizing capabilities of the VM are another. Finally, there's the persistence of the VM environment. This adds up to a big (in terms of performance benchmarks) difference between short-run application performance and long-run application performance for Java. If you think MS Office is slow when you just want to quickly edit a document and shut down again, you should try writing it in Java and try that again. Perl actually outperforms Java in a significant number of cases for short-run benchmarks -- Perl, which actually does its parse-tree compilation (which is a more-than-single-pass compiler, by the way) at runtime! Count the startup time of both languages in a several second computation benchmark, and Perl will probably win. On the other hand, if you're running a server process that churns along for months at a time, and measure its performance on day three, it will rival roughly identical code in C++ for speed (though anyone claiming it matches C doesn't understand the differences between C and C++, or writes incredibly bad spaghetti C). Not only does measuring at the beginning capture VM startup time -- it also measures unoptimized code. Java is a bit like vehicle with a big engine and a huge flywheel: it doesn't accelerate worth a damn, but it sure does burn up road once it gets up to speed. In general, I agree that the actual startup time of the JVM is trivial, and mostly ignorable -- and thus, might as well be measured. Sure, it can prove significant if you're measuring something so brief that a 0.1 second startup is problematic -- or more, on an older machine (which most of us are using, believe it or not, as I know from personal experience since a JVM would start up somewhat slower on this 1600MHz AMD than on your Core Duo 2). That's of little import when measuring actual human wait time, though, as you point out. The real problem is the code optimization, which doesn't happen instantly. Let it run through its paces a few dozen times, and it's much snappier. Basically, I guess what I'm saying is that it's not really that simple. You just have to measure all the parts that are relevant to your purpose, adjusted every time your purpose changes. Saying Ruby does some calculation faster (or slower) than Java requires context to be worth crap, simply because of the strange (as compared with other languages) way that Java works. The actual VM startup time isn’t the only thing at issue here, though. Java suffers (or benefits, if you’re a Java-head) from a number of phenomena in the way it’s implemented that significantly affect performance over the lifetime of a process. The startup time of the VM is one factor: the optimizing capabilities of the VM are another. Finally, there’s the persistence of the VM environment.

This adds up to a big (in terms of performance benchmarks) difference between short-run application performance and long-run application performance for Java. If you think MS Office is slow when you just want to quickly edit a document and shut down again, you should try writing it in Java and try that again. Perl actually outperforms Java in a significant number of cases for short-run benchmarks — Perl, which actually does its parse-tree compilation (which is a more-than-single-pass compiler, by the way) at runtime! Count the startup time of both languages in a several second computation benchmark, and Perl will probably win. On the other hand, if you’re running a server process that churns along for months at a time, and measure its performance on day three, it will rival roughly identical code in C++ for speed (though anyone claiming it matches C doesn’t understand the differences between C and C++, or writes incredibly bad spaghetti C).

Not only does measuring at the beginning capture VM startup time — it also measures unoptimized code. Java is a bit like vehicle with a big engine and a huge flywheel: it doesn’t accelerate worth a damn, but it sure does burn up road once it gets up to speed.

In general, I agree that the actual startup time of the JVM is trivial, and mostly ignorable — and thus, might as well be measured. Sure, it can prove significant if you’re measuring something so brief that a 0.1 second startup is problematic — or more, on an older machine (which most of us are using, believe it or not, as I know from personal experience since a JVM would start up somewhat slower on this 1600MHz AMD than on your Core Duo 2). That’s of little import when measuring actual human wait time, though, as you point out. The real problem is the code optimization, which doesn’t happen instantly. Let it run through its paces a few dozen times, and it’s much snappier.

Basically, I guess what I’m saying is that it’s not really that simple. You just have to measure all the parts that are relevant to your purpose, adjusted every time your purpose changes. Saying Ruby does some calculation faster (or slower) than Java requires context to be worth crap, simply because of the strange (as compared with other languages) way that Java works.

]]> By: Assaf http://labnotes.org/2007/02/10/how-well-do-you-know-java/comment-page-1/#comment-137596 Assaf Sun, 11 Feb 2007 07:06:56 +0000 http://blog.labnotes.org/2007/02/10/how-well-do-you-know-java/#comment-137596 cyu, Ruby is not immune from measuring the wrong thing, and I think there's enough material for future posts on that. But Java has a particular obsession, a doctrine for only measuring that which was JITted, probably dating back to when Java was fighting the performance war against C++. Right now, for most applications, it's fast enough that we can start worrying about other things. Some people don't think the "waiting for the JIT" doctrine is a problem. But hopefully enough people will recognize it and do something to fix it. Before Java ends up executing as fast as C++, and being as painful to work with as C++. cyu,

Ruby is not immune from measuring the wrong thing, and I think there’s enough material for future posts on that.

But Java has a particular obsession, a doctrine for only measuring that which was JITted, probably dating back to when Java was fighting the performance war against C++. Right now, for most applications, it’s fast enough that we can start worrying about other things.

Some people don’t think the “waiting for the JIT” doctrine is a problem. But hopefully enough people will recognize it and do something to fix it. Before Java ends up executing as fast as C++, and being as painful to work with as C++.

]]> By: http://openid.claimid.com/cyu http://labnotes.org/2007/02/10/how-well-do-you-know-java/comment-page-1/#comment-137594 http://openid.claimid.com/cyu Sun, 11 Feb 2007 06:43:08 +0000 http://blog.labnotes.org/2007/02/10/how-well-do-you-know-java/#comment-137594 Here's the problem with doing the measurement this way. Say you benchmark some calculation in Java that took 0.1 seconds, bringing the total real time to 0.2 seconds (startup time + calculation time). Not so bad, but what happens when you bring that calculation in your server in loop 1000 times, or is requested 1000 times simultaneously? Using the real wait time measurement as your basis, you would estimate that the calculation executed in 200 seconds, when in reality it is probably about 100 seconds. Then the measurement becomes significant. So then you could ask, 'why not just include the looping into benchmark?' - but that's not what you should do when benchmarking, you should only test the code you are concerned about, and in this case it is the calculation. I don't see this as an issue of Ruby v. Java, as I can make the same statement above about Ruby and it would still apply (the number just wouldn't be as significant). The issue is your criticism of a very valid benchmarking technique, and your statement that using it would lead to poor design decisions. Here’s the problem with doing the measurement this way. Say you benchmark some calculation in Java that took 0.1 seconds, bringing the total real time to 0.2 seconds (startup time + calculation time). Not so bad, but what happens when you bring that calculation in your server in loop 1000 times, or is requested 1000 times simultaneously? Using the real wait time measurement as your basis, you would estimate that the calculation executed in 200 seconds, when in reality it is probably about 100 seconds. Then the measurement becomes significant. So then you could ask, ‘why not just include the looping into benchmark?’ – but that’s not what you should do when benchmarking, you should only test the code you are concerned about, and in this case it is the calculation.

I don’t see this as an issue of Ruby v. Java, as I can make the same statement above about Ruby and it would still apply (the number just wouldn’t be as significant). The issue is your criticism of a very valid benchmarking technique, and your statement that using it would lead to poor design decisions.

]]>