39

Recently at work we were playing around with the following quiz question asked by IBM https://www.research.ibm.com/haifa/ponderthis/challenges/May2015.html

After a bit of effort a colleague and I have arrived at two solutions, one in GoLang https://gist.github.com/walesey/e2427c28a859c4f7bc920c9af2858492#file-main-go-L57 and the other in Java https://gist.github.com/boyter/42df7f203c0932e37980f7974c017ec5#file-puzzle-java-L63 with the performance critical method for both being playGames in Java and game in GoLang (both linked in above).

The Go program is almost a literal copy of the Java one, and yet its runtime is ~6 seconds whereas the Java one is about ~26 seconds (on my local machine). Similar numbers were replicated on a few other machines with the Go program being about ~5x faster.

The Go program is compiled using 1.7.5 and Java using version 1.8.0_65 both running on macOS Sierra 10.12.3 on a late 2013 retina Macbook Pro with 2.6GHz i5 CPU.

Why is it that the Go program is 5x faster then the Java one when most benchmarks indicate that Java should about the same runtime? It is just basic math in a loop so it seems that they should run about the same time. I could understand a second or so for the JVM start time, but this seems off.

Both programs use pretty much the same loop. All of the possible permutations of game results are created and iterated over for each starting amount of money. It just seems that for any number of looping operations in the main loop that Go is running rings around Java.

I understand that this is a "micro" benchmark, but I am wondering why exactly the Go code is massively outperforming the Java code. Is it just that Go for simple loops/math is more efficient and hence faster? Is it able to unroll the loop perhaps (although this seems unlikely to produce such a massive difference)?

If not how should you structure a Java program to get the most performance out of a simple loop and math operation?

EDIT - Thanks to Dolda2000 I have modified the Java version. It is now about the same speed as the GoLang version. Indeed the issue was the the games were created causing the Java version to have to simulate more games to determine if the game went long enough. With the changes it is running in about ~6 seconds now and has restored my faith in Java.

Update - Here is an expanded essay which discusses the background of this question in further detail.

7
  • 2
    I think simply measuring the execution time of a program might not be good enough to interpret the results in any meaningful way. There are heaps of unknowns on the path from your code down to the processor (-core) and back to your terminal window.
    – Jan Groth
    Mar 29, 2017 at 1:48
  • 2
    Micro-benchmarks like this are a fool's game. This comparison benchmarksgame.alioth.debian.org/u64q/go.html shows one benchmark where Go has a 2.2 to 1 speed advantage, but several others where Java wins handily. Details of code generation and runtime environment easily have those kinds of effects.
    – Gene
    Mar 29, 2017 at 2:21
  • You have a lot going on, maybe you can isolate parts and test them individually.
    – Akavall
    Mar 29, 2017 at 4:24
  • So the performance critical areas for both are in playGames and game. If you look at them they are both almost 100% the same.
    – Ben Boyter
    Mar 29, 2017 at 4:43
  • 1
    Then either (a) your performance-critical analysis is flawed or (b) the issue as actually elsewhere. Either way the other comments about the conceptual issues involved in benchmarking or comparing two different languages are accurate. It's a bit like asking "Why is a Chevy faster than a Honda?" which - I hope you can appreciate - doesn't really make sense to ask.
    – dimo414
    Mar 29, 2017 at 6:48

1 Answer 1

48

As it turns out, your programs aren't as equal as you believe them to be. I instrumented them to see how many games (that is, individual betting rounds) they simulated, and while the Go version simulated 1 612 629 805 games, the Java version simulated 12 323 903 502 games, almost an order of magnitude more.

On my machine, turning off multithreading for more predictable results, the Java program clocked in on around 75 seconds, and the Go program on 12.5 seconds. Matching that against the total runtime, it appears that the Java program is actually slightly faster per simulated game, at about 6.1 ns, as compared to 7.8 ns for the Go program.

Not sure yet why they simulate such vastly different numbers of games, though. Perhaps the way the Go version generates the rounds simply happens to find a lot more quicker terminations.

EDIT: Actually, that last guess makes a lot of sense. The Go version starts off by modulating the initial rounds of a game, while the Java version starts off by modulating the last rounds of a game (in other words, looking at the list of rounds as a list of increasing 11-digit base-3 numbers, the Go version is little-endian, while the Java version is big-endian, so to speak), so the Java version will have to simulate through a lot more identical starts to get to the variations that terminate. I haven't tried to verify this hypothesis, but I'm sure enough about it that I don't feel the need to.

2
  • Thank you so much! I was going crazy wondering about this and never even considered that it might just be doing more work.
    – Ben Boyter
    Mar 30, 2017 at 0:19
  • What a great question! And what a great answer! Usually, we tend to resist instrumenting our code, however, it is a wiser move (measuring first) than optimizing (blindly) first. Dec 21, 2021 at 18:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.