Both the JVM and the CLR are regularly beaten by C/C++, despite claims that they are "as fast". What gives?
In my view the whole problem comes down to whether or not you are using an idiomatic approach or not. In idiomatic C#/Java you use a lot of heap allocations, garbage collection, you may be using dynamic dispatch and so on.
If you write a C# program that uses stack allocation only (no classes, only structs and primitives), no inheritance/polymorphism, no exceptions, you should find that the CLR stands up pretty well to a C++ program.
Sadly, what you have done then is essentially code in a very small subset of C#, OR you have achieved something that is so hard and so prone to foot-shooting you could just as well have used C++ to begin with.
To reverse the argument: if you use C++ with loads of garbage collected objects etc. you will end up with performance similar to a java/C# program. But in idiomatic C++, you usually don't.
C# lacks the language features to make dealing with such coding style sane and the libraries use the "idiomatic" approach - if you're willing to put on a straitjacket and throw away the libraries then why bother with C# ?
That's not saying that C# can't be made more efficient by avoiding allocation in performance critical sections, but overall it's going to perform worse than C++, both idiomatic and non idiomatic versions. It's just that for most use cases C# is used the performance difference isn't relevant anyway.
Java doesn't even have value types, so it can't even come close in terms of memory layout efficiency and avoiding allocations without perverse levels of manual data structure expansions - for eg. consider how would you do the equivalent of : struct small { int foo; float bar; char baz; }, std::vector<small>.
I agree completely. For an "inner loop" scenario in C# you have to use the straight jacket version of C#. Luckily you can use idiomatic C# for the rest of the program.
The option to using "straight jacket C#" is using C/C++ interop for a part of the program. If that part is large enough, such as in most high end games, it's usually worth biting the bullet and go C++ throughout. Luckily again, those programs are rare.
Point is still that C-style C# is almost as fast as C++ (modulo compiler specific optimizations) but for the reasons above, that fact isn't very interesting.
Java will have value types soon. Also, C++ does some heap allocations behind-the-scenes quite often (e.g. with vector or string) which are hard to get rid of and they are much more expensive than in JVM/CLR. So not always idiomatic C++ smashes JVM/CLR in performance. YMMV and typical differences are small enough to not matter for most of server-side software like web-apps, web-servers or databases.
What gives is this. The people who right these fast C/C++ programs that beat Java/C# are usally far more skilled and trained.
Any programmer who know neither language well and has to write a big application in it will probebly find java/c# far easier.
I remember a long blogpost where somebody set out to test this on himself. And that guy was a very good C++ programmer. He found that his C++ prgrogrammer was slower, but he then set on improving the speed and in the end beat java by quite a bit. However the amount of effort was completly unreasonable for most programmers.
So "What gives" is this, and this has been true for a long time. If you are a expert in a low level language and spend time optimizing you will probebly beat Java/C#.
I would suggest that you should look into what a JIT or GC can and can not do. Some of the performance problems you identivy are really almost never a bottleneck anymore.
I agree, my point is merely that the bulk of the difference between idiomatic Java programs and idiomatic C++ programs is due to the wildly different ways of of programming idiomatically in Java vs. C++.
The small (steady state) performance difference remaining when doing the "exact same thing" in both programs is just down to how good the C++ compiler is vs. the VM JIT at optimizing (usually better, sadly).
What intrigues me about Rust is that hopefully we won't have to choose between readability and elegance vs. performance and safety. Keep up the good work.
In my view the whole problem comes down to whether or not you are using an idiomatic approach or not. In idiomatic C#/Java you use a lot of heap allocations, garbage collection, you may be using dynamic dispatch and so on.
If you write a C# program that uses stack allocation only (no classes, only structs and primitives), no inheritance/polymorphism, no exceptions, you should find that the CLR stands up pretty well to a C++ program. Sadly, what you have done then is essentially code in a very small subset of C#, OR you have achieved something that is so hard and so prone to foot-shooting you could just as well have used C++ to begin with.
To reverse the argument: if you use C++ with loads of garbage collected objects etc. you will end up with performance similar to a java/C# program. But in idiomatic C++, you usually don't.