It's interesting to compare the issues that Shapiro found with the issues that we found during the development of Rust. It turns out that we ran into many of the same issues. Since Go is being mentioned here too, I'll compare the way Go dealt with the issues as well.
(0) Objects and inheritance: Rust had support for objects (via prototype-based inheritance) in the first version of the compiler, but we found that they weren't being used. We attempted to be as minimalist and simple as possible regarding objects, and as a result we ended up with a system that didn't have enough features to be useful. It also didn't fit well with the rest of the language, so we scrapped it and added typeclasses instead. Those worked a lot better, and now most of our standard library is using typeclasses for OO-like patterns. Recently, we've found that we really do want objects, but mostly as a way to achieve code reuse, privacy, and a more direct association between a type and its method than typeclasses alone provide. The current system that is being implemented is a nice way, in my opinion, to unify typeclasses and object-oriented programming. There are just a few concepts to learn, and it all meshes together quite nicely.
Go's interface system is quite similar to Rust's typeclass system. The main things that Rust has that Go doesn't are first-class constructors, the trait system (not yet implemented) to facilitate method reuse, and the ability to define multiple implementations of an interface per type. The things that Go has that Rust doesn't are duck typing (which is a quite good idea, and I'd like to add it to Rust as well) and downcasting (which we don't want to support because we have more type-safe mechanisms for the same thing).
(1) The compilation model: Rust uses dynamic linking pervasively, because OS's have good support for it and it helps keeps binaries small. It also has strong support for separate compilation, because we want to make compile times fast. So far, so good, but, just like BitC did, we discovered that type abstraction (which you use generics in Rust to achieve) doesn't mix well with separate compilation. We didn't want to have a uniform value representation like the MLs do (supporting only 31-bit ints and boxing everything else doesn't fly in a systems language), so we tried to use dynamic size calculations for all of the values. It resulted in a huge amount of complexity (we never shook out all the bugs), and it also had a large runtime performance penalty. Unlike C#, we couldn't fall back on a JIT, because Rust is an ahead-of-time-compiled language. So we moved to a "monomorphization" scheme for Rust 0.2, which is basically like C++ template instantiation, only without the overhead of reparsing all the code from scratch. Even with this scheme, you only pay for monomorphization when you use generics, you can still dynamically link all non-generic code (which is most of it), and your runtime performance is unaffected by your use of generics.
Go, of course, doesn't have generics. I don't personally believe that buys them much though; the programmer ends up working around it in a way that either involves boxing (via interface{}) or by-hand monomorphization (by duplicating the code for each type). To me, generics are just a way for the compiler to do work the programmer would end up having to do anyway.
(2) Insufficiency of the type system regarding reference and by-reference types. It's spooky to read this, because it's precisely the problem we ran into with Rust. At the moment we have by-value and by-reference modes for parameters, and we've found that this isn't sufficiently expressive. (We also tried making the only difference between by-value and by-immutable-reference internal to the compiler, which didn't work well due to higher-order functions and interoperability with C code.) We also found that parameters really aren't the only place you want by-reference types; you really want to be able to return references and place them within other data structures. Whenever we wanted to do this, we had to fall back onto heap allocation, and that was significantly hurting our performance, especially when unique types were involved (since aliasing a unique type is impossible, you have to copy it). Profiling Rust programs showed an alarming amount of time spent in malloc and free. So we're in the process of bringing up a new regions system that I'm excited about: it's too early to say for sure, but I think we've stumbled upon a way to make regions not require a doctorate in type theory to understand. Regions allow you to have safe pointers into the stack and into the heap and pass them around as first-class values.
Go doesn't have zero-cost reference types at all; it just does simple escape analysis to allocate structures on the stack when it can and falls back on tracing GC for the rest (note that this is what Java does nowadays too). This is one of the most significant differences between Go and Rust; Go's memory model is essentially identical to that of Java, plus the ability to allocate structures inside other structures, while Rust has a much more C++-like memory model (but safe, unlike C++). This decision is based on our experience with Firefox; fine-grained control over memory use is so important that we didn't want to place our bets on pervasive use of GC.
(3) Inheritance and encapsulation: Rust still has no concept of inheritance; it's our hope that a combination of enums (datatypes like in Haskell or case classes from Scala) and traits will allow us to avoid introducing the complexity of inheritance into the language. Time will tell, of course. As for encapsulation, we thought we didn't need it, but it turns out that we really did want private fields. This we're solving with the class system, mentioned above.
Go achieves inheritance through anonymous fields. Anonymous fields are multiple inheritance in all but name, complete with the "dreaded diamond" problems of C++. We were hoping to avoid that. Go has standard privacy through "unexported fields".
(4) Instance coherence. Since you can define multiple typeclass implementations for a given type, and the caller chooses which implementation to use, you have to make sure in many contexts (for example, the hash method of a hash table) that the same implementation gets used for the same data. That's one of the reasons we introduced classes -- they tie implementations of methods to a type.
Go doesn't have this problem, because it only permits one implementation of an interface for each type, and it has to be defined in the same module as the type. Basically, Go's interfaces are much like our classes in that regard. We wanted to allow people to add extra methods to types -- for example, to add extension methods on vectors (think what Rails does to the Ruby standard library, but in a way that doesn't involve monkey patching) -- so we didn't want to force this restriction on users.
I think that one of the most important things to underscore is that we would have never found these things so early unless we had written the Rust compiler in Rust itself. It forces us to use the language constantly, and we quickly find pain points. I highly encourage all languages to do the same; it's a great way to find and shake out design issues early.
>Go achieves inheritance through struct nesting. I find this a bit clever for my taste, and it only admits single inheritance, but it's certainly simple.
Is that really so? I thought that Go could nest multiple structs at once, and would complain if there were method name conflicts, but allow these to be resolved by defining a new method (based on the "outermost method wins" resolution rule) that could decide how to explicitly call included methods.
> Go has standard privacy through "unexported fields"
And via the "all methods must be defined with the struct" rule which prevents out of control access to internals.
Hmm, hairy how? You get forced to resolve clashes by defining a new top level method, and you can always call the included anonymous methods directly.
Seems to me, "in type T, keep included type A's implementation of X and rename B's implementation as Y" is identical in effect to "define T.X as calling self.A.X() and define T.Y as calling self.B.X()"
I suppose they aren't that hairy if you introduce type-specific namespacing like C++ does; i.e. allowing method names to be qualified with the name of the class or trait they belong to to resolve conflicts (A::X and B::X). Renaming seems like a way to avoid having to create an extra delegation function in case you want to delegate to one or the other, though.
It's important to keep in mind that traits aren't the same as inheritance, though. Traits combine methods and fields into one final class instance (they're like mixins), while anonymous fields are more like C++ inheritance: there's actually an instance of each superclass embedded inside each subclass. This starts to matter especially when you have diamond patterns (A derives B and C, B and C derive D). When you have an instance of D and are calling methods of A on it, does each method of B's embedded A clash with that of C's embedded A in Go?
Go's anonymous fields are always accessible as a struct member named after their type, basically. It isn't namespacing, it's syntactic sugar that kinda-sorta removes their anonymity. Diamond inheritance in Go means you have physical nesting like [D [C [A]] [B [A]]] so necessarily the A's will clash, but you can override their methods and manually resolve it - yeah, at the cost of a delegating call (which the compiler could probably strip out).
Interfaces subsume the feature of traits to define an operation abstractly against the methods of its operand without defining its representation. Unlike traits though they can't mess with member variables directly (of course methods/interfaces can be defined on things that lack members, such as a renamed int32).
Traits are actually about code reuse, not about polymorphism. Indeed, in the original traits literature (and in Rust), traits are not types at all. You use interfaces in Rust for that, just as you do in Go (they work very similarly).
Think of traits as little pieces you can glue together to make a class.
Pure-code mixins and methods on Go interfaces are sort of duals of one another. Interface methods are parameterized with a type when called ("calling outwards"), mixins are parameterized by a type when compiled, and merged into it ("calling inwards").
Wow that was very interesting to read and shows you guys are pushing boundaries even when the goal is a pragmatic language.
One last big thing that Shapiro mentioned was criticizing purity alone as a useful way of attacking concurrency and losing faith in verification. Can you or someone else knowledgeable comment on this?
>The last reason we left objects out of BitC initially was purity. I wanted to preserve a powerful, pure subset language - again to ease verification.
The object languages that I knew about at the time were heavily stateful,
and I couldn't envision how to do a non-imperative object-oriented
language. Actually, I'm still not sure I can see how to do that
practically for the kinds of applications that are of interest for BitC.
But as our faith in the value of verification declined, my personal
willingness to remain restricted by purity for the sake of verification
decayed quickly.
The other argument for a pure subset language has to do with advancing
concurrency, but as I really started to dig in to concurrency support in
BitC, I came increasingly to the view that this approach to concurrency
isn't a good match for the type of concurrent problems that people are
actually trying to solve, and that the needs and uses for non-mutable state
in practice are a lot more nuanced than the pure programming approach can
address. Pure subprograms clearly play an important role, but they aren't
enough.
Sure. Regarding concurrency, we're trying to eliminate data races by avoiding shared mutable state. Right now, our unique pointers attack the "shared" part of that -- since data in the exchange heap is localized to a single task, you don't have data races, regardless of mutability. So mutable structures aren't a concern there.
There is the concern that unique pointers are too limited to do a lot of the kind of tasks you want to do in a parallel way, and that's where the experiments we want to do with the "patient parent" model come in -- you can see my colleague Niko's blog here [1] for info on that. That approach allows shared immutable data.
Verification has never been a goal of ours, except for the kinds of invariants you can address relatively simply in a type system. Dependent types are cool, but at the moment they're so complicated as to place the language out of reach of most programmers. We have more interest in making dynamic checks (assertions, design-by-contract) easy to use.
Go achieves inheritance through anonymous fields. Anonymous fields are multiple inheritance in all but name,
Anonymous fields in Go are syntactic sugar for delegation. Because anonymous fields do not create a "subtype" relationship between types, anonymous fields are inheritance.
(0) Objects and inheritance: Rust had support for objects (via prototype-based inheritance) in the first version of the compiler, but we found that they weren't being used. We attempted to be as minimalist and simple as possible regarding objects, and as a result we ended up with a system that didn't have enough features to be useful. It also didn't fit well with the rest of the language, so we scrapped it and added typeclasses instead. Those worked a lot better, and now most of our standard library is using typeclasses for OO-like patterns. Recently, we've found that we really do want objects, but mostly as a way to achieve code reuse, privacy, and a more direct association between a type and its method than typeclasses alone provide. The current system that is being implemented is a nice way, in my opinion, to unify typeclasses and object-oriented programming. There are just a few concepts to learn, and it all meshes together quite nicely.
Go's interface system is quite similar to Rust's typeclass system. The main things that Rust has that Go doesn't are first-class constructors, the trait system (not yet implemented) to facilitate method reuse, and the ability to define multiple implementations of an interface per type. The things that Go has that Rust doesn't are duck typing (which is a quite good idea, and I'd like to add it to Rust as well) and downcasting (which we don't want to support because we have more type-safe mechanisms for the same thing).
(1) The compilation model: Rust uses dynamic linking pervasively, because OS's have good support for it and it helps keeps binaries small. It also has strong support for separate compilation, because we want to make compile times fast. So far, so good, but, just like BitC did, we discovered that type abstraction (which you use generics in Rust to achieve) doesn't mix well with separate compilation. We didn't want to have a uniform value representation like the MLs do (supporting only 31-bit ints and boxing everything else doesn't fly in a systems language), so we tried to use dynamic size calculations for all of the values. It resulted in a huge amount of complexity (we never shook out all the bugs), and it also had a large runtime performance penalty. Unlike C#, we couldn't fall back on a JIT, because Rust is an ahead-of-time-compiled language. So we moved to a "monomorphization" scheme for Rust 0.2, which is basically like C++ template instantiation, only without the overhead of reparsing all the code from scratch. Even with this scheme, you only pay for monomorphization when you use generics, you can still dynamically link all non-generic code (which is most of it), and your runtime performance is unaffected by your use of generics.
Go, of course, doesn't have generics. I don't personally believe that buys them much though; the programmer ends up working around it in a way that either involves boxing (via interface{}) or by-hand monomorphization (by duplicating the code for each type). To me, generics are just a way for the compiler to do work the programmer would end up having to do anyway.
(2) Insufficiency of the type system regarding reference and by-reference types. It's spooky to read this, because it's precisely the problem we ran into with Rust. At the moment we have by-value and by-reference modes for parameters, and we've found that this isn't sufficiently expressive. (We also tried making the only difference between by-value and by-immutable-reference internal to the compiler, which didn't work well due to higher-order functions and interoperability with C code.) We also found that parameters really aren't the only place you want by-reference types; you really want to be able to return references and place them within other data structures. Whenever we wanted to do this, we had to fall back onto heap allocation, and that was significantly hurting our performance, especially when unique types were involved (since aliasing a unique type is impossible, you have to copy it). Profiling Rust programs showed an alarming amount of time spent in malloc and free. So we're in the process of bringing up a new regions system that I'm excited about: it's too early to say for sure, but I think we've stumbled upon a way to make regions not require a doctorate in type theory to understand. Regions allow you to have safe pointers into the stack and into the heap and pass them around as first-class values.
Go doesn't have zero-cost reference types at all; it just does simple escape analysis to allocate structures on the stack when it can and falls back on tracing GC for the rest (note that this is what Java does nowadays too). This is one of the most significant differences between Go and Rust; Go's memory model is essentially identical to that of Java, plus the ability to allocate structures inside other structures, while Rust has a much more C++-like memory model (but safe, unlike C++). This decision is based on our experience with Firefox; fine-grained control over memory use is so important that we didn't want to place our bets on pervasive use of GC.
(3) Inheritance and encapsulation: Rust still has no concept of inheritance; it's our hope that a combination of enums (datatypes like in Haskell or case classes from Scala) and traits will allow us to avoid introducing the complexity of inheritance into the language. Time will tell, of course. As for encapsulation, we thought we didn't need it, but it turns out that we really did want private fields. This we're solving with the class system, mentioned above.
Go achieves inheritance through anonymous fields. Anonymous fields are multiple inheritance in all but name, complete with the "dreaded diamond" problems of C++. We were hoping to avoid that. Go has standard privacy through "unexported fields".
(4) Instance coherence. Since you can define multiple typeclass implementations for a given type, and the caller chooses which implementation to use, you have to make sure in many contexts (for example, the hash method of a hash table) that the same implementation gets used for the same data. That's one of the reasons we introduced classes -- they tie implementations of methods to a type.
Go doesn't have this problem, because it only permits one implementation of an interface for each type, and it has to be defined in the same module as the type. Basically, Go's interfaces are much like our classes in that regard. We wanted to allow people to add extra methods to types -- for example, to add extension methods on vectors (think what Rails does to the Ruby standard library, but in a way that doesn't involve monkey patching) -- so we didn't want to force this restriction on users.
I think that one of the most important things to underscore is that we would have never found these things so early unless we had written the Rust compiler in Rust itself. It forces us to use the language constantly, and we quickly find pain points. I highly encourage all languages to do the same; it's a great way to find and shake out design issues early.