While go has some nice features-- standalone executables and fast execution are two. I don't really see a compelling reason to switch to it from Elixir/Erlang.
Biggest downside seems there's no real easy way to handle errors being returned from function calls.
Easier deployment would be good, but once you solve it for elixir it's not a big issue. On the other hand Erlang is very well tested and established and pretty complete-- though go's libraries and open source support is growing by leaps and bounds.
So really, it's that error issue. Oh, and pipe. I really love elixir's pipe ( |> ) operator.
I've recruited and built a team which was writing code in Elixir, even more obscure than Erlang. It wasn't really a problem finding people who knew it or were strong programmers who were interested in it.
Good languages attract good programmers.
The idea that we need to stick to popular languages is, I think, driven by business guys who want commodity programmers.
If you want 100 engineers added to your team in a year, then sure, use Java. But you will still be less effective, I bet, than 10 engineers doing erlang in that same year.
I would argue that you can pick any new'ish (last 5 years) language that is suitable for development and find the same results. Anyone who knows the language well taught themselves how to use it without the guarantee of being able to use it at a job.
This will be a fairly small subset of developers, the ones that have motivation and time to put into growing themselves. So this probably means that the devs that know new language X are just more interested in software development (compared to devs that just know the language(s) they learned in school, and whatever they were taught on their jobs). So they have a much higher chance of being good developers.
Basho, Klarna, Cloudant (now IBM), WhatsApp, T-Mobile for some SMS stuff, many others.
You also use Erlang without even knowing ;-) Chances are every time your access the Internet via your smart phone, there is Erlang code managing that. RabbitMQ broker is written in Erlang.
I've never really understood the "no easy way to handle errors" complaint. I'm not saying it is easy, nor that it isn't verbose (hell, it is super verbose!) but I'm seriously asking: what would be an easy way to handle errors? I'm not asking about how to do it in Go, just how to do it in any programming language.
To me errors are very special constructs, is not a typical return value, as for example a method returning true or false whether the arguments are valid or not, so I expect errors to be something similar (at least semantically) as exceptions in other languages, and in that case, that is, return values that should be deal with in a certain special manner, so I don't expect to deal with them in an "real easy way"™, but in a very conscious and deliberated way.
The Erlang way would be to code for the happy path, and not try to anticipate most errors. If your process encounters an error, it will crash, the supervisor will restart it, and it will continue processing. Of course, you can choose to handle anticipated errors if you would like to be more user friendly (e.g. provide a user-friendly message if a file is missing), or if you want to log the error or something. But generally it's considered defensive programming and an antipattern to try and handle every error condition in an Erlang program like you have to do with e.g. Java exceptions.
In languages with powerful type systems like Haskell, errors can be handled by using (for example) the Either[1] type. That way, you are forced to pattern match on the type (it will be a Right if successful or a Left if unsuccessful/errored), so it's not possible to "forget" to check for a null reference like you do in Go (i.e. checking err != nil all over the place).
Go's panic() acts similarly to what you're expecting out of Erlang. You still have to call it for other libraries.
> so it's not possible to "forget" to check for a null reference like you do in Go
The Go compiler will complain if you don't use the `err` variable after creating it, helping to make you remember to check it, and letting you branch accordingly. This doesn't help with someone who re-uses the err variable or assigns the error return to `_`, but those are conscious choices to bypass the checks Go adds for you.
> Go's panic() acts similarly to what you're expecting out of Erlang. You still have to call it for other libraries.
Go uses a shared heap. So even though one go-routine panics, you can't safely assume the state of your system is still predictable. If it is not predictable you can't necessarily safely restart that go-routine.
Without Erlang/Elixir I would actually do it with OS processes / containers at a higher level. Erlang's processes are only a few K of memory and are very easy to restart and handle so you get all that built in.
To be fair, if you're modifying the heap from a goroutine, you're "doing it wrong". Go provides channels, which work quite well at reducing the need to mutate the global state. I use such a pattern quite frequently (the http.Server), and have never had a problem with heap corruption due to a handler goroutine panicing.
That's true. To that extent C, C++, Java work took. The Threads + thread safe queues is a common paradigm I've used. The problem comes with using other libraries, sharing code with others in a large code base. But errors and concurrency bugs still happen.
But just like type systems, they are there not just to make code faster but add some guaranteed safety. The guaranteed is important. In case of concurrency, errors and state, the isolated heap also adds that guarantee. That is important.
There is a cost, though. Low enough to bear in most cases, but transferring more state through IPC, the overhead of starting new processes, context switching between processes, duplication of in-memory structures; these can add up to make an equivalent Erlang program much more heavyweight on a system than a Go program using many goroutines.
Erlang processes are nothing at all like Unix processes. They are much, much smaller memory-wise as well as in terms of spawning/killing speed as they do not rely on system calls but rather the Erlang VM (which is highly tuned for these purposes).
Yes, there is a small overhead incurred when copying data between processes because the VM is truly copying the data[1], but the benefit gained is that you get per-process heap isolation (references can't cross process heap boundaries). This is important because in languages like Java or Go, one misbehaving bit of code that is causing a lot of GC to happen can "stop-the-world", and make your entire system pause (and it does happen[2]). This is not acceptable when writing soft real-time code, like say an auction system.
I'm not sure why you think an Erlang program is more heavyweight than Go. Is it due to Go programs being compiled to binary? Erlang can do that as well,[3] although it's not often done as Erlang requires a runtime, and it's often simpler to use a separate runtime. But you should know that Go also requires its own runtime to run Go programs[4], it's just that it is statically linked into your compiled code (which is what makes the resulting binaries so huge). But Erlang could do the same thing, it's just not really an industry practice.
Thank you for the explanation. I think it makes sense somehow, however, and high likely because I've never dealt with errors in that way, the _encounter error, crash, restart, continue_ workflow seems a bit awful to me :)
The Haskell solution seems really nice. I've only used Haskell for pet projects and it's one of my favorites languages (though I don't have any proficiency with it,) so thank you for teaching me something else about it.
> high likely because I've never dealt with errors in that way, the _encounter error, crash, restart, continue_ workflow seems a bit awful to me
It is a different way to see things, but makes sense when you consider that Erlang comes from the world of distributed high-availability telecoms where "one machine" is the same as no machine (because when — not if — the machine crashes, the system is dead).
Applied to the development environment, that means of course erlang developers try to avoid errors, but its designers know it's not possible to have 0 errors so they've built support for independent error recovery into the system: when an error occurs, the whole agent may be corrupted (in an invalid state or whatever) so the erlang way is to throw it out and have an independent "auditor" (the supervisor) decide what should be done.
At scale, that's essentially what Netflix builds (and exercises with their Chaos Monkeys)
Something to think about: this is similar what people are building (outside of Erlang/Elixir) using Docker and CoreOS. Deploy a bunch of instances of your app both to scale and to handle failures. Need to update the OS on a box? Restart it, there's other app instances to cover it. A container crashed? Restart it -- it's not sharing state with other instances so things will be fine.
Another way to handle error propagation transparently is using monads. Even if you haven't used monads in Haskell, you've probably used them in Javascript: promises are monads. Consider how both errors and exceptions propagate through .then chains in promises. There might not be as much syntactic sugar in JS as there is in Haskell, but the idea is pretty much the same.
> _encounter error, crash, restart, continue_ workflow seems a bit awful to me :)
Granted it is better and less awful than error, everything stops, get calls from customer at 4am, fix, continue cycle ... ;-)
It is a bit different. Learn about it some more and you'll love it. The reason you can't easily do it in other languages is that errors are not isolated. That crash that happened, it might have left some global variable some place in a strange un-expected state. That is why Erlang/Elixir processes do not share memory.
That way it lets you feel a bit safer about crash-restart parts of the system.
> "real easy way"™, but in a very conscious and deliberated way.
If your don't deal with errors in a "real easy way"™, your customers will deal with errors in "a harder way"™ and they'll call you at 4am about it too.
You can't avoid dealing with errors basically.
> deal with in a certain special manner,
Ok is this "manner" easy or hard to handle.
You can also look at it this ways, errors will happen. Some will be predictable (as in you expected a "connection refused" exception or error value. Some will be unpredictable ("this library I downloaded return -5 and just silently doesn't send the value" or throws some un-expected exception).
There are 2 good ways to deal with errors:
1) Use a stronger/better type system or other static checkers. Something like Haskell or Rust, hoping the type checker will prevent some class of errors. You hope to avoid errors ahead of time here. This approach is taken for some critical systems. Navigation, medical equipment, military hardware etc.
2) Isolate errors when they happen. Errors will happen at runtime, even in a typed-checked language. How do you want to deal with them? You can deal with them in an easy way -- isolate the component that fails and degrade the system slightly without completely stopping service. Or even better restart just that one sub-components. Or, have everything come crashing down with an exception or tracelog and get a call from a customer. Erlang and Elixir (and other BEAM VM) languages do this best. You can to a certain way replicate this with OS processes, but Erlang has that built-in. The reason for that is concurrency primitives (processes in Erlang, others have tasks/goroutines/threads) do not share a global heap. So that lets you both isolate the errors they happen to one component, and lets you separate main code from error handling code (you can have supervisors that watch other processes and if they crash they can deal with it better).
It's the verbosity that I'm talking about. In go I end up with %70 of my code being error handling or error related... with Elixir it's close to %0.01. I'm giving estimates here, of course, but they aren't off by much. With elixir, using try catch is rare, because it should never crash, and when it does, you lose a process and debug it.
With go, every line that calls a function needs to check to see if there was an error. And handling that error takes at least another line or two.
Ok, verbosity I agree. That doesn't have anything to do with being easy or not, I think. I might be wrong. Verbosity can be a pain. Even after reading Rob Pike's article on "Errors are values" [1], I agree with you that the solution is still verbose.
Those numbers you gave, even when you said they're estimates, still seems like made up numbers. I've never used Elixir, so if you could point me to an example so I can be more informed, I'll be grateful :) but also I've read some Go programs (not many, I'm not an expert) and it didn't seem to me that 70% of the code is dedicated to error handling.
Note that I'm not defending Go's position on error handling, I'm asking about error handling in general, what does people mean with "easy way to do it".
The Erlang/Elixir way can be hard to appreciate because it's not just a very different way of handling errors, it's a completely different way to architect your application. The following tutorial only takes about 45 min to go through, but you'll get a taste for how error handling (across distributed process, at that) works in Elixir: https://howistart.org/posts/elixir/1
Special how? Surely they are not special in a way that can't be captured by a type system. You say they should be handled in a special/deliberate way, but that's what it means for something to have a distinct type: it can't be handled like everything else. You just define special error-handling functions and they will work on errors and nothing else, and nothing else will work on errors.
As far as syntax goes, the I feel the big issue is about deferring error handling. You don't always want to mix error handling code with your algorithm logic, so you need to defer. We also talk about 'errors' so generally that we never bother to qualify which kinds of errors should be deferred or not in a way that is encapsulated in the type system. Or what kind of deferring should be available -- return errors or just move them later in the same scope? (Java's checked exceptions are a good example of attempting to do this, but it doesn't work out in practice. Programmers don't think like programs do: deferred error handling isn't the same as deferred error-handler writing, and both are needed in different ways.)
From what I can tell about error handling in Go, it isn't really a step forward. (And we really want a step forward.) Go just kind of falls back on "this was the last thing that worked, and making significant progress is probably too hard, so we won't bother trying."
That's the impression I get, at least. I don't actually do any Go programming, but I'm not inspired by its approach, either.
I think it's funny, but not surprising, that you assume it will be easier to grow the team with Go. It's not a more popular or established language, yet.
But also, I've recently built a team using Elixir before Elixir was even 1.0. About the hardest possible example of that argument and we found great people, grew a great team in Austin Texas.
Having a really good language as part of your stack works as a filter-- you get the best programmers that way. Not saying go is bad, but it's popular, and so you're going to get a lot of people who chose it because it was popular.
As for re-inventing the wheel, erlang has a lot of 3rd party libraries, and is ahead of go in that regard, for the time being.
Most interesting to me is your perception of the situation, which I'm guessing is driven by the fact that go is very popularly being evangelized right now.
>It's not a more popular or established language, yet.
Github would disagree as one data point.
I don't have my own numbers to back this up but I firmly believe based on my communication that there are more people in the world that have written Go right now than Erlang.
>Having a really good language as part of your stack works as a filter--
I think this is good for Go and Erlang... So I don't think it helps distinguish.
Not to mention anyone that smart knows C/C++/C#/Java can very quickly become a gopher if needed; quicker I would contend than you will ramp them up on Erlang.
>Most interesting to me is your perception of the situation
I've been writing Go full-time since before Go 1.0 RC-- I think you don't have an accurate perception of where Go is at.
> Having a really good language as part of your stack works as a filter-- you get the best programmers that way.
Agreed. At one point I felt this way about Scala, but it's starting to change as the boulder rolls downhill.
I still use Scala, because it's fantastic, but it's no longer a safe assumption that a Scala programmer can be trusted with sharp objects from the jump.
Nope, that's what I'm talking about. That article shows how to optimize this constant checking of errors.
What I'm trying to say is that in elixir you handle this at the level of the process. In go the whole thing is one process so you can't do that. In elixir you let it crash, and if the process crashes you report the error and conditions of the crash in the log.
In a real system you would only have a couple things where you do a try/catch or "is error not null" type checks-- for the whole program.
Can you be more specific in how handling errors is difficult? I've found it to be generally forced and explicit, which, while sometimes unpleasant, is quite comforting.
The alternative seems to be exceptions flying unhinged.
Let it fail, let it fail, let it fail! (to the tune of "let it snow, let it snow, let it snow!) Supervisor Trees!
Exceptions are not much better, but checking the return every time is no fun. And if you crash the whole thing crashes.
In erlang you can have a once in 1x10^9 error and not even catch it, but only that process crashes (and your system keeps running) and that process gets relaunched. With go, it's the whole program that will crash, necessitating checking every return value.
I say this, though I might be missing some error handling system in go that I'm simply not aware of.
You can set a `defer recover()` which will stop the panic from crashing the entire program, letting you recover & take whatever action you deem appropriate.
> The alternative seems to be exceptions flying unhinged.
…
The commenter you replied to explicitly mentioned Erlang and Elixir, you may want to have an idea about where they're coming from before replying to their comment.
Biggest downside seems there's no real easy way to handle errors being returned from function calls.
Easier deployment would be good, but once you solve it for elixir it's not a big issue. On the other hand Erlang is very well tested and established and pretty complete-- though go's libraries and open source support is growing by leaps and bounds.
So really, it's that error issue. Oh, and pipe. I really love elixir's pipe ( |> ) operator.