I agree that using selective receive helps in dealing with messages that arrive out of the order of a specific state transition. Akka gives users the ability to stash messages if they want to. On the JVM, a long-running actor-based application (which is one of the reasons for using actors in the first place) can struggle with it. It's one of the reasons the original Scala Actor library is no longer in use, though there are other important reasons - such as Akka's use of ActorRef, analogous to Erlang's PIDs, which mask the instance of an actor from those who wish to communicate with it, as well as it's physical location. As you scale actors across a cluster of machines, that becomes really useful.
That's great about assigning the fiber to a FJP. A dangerous task would be anything that could take down an actor, which can be worrisome depending on what state the actor is holding. There are varying kinds of such state, including that which can easily be retrieved again from an external source, that which is "scratch" data and inconsequential if lost, and that which cannot be recovered if lost. In actor-based applications, we want to encapsulate mutable state and use message-handling single-threaded interaction to prevent concurrency issues, right? If we're going to do something that could cause the actor to fail and risk losing data, we want to export that work along with the data needed to perform the task to another actor and let IT fail rather than risk the important one. There are ways to pass such data between incarnations of an actor on the JVM by carrying it with the Exception, but it's not free and you have to know when to use it.
So a dangerous task could be asking for data across an unreliable network or non-replicated source, it could be dividing by 0, anything that could cause typing errors (even in Erlang), you name it.
But how would a dangerous task affect the entire pool?
Also, I don't know if there should even be "important actors". Like in Erlang, we want to let it fail. Important data should be kept in a shared data structure that supports good concurrency, not in actor state. Like I said in the post, I don't think every aspect of the application should be modeled with actors.
With a thread pool shared by actors, even yours, if one of the actors fails, that thread is gone until the pool creates a new one (as needed). That's one less available thread until the recreation occurs. To minimize the impact on other actors, you put known dangerous tasks on their own small pool so that their probable thread death has no impact on others.
What you're not seeing is the relevance of a supervisor hierarchy and OTP. Yes, you want to let it crash. But you want isolation of failure as well, and only with failures of increasing criticality do you want it to be escalated through the hierarchy. There is a difference between a database call failing because a SQL command failed and all database interactions failing due to network partition. OTP via Erlang and Akka allow you to model that in your actor tree.
Important data kept in a shared data structure? Globally visible? Managed with what, STM? That won't fly at scale - STM is great only when you're dealing with small datasets that don't change very often. Immutable, persistent data structures? Also not good at scale due to fixed size allocations constantly happening as structural sharing is enforced. Allocations are cheap and hopefully the objects are short-lived and GC-friendly, but it's still far from a free ride.
The whole point of actors is a concurrency abstraction. They are meant for isolating mutable state from being affected by multiple threads at the same time. OTP and supervision is just a nice way of organizing them and learning when failure occurs on another thread.
Should an entire app be modeled with actors? Probably not. A subsystem certainly can be. Depends on what you're doing, of course.
That's great about assigning the fiber to a FJP. A dangerous task would be anything that could take down an actor, which can be worrisome depending on what state the actor is holding. There are varying kinds of such state, including that which can easily be retrieved again from an external source, that which is "scratch" data and inconsequential if lost, and that which cannot be recovered if lost. In actor-based applications, we want to encapsulate mutable state and use message-handling single-threaded interaction to prevent concurrency issues, right? If we're going to do something that could cause the actor to fail and risk losing data, we want to export that work along with the data needed to perform the task to another actor and let IT fail rather than risk the important one. There are ways to pass such data between incarnations of an actor on the JVM by carrying it with the Exception, but it's not free and you have to know when to use it.
So a dangerous task could be asking for data across an unreliable network or non-replicated source, it could be dividing by 0, anything that could cause typing errors (even in Erlang), you name it.