Despite the nice packaging, it appears to be a single-person maintained project. There are quite a few reasons why no opinionated PaaS built on Docker Swarm or Kubernetes (maybe surprisingly) came even close to being a popular choice in the market. Usually, anyone serious who's going to bet on a PaaS that runs their stack will also need a serious corporate entity (i.e. software vendor) supporting the tools in case things go wrong. Similarly, often many serious companies are better own building their own PaaS on top of these platforms as they realize sooner or later, they want a lot of customization and control over the platform. So far I'm adding this to the pile of people who’ve tried to build a PaaS on top of docker/k8s. I'm not seeing a lot of features to make it stand out, what I'm seeing in the features section is more of an opinionated stack.
I remember first time seeing Elastic Search around 2012. Shay Banon was the sole commiter for over two years at the time. Never underestimate determined founders.
While you're right that I am the main maintainer of this project, you might underestimate its traction.
apollo is being developed and professionally supported by my company and runs in production for many of our customers (ranging from single-node spaces to 5+ node Webhosting clusters, mostly Docker Swarm, a few on k3s).
While I put a lot of my personal wishes for "good IT systems" into apollo, it has originally been built as an open platform (open as in "no vendor lock-in, no custom tooling we can't leave behind later, industry-standard, best-practice, easy to grasp, don't stand in the way, integrate with everything") for a customer that decided to let my company open-source the product afterwards and build a team around ongoing development. Which we have done and it's a self-sustaining business since day 1.
What I try to say here is: we are a serious company, building this for other serious people with a similar understanding of "open" to use. It's also explicitly built to be customized to anyone's needs. We're betting on the software. Our clients can bet on us.
I understand that it might not be a popular choice on the market, it might not even become one. But it's free for everyone to build something with it, no questions asked. I guess that's at least something!
Your competing product's org has a bad track record here of canceling new tools, so from that perspective, the small, neutral, and fully open solution seems safer choice given the two. I'd reconsider barking up this tree.
More objectively: It's new, production architects of "serious" code shouldn't depend on either. However, one of them represents a lot of "boring" stacks, so if a team is comfortable forking vs NIH'ing, this may be closer to "serious" production-worthy.
The idea is pretty cool IMO - we were looking at Replicated years ago, and if they had gone open like this, we would have considered them more seriously.
The fact that it works with an existing docker-compose.yaml and will provision itself on a cloud for you is pretty compelling I think.
I have tried the Docker AWS ECS plugin for deploying your compose stack, and it's buggy.
The AWS ECS CLI would deploy docker-compose files in V1. V2 (now called Copilot) has shifted away from this, and it uses a custom manifest YAML file. Copilot isn't bad.
Most of the platforms I work on have a docker-compose stack with Postgres, a backend API, a frontend, and potentially ancillary services. Anything that integrates well with this is a godsend tbh, because otherwise you need to maintain two sets of configuration and keep them in sync.
It definitely is a single person maintained project (he's responsible for 99% of the code churn)...not that there is anything wrong with that. Here's a breakdown of his contributions:
In my comment I was referring to open source PaaS projects building on Kubernetes/Docker. Those are managed services where you aren’t exposed to the underlying infra (to varying degrees). So they aren’t apples to apples with this project.
This looks incredible! Has some serious offerings against the current Self-Hosted PaaS scene, hope I can check it out over the weekend.
I have used Dokku[0] in production, and played with CapRover[1] on hobby projects. Flynn was like a better Dokku, before development died.[2]
CapRover is a great experience, and it's not a toy IMO. It nails the role of "I want to press a button and have a self-hosted Heroku, with a nice dashboard, auto-provisioned HTTPS domains, and be able to deploy a bunch of Docker images or Buildpack apps." A single-node $10 box can run a good number of services.
This looks well-positioned and closer to something like Nomad, but going through the readme I had a few questions:
"Apollo requires a manager- or control-node. We call this manager-0. This node runs the entire controlplane and monitoring stack for a cluster and should be sized appropriately (8GB Memory, 2-4 vCPUs)."
Is there a way to run this single-node and disable some of the peripherals for cheap/play $5-10 instances (disregarding best-practices)? An 8GB mem + 4vCPU DO droplet is $40/mo. Not an arm and a leg, but if you have to add at least a secondary server, seems like minimum would be ~$50/mo?
"Your space has been created. Now let's cd to its directory: cd $HOME/.apollo/.spaces/demo-1.space"
Are the Apollo Spaces not meant to be checked in to version control and used for infra + shared between people on your team?
And final question (sorry if this is dumb):
Automated infrastructure (currently only DigitalOcean and HETZNER Cloud supported)
Does this mean I can still deploy to AWS/GCP, I just need to do it manually, or are these infrastructure templates required?
author of apollo here. Thanks for the feedback and questions, I'll try my best to shed a little light here.
- Yes, you can run apollo even on nodes with only 512MB RAM and 1 core. There's no programmatically imposed limit - it's just an advice to bring beefy infrastructure to production setups as metric-collection/logging consumes quite a lot of memory. You can just use apollo with a 2GB HETZNER node (2,98€/m) and you're good to go. Additionally, you can switch of metrics, logs and analytics in your Spacefile.yml - these are the main memory-hogs. Without them, you still have a nice docker swarm or k3s.
- apollo spaces are totally meant to be checked into version control for collaboration and CI/CD. Once created, just enter you space directory and hook it up to any git-repo like you would to with any other code (`git init` etc). We're planning to implement this workflow into the CLI for 2.1.0 to improve UX, but basically apollo config lives in a directory which you can do with as you please in terms of git.
- The infrastructure templates aren't required. As my company is not using other providers than hcloud/digitalocean, we have no ready-made templates for them. But you can provide your own Terraform code to spin up infrastructure. You only need to work in the `output.tf` code that saves a special formatted Nodesfile.yml as an artifact of your TF code. Additionally, you can of course write up your Nodesfile.yml manually.
Thank you, that does answer everything! I also see now after going through the readme again "Single-node clusters are possible and can be scaled up", so apologies for that question.
Dokku user and now CapRover user. I have to say it's my favorite for spinning up a new project. I don't know if I'd use CapRover for anything production just yet but I love it for development. For me it's the perfect "stick this server, client, DB combo on a server, add HTTPS, and don't make me think about it".
I feel confident that anything I throw on CapRover I could move to "more production-level" setup easily if I wanted; using managed db's, containers, block storage etc on AWS/Google/etc. Like, if I can get it working in CapRover I know I can run it anywhere and CapRover makes it super easy to get my idea hosted and running "in the cloud" quickly. I don't know why but seeing my project run full-time, verses just when I'm developing locally, is a huge motivator for me so CapRover is perfect for me in the same way I use Caddy for my local services. I want fast, safe, and easy with the knowledge I can scale up with some work in the future but I don't need it for day 1.
Author of apollo here. Using multiple Cloudron instances for production stuff and am still heavily inspired by the operational stability and ease of app-onboarding the Cloudron team has managed to maintain. apollo strives to deliver a Cloudron-like experience (in terms of stability) over time.
I was kind of forced into using Dokku in production. I inherited a 4-year old Rails codebase previously running on AWS EC2 and that was how the infra was set up.
But given the timeline, I think it was probably the best available option at the time. Flynn, CapRover, etc weren't really a thing, so Dokku was innovative in this space.
I would say that there's absolutely nothing wrong with Dokku, but experience-wise CapRover is miles ahead.
> Is there any reason why you wouldn't use Caprover in production? (other than you haven't needed to)
If you use the Docker Swarm setup multi-node, I feel like it's a solid choice for a small company. If you get big/complex enough, you'll probably eventually wind up needing k8s/Nomad but as far as tradeoffs for development velocity vs stability, I feel good with CapRover.
Going to try Apollo out soon though. It appears to address things like multiple environments better than CapRover. And as mentioned, the docker-compose.yaml integration is a huge one for me.
And let’s not forget the well-liked iOS Reddit client as well.
Or the Greek god
Or the moon missions.
Or Counter Culture’s Light Roast beans.
Basically the name is a bit tired and few actually have a reason to call it that. (Apollo the Reddit client to browse the world of Reddit which has an alien as it’s avatar almost makes sense.)
Been using Dokku for a year now, and it is quite a joy.
Admittedly, I'm not in the 1000s of requests per second space that many other people operate in (I'm in the internet equivalent of the Mom & Pop website hoster space).
I've found Dokku does everything Heroku does, but I get to control it all, which is really nice.
Never tried Caprover or the other container solutions, because I'm not big enough to even exceed single-instance sizes at Hetzner.
My main worries nowadays are:
- What if popularity shot up? How far does my solution scale?
- Hackerzzz. Who is out there just trying to damage something. Not sure it's as bad as the media will have you believe it is, but I still want some relatively strong opsec.
- Disaster recovery. I want to control as much of my stack as possible, so I can fix things when they go wrong, but I just can't seem to shake that eternal worry of 'what have I missed?' in the production DB
I find it quite hard to strike a balance between over engineering things (boy do I wish I had a load balancer that automatically detected an issue with a node somewhere and then did some sort of seamless failover without losing whatever transaction was in progress at the time lol) and just trusting that the thing will work when it has to.
The promise of platforms like Heroku is undeniably less hassle, or a worry-free life, but I don't know whether black-box models can ever truly provide that.
I've been using Dokku for years, and I have only positive words for it. Here is my experience regarding your questions:
- Scaling: Dokku can scale your application (via "dokku ps:scale" command) on the same machine. If you are concerned with spikes, put Cloudflare (free tier) in front, it will solve most of the problems.
- Hacking: Dokku will expose Nginx and proxy requests to your application from the public. Assuming the rest of the system is behind firewall (eg. iptables with exposed only ssh/https/http ports) and the system is regularly updated, maybe the only critical part is how your application is secure (eg. XSS or SQL injection attacks). If you are using a sound framework and don't abuse it too much, things should be sufficient. Again, if you are behind Cloudflare, it can help a bit here.
- Recovery: All your code is on git and you can easily replicate it on the new Dokku instance. If you happen to use a database as storage, frequent database dumps are advised (eg. daily cron job that will send dumps to Hetzner backup storage).
- Load balancing: Because Dokku is using Nginx in front of your application, starting two (or more) application instances ("ps:scale") will be good enough. Notice however that this will be done on one machine only. If you want to spread it over 2 machines for example, install Dokku on both, git deploy on both (make or shell script will help) and put Cloudflare load balancer in front of it ($5/mo for two origins). Or you can use DNS instead; look for DNS load balancing.
Load balancing: This is a new one to me. Excellent! I had wondered about how to do some kind of two-machine load-balancing and eventually, failover. Still finding it tricky to wrap my head around how I'll keep my DB consistent between the two systems, etc, but so far I'm nowhere near the size where this is needed from a performance perspective. This is about redundancy only, so far.
Unfortunately I've never used Dokku or Heroku myself, but from what I know of them, apollo is different in that it doesn't provide any platform services developers can consume (like DBs, Caches, S3, ...). apollo is explicitly made for production use which is why the focus is on operations and infrastructure and we leave the tooling in the hands of the developer (hence we support native docker-compose/docker stack deployments without custom logic).
It's a very DevOps-y platform that can grow with your needs and supplies a good toolchain that helps with operations. Focus is clearly not on "abstracting as much as possible away from the end-user" but more on "trusting that the thing will work when it has to".
Besides, apollo is a great platform to build upon that you can simply leave without reinventing the wheel. Since you don't need special manifests to deploy but build with standard "docker" components like stacks and services, you can deploy your stuff elsewhere in a minute and don't risk vendor-lock in.
For the time being I am probably not doing anything complex enough to warrant switching away from Dokku.
As a single-person operation, I have to keep a pretty fragile balance between effort-of-setup (I can deal with a bit more effort than Heroku) / control-over-setup (I need Dokku's finer-grained control, with more effort).
One day I may outgrow those requirements, however. Who knows. I will keep Apollo in mind for that :)
Looks like the main difference is docker compose compatibility but admittedly I have more reading to do. Still tho that is a good difference! I have multiple caprover nodes in prod for hobby stuff so absolutely not knocking it.
I think conceptually apollo and cap rover are pretty similar in that both try to deliver some degree of abstraction to the developer when it comes to application deployments.
Where I see a great difference is that apollo doesn't imply a special deployment or release workflow. You just use plain docker tooling (or helm for k8s). We also don't provide DBs or other platform-services. With apollo, you're required to take care of your stack's needs (if you need a database, create a service in compose and add a volume).
apollo's focus is on stable day-2-operations and giving great insights to what's under the hood (like direct access to metrics and central logs). We also don't invest time in writing custom code for web-interfaces and alike. apollo includes portainer (or rancher on k3s) which you can use to manage your containers; and grafana for analytics.
Another thing we focus on is to simply support industry-standard tooling and workflows (apollo can has a CI/CD workflow based on SemVer you can use to continuously validate and deploy your clusters) - we want to avoid vendor-lock-in as much as possible.
apollo is meant to be used in production and can replace the need for a full-blown DevOps team if used correctly. Not sure if Caprover does that.
I know the docs make it look like it does, but it doesn't really.
What it actually supports is copy + pasting sections of Docker Compose for prebuilt images.
So let's say you have a docker-compose.yaml where one of your services uses "build"/"context" and points to a local Dockerfile. You cannot deploy this to CapRover (in this way, through docker-compose.yaml).
The docker-compose support is more geared towards the "one-click" apps they have when you want to deploy something like pgAdmin or MySQL which uses a prebuilt image.
Because of this, it means you can't really deploy your existing setup. From the instructions:
1. Navigate to Apps
2. Click on "One Click Apps/Databases"
3. Navigate to the very bottom of the list, and click on the last item, called >> TEMPLATE <<
^^ Because this is happening on your serverside instance, in a web app, it has no context for local files.
The "Alternative Approach" is to connect to your instance and upload the source + manually run "docker-compose up" with it wired to the overlay network. This is not a good or scalable solution IMO.
I've been wanting to build something like this for a long time, very neat. I haven't had a chance to use it yet but looking through the readme this is a very comprehensive project.
For Hobby projects I'm loving Lambda proxy from AWS Labs. I'm able to use native Golang frameworks like gin/echo without changing anything much and deploying to Lambda + API gateway. This runs for basically free, and is easy to port out.
Is this the first self-hosted PaaS solution also running on Kubernetes? As far as I know neither Dokku nor CapRover do, which always deterred me from these applicationsas I'm running K3s at home.
I assumed that Knative [1] is the canonical self-hosted PaaS for Kubernetes. It is branded as Cloud Run [2] by Google. I'm not sure how it compares feature-by-feature with Dokku nor a 12-Factor App PaaS like Heroku.
If Knative is more self-hosted FaaS than PaaS, that explains the core use case for Apollo; a traditional 12-Factor App PaaS for Kubernetes/Docker-Swarm. Knative supports a Serving Spec for routing (like AWS API Gateway) but doesn't seem to have buildpacks for 12-Factor App frameworks, as far as I can see.
Apex Up [1] deploys web apps on top of AWS Lambda using Amazon's API Gateway as the RESTful routing proxy. Maybe something like Apex Up is required to transform Knative into a PaaS for historical 12-Factor App frameworks. ZEIT Now 2.0 did the same thing for Serverless (Cloudflare Workers only?) but the company has been renamed Vercel and has pivoted around Next.js.
CapRover uses Docker Swarm internally, even if you're only running one node, which really sucks because it invokes the docker userspace proxy, killing the L3 source IP for connections in to your containers.
