You don't start by choosing an architecture. You start by understanding the problem and the pieces in the solution space. You then figure out how those pieces can work together to solve the problem. After that you can see the system as nested subsystems and interactions, which you can draw as diagrams and call it an architecture.
So I see "architecture" more as a teaching and communication technique, applied after the fact. Altho since we're always moving between the details and the big picture, having organized abstractions (ie an architecture) in mind will help keep the design clean.
The architecture evolves and sharpens as a system comes together. And this evolving architecture provides the team with a common terminology and overall direction.
But you rarely start by choosing a specific architecture. Unless you already know a lot about your solution.
It really depends. Some patterns might be specific to problem domains, but a layered architecture applies pretty much across all types of software projects.
Between defaulting to a layered architecture and just mindlessly piling up ad hoc decisions without any coherent criteria, a layered architecture always wins.
Honestly - a layered approach ends up paying dividends even if used solely for code organization. Options like microservices and event frameworks work quite well when using layered approaches internally.
Basically, I've seen SQL in controllers and I don't ever want that again.
depends, some systems are such that all they are doing is pumping SQL results into JSON, in that case, I don't want to see some general purpose programming language in the controller. Which is kind of the idea behind https://postgrest.org/ ( written in haskell if anyones interested )
If all you're doing is pumping SQL into json, you should either be generating code, or if you don't care about performance using a dynamic language with meta-programming techniques such that you write almost 0 code to achieve your results.
> But you rarely start by choosing a specific architecture. Unless you already know a lot about your solution.
That's how it probably should be, but in my experience it rarely is.
Generally, businesses decide that they need to re-/implement x, then the enterprise architect shows up and decides on the pattern and then the developers are required to somehow make it work, even if it objectively doesn't.
These patterns are so vague and high-level that you could just throw a dart in one of them and it'd work OK for whatever some company is doing. Almost no one is solving issues that can only be described in one pattern.
If anything people are too caught up in the idea that there decisions are make or break for a business.
I guess theoretically if the enterprise architect knew about the project and its requirements, the pattern should probably have matched the intended outcome...
Don't pick the right tool for the job, pick the right framework which secures you extra work for being a tool. - @iamdevloper ... via https://github.com/globalcitizen/taoup
It's been that way forever. I live in a third world country, for me $40 is a week's expense (total, including food, bills, and so on). If it wasn't for some, ahem, less reputable means of obtaining learning material, I would be carrying boxes in some warehouse right now.
* How to disconnect the domain model from persistence.
Some solutions add the OR Annotations to the domain model. Some other map the domain model to persistence with a Mapper-layer.
But all of them have some constraints. For e.g. on a web api I receive a dto, map it to a domain model, map it to a persistence model and vice versa. Not much gained imho. In this example I would add the OR-annotations to the domain model and configure many-to-many by "text" so that they are not contained in the domain model.
Depends what "disconnected" means. Before Object Relational Mappers came along, you wrote a load of SQL in Stored Procs or in some other framework. This was super tight coupling because updating the database version could ruin everything. Not everything was in compiled code, so it was hard to track.
If you use an ORM (such as Entity Framework I'm about to mention but there are lots) for the write/mutation part then the ORM provides you with that disconnect. You might want to use a different mapper for read, that allows you to write efficient SQL that meets particular needs.
For example, a few years back in .NET Framework I swapped a system that designed/built on Sql Server to run on Postgres. Took me a few days. I didn't change any domain logic during that time. I'd say that the ORM provided a disconnect between the actual persistence and my domain.
I do like a disconnect between my API and the database so that I am able to update the database without causing issues with consumers of my API. That usually requires some kind of mapping but only once. My preferred model right now is a CQRS (don't need ES) using Paramore's Brighter command pattern.
That's for an RDBMS, I think with schemaless/DocDB you can go even further but then the code that loads out the data needs to deal with missing values. That's a different sort of constraint.
I usually create "Service" classes in the domain layer, which use Repositories. These repositories are a wrapper for the DbContext to allow unit testing. Where do you put your domain logic? Do you have a domain model or is your persistence model your domain model?
Domain model is the peristence model. Splitting them apart is often known as an anemic domain model (in Domain Driven Design).
All the domain logic is on the entity. So our command handlers (akin to your service methods) look like:
1. Load entity
2. Call method on entity, passing in data
3. Save entity
4. Raise events
So we only unit test 2. because that's the domain code. Loading entities from the dbContext is someone else's code. So is saving. So is raising events.
If your entity class gets big, split out the functionality into other classes so that you end up with a fair portion of bodyless methods.
The more complex your system gets the more you will appreciate this division. However at the outset it just makes things more complex than the situation warrants and it's not clear if the system will ever grow large and complex enough to pay for this expense.
I recommend creating your domain model as a simple wrapper around the persistence model. This way if the whole system never gets complex you will retain 95% of simplicity and if it does get complex you're one refactoring away from making your "shallow" domain model into a full-blown domain model.
Likewise the outbound part of your view/DTO model can be a simple wrapper around the domain model. Inbound models will probably have to be their own classes with extra mapping. The latter might be a bit of a pain (in ASP.NET MVC, for example), however I suspect the pain is instructive - making your inbound DTO a copy of the entire object could be simply the wrong path altogether.
So you create a class in the Domain which is responsible to save the domain object? And inside of this class you map the domain object to the persistency model? Is that not a forbidden by the layer principle because now the domain model layer has a dependency to the persistency layer.
> So you create a class in the Domain which is responsible to save the domain object?
Yes.
> Is that not a forbidden by the layer principle because now the domain model layer has a dependency to the persistency layer.
In my book it's fine. I find that in practice one-way dependencies are ok, so domain->persistence dependency is fine so long as there is no persistence->domain dependency. Simply avoiding loops will take you a very long way.
I see know that one can think of the domain model as most changed layer and if there is a bigger change it is because the domain model changes (e.g. business requirements). So a dependency from the domain model to other parts below are mostly fine.
You will have a bad time with real software engineering if you take those rules as mandatory. Software engineering rules are like algebraic math postulates, they create common ground that let you explore and communicate some things. Not like legal rules that disallow you from doing something.
Anyway, "layers" imply on single-way dependency. If you have completely independent code, two-way or multi-way dependency, it's something else.
Decoupling the physical storage of the model from its logical representation is a very important capability if you want any sort of agility down the road. The way we do this is fairly simple:
- Maintain a common domain model stored in a shared dll. This is a POCO type without any methods or mappers of its own. All it does is contain all the facts you might need. To encapsulate everything into a serializable structure, we place all top-level domain types as collections into an aggregate type simply called "Domain".
- Maintain mappers to/from the domain model in the locations most appropriate for these to live. These mappers could talk to SQL, memory, some noSQL garbo, the user's web UI, etc.
The advantages with this layer of separation almost universally outweigh the downsides unless you can convince your development team to build a more standardized single-process/monorepo product. In a simpler product, I would be more inclined to just directly interact with SQLite via anonymous types.
We have used ORMs like EF in the past, but these get in the way really badly and prevent you from building your logical model in precisely the way you want to.
MS Office back in the day was well known for being horrible to interoperate with. Their file format was literally a memory dump. So disconnecting enough that the data on disk is at least serialized in to something that can be understood by others is good.
Being disconnected enough so that V2, can load V1's persistence without too much pain is good. Again looking at you office.
Persistence and domain models are fraternal twins, not identical twins. In general recognize that domain model, and persistence model are different things that have slightly different needs. For example the domain model of a person might only care about age, the persistence model is only going to care about DOB. So yeah 95% of the time they are the same and nothing is gained from separating them but having them be too close is often the source of problems.
I'm building an application in this fashion at the moment in Go, and honestly, I think it's kinda painful. At some point I already merged the persistence with the domain model, so it's just the domain model with SQL related struct tags. I am tempted to merge it with the HTTP model as well, add JSON tags to it.
But everyone is telling me it's a bad idea, so I'm really not sure. A more convenient way to copy properties over would be nice.
> I am tempted to merge it with the HTTP model as well, add JSON tags to it.
If you add a property which shouldn't be public you have to separate them. For example in the domain model you have a user with email but do not want to expose the email.
Second argument: I usually create a simple struct for each method. For example I do not want, that they provide the id for post requests. I want to send the id on get requests so that they can send an update request with the id. I prefer only "non-null" arguments and as a bonus you get a clean (auto-generated) api documentation.
> in the domain model you have a user with email but do not want to expose the email.
You can usually configure that out with @JsonIgnore or something like that. Although this works only for this particular (very simple) example and won't help with e.g. flattening multiple nested objects.
At this point you already have 2 "models", in the sense of 2 "concepts" or 2 "types": the one with the field and the one without. The second one is implicitly defined with these annotations, which has a good chance to be annoying sooner or later when you need it to be explicit.
>If you add a property which shouldn't be public you have to separate them. For example in the domain model you have a user with email but do not want to expose the email.
Another possible problem: having to support two versions of the API at the same time.
All you need is a little conversion function which takes a domain struct and converts it to a DB struct with tags (for example)? Unless it's a very complex entity multiple levels deep, I usually don't find it all that painful. What is it about DTO conversions that makes it painful for you? Being boilerplate, it's probably not very aesthetically looking, but it usually doesn't take more than a minute to write the conversion function.
- DB objects: A poor flat view of those DTOs, for example complex subobjects can be jsonized, or mapped as a foreign key.
Basically it is the role of the DAO to transform the DTOs into DBOs and store them immediately, or read the tables and transform them into structured DTOs. Why? Because the DB representation is always ugly.
So our application works mostly only with the DTOs. The trick is to make the DTOs rich: Adding methods onto them to ease manipulation, and add validation so that a DTO can only exist in a valid state (as much as possible, modulo when they come from the front-end). So it’s not “setDocument(abc)” then “setState(HAS_DOCUMENT)” then “setLastUpdated(now())”. It’s “.addDocument(doc)” and it changes the state accordingly, so the state is always coherent.
So, our DTOs only have Json annotations on them.
And merging DTOs and DB objects? Never succeeded even on a simple app. I wish we could write into tables without copying data into the DB objects.
I don't think the term "Microkernel Architecture" should be used in this context. I think "Modular Architecture," (or Plug-in like is mentioned) gets closer to this extension-based pattern.
The reason being that there's no relevance to the kernel, and modular kernels, also take this approach with replaceable plug-ins or extensions.
I think people gravitate to it because "kernel" feels like a cool word and some people have heard of OS microkernels being modular. As you say, "modular architecture" is a much clearer way to express the intent and conveys the purpose without being pretentious.
I don't think "Microkernel Architecture" is the same as "Modular Architecture". I've only heard the term "Microkernel Architecture" used for systems that have clear public extensions points that enables users to choose which plugins to run or even add third-party plugins.
"Modular Architecture" is more broad in my opinion and rather a description of internal structure. A "modular monolith" for example is modular but doesn't necessarily have a "core" nor is it required to be extensible with plugins by users.
Right, although it seems to be a common misconception that such architectures are mutually exclusive alternatives. And terms are relative: Isn't a single microservice a small monolith on its own? Isn't a bunch of monoliths in an enterprise IT landscape that communicate via network so much different from a microservice architecture.
That's what I was thinking as I looked at the list. It a nice little overview, perhaps for starting a discussion, but it doesn't capture how you'd use them in practice. When presented separately they appear like they should be applied in a standalone fashion.
I agree DDD doesn't belong in this list, but this seems a little harsh. Form follows function. DDD is more about understanding the function, and then architecture patterns are the form
DDD works well enough for translating a business domain to computer code, and if your main obstacle is byzantine business requirements it's great, but I wouldn't want to develop something like an MPEG encoder or an operating system based on that principle. They have quite different types of difficulties that don't benefit from DDD.
There are also many variations in between, something like SCS [0], for example.
Nowadays, people started doing such interesting (although sometimes over-engineered) things with architecture, I couldn't even tell what would be the closest from this list.
Good introductory post. For those relatively new to Soft Architecture I always recommend Software Architecture in Practice, great to read and filled with good tactics focused on Quality Attributes.
This part was confusing and possibly useless. The source oreilly article (https://www.oreilly.com/content/software-architecture-patter...) cites a web browser-based auction site. That doesn't sound very "data decoupled" to me when each browser needs up-to-the-second updates on bids.
There's a MMO which uses the approach described in the article. Almost everything happens and is stored in memory, in various independent processing units. Data is dumped to disk only occasionally, in case there's complete system failure (so it's more like a backup). I guess they can afford it because losing some progress in a casual game is not very critical.
So I see "architecture" more as a teaching and communication technique, applied after the fact. Altho since we're always moving between the details and the big picture, having organized abstractions (ie an architecture) in mind will help keep the design clean.
The architecture evolves and sharpens as a system comes together. And this evolving architecture provides the team with a common terminology and overall direction.
But you rarely start by choosing a specific architecture. Unless you already know a lot about your solution.