This is still a json API comparison, not a full framework comparison.
In the first test it sees how fast frameworks can serialize json and in the second test it sees how fast frameworks can access the database AND serialize json.
An inefficient json library and a framework is doomed across all tests. There are a lot of things frameworks do beyond serving json. These benchmarks are presented as representative of broad use when really they are only representative of the frameworks if used as a json api.
A raw HTML hello world, and removing the json encoding step in the db test would go a long way towards getting results that can mean what the authors seem to want them to mean.
There's nothing wrong with a json API benchmark, in fact its quite valuable in a lot of cases. But if that's what this is intended to be, the authors should say so.
Agreed; for example, in our (in-house) use case, switching a django app from using either stdlib's json or simplejson to using ujson[0] was a significant performance increase when serializing/deserializing large-ish (~100MB) JSON datasets.
In the first test it sees how fast frameworks can serialize json and in the second test it sees how fast frameworks can access the database AND serialize json.
An inefficient json library and a framework is doomed across all tests. There are a lot of things frameworks do beyond serving json. These benchmarks are presented as representative of broad use when really they are only representative of the frameworks if used as a json api.
A raw HTML hello world, and removing the json encoding step in the db test would go a long way towards getting results that can mean what the authors seem to want them to mean.
There's nothing wrong with a json API benchmark, in fact its quite valuable in a lot of cases. But if that's what this is intended to be, the authors should say so.