It's 2 different syntactically identical API's under an umbrella.

1: IEnumerable<T> that works lazily in-memory (and similar to the authors improvement) can be done in any language with first class functions, see the authors linq.js or Java's streams library (it's not entirely the same as a chain of map/reduce/filter since it's lazy but that's mostly not a drawback since it improves performance by removing temporary storage objects).

2: IQueryable<T> is the really magical part though, by specifying certain wrapper types the compiler is informed that the library expects an bound syntax tree(AST) of the expression you write, the library can then translate the syntax tree to SQL queries and send the query to the server to be processed.

Thus huge tables can be efficiently queried by just writing regular C# and never touch SQL. In most ORM's it's annoying or have impedance mistmatches but with EF you can write code and be fairly certain that it'll produce a good SQL query since the entire Linq syntax is fairly close to SQL.

It's part of the compiler - ast. Linq has two forms - one in the linq ordinary syntax

from x select x.name

And other is just lambda with anonymous types and so on.

For the lambda syntax, you can just do this: https://www.npmjs.com/package/linq

Of course, if you want to run this against a query provider, you do need compiler support to instead give you an expression tree, and provider to process it and convert them to a language (often sql) that database can understand.

There seems to be some transpilers, or things like that - but i don't know what the state of the art is on this: https://github.com/sinclairzx81/linqbox

C# can turn lambdas into expression trees at runtime allowing libraries like EF to transform code like `db.Products.Where(p => p.Price < 100).Select(p => p.Name);` right to SQL by iterating the structure of that code. JavaScript ORMs would be revolutionized if they had this ability.

Good answer. To elaborate on it and provide examples.

In languages that don't have expression inspection capabilities you have to replace the `(p) => p.Price < 100` part with something that is possible for the language to inspect.

Normally it's strings or something using a builder pattern.

For example, in TypeORM:

    queryBuilder.where("product.price < :price", { price: 100 })

And in Mongoose:

    Product.find({ price: { $lt: 100 } });

The LINQ-ish version would be:

    Product.find((p) => p.price < 100);

--

Similarly, for Ruby on Rails:

    Product.where("price < ?", 100)

Ruby's Sequel overloads operators to have a more natural syntax:

    DB[:products].where { price < 100 }

But the "lambda" syntax would be:

    Product.where { |p| p.price < 100 }

Sadly expression trees got out of love in modern .NET and it remains to be seen how much they will ever improve them.

https://github.com/dotnet/csharplang/discussions/158

The last comment thread by agocke is interesting. I've thought before that it's unfortunate that LINQ and expression trees were implemented before the move to Roslyn, because if they'd been implemented afterwards they could maybe have just directly used the same object model that the compiler itself uses, which could make it more sustainable to keep them in sync with language changes.

Java has two somewhat related projects in this space, and it does add a substantial cost to language changes (assuming you commit to keep expression trees up to date). https://openjdk.org/projects/babylon/ is the most interesting to me, as a linq+++ potentially.

Yes, being on both ecosystems most of the time, means I get to have lots of nice toys both ways, with them counter influencing each other all these years.

An interesting thing about expression trees is that with JIT they can be compiled at runtime, but with AOT they are interpreted at runtime.

> JavaScript ORMs would be revolutionized if they had this ability.

Is this possible in JavaScript?

Not easily. There's no built-in way to access the abstract syntax tree (or equivalent) of a function at run time. The best thing you can do is to obtain the source code of a function using `.toString()` and then use a separate JS parser to process it, but that's not a very realistic option.

There is a limited form of such "expression rewriting" using tagged template strings introduced in ES2015. But it wouldn't be particularly useful for the ORM case.

C# is definitely not the only possible language, but some things stand out:

1. You can extend other people's interfaces. If you care about method chaining, _something_ like that is required (alternative solutions include syntactic support for method chaining as a generic function-call syntax).

2. The language has support for "code as data." The mechanism is expression trees, but it's really powerful to be able to use method chaining to define a computation and then dispatch it to different backends, complete with optimization steps.

3. The language has a sub-language as a form of syntactic sugar, allowing certain blessed constructs to be written as basically inline SQL with full editor support.

Expression trees are highly underrated.

Compare C# ORMs to JS/TS for example. In C#, it is possible to use expression trees to build queries. In TS, the only options are as strings or using structural representation of the trees.

Compare this:

    var loadedAda = await db.Runners
      .Include(r => r.RaceResults.Where(
        finish => finish.Position <= 10
          && finish.Time <= TimeSpan.FromHours(2)
          && finish.Race.Name.Contains("New")
        )
      )
      .FirstAsync(r => r.Email == "ada@example.org");

To the equivalent in Prisma (structural representation of the tree):

    const loadedAda2 = await tx.runner.findFirst({
      where: { email: 'ada@example.org' },
      include: {
        races: {
          where: {
            AND: [
              { position: { lte: 10 } },
              { time: { lte: 120 } },
              {
                race: {
                  name: { contains: 'New' }
                }
              }
            ]
          }
        }
      }
    })

Yikes! Look how dicey that gets with even a basic query!

I get that Go maintainers want to keep things simple, but this stuff is pretty useful.

A simple language can make written in it code complex. A complex language can make code simpler. It's not a perfect rule or anything but it's been my experience that attempts at making simpler programming languages just put more demands on the programmer. The lack of expressive power has to be paid for somewhere.

(Note: A lot of answers discuss LINQ to SQL, which ZLinq does not appear to optimize.)

Iterators: LINQ works on any type that supports iterators. In most languages, this is any type that you can write a for (foreach) loop on and perform an operation on each item in a collection / array / list. (In C#, the collection must implement the IEnumerable<T> interface.)

Lambda functions: LINQ then relies heavily on Lambda functions, which are used as filters or to transform / narrow down data. Most languages also have something similar to these.

Generics: C# allows for "list of foo objects" instead of "list of objects that I have to typecast to foo." Although not explicitly required to implement something LINQ-like in other languages, the compiler verifying type helps with autocomplete and in-IDE suggestions; and helps avoid silly typing bugs.

Generic inference: C# can infer the return type from a lambda function, and infer the argument type in a lambda function. This means you don't need to decorate LINQ syntax with type information; except in some rare corner cases.

This is why, for example, there are LINQ-like libraries in Javascript and Rust. Java supports something that is LINQ-like, although in my limited Java experience, I didn't use it enough to really "get the hang" of it.

---

Note that LINQ has a very serious pitfall: It's easy to accidentally build a filter, and then have a lot of overhead re-running an expensive operation to re-load the source collection. The simplest way to avoid this is to call .ToArray() or .ToList() at the end of the chain to ensure that you store the result in a collection once.

Unchecked Exceptions make a big difference as well. Otherwise, some kind of exception forwarding would need to be handled.

Extension methods allow LINQ to be implemented as library over all the existing collection types instead of needing child types or refactoring the core collections library.

(Perhaps that is why I never got the hang of Java's LINQ equivalent?)

Java has RuntimeException, which is unchecked.

Granted, I can understand why some developers don't understand why some exceptions need to be checked versus not checked. Rust got this right with its Result type and panic.

Basic LINQ on in-memory collections isn't really that different from what you have in other languages. Where things get special is the LINQ used by Entity Framework. It operates on expressions, which allow code to be compiled into the application and manipulated at runtime. For example, the lambda expression that you pass to Where() will be examined by an EF query provider that translates it into the where clause for a SQL query.

I feel like pretty much every language with generics has a LINQ, like functools/itertools in Python, lodash for javascript. It’s just a different expression of the same ideas.

Nope, very different. Depending on whether the expression is on an Enumerable or a Queryable, the compiler generates an anonymous function or an AST. That is, you can get "code as data" as in say Lisp; and allows expressions to be converted to say SQL based on the backend.

You can do exactly the same with Smalltalk metaclasses and reflection.

However I do conceed most developers will only see them for the first time in .NET languages.