design goal: speed
    write fast, run fast, modify fast.
    minimize product of vocabulary and program size.

    comprehensive programming environments 
    (programming, files, comms, procs, ...)
    are depressingly complex.
    commonlisp, c/lib/syscalls, java, winapi
    are all in the 1000's of entry points.
    many with multiple complex parameters.
    with k we try to do more with less
    in order to write correct programs faster.

    > Just curious, but does the state of California charge you a large tax
    > for each character in a variable name?
    
    yes.
    
    we also try to fit our algorithms onto licence plates.

For some 'licence plate' K algorithm dissections, you might like this Twitter account:

https://twitter.com/kcodetweets

    print<>!~($/=$")+2*map 1..$s{$_%$'}++,<>

I can easily skim thousands of lines of Perl and Java in seconds and have a rough idea of what the code does. Reading K, however, requires careful and deliberate attention to every single character. I found it useful to maintain documentation on how to /read/ the code (almost a literate programming approach).

I write a line once for a 100 times I read it.

Conifer forest

Vs

Pine Pine Cedar Spruce Pine Cedar ...

Also it has (or had, maybe they fixed it) a bunch of weird quirks, like "sum foo" skipping all the NAs in a list while "+/foo" didn't, resulting in NaN. (Or maybe it was +/ over an int array didn't work while a double array did. I forget.) Arrays of dicts were immediately turned into tables, which didn't work the same as an array of dicts, etc.

But, yeah, it's not crazy implementing things in q/K, as long as they're mostly linear things. Trees are hard, iterative methods are hard, and it's really hard to skim code, other than to just rely on names and comments.

  q)a: 1 2 3 0N 3 4
  q)sum a
  13
  q)+/[a]
  13

However, I still find it problematic, and it's not really about the language itself: it's more about how it's promoted in the article, by the people in this thread, and in general. Specifically, the claims it's perfectly readable, and anyone who says anything otherwise is either stupid, lying, or too poor to understand it are not useful. (Yes, you know who you are for that last one. You're not helping your case.) The fact of the matter is that most programmers are used to ALGOL-esque syntax, and there is significant value in being able to lift concepts and interoperate between those languages at a superficial level. Every "inspired" languages hits these issues, depending on how "strange" it is: Objective-C is ugly, Lisp is alien, and on the far end APL derivatives are simply labelled as unreadable. That doesn't mean they're bad languages, but to those promoting it: drop the attitude, and understand why we have reservations. Your language isn't the "true" way to program, that only the chosen elect can understand and lecture us on. Don't ignore or gloss over languages that have similar features to yours. We want to hear what the interesting things are that you can do in the language, actual guidance for how we can adapt to the language (not: "one-letter identifiers are just what we do; deal with it"). Thanks.

It's like learning Chinese. Everyone learns it in China, and everyone there will tell you that it's perfectly readable and you don't need to be extraordinarily intelligent to speak it. But if you expect to figure it out based on your knowledge of English, it will be terribly frustrating. This does not mean that German or Dutch (which share roots with English) are better languages, or even easier to learn. Do not expect your Chinese teacher to give you a different answer from "these weird symbols are just what we do; deal with it".

This article is quite good. Clearly the author understand your reservations and he is saying: give it a try and it will click. The learning material might be better, but it's there (start with Iverson and work from that). It's much easier than learning Chinese!

I'm not sure Chinese is necessarily a good example for the point you're trying to make.

Native Chinese speakers _probably_ spend more effort acquiring full competency in Chinese than native English speakers need to learn English. For one thing, reading and writing is harder to learn, even for natives (source: am Chinese and haven't acquired full literacy, despite years of half-hearted effort).

English is generally easier to learn, not only for German and Dutch speakers, but for most people in the world who speak languages that are unrelated to both of them.

One hypothesis[1] is that English has been learned by foreigners multiple times in the past and slowly lost it's more difficult native features (noun declension, many forms of inflection). Chinese has tones, which are rare amongst world languages and, as many learners will tell you, difficult to achieve proficiency with as as an adult.

On a practical level, the British and Americans have had hegemonic influence over the world for about 2-3 centuries, so many idioms and concepts in the language will be familiar to a learner; less so Chinese, which is far more coupled to the culture of China and its history.

Like, every schoolchild in China learns a bit of Tang poetry as part of their education, and 成語 idioms are part of daily speech across all social classes.

https://en.wikipedia.org/wiki/Middle_English_creole_hypothes...

The claim that noun declension and inflection is harder to learn isn't well founded. If your native language has these, it'll be more natural for you. All languages have semantic declension and inflection, in the sense that context signals the kind of each word even if it's not syntactically inflected. For example in English the position of the word can signal whether a word is an object or subject "John hit Sally" vs "Sally hit John", the order of John and Sally signals who hit whom. In an agglutinate language you could signal this by putting a prefix like "John hit Sallyum" or "Sallyum hit John" would be parsed the same. My native language is Turkish, I've been learning English since I was 5 years old, but this "order matters" thing eternally confused me. It doesn't make sense to me when people say suffixes are more confusing than this order thing.

Ohoh, this line sparked a thought in my head:

"English is the JavaScript of human languages."

> ..slowly lost it's more difficult native features

That may be true of creole and English spoken as a second language (which, I believe, is more dominant in the world than English spoken by natives). Maybe also true about losing inflections, thank goodness, that surely must have made the language easier.

On the other hand, English as spoken by natives do contain plenty of quirks, exceptions, phrases that don't make sense explicitly (with implicit meaning you have to have learned/memorized)..

Microsoft's sample application "Your First Windows Program", written in C, using Win32 calls, can be translated into TXR Lisp literally, almost expression for expression:

https://docs.microsoft.com/en-us/windows/win32/learnwin32/yo...

The "meat" part, without the FFI declarations that precede it:

  (defun WindowProc (hwnd uMsg wParam lParam)
    (caseql* uMsg
      (WM_DESTROY
        (PostQuitMessage 0)
        0)
      (WM_PAINT
        (let* ((ps (new PAINTSTRUCT))
               (hdc (BeginPaint hwnd ps)))
          (FillRect hdc ps.rcPaint (cptr-int (succ COLOR_WINDOW) 'HBRUSH))
          (EndPaint hwnd ps)
          0))
      (t (DefWindowProc hwnd uMsg wParam lParam))))

  (let* ((hInstance (GetModuleHandle nil))
         (wc (new WNDCLASS
                  lpfnWndProc [wndproc-fn WindowProc]
                  hInstance hInstance
                  lpszClassName "Sample Window Class")))
    (RegisterClass wc)
    (let ((hwnd (CreateWindowEx 0 wc.lpszClassName "Learn to Program Windows"
                                WS_OVERLAPPEDWINDOW
                                CW_USEDEFAULT CW_USEDEFAULT
                                CW_USEDEFAULT CW_USEDEFAULT
                                NULL NULL hInstance NULL)))
      (unless (equal hwnd NULL)
        (ShowWindow hwnd SW_SHOWDEFAULT)

        (let ((msg (new MSG)))
          (while (GetMessage msg NULL 0 0)
            (TranslateMessage msg)
            (DispatchMessage msg))))))

If you look at this and the C side by side, you can easily see where the parts match up.

It's not really meaningful to compare the readability of sentences in English versus Chinese. On the other hand, it's both meaningful and important to compare readability of program snippets - and readability is more than just the first-time learning burden.

Overly-concise code being difficult to maintain is a well-accepted fact. The response shouldn't be to demand fluency - that's just sticking your head in the sand.

> Overly-concise code being difficult to maintain is a well-accepted fact.

Your well-accepted facts do not apply to this totally different paradigm. You seem to think those well-accepted facts are universal. They are not.

I have spent the last two years learning about array languages and think it is a fascinating topic, so I tried to give some advice (based on my own experience) to those who want to make the same journey. My point is not that you need fluency (you need some fluency to understand anything complicated), it is that the fluency you have in other languages may be counterproductive. Your comment is an excellent example of this.

Try to read a tutorial. Rewrite it using "meaningful names". All we have done that at some point, and all we started finding the sort names more comfortable to work with after a while. Don't take my word, give it a try.

Why do I need to do that?

I'm interested in solving problems for people, especially using computers. To do that, I need to build teams and build software. I need to think about costs and benefits, about longevity and sustainability. A great deal of that thinking is driven by the world as it is, by the historical accidents that have gotten us to where we are.

If this is a recreational activity, not meant to be practical, then great. I know people who teach themselves Esperanto and Klingon, and they seem to have a good time. Extreme Ironing looks fun, too. But if it's going to have some bearing on my profession, then I'm unlikely to forget what I already know, because in practice the people I will work with already know quite a bit, and brains being what they are, they won't be forgetting it.

Because those are the standard terms of intellectual discussion. People going into such a conversation without an open mind are called idiots, and they're not worth taking too seriously.

> I need to think about costs and benefits, about longevity and sustainability

Isn't it weird that people don't value that?

I mean, look at how many python programmers are out there! I have programs written just five years old that don't work in a new python interpreter and that's terrible for "longevity and sustainability". Good thing there are so many cheap Python programmers so we can keep the costs down: We're going to be rewriting this thing until we're old and gray!

I personally find this line of thinking quite unproductive though.

> I know people who teach themselves Esperanto and Klingon, and they seem to have a good time.

Hey that's great. I know a few Go programmers too. Loads of fun those chaps.

If only Esperanto or Klingon or Go were useful in some way....

You asked why you should keep an open mind when encountering and talking about a strange language. He pointed out that it's necessary for a good discussion on it. He's not wrong.

You've dismissed every point he made while not knowing a single thing about the topic at hand. That's not grounds for a proper discussion.

What I'm responding to in the thread is a question of pragmatism. In particular, I'm taking issue with the notion that one can only approach this "forgetting what you already know from other languages".

That's very much distinct from the notion of an open mind. And it's especially distinct when we're talking about tools that must fit into a professional context that involves decades of history, millions of people, and a complicated set of economic factors.

For example, Imagine that you were working in a machine shop and I came to you excited about a new screw-head. If you dismissed it out of hand, I might ask you to keep an open mind while I explained why I thought it was better. But if my explanation ignored that the Phillips head was the dominant choice, with billions of drivers and hundreds of billions or perhaps trillions of screws in existence, that would be another thing entirely.

The same logic applies to conlangs like Esperanto and Klingon. Is it possible it would be better if we switched the whole world to Klingon? Sure. Might I learn interesting concepts if I studied Klingon? It's possible. Should Klingon be considered for a moment as an important thing for most people to learn? No, not at all. It's utterly impractical.

So yes, keeping an open mind is valuable. But that's not something I've ever heard seriously disputed, so editing me down to saying that is constructing a straw man. Which also isn't grounds for a proper discussion.

Many seasoned APL/J/K programmers state that conciseness is an asset. I would trust their judgment on that. There are ways in which that and the quoted statement could both be true at the same time.

Terse logic is also very different from terse names; how much of K brevity is just from name compression vs logical succinctness? Haskell people could also use single character names everywhere. When will they adopt the conveniences of fitting everything in as few pages as possible?

And what’s the point of fitting as much content as you can into one eyeball when working memory can’t consume a whole page of a non-succinct language like Python?

it is if one speaks both languages and is qualified to comment. part of 'overconciseness' is domain/cultural experience - what is overconcise in one context is not in another.

I like the choices of primitives in K and the novel ways they can be combined to do things more efficiently (and with fewer pieces!) than you may expect. Good examples are 'where' (&), the grade twins (< and >), scan (\), and eachprior (':).

I love the adverb syntax: each (') lets you map a function over a data structure with zero syntactic effort, and 'over' (/) and 'scan' (\) make reductions trivial.

With regards to an on-ramp, I started with John Earnest's oK REPL and did a bunch of Project Euler and Advent of Code problems. I then moved to trying to understand the examples on kparc.com. I then converted a spreadsheet application from JS to k7. In all of this, John's documentation was essential.

For me, having single variable names isn't paramount, but in some cases - particularly functions - I can see it helps to 'drive' your mind towards finding simpler ways to do things. It lets the characters that do the 'real' work come to the forefront.

You also would be much better off learning X's idioms. These are standard ways of saying something. They're standard for a reason - because they work, and because everybody understands them. If you can't read them, then a lot of code is going to be difficult to understand.

That's just table stakes for using X. If you want to use X, but don't want to learn the syntax and the primitives, then you don't really want to use X. You may want the benefits that you think X would get you, but you don't actually want X.

Now for the other side: If you're a proponent of X, you need to remember that many languages are magic within certain domains, and anywhere from "meh" to horrible outside it. Take K, for instance. I'm pretty sure it wouldn't be a good fit for most embedded systems. I'm fairly sure it wouldn't be a good fit for web programming. I'm not sure I'd want to do text processing in K. So it's not "the one way" (except possibly within certain domains). It's not "the way of the future" (except possibly within certain domains). It's not "what all the smart and enlightened people are using" (except possibly within certain domains). And, unless we're working in those certain domains, it really isn't all that important whether the rest of us learn it or understand it.

I suspect that looking like C or ALGOL would not be all that useful for K (or Lisp, or Haskell, or Forth). The semantics are too different. A surface similarity in syntax isn't enough to bridge the gap in a meaningful way.

Yes: It is perfectly readable once you know how, and anyone who says anything otherwise is just plain wrong. Including you 58 days ago.

I don't know if you are wrong because you are lying, or you believe this because you are stupid. I wrote off at the time that it was well-meaning ignorance, but it doesn't really matter: The key thing, that is, the thing holding you back from reading it, is you.

I believe you can learn to read it and read it as well as me as quickly as with a few months of study. If you think anyone else is telling you otherwise, ignore them.

I don't believe you need to not be "poor" to do it - everything you need is available to you.

It's all about whether you want to do this or not.

> Every "inspired" languages hits these issues, depending on how "strange" it is: Objective-C is ugly, Lisp is alien, and on the far end APL derivatives are simply labelled as unreadable. That doesn't mean they're bad languages, but to those promoting it: drop the attitude, and understand why we have reservations.

We understand because we were once you: Every X-evangelist you're talking to (where X is Lisp, or Objective-C, or in this case K) was once like you in at least the way of not knowing X, so we know you are wrong in a way (as we now know X) that you simply cannot comprehend yet. Think about that. Maybe re-read Paul Graham's blub article and appreciate really what it means to be the blub programmer.

Many K programmers have some familiarity with APL. Some (like myself) have used Objective-C and Common Lisp (I've got over a decade of professional history in CL), and loads of other languages. Until you learn K, all you have is our testimony; our gospel that this is worth learning. You can literally do anything in blub, being able to do it easier is the point.

Now do the same with well-written, verbose C-style or Python code (no shorthand/abbreviations, no lambdas, no ternary notation, etc.).

Shorthand code in any language is bad in the long run because it's unmaintainable except by people who work with it on a regular basis. You are making the same arguments as Perl fanatics who used every possible abbreviation in their language 25 years ago. How much of that is even comprehensible now without a major reverse-engineering effort?

Perl is inscrutable in a different way: It is impossible to insert an instruction reliably into a statement allows the student to observe what is going on without changing it: Program flow can change mid-sentence. This isn't true of K.

> Take a break from K (or any other shorthand language of your choosing) for a year or five, then come back and try to figure out what the code you wrote when you were using it frequently is actually doing.

I've been looking at K code for more than five years at this point, and I don't have any problems reading code written by K programmers a decade ago or more.

I think you're completely wrong about this.

I think you're completely missing the point here. The assumption wasn't that old code is unreadable, but that you have trouble understanding it after NOT looking at ANY K code for five years.

Many other skills require a period of re-learning after a five year gap. Where do you believe K sits on the scale of re-learning effort involved, riding-a-bike seconds to minutes, or months to years of effort?

I've only learned a little APL so far, and that really didn't take much effort so I'd be surprised if it would take me more than a month of dedicated learning to get fluent in the language. If that's what it takes to learn from scratch (the language, not the concepts common to maths and many programming languages), I would expect re-learning to be possible in less than a week. That does not strike me as too big an obstacle in many cases.

Fair enough.

I don't see any evidence why that would be true, or even more true than other languages.

Do you have any?

A slightly weaker but maybe more realistic example, albeit in mathy terms: pick a language L from the set of all programming languages as your next language, according to a probability distribution of commonality of use in 2020 or so, possibly skewed for your field/interests, and work on it exclusively for 5 years.

If your initial language (before L) was K, you're not terribly likely to have picked a successor language that has any relation to K. If your initial language was C, your successor is likely ALGOL-like, and you probably won't have rewired your brain totally. So coming back to old work after 5 years will likely be less of a shock.

Granted, this is like saying "k is different!", but it ties it to a real-world situation where it could even bother a dedicated k programmer.

However even if it's true for most programmers, why does this definition of "readability" have value?

My other comment on the nearby thread likely best-addresses why I think coming back "cold" is useful, based on an assumption that being a novice and being cold are similar, and maybe they're not.

With regard to picking up postscript specifically: I'm not intimately familiar with the language, but the bits I found in a quick search look a) less approachable than javascript to me, so fair point on that, but b) somewhat algol-like. At a glance I felt like I could pick up the flow-of-execution without a lot of work, though I could be wrong.

If it is kind of algol-like, I feel like that plays in favour of my argument a bit -- algol-like languages are so ubiquitous as to be hard to really not use the mental model for 5 years, so we're probably not really coming in "cold" to one.

In my opinion, it is a fair comparison.

Excellent question! Personally, I'd argue that a language cannot be viewed in isolation from the problems it's trying to solve.

5 years ago I maintained decades old HLASM programs for IBM mainframes during a consulting gig and haven't touched anything like it since.

I'd have very little issues reading and understanding the source code today. I couldn't say the same about the domain logic and business processes behind it, because it was very specific to the company and I worked in a totally different field since then.

This leads me to think that it's a combination of working with similar languages or in comparable application domains that makes coming back "cold" easier.

IMHO people tend to underestimate the influence of domain knowledge in the programming world and greatly overestimate the value of particular language and framework choices...

I'm not sure I'd agree with that, at least not entirely.

A language is a tool for thinking. It doesn't solve problems by itself, but it is a good language if it makes thinking easier. k is a great tool for thinking! With only 25 (or so) easily accessible symbols, it's amazing we got about a hundred primitive operations out of them, but what's really incredible is how tastefully Arthur selected those primitive operations such that the symbol could indicate an entire class of operations.

Some work well: - means subtraction and negation. Others are cute: < means grade (ascending) and less-than (and the grade of a grade turns out to be pretty useful!). Some are (however) unfortunate: + means flip and addition (which aren't the same thing at all), and I don't think + would be flip if flipping wasn't so useful.

And it's the tool that matters to me: I don't think I ever used array_flip in php until I learned flip in k, and now it's "obvious" (although PHP's array_flip is limited to a single use-case and requires other functions for other use cases, oh well). And I'll probably know how flip works until the day I die, and if I need it (in another language), I'll probably just make it again.

The tool has survived! And it's the tool that I go rooting in languages to find.

Lisp and Erlang have some really great tools (concepts) like this.

> 5 years ago I maintained decades old HLASM programs for IBM mainframes during a consulting gig and haven't touched anything like it since.

How funny!

I've had almost the same exposure to HLASM (although it was more than a decade ago)- consulting gig, never again. Any language designer who thinks comments can begin only on the first column needs to be taken out and shot.

However I think having to refresh myself on exactly how BALR works, or what types dyadic ! takes isn't going to waste much time. Are these real concerns you have about forgetting the tools you learn?

Then it's not perfectly readable. That, or your definition of readable is useless because then any programming language is readable given enough study time.

> being able to do it easier is the point.

I still haven't seen any example where it is actually easier. Shorter? Yes. Easier? Very debatable.

Given a fragment of Chinese, if you do not know Chinese, would you say it is unreadable?

What possible definition of "unreadable" could you have that would make that a useful statement?

No, I don't think speaking to a language as being "unreadable" is particularly helpful. A very careful reader might interpret your claim that "K is unreadable" as "gjulianm can't read K" but I wonder what exactly is the point of even saying it in the first place? Maybe you mean "approachable" -- a fully reasonable statement, but one reflecting your inability to say anything else meaningful about it.

On the other hand, if I know Chinese, it's entirely possible for me to point to a gibberish statement (for example, on a tattoo), and say that it's unreadable. I'm definitely referring to something else here.

Now maybe we just accept having two different definitions for "readable", but I think yours has real harm. I think referring to something that you do not understand with the same word that means nobody can understand it might confuse people, and I just cannot understand for the life of me, why so many programmers would want to intentionally confuse people like this.

If I pick up some random K code, I can read it. I could make a change to it, and other K programmers will understand what I did and why. That's good enough for me.

> I still haven't seen any example where it is actually easier. Shorter? Yes. Easier? Very debatable.

I think the max of list example is a good one. I've also previously observed some examples that might be less mathematical.

[1]: https://news.ycombinator.com/item?id=22467866 - Here I beat a custom "database" by 50x with one line of q (a very similar language to k)

[2]: https://news.ycombinator.com/item?id=21799376 - A six-character solution to the rock-paper-scissors game, using q to analyse the creation of the solution

[3]: https://news.ycombinator.com/item?id=21680743 - Here's our friend the bin operator again, showing how it appears in production.

[4]: https://news.ycombinator.com/item?id=19659590 - One of my favourite k/q features is views.

[5]: https://news.ycombinator.com/item?id=16851862 - String parsing is "obvious" in k/q where it isn't in other languages

In these cases (and many others) q/k is "easier" because it gets the job done faster. If you can beat my k with your JavaScript then good for you (And I'd like to see it! I can always get better!), but I usually can't. I find the solution faster with k, the program runs faster with k, and those are the things that matter to me when I say "easier".

I absolutely do not mean "more approachable".

Your points are really good here, and writing code that runs faster and finding a solution yourself faster are clear wins for a language. I think the value of those wins is subjective, though.

If you need speed of execution, or if you're working in a green-field scenario and need to get your work done super fast, your wins are the most valuable ones.

But for example, if you work on a CRUD application that handles edge cases for specific customers that someone wrote 11 years ago when you used a different ORM but that still matter today, concision and execution speed aren't really the triumphal traits of the code. Clarity to others who may not be as well-versed as I am is the key thing I care about.

Does Javascript, or Python, or whatever else achieve that? Gosh, I don't know. But I'm scared of thinking about k as a total novice, so... I think they win against it up to some point. Maybe the level at which I'd know enough k to not be bothered by it is low enough that this doesn't matter, but ... it's still scary!

If I feel that way, why would I choose to use it knowing that I'd need to find the rare someone who overcame that fear, just to work on my CRUD app? This isn't k's fault, but k is not the right choice here, and so k has lost on this attribute of approachability, which is "readability for novices", and is the most valuable for a huge number of people.

What you've got that's similar: Block structure, single dispatch, everything's an atom, and a few seemingly familiar keywords like "for" and "if" that despite working differently, often appear in familiar patterns. I mean, how different is for k in x: than x.forEach(k =>? It seems like enough of these things dominate those first few steps into the unknown that they provide a comforting handrail into the descent into the coming madness, but it's difficult for me to believe it provides anything more than that.

And it's my experience that kind of comfort can be a lie: To read a fragment of well-documented code, to be amazed in the work and thought that clearly has gone into it, only to discover that fragment contained a bug I skipped over (since that area was so well tested and so clearly good) simply because I didn't bother to actually read it. So now I read. And maybe I read JavaScript and Python a little slower than others, what's important to me is that I also read JavaScript and Python slower than I read k.

I think that's a valuable trade.

But I think you're touching on something important: It's very difficult to mix in a little bit of k into an existing application to "try it out". I hope that there may be things we can do to the environment to make k more accessible and approachable that don't involve trying to cut out this valuable thing, because this thing that makes k unique is just so good I don't want to give it up.

Regarding your examples:

1. If you are using a database then it's not an argument for the language, I think. 3. A binary search is hardly a selling point of a language. 4. Views are cool, I reckon that. Although K is not unique in that regard. 5. That solution is, from my point of view, suboptimal and non-obvious. It's not only harder to understand than the naive search but it also performs algorithmically worse in all situations: it always performs nm comparisons (n, m being string, substring lengths resp.) plus the hidden cost of the rotation (which could be zero, could be not-zero depending on how is that implemented) plus the AND of every bit, plus the search of the match position.

I also left the second one as the last, because that, to me, is an anti-example. See the things I need to have "in-memory" in order to understand the snippet `Paper`Rock`Scissors@&/"ki"?

"ki"? means search for "k" or for "i"? I have to assume it's what that is, but I can't reconcile that with the documentation of the find function https://code.kx.com/q/ref/find/ because I don't know where comes the function argument.

* / means 'over'. Together with '&' (which somehow is both 'and' and less than? that's very confusing) it looks like it takes the minimum? I honestly do not know what it's meant to do at this point because I don't know what kind of object is "ki"?.

* @ means 'apply' and here I don't get how you are applying three actions to a what, a list? Because I get that ` means 'print'. Knowing what the code does I assume that when it finds 'k' (index 0) does the first action (paper) and so on. Looks like the 'find' function returns the length of the list when an element is not found (weird, because it is a very good way to facilitate mistakes and induce race conditions) so that's how you get 'scissors' in the other cases.

To me it's weird that you put this as an example. A code example that, despite being pretty simple (if the input has k, return Paper, if it has i, return Rock, else return Scissors) I am not able to understand it even with the documentation at hand. Given this example, I find hard to believe that you can find any K code and understand it quicker that you'd do with other programming languages. And I honestly doubt that it's faster than a for-loop in C.

Yeah, you can say you understand it. But you need a bigger "translation table" than in other languages, which adds to the mental concepts that you also need to keep in mind with programming. It's not to say that it's impossible, but it's more mental work for, as far as I have seen, very little benefit. It can work better for you, yes, but it's very possible that it's just that you're used to it and you don't notice the extra burden of dealing with that code.

How do you figure? Programmers too, have a "native language" (usually the one they use the most, or that has the most powerful features they know how to implement).

> Usually, when people talk about "readable programming languages" the definition is taken as the amount of effort an average programmer needs to read a certain "standard" piece of code.

I think that's a load of shit. What would be the point of communicating such an opinion to others?

> 1. If you are using a database then it's not an argument for the language, I think.

It's built-in to the language. You don't call Python a database because it can unpickle things.

> 4. Views are cool, I reckon that. Although K is not unique in that regard.

Excel has views. I'm not aware of any other language that has them. Do you have some examples?

> I can't reconcile that with the documentation of the find function https://code.kx.com/q/ref/find/ because I don't know where comes the function argument.

See the first example under type-specific uses of find. x (the left argument as given in the example) is a simple list.

> / means 'over'. Together with '&' (which somehow is both 'and' and less than? that's very confusing)

https://code.kx.com/q/ref/accumulators/#unary-application

https://code.kx.com/q/ref/lesser/

To x&y is the lesser of x and y. You can see in binary this harmonises with logical and:

    00001110b&11110111b
    00000110b

No. `foo is a symbol foo. A symbol by itself is an atom, but `a`list`of`symbols can be written just by writing them together.

https://code.kx.com/q/basics/datatypes/#symbols

> code example that, despite being pretty simple (if the input has k, return Paper, if it has i, return Rock, else return Scissors) I am not able to understand it even with the documentation at hand.

Actually the point is to look at the analysis to see how I chose k and i. If you already know you can solve the game by searching for "k" and "i" then you've missed the point: Array programming languages make this obvious.

Unless everybody is born learning a language, no we don't. Even 'The one they use the most' varies over time.

> What would be the point of communicating such an opinion to others?

An easy use case: you're in your company and you need to do a simple script for some computation. The more readable the language you use is, the more coworkers will be able to find a bug/reuse it/modify it without requiring your help.

> It's built-in to the language. You don't call Python a database because it can unpickle things.

I read more of the parent comments on that link. I didn't notice that the OP started with an unsorted dataset, you did binary search on a sorted one. I thought that it the "kdb native format" meant more database magic that just serialization.

> Excel has views. I'm not aware of any other language that has them. Do you have some examples?

--C++ has string_view at least. I think C# had also views of some sort but whenever I search 'C# list/string view' I get MVC or WPF examples about unrelated things.--

Misunderstood what a view was, IDK why I thought it was string/list views.

> To x&y is the lesser of x and y. You can see in binary this harmonises with logical

"harmonises" in the sense that it takes one of the most common programming symbols and changes it? I don't know of anyone that sees "2 & 3 = 2" and thinks "yeah this makes sense".

> Array programming languages make this obvious.

How? First, your analysis line is even more cryptic than the resulting code. And I don't think the idea "find the minimum distinguishing characteristics between the outputs" can only be born out of array programming languages. Don't get me wrong, I think it's smart, but I don't see how the language makes that obvious.

Edit: Sorry, I misunderstood what a view was.

Nobody is born having learned a language. It's something even babies have to learn and they know nothing!

People can even "switch" their native spoken language up to a point although it seems to get much harder as they get older and more experienced with one language, as they're constantly comparing their aptitude to their "native" one -- just as they do with programming.

> An easy use case: you're in your company and you need to do a simple script for some computation. The more readable the language you use is, the more coworkers will be able to find a bug/reuse it/modify it without requiring your help.

The decision to bring another language into a company requires different considerations for every company. The one I work for has a lot of q/k programmers, so this isn't a problem for me. I like programming in k, so it makes sense I might want to be around other k programmers.

I think if you decided to write your "simple script" in even a well-known language like JavaScript, you'd cause some grumbles in a Python+C# shop; k isn't special in this regard.

> I read more of the parent comments on that link. I didn't notice that the OP started with an unsorted dataset, you did binary search on a sorted one.

HIBP offers the data pre-sorted, so I used that. I could have used asc (actually sort) instead of `s# (an assertion that it is already sorted) if I'd used a different file, but I value my time.

> I thought that it the "kdb native format" meant more database magic that just serialization.

No. It's literally just mmap. There's a header on the file describing the array shape, and indicating that it is sorted (so bin will be accelerated), but that's it.

kdb is a database in exactly the same way that Python could be if you just pickle/unpickle everything, except it's actually fast enough that people do this, even for large data sets.

> "harmonises" in the sense that it takes one of the most common programming symbols and changes it? I don't know of anyone that sees "2 & 3 = 2" and thinks "yeah this makes sense".

If you write something like (f=42)&(g=69) you expect "&" to mean "and" here, right? Well if f is an atom, then f=42 returns either a 0b or a 1b based on whether f is 42. What if f is a vector? Well, let's see:

    f:1 42 2
    g:4 69 4

"&" is just generalised from here to work the same way non-binary values. Having it convert to binary-and would be very confusing for vectors, and although I can see a consistent way it could work, I don't see the value in that operation.

I meant a programming language (it gets confusing, sorry). But anyways, our native language is much more embedded in the brain than programming languages.

> I think if you decided to write your "simple script" in even a well-known language like JavaScript, you'd cause some grumbles in a Python+C# shop; k isn't special in this regard.

Yes. And I'd cause even more grumbles if it was written in Haskell, and even more if it was on K. That's the point.

> HIBP offers the data pre-sorted, so I used that.

Yes, and that changes how you access the data. So the evaluation was not fair, as the OP was not using sorted data.

> db is a database in exactly the same way that Python could be if you just pickle/unpickle everything, except it's actually fast enough that people do this, even for large data sets.

That's nice, I'd actually wish Python had better serialization facilities.

> f=42 returns 010b and g=69 returns 010b -- do you see why

What if "f:42 1 2"? f=42 returns 100b (is that a value or a vector?) so under the common "and" definition "(f=42)&(g=69)" should return 000b, under the "less than" returns 010b?

I think it's possible that might have more to do with the fact we use "our native language" more.

> Yes. And I'd cause even more grumbles if it was written in Haskell, and even more if it was on K. That's the point.

I don't know. I work in a place with a lot of q/k programmers, so this isn't a problem I face.

> Yes, and that changes how you access the data. So the evaluation was not fair, as the OP was not using sorted data.

You're mistaken. They did use sorted data, they just didn't know about the presorted file so they also wrote code to sort it.

> is 100b a value or a vector?

It's a vector of three elements (bits: zeros or ones).

> What if "f:42 1 2"? f=42 returns 100b … so under the common "and" definition "(f=42)&(g=69)" should return 000b

Yes that's right, and q/k would return 000b in that case.

> under the "less than" returns 010b?

100b<010b is 010b. We need something that gives us 000b and that operation is "and". Say I have a table that looks like this:

    f  g  k 
    --------
    10 9  kf
    20 8  je
    42 69 kf
    42 0  lk

    t:+`f`g`k!(10 20 42 42;9 8 69 0;`kf`je`kf`lk);

    q)a:`f`g`k!(10;9;`kf);
    q)b:`f`g`k!(20;8;`je);
    q)c:`f`g`k!(42;69;`kf);
    q)d:`f`g`k!(42;0;`lf);
    q)(a;b;c;d)

    f  g  k 
    --------
    10 9  kf
    20 8  je
    42 69 kf
    42 0  lf

I can write t.f=42 and get 0011b and t.g=69 and get 0010b. I can do queries like this:

    q)select from t where (f=42)&(g=69)

      t@&:(t.f=42)&(t.g=69)
    +`f`g`k!(,42;,69;,`en)

And from there it should be clear why we need this "and" behaviour. The question is what should a&b do when they're not bit vectors that would be compatible with this behaviour? 42&69 should return 42 because it's the lesser; 0b&1b returns 0b because it's the lesser... Mathematics is already quite comfortable with the relationship between logical-and and min, so this has a precedent.

(But: I am a pure mathematician by background, and the idea that logical AND is a special case of MIN is pretty familiar to us. The thing whose only values are TRUE and FALSE is a particular Boolean algebra. A Boolean algebra is a particular kind of lattice, and lattices have MIN and MAX operations, though we usually call them MEET and JOIN instead. I would expect & = min to be less comfortable to people with different backgrounds.)

[EDITED to fix an inconsequential typo.]

There are a few of us in this thread, trying to explain why.

You find outrageous that 2&3=2, almost everyone (all the non-programmers of the world) will find more outrageous that i=i+1. You get used to it quite fast. It's obvious as soon as you realize that, with booleans, min is and, and max is or.

Yep, it is hard to understand. That's precisely why I don't like having even more unintuitive things.

> It's obvious as soon as you realize that, with booleans, min is and, and max is or.

But it's not min and max with the rest of types.

Yes it is: & is min for all types including boolean, where it is "and" as we can see above. Similarly | is max for all types including boolean, where it is "or". You can convince yourself in binary, or pick up

> it is hard to understand. That's precisely why I don't like having even more unintuitive things.

If I tell you something is "intuitive" I do not mean you should find this approachable knowing what you know. Just as statements on "readability" I find this personalised definition doesn't promote a meaningful discussion, as it depends too heavily on social factors.

Now I don't deny those social factors are important, but if you really mean that, you're basically rejecting learning anything about X because (as at least as a factor) nobody else knows it.

Are you sure you want to be that person? Does anyone?

No of course not. We want to be scientists and evaluate these tools on their own merits. If they are truly new and novel, then we would not expect them to be popular, and so we cannot (if we are to be good scientists) reject them for that reason.

It is for this reason I recommend when someone says "it is readable" we always interpret it to mean if one knows how, and similarly, "it is intuitive", should have an implicit when you are thinking a certain way.

If we do not do this, then we cannot get any value whatsoever from what the other person is saying; if I truly imagine this is what the mean -- that they mean they do not have any interest in being a good scientist and having a discussion about the merits, then they're definitely not worth talking to.

So let me say something about "intuition" under this definition: To me, it is important to understand the relationship between the intuition I can have (or generate) about something, and the memorisation and taxonomy that something requires. "&" is extremely intuitive in this sense because it always has the same definition for every type (It always means the lesser). Most things in k are intuitive like that, and those things that are not, tend to be useful enough that they're worth memorising.

An example of something like the other is "?" which usually means "draw" (As in, that's the most common meaning). You might say 10?100 to draw ten numbers from 0-99. But what would `x?`y? How many is an `x? So k gives another definition to this form (enumerate). This is not intuitive, but if the definition is useful, we will memorise it. This one turns out to be useful.

For the most part, and in the way that I mean, k is very intuitive, and where it is not, it tends to be useful. These are good things!

The database is a piece of the language, though, and the language ships with it. Python has a lot of very complex things in its standard library; C without a standard library is foreign to most programmers. Why do they get a pass while k doesn't?

It can work better for you, yes, but it's very possible that it's just that you're used to it and you don't notice the extra burden of dealing with that code.

It's not like 'geocar is foreign to writing in other languages. His GitHub (github.com/geocar) is full of projects in other languages, some fairly recent. This may make sense as an accusation for someone who doesn't write any other code, but I don't think it's a fair accusation here.

If there was some intense burden from writing k in comparison to ALGOL-like languages, would 'geocar have written an operating system in it rather than C? That doesn't seem apparent to me.

When we are discussing languages, they don't get a pass. Python has a great math library with Numpy, but it doesn't mean that the language itself is optimal for math-related problems. And specifically in that example, using an external database when defending the speed of the language... meh. It's as if I said that Python is very fast and showed a Numpy calculation.

> This may make sense as an accusation for someone who doesn't write any other code, but I don't think it's a fair accusation here.

It's not about writing or not other languages. It's about being so used to something that you don't notice the burden. I am very familiar with some of the projects and languages I work in and I feel comfortable with the codebase, and I get used to the mental burden of working with them and I end up not noticing it. It doesn't mean it is not there.

> would 'geocar have written an operating system in it rather than C?

Has he? Couldn't find anything (I'd like to see it, honestly, seems a fun experiment).

kdb isn't written in an external language.

Has he? Couldn't find anything (I'd like to see it, honestly, seems a fun experiment).

Not entirely him, of course. There are a few remaining files on kparc.com:

kparc.com/z/ kparc.com/$/

so on.

Not free software, and they seem reluctant to show it off. 'geocar has done a live demo of a primitive version, though: https://www.youtube.com/watch?v=kTrOg19gzP4 (2014)

The archive is more interesting:

https://web.archive.org/web/20130830071750/http://kparc.com:...

You're completely mistaken. It's not an "external database". That's a regular, ~600kb interpreter download from kx.com

Those lines are all the lines typed into a q) prompt one at a time on my laptop (A 1.6ghz macbook air).

> In this case it seemed that it was just doing a binary search,

Yes. That's all okon does as well, they just do it over a B-tree instead of a contiguous list. If you read the writeup you'll see the author actually eschewed using a contiguous list for reasons of random insertion then took to a B-tree generation algorithm that couldn't handle unsorted input. I suppose they just forgot what they were doing at some point.

> but it was using an already sorted dataset that the OP of that comment wasn't using.

Again, you're mistaken. The author (Stryku) might not have known the sorted file existed, but the first thing they tried to do was sort it. They just had a problem sorting a 22GB file because they were using terrible tools. They had to make tools because they didn't have any good ones. The rest of their algorithm took advantage of the fact the input (to these later stages) was sorted.

> Well, I can't get anything out of this.

> I can't honestly believe anyone that says that's a good way to create code.

I spent 5 minutes to write something (in q) someone else wrote over twenty days (in C++).

I can't honestly believe anyone that thinks spending twenty days on a problem is better than five minutes.

If you can't get there, it's going to be really hard to talk about what's amazing in k!

Nobody thinks that and I didn't say that. I was talking about that specific file (although the rest of them are equally unapproachable).

> If you can't get there, it's going to be really hard to talk about what's amazing in k!

Well, the thing is that at this point the only amazing things that have been talked about is that it's fast (in some specific use cases of data queries) and extremely concise. What more there is to K? What more reasons are there to use it, and with which downsides does it come with?

I hope nobody thinks that, but unless you can come to terms with the fact that you're wrong, and that this is a completely fair comparison, you're going to be the guy who writes okon2 at some point instead of using a better tool.

> the only amazing things that have been talked about is that it's fast (in some specific use cases of data queries)

Those weasel words are preventing you from seeing what should be obvious:

The k program is faster, shorter, obviously correct, and it took less time to write.

That's the amazing thing. Who doesn't want that?

> What more reasons are there to use it?

This is the wrong way to think about things, because there's an infinity of such reasons. Instead, invert it: You should always use the best tool you can. If you don't know k, it can never be used even if it would otherwise be the best tool.

I don't recommend people use k (except when it works), but I recommend people learn k because it'll make them better programmers.

> which downsides does it come with?

The biggest downside is that you don't know it, and the only person who can fix that is you

I again point to the C standard library. Modern standard-compliant C can do almost nothing without it. I don't even think it can handle IO without it.

The language is cryptic and the variable names are even more cryptic.

You're thinking of it in a light that doesn't help you understand; don't think of it as a variable, think of it as a definition. "jk is defined as..."

It doesn't increase linearly though. #:'=: is seven characters that mean as much as 68 separate lexemes in lisp:

    (defun count (list)
      (let ((hash (make-hash-table)))
        (dolist (el list)
          (incf (gethash (cadr el) hash 0) (car el)))
        (let (result)
          (maphash (lambda (key val)
                     (push (list val key) result))
             hash)
          result)))

The choice of operators (and their definitions) was made with great care in an effort to maximise this effect.

The hashing stuff feels a little "bolted on" in ANSI CL. Add-on libraries can round out the functionality.

In another dialect, TXR Lisp, the whole (let ...) part above reduces to this:

   (hash-pairs hash) ;; get key-value pairs as two-element lists

To reduce a sequence into a histogram returned as a hash in TXR Lisp, we would do a group-reduce

   [group-reduce (hash) identity (op succ @1) INPUT-LIST-HERE 0]

Within each of these buckets, an independent reduce (i.e. left fold) is going on, whereby a hash table holds the reduce accumulators.

For maximum flexibility, group-reduce asks the caller to specify the hash table rather than making one implicitly; hence the (hash) argument.

Likewise, the initial value for the reduce (shared by all the buckets) is specified explicitly. reduce is too general a concept to justify making it a defaulted optional argument that goes to zero.

Here, we somehow abuse reduce. We start the accumulator at 0, and at each reduction step, we ignore the new value coming in, and instead produce the successor of the accumulator as the new accumulator value, so all we do is count the reduction steps in each bucket, thus producing a frequency histogram.

(op succ @1) is needed rather than succ because the function it denotes takes two or more arguments (it gets called with two), whereas succ requires exactly one argument. If succ ignored arguments after the first one, it could be like this:

  [group-reduce (hash) identity succ INPUT-LIST-HERE 0]

  [group-reduce identity succ INPUT-LIST-HERE]

  [group-reduce succ INPUT-LIST-HERE]

The traditional, mainstream Lisps assume that the developers will make these for themselves, especially for tasks outside of list processing.

Could K become a general purpose language? Maybe, but it always strikes me as odd that that feasibility as a mainstream language comes up so frequently when K is discussed. It's not the high-order bit of the conversation.

The idea that concise formulas provide this (and that more modular solutions can't) appears to be a conceit.

    + / ! 1000

+1. All of these "compact" languages (I'd even extend it to Perl to some extent, the way some write it), are seriously lacking in the ergonomics department. Sure, J Leet Hacker might have no problem parsing pages of the stuff. What about the coworker with dyslexia, ADHD, vision problem, etc?

Even if you aren't, what are the real gains? It seems resistant to tabcomplete, git diffs are probably near-inscrutable, grepping for snippets is probably an exercise in judicious application of backslashes... it's cute, but I don't see where this holier-than-thou is coming from (not that ANY language justifies that attitude)

Edit: oops, touched a nerve, I see. I'll add to the above the 7 +/-2 rule [0]. Sure, lets say even you COULD write a program in 1/100th or 1/1000th of the lines in k. I don't WANT a source code that looks like that. Way too much information density.

I'll give them this: perhaps the criticism is that ALGOLikes are too sparse, could stand to be denser, and k is the manifestation of the far other end of the spectrum.

- https://en.m.wikipedia.org/wiki/The_Magical_Number_Seven,_Pl...

Beyond a point however, I cannot read my own regex's after a month's absence. Which is why I use perl's /x modifier extensively to split up regex components onto multiple lines and to document them thoroughly, even if they are for throwaway scripts, because I don't always throw them away!.

For example:

    $_ =~ m/^                         # anchor at beginning of line
            The\ quick\ (\w+)\ fox    # fox adjective
            \ (\w+)\ over             # fox action verb
            \ the\ (\w+) dog          # dog adjective
            (?:                       # whitespace-trimmed  comment:
              \s* \# \s*              #   whitespace and comment token
              (.*?)                   #   captured comment text; non-greedy!
              \s*                     #   any trailing whitespace
            )?                        # this is all optional
            $                         # end of line anchor
           /x;                        # allow whitespace

This is where K fails me. It may not be a fault of the language, but everyone in the community has bought into this strange idiomatic style. I can't imagine debugging it, or checking it for correctness, or foisting it on a less experienced developer. Here's a canonical example an xml parser, on their website.

https://a.kx.com/a/k/examples/xml.k

Where's the pedagogy? Where are the comments? Why is this line noise considered acceptable?

> Where's the pedagogy? Where are the comments?

Most of that document _is_ comments. There's a comment on almost every line, very similar to your perl example. Comments begin with a "/" character which doesn't have a function to the left (e.g. whitespace).

First we have some constants (L,W,B,S,R) which refer to the left-bracket, whitespace (which includes blank), blank space, and slash and right-bracket. We've also got some utility functions (cut;join). These are simple enough they don't require any special explanation to the K programmer who reads this.

Then we have a function that produces an xml-entity from a character. The author assumes octal is required, so (needlessly) converts to that. The octal string (with a leading zero) is concatenated onto ";&#" then rotated so the ";" appears at the end (1! is cute). I would probably write this differently, because: 1!";&#",$_ic is shorter.

We then have a function that does the reverse, cutting off the first three characters after rotating (which is the ";&#" string again) and converts the octal digits back into decimal. This is probably wrong because real XML documents will probably prefer decimal entities, but perhaps the author wasn't dealing with these. I would certainly write this differently if I changed oc (as above).

Now we have the helper function xc and cx (whose names suggest they are converting from character-to-xml and xml-to-character respectively). This is a stylistic observation, we can also see this from the comment, or by reading the code (if we know what XML is). These implementations are pretty basic, just using ssr to do repeated search/replace on the entities (note that ssr knows that ? character means any).

You get used to it.

> Why is this line noise considered acceptable?

One major challenge reading inscrutable perl scripts is knowing where the execution begins. Perl just has so many rules for parsing it you really need either wizardry or patience to know how to pull it apart, but K is extremely regular: there's only one way to parse it, and shortly after learning it you also learn (quickly) you can insert trace statements that don't change the meaning of the rest of the statement to learn a new operator (or a new use of an operator you didn't know). I note this especially as it is extremely hard to do in perl (and even other Iverson languages including APL and J).

Line noise is a subjective quality that goes away (at least in this case) when you become more fluent in K. I don't believe this is necessarily true of all compact languages though.

It seems like K would be a particularly suitable language for having more than one syntax. The short syntax, once you get used to it, would be better for keyboard input and expert whiteboard discussions, but it might also be nice if there were also a standard syntax that was longer and closer to what most people expect? An editor could automatically translate between short and long syntax, and this would be helpful for making sure you typed what you think you did.

But whilst the k/q operators are certainly useful, the Key thing is the notation. The notation is really valuable, and it seems hard to get it until you understand the notation well enough that it starts changing how you think about programming: numpy.matmul(x,y) might do the same thing that x+.y does, but the latter suggests more. I recommend reading Iverson's paper[2] on the subject, although you might find reading §5.2 before the beginning to be helpful in putting into context what exactly is meant by notation here.

[1]: There's a lot missing still. Good tables, efficient on-disk representation of data, IPC, views, and others-- all of which will be hard to do in Python without limiting yourself to a subset of Python that might not feel like Python anymore anyway.

[2]: http://www.eecg.toronto.edu/~jzhu/csc326/readings/iverson.pd...

I've read a lot of human oriented, commented code which meant nothing to me, because the overall state space / architecture was fuzzy.

Whether it's a oneliner or a framework, readability is quite secondary IMO

ps: this also connect to the mathematician views about naming vs structure.. names are mostly arbitrary, it's the structure that drives the logic and the device.

    $_ =~ m/^                                     # anchor at beginning of line
            The\ quick\ (?<adjective>\w+)\ fox    # fox adjective
            \ (?<verb>\w+)\ over                  # fox action verb
            ...

While I can understand how that code can be intimidating to a programmer with a more “traditional” background… there are 26 (non-empty) lines of comments, and only 15 lines of code.

In this context, "intimidating" and "less tradational" are euphemisms that stem from cognitive dissonance reduction.

/ xml from char

You don't have to like it, but the existence of K, et. al. shows that most of us are wasting a huge amount of time and energy using bad tools. These things are so powerful and not that hard to learn.

This little system runs circles around entire sub-industries of other software. The fact that it exists and uses one or two orders of magnitude less time and code than other systems is significant.

It's like axes vs. chainsaws. If the job is to log a forest the latter will be better than the former.

If people can make working, maintainable software other ways, particularly if those other ways have large, established ecosystems of dependencies, then they are likely doing it right regardless of how much you like apl.

Less code -> fewer bugs, all other things equal, eh?

> large, established ecosystems of dependencies

...are actually a symptom of failure to refactor. Ideally, over time, you code base approaches the Kolmogorov complexity of the problems it solves.

> If people can make working, maintainable software other ways ...

...but it takes 10x-100x more code, RAM, CPU power, developer time, etc. than if you hadda used K, then you're leaving money on the table.

The example with a for-loop for summing a range is a blatant strawman. In which modern language would that be idiomatic?

Furthermore the examples only show a particular use case: Processing lists of numbers. How does the benefits measure up for all the other stuff a million-line program does?

That said, if you really have a million-line program doing mostly numerical processing, then I'm sure it would be a massive benefit to switch to a programming language optimized for this task.

Golang?

Oh, you said modern. Never mind.

It adds up! Consider something like this:

    (defun count (list)
      (let ((hash (make-hash-table)))
        (dolist (el list)
          (incf (gethash (cadr el) hash 0) (car el)))
        (let (result)
          (maphash (lambda (key val)
                     (push (list val key) result))
             hash)
          result)))

  (defun count (pair-list)
    (let ((hash (make-hash-table)))
      (loop for (val key) in pair-list
        do (incf (gethash key hash 0) val))
      (loop for key being the hash-key of hash
        collect (list (gethash key hash) key))))

But we can play this game both ways. What's the K for (Counter(a) & Counter(b)).most_common(3)?

Counter could be #:'=: (count each group)

    c:#:'=:

Counter[x]&Counter[y] is a bit tricky to write, because while the documentation says set intersection, what they really mean is the set intersection of the keys with the lower of the values.

This is entirely clear in k; First, understand I have a common "inter" function that I use frequently (it's actually called "inter" in q):

    n:{x@&x in y}

    i:{(?,/n[!x;!y])#x&y}

most_common could be: {k!x k:y#>x} but if I'm going to want the sort (in q) I might write y#desc x which is shorter. In this I can save a character by reversing the arguments:

    m:{k!y k:x#>y}

    m[3] i . c'(x;y)

    from collections import Counter

All of these functions operate on data, so whether the data lives on the disk or not makes no difference to k.

That's really powerful.

In Python I need more code to get it off the disk if that's where it is.

I also may have to deal with the possibility the data might "live" in another object (like numpy) so either I trust in the (in)efficiency of the numpy-to-list conversion, or I might have to do something else (in which case really understanding how Counter[a]&Counter[b] works will be important!).

I might also struggle to make this work on a big data set in python, but k will do just fine even if you give it millions of values to eat.

These things are valuable. Now if you're interested in playing games, let's try something hard:

    "j"$`:x?`f

    q)"j"$`:x?`f
    0
    q)"j"$`:x?`g
    1
    q)"j"$`:x?`h
    2
    q)"j"$`:x?`f
    0

This is pretty much my point: any ‘reimplement this weird built-in thing in $OTHER_LANG’ is going to involve overheads unless $OTHER_LANG also has that thing. I don't think keeping most types out of prelude should be a downside; you can always ‘from myprelude import * ’ if you disagree.

> How would you do this in Python?

I don't really know what "persists the unique index of f" means, but this seems similar to shelve. I can misuse that to give the same effect as what you showed.

    with shelve.open('x') as db: db.setdefault("f", len(db))
    >>> 0
    with shelve.open('x') as db: db.setdefault("g", len(db))
    >>> 1
    with shelve.open('x') as db: db.setdefault("h", len(db))
    >>> 2
    with shelve.open('x') as db: db.setdefault("f", len(db))
    >>> 0

I don't understand what that means.

> I don't really know what "persists the unique index of f" means, but this seems similar to shelve. I can misuse that to give the same effect as what you showed.

I think you got it, but it seems like a lot of typing!

How many characters do you have to change if you want it purely in memory? In k I just write:

    `x?`f

> How many characters do you have to change if you want it purely in memory?

When the question is if ‘K can reduce line count by a factor 1000’, shorter keywords hardly cut it. ‘setdefault’ is wordier than ‘?’, but as something you're only going to use on occasion, so what? And d.setdefault(_, len(d)) is something you'll use maybe once a year.

shelve acts like a dict, so you can do the setdefault dance the same way on in-memory dicts.

I might not understand this about python.

    $ python3
    Python 3.7.6 (default, Dec 30 2019, 19:38:26) 
    [Clang 11.0.0 (clang-1100.0.33.16)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.
    >>> Counter
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    NameError: name 'Counter' is not defined

> but as something you're only going to use on occasion, so what? And d.setdefault(_, len(d)) is something you'll use maybe once a year.

If it takes that many keystrokes, you're definitely only going to use it once per year! This is the normal way you make enumerations in q/k.

The low number of source-code bytes is the major feature in k. I've written some on why this is valuable, but at the end of the day, I don't really have anything more compelling than try it you might like it, and it works for me. This is something I'm working on though.

- https://news.ycombinator.com/item?id=8476113

- https://news.ycombinator.com/item?id=10876975

My comment about prelude was in response to you saying I was using Python's “language+plus-its-entire-ecosystem”. This is not true; Counter is part of the standard library that comes with the language. It is not from a third-party package.

> If it takes that many keystrokes, you're definitely only going to use it once per year! This is the normal way you make enumerations in q/k.

You don't use this frequently in other languages because you almost never care about the index of a value in a hash table (because it's ridiculous), and you almost never want to brute-force unique a list (because it's a footgun).

I'm not sure I agree "third-party package" is a good/useful place to draw the line, but I don't think it's terribly important. Sorry.

> you almost never care about the index of a value in a hash table (because it's ridiculous)

So I want to query all of the referrer urls I've seen recently. I only see a few of them, so I want to have a table of all the URLs, and another table with one row per event (landing on my page). In SQL, you might have a foreign-key operation, but in q, I can also make the enumeration directly. I don't really care about the value of the index, just that I have one.

For temporary calculations I'd just `{u: Url(u) for u in url_strings}` for the table of URLs, and stick the `Url` objects in the table of events.

I have fifty billion of these events to handle every day, and I do this on one server (that is actually doing other things as well). It is not an option to “just” do anything at this scale if I want to keep costs under control.

In Python, although you can just use shelve to store data on disk, in practice this is considered a bad idea beyond very simple cases. Valuable data wants the guarantees that real databases provide, like ACID. shelve doesn't provide this, and IIUC nor does kdb+.

So if you're handling 50 billion events a day, live, and you need these to persist, you'd use SQL or something similar. That would then ultimately determine how you add and manipulate records.

If you don't care that much if you lose the data on a crash, that's when we're talking about temporary calculations. In Python, rather than having two tables like (eg.)

URLs:

    url_id  url          count
       1    google.com    300
       2    python.org    400
       3    example.net   200

    request_id   data   url_id
        1       'spam'     2
        2       'eggs'     2
       ...
       900       'ham'     1

    urls = {
        "google.com":  Url("google.com",  300),
        "python.org":  Url("python.org",  400),
        "example.net": Url("example.net", 200),
    }

    requests = [
        Request("spam", urls["python.org"]),
        Request("eggs", urls["python.org"]),
        ...,
        Request("ham",  urls["google.com"]),
    ]

ACID is overrated. You can get atomicity, consistency, isolation and durability easily with kdb as I'll illustrate. I appreciate you won't understand everything I am saying though, so I hope you'll be able to ask a few questions and get the gist.

First, I start write my program in g.q and start a logged process:

    q g -L

    upd:{r[`u?y`url;y`metric]+:1}

I checkpoint daily:

    eod:{.Q.dd[p:.Q.dd[`:.;.z.d];`r] set r;.Q.dd[p;`u] set u;r::0#r;system"l"}

    total 24
    drwxr-xr-x  4 geocar  staff  128 11 Mar 16:59 2020.03.11
    -rw-r--r--  1 geocar  staff    8 11 Mar 16:59 g.log
    -rw-r--r--  1 geocar  staff  206 11 Mar 16:55 g.q
    -rw-r--r--  1 geocar  staff  130 11 Mar 16:59 g.qdb

    geocar@gcmba a % ls -l 2020.03.11 
    total 16
    -rw-r--r--  1 geocar  staff  120 11 Mar 16:59 r
    -rw-r--r--  1 geocar  staff   31 11 Mar 16:59 u

If I've made a mistake and something crashes, I can edit g.q and restart it, the log replays, no data is lost. If I want to do some testing, I can copy g.log off the server, and load it into my local process running on my laptop. This can be really helpful!

I can kill the process, turn off the server, add more disks in it, turn it back on, and resume the process from the last checkpoint.

You can see some of these qualities are things only databases seem to have, and it's for that reason that kdb is marketed as a database. But that's just because management has a hard time thinking of SQL as a programming language (even though it's there in the name! I can't fault them, it is a pretty horrible language), and while nobody wants to write stored procedures in SQL, that's one way to think about how your entire application is built in q.

That's basically it. There's a little bit more code to load state from the checkpoint and set up the initial days' schema for r:

    u:@[get;.Q.dd[.Q.dd[`:.;last key `:.];`u];{0#`}];
    r:([url:`u$()]; req:0#0; imp:0#0; clk:0#0; vt:0#0);

I'm so glad k isn't a "database" like that.

Calling them expensive is ironic AF. Most of them are free and open source.

Yes, but I understand you keep doing it because you don't understand this stuff very well yet.

> RocksDB would be more comparable

How do you figure that? RocksDB is not a programming language.

If you combine it with a programming language like C++, it can, with only 4x the hardware just about keep up 1/6th of my macbook air[1].

RocksDB might be more comparable to gdbm, but it's not even remotely like q or k.

[1]: And that's taking facebook's benchmarks at their word here, ignoring how utterly synthetic these benchmarks read: https://github.com/facebook/rocksdb/wiki/Performance-Benchma...

> much stronger resilience guarantees,

You're mistaken. There is no resilience guarantee offered by rocksdb. In q I can backup the checkpoints and the logs independently. It is trivial to get whatever level of resilience I want out of q just by copying regular files around. RocksDB requires more programming.

> gives you access to all your data at once

You're mistaken. This is no problem in q. All of the data is mmap'd as soon as I access it (if it isn't mmap'd already).

> Calling them expensive is ironic AF. Most of them are free and open source.

If they require 4x the servers, they're at least 4x as expensive. If it takes 20 days to implement instead of 5 minutes, then it's over 5000x as expensive.

No, calling that "free" is what's ironic, and believing it is moronic.

I'm comparing to the code you showed. You're using the file system to dump static rows of data. All your data munging is on memory-sized blocks at program-level. Key-value stores are the comparable database for that.

> You're mistaken. This is no problem in q. All of the data is mmap'd as soon as I access it (if it isn't mmap'd already).

Yes, because you're working on the tail end of small, immutable data tables, rather than an actually sizable database with elements of heterogeneous sizes.

> In q I can backup the checkpoints and the logs independently. It is trivial to get whatever level of resilience I want out of q just by copying regular files around.

Yes, because you don't want much resilience.

---

What you're doing here is incredibly simplistic. It's not proper resiliency, it's not scalable to more complex problems, and it's not scalable to larger workloads. An mmap'ed table and an actual database are different things.

It works fine for you, but for many other people it's not.

That's what MySQL, PostgreSQL, SQL Server, and Oracle all do. They write to a logfile (called the "write ahead log") then periodically (and concurrently) process it into working sets that are checkpointed (checked) in much the same way. It's a lot slower because they don't know what is actually important in the statement except what they can deduce from analysis. Whilst that analysis is slow, they do this so that structural concerns can be handed off to a data expert (often called a DBA), since most programmers have no fucking clue how to work with data.

That can work for small data, but it doesn't scale past around the 5bn inserts/day mark currently, without some very special processing strategies, and even then, you don't get close to 50bn.

> All your data munging is on memory-sized blocks at program-level.

That is literally all a computer can do. If you think otherwise, I think you need a more remedial education than the one I've been providing.

> What you're doing here is incredibly simplistic. It's not proper resiliency, it's not scalable to more complex problems, and it's not scalable to larger workloads. An mmap'ed table and an actual database are different things.

Yes, except for everything you said, nothing you said is true in the way that you meant it.

Google.com does not query a "database" that looks any different from the one I'm describing; Bigtable was based on Arthur's work. So was Apache's Kafaka and Amazon's Kinesis. Stream processing is undoubtedly the future, but it started here:

https://pdfs.semanticscholar.org/15ec/7dd999f291a38b3e7f455f...

Not only does this strategy get used for some of the hardest problems and biggest workloads, it's quite possibly the only strategy that can be used for some of these problems.

Resiliency? Simplistic? I'm not even sure you know what those words mean. Putting "proper" in front of it is just weasel words...

I’ve personally found 20-30x compared to Java - admittedly not the most compact language.

I assume you are comparing this to deciphering and maintaining K-code written by developers who uses all the idioms just a bit wrong?

In a 'code golf' language where you reduce a 1 million line program to 1000 lines, the 1000 line program is going to be just as hard to read. More concise syntax doesn't help as the same number and complexity of concepts are going to exist in both programs, and that's what the human has to understand.

There are languages where more powerful concepts are available for less code. Erlang for instance, with concurrency. In those cases, yes, readability is aided and maintainability is improved.

[1] https://www.sciencemag.org/news/2019/09/human-speech-may-hav...

Claim B: The intelligibility of a computer program is proportional to its complexity of the problem it solves, not to the number of characters in its source code.

First off, I don't see how B follows from A.

Secondly, even if there does seem to be some similarity, one of the reasons any comparison breaks down is that human languages are evolutionarily adapted to our cognitive capacities, whereas programming languages are designed, for the most part without a very rigorous understanding of how they interact with our cognitive capacities (this is an active area of research, but not very advanced, I think.)

Finally, I reject claim B. The intelligibility of a computer program is heavily affected by how clearly the underlying concepts that define a solution to the given problem are mapped into the structures available in the given programming language, and that is obviously heavily affected by the choice of programming language. And I believe that in general more concise languages are more concise precisely because they make available more and more straight-forward mappings from problem-space to language-structure space. So more concise languages make it possible to write more easily comprehensible programs once you are over the barrier of holding all the mappings they make available in your head.

> The intelligibility of a computer program is heavily affected by how clearly the underlying concepts that define a solution to the given problem are mapped into the structures available in the given programming language, and that is obviously heavily affected by the choice of programming language.

I don't think so. Nobody uses "just" a programming language. They use a generic programming language plus libraries (or DSLs) plus their own code that maps the concepts of their problem to code. Unless the problem to be solved is really generic and/or simple, a programming language alone is very very far of having direct mappings from problem to language.

And you can't escape the complexity of the problem at hand. You can hide it, but in this regard there's no difference between hiding it behind a compiler, behind libraries or behind your own code. You still need to understand the problem and the underlying concepts.

That's what they're designed for, but why does that mean they aren't also evolutionarily adapted to our cognitive capabilities?

When my assumptions about the underlying problem are wrong a million line program is much easier to fix than a thousand line one.

Or rather, a thousand line concise program in a quirky language will become a million line program in a quirky language when exposed to changing business requirements.

>Languages that are often derided as write-only include APL, Dynamic debugging technique (DDT), Perl,[2] Forth, Text Editor and Corrector (TECO),[3] Mathematica, IGOR Pro and regular expression syntax used in various languages.

As a thought experiment, would there be benefit to letting functions only be written once?

No-one could come along and break code by changing a function. If you wanted to fix a bug in a function you would have to write a new copy and explicitly update callers to use the new version.

A lot of maintenance overhead? Possibly, but tooling would take care of the majority of it. It would be useful to be explicitly know not just when a function has changed but when the functions it calls have changed.

You would need a naming convention, instead of Main you would need something like Main#23145. It would tick over very regularly.

Library functions like CalculateTax#12 would tick over less frequently.

By calling the old revision you could call the old code if you didn't want to update a module or function.

Perhaps you could extend that name to be something like Namespace.Name#IndirectRevision.DirectRevision.Hash which would allow tools to more quickly extract when the change was made in the logic of the function itself or when the change was in dependents.

This would be an alternative strategy to maintenance to IoC. Instead of treating dependents like they don't affect the code being executed, we can control instead the version of code

By forcing code change in all callers, you build up an explicit picture of where bugs happen but more important see also at the impact of those bugs on other places.

It would need great tooling to get over the paradigm shift of moving away from IoC but I think it would be interesting.

[1] https://www.unisonweb.org/

> If you wanted to fix a bug in a function you would have to write a new copy and explicitly update callers to use the new version.

Finding all callers is the hard part though! And in modern languages, that need to be addressed both at compile (or "compile") and run time.

Assuming you've got all the call sites with what constraints you were given and have imposed, the second part, where you call a specific version of a function, is where it gets tricky.

Is the bug in the calling code or the called code? because huge software maintenance is obtuse. The original programmers have all long since left, so what we're left with is a smattering of random specific version calls, with only the tiniest of tidbits of history, locked away in someone's email, and saved in the previous ticketing system.

So it sounds interesting, but I worry (though I worry about a great many things), that the programmer three years later, ends up coming across calls to three different versions of some code, with no guidance on which ones can be upgraded, should be upgraded, and must not be upgraded. I can't say which one would involve digging up more ancient history though.

Only because we made it hard by storing code in files instead of a database.

In a genuinely write-only language, how do you update those callers? You can't, you'd have to write new ones to call your new bugfixed function. And then you'd have to write new callers of those callers...

You'd have to rewrite the whole program every time you wanted to make a change. Which would make bug-fixing a much more cerebral activity since you'd want to work out as many as possible before the rewrite to save time.

You'd absolutely need tooling to automate some of this (present a list of call sites; you select which should use the new function). It would also increase the size of your code base by a lot, although I'm not sure that's as much of a problem since we may be spending less time reading code and more just writing a new function.

We have this style in the programming world. It's called Literate Programming. How many K programmers write in that style?

More than you would think. I had a very nice gig at a hft years ago where we were using noweb+k extensively.

The only place were I edited source code by hand with paper, pen and paste.

A mathematical paper introducing some meaningful worker function uses English because mathematical notation is insufficient for describing computation. That's one of the important goals of Iverson's notation.

> We have this style in the programming world. It's called Literate Programming. How many K programmers write in that style?

Quite a few! I would actually suggest there are more literate K programmers than there are literate C programmers! Every 4-5 lines of code probably has 2-3 lines of prose in a large application that I work on, but in another language that might be something like 500 lines of code for 2-3 lines of prose which puts the prose offscreen for most of the code it's describing.

Some C programs will try to keep the prose-to-code density higher than that, but even amongst Knuth's code that's rare.

In your example, assuming that a million-line program is well done in its language, you'd have exactly the same complexity but now in a thousand lines, meaning that now each line does more, is more complex and changing it/understanding it is more difficult and prone to errors. It would be equally (if not more) impossible to understand.

If it were me, I'd choose the million-line program and keep the ability to understand parts of it and performing maintenance with just the program reference by my side, not the language reference too.

Disagreed. Reductio ad absurdum: It's not a good idea to force everyone to program in machine code, writing sequences of 1s and 0s. Why? Because the right abstractions (and associated notation) do make complexity more manageable.

Of course, assembly is far better than machine code. C is almost always better than assembly, because it maps better the mental ideas to code than assembly. After that, you have higher-level languages and then the change is not that clear. High-level languages tend to hide the complexity of dealing with the computer: in Python, JS, Haskell or K you don't need to worry about memory allocations, or how objects are defined, or how to pass arguments to functions. Sometimes you need to worry about that, and having a language that does that for you makes your program harder to understand (how does JS work with parallelism? When is an object an object and not a copy? Suddenly you need to know how does JS do certain things and that complexity reappears in your program).

Now, for the example, when I said that the million-line program is well done, I was thinking about some JS, Python, C++, or Java application without too much unnecessary cruft and that maps properly problem concepts to code concepts. If the 1000x line reduction (if that even exists) is done on the basis of smart one-liners, unreadable code and constructs and language-specific, it's not worth it. Say your application was a physics simulator of a certain situation. Most people would want the million-lines Python code where each function and each part is understandable and lets you focus on the problem at hand, instead of a thousand lines where you not only have a difficult problem but also a difficult code.

Alan Kay' VPRI's STEPS project used PEG parsers to create math-like DSLs to in their attempt to go "from the desktop to the metal in 20,000 lines of code."

      f:{11 22 33 x}
      .[f;,1;:]
    0 22
      .[f;,4;:]
    (1;"index")

      f 4
    index error
    {11 22 33 x}
     ^
    >  x
    4
    >

> If each punctuation character in the K is a separate part of speech, and you reverse their order

     +      /     !    100
     plus reduce range 100

https://en.wikipedia.org/wiki/Maxwell%27s_equations#Formulat...

You would not expect them to read them like a paragraph. You are expect to read them symbol by symbol and unpack from the inner out.

K seems to be a dense language. Expect to read it symbol by symbol, not word by word or line by line.

If will be come more readable if you (1) slow down (2) get used to the programming patterns.

Not saying K is a good language or anything. But the frustration of the author seems to have more to do with impaticence and mismatched expectations than with flaws in the language itself.

As someone who has a bunch of letters before my name because of physics, no. You don't read physics symbol by symbol. You expand those symbols to pages and pages, meditate on those pages and achieve enlightenment on one small aspect of the problem. There is so much implicit state in those equations they are basically meaningless.

A nice book that deals with all the implicit state and terrible unclear notation used in physics: https://en.wikipedia.org/wiki/Structure_and_Interpretation_o...

So for the k/q/APL/array language gurus out there: in what situations would I not want to use k or another array language, if the relative obscurity of the syntax and mental model were not a factor (e.g., assuming you, your team (if any), and anyone who would ever look at your code would be very familiar with the language and idioms)?

I may not want to choose Go or Java because of latency concerns, or Rust because of compilation speed/iteration time concerns, or C/C++ because of safety issues, or Python/Ruby/Javascript because of performance issues. Why would I not want to pick k?

Also, think about distribution. With most of the array languages, it is commercial, so I'd be cautious about building a business around it. With other languages, it is free to use and deploy as you see fit and some have binary executables that don't need a runtime. K is very expensive. Dyalog APL is pretty reasonable, but still costs money to use and royalties in certain situations.

I think any decent developer can pickup array languages, but many won't want to as they'd rather have something more mainstream (Java, JS, C++, Python...etc) on their resume. So it might be difficult to hire.

Do APL/j/k/q work well for some performance-sensitive situations but not others? Some of the comments here by people who appear to be very familiar with the language make it seem like it performs quite well compared to the traditional high-performance languages.

Distribution is definitely something I hadn't considered; I guess I've been spoiled by the variety of free/open languages available these days.

I've always wanted to try to pick up an array language to expand my horizons a bit (e.g., that one bit of advice that programmers should learn one of the four main types of languages --- ALGOL-based, Lisp, functional, array-based), but I just haven't had the time.

Arthur Whitney's a biased source, of course, but the kparc site has a bunch of little programs in which he beats the equivalents from rather influential people (like the UNIX/Plan 9/Inferno authors http://www.kparc.com/z/bell.k ) for bragging rights.

k losing to some really nicely-written C and most Fortran seems plausible, Java and C++ not so much.

K/Kdb/q seems very happy with this state of affairs, which is really strange: if they O/S'd it I could see a lot of goodwill/mindshare going their way, but I think their Morgan Stanley roots view O/S as commie nonsense. Meanwhile, it's definitely viewed as legacy in the bulge-bracket banks that use it b/c it's so hard to hire for, not to mention its limited applications for teams (kdb is NOT a good multi-user database). However elegant it may be, it's doomed to be a COBOL if this is where its main userbase stays.

J is open source, but I can't get a handle on how battle-tested it is. It always seemed like Kdb was the real product that got used in anger.

Personally I think array-heads should spend more time with Haskell, since it has the ability to be massively terse and point-free, has type inference, loves infix gobbledygook, can be tuned for great performance, and is thoroughly open-source.

It's not too hard to grok: if they freed it, they'd lose money. $200,000,000 in revenue last year from Kx alone. Goodwill compared to cash, executive picks cash every time.

I have no idea what Shakti's doing, but they, too, are probably making a substantial amount of money.

I'd think AW would be more concerned about legacy and impact than just $$$ at this point -- tradetech has long stopped being a significant source of innovation in computer science as it's essentially iterating on the same problemset from the 00s.

OTOH I suppose having a small cult has its benefits, as opposed to actually giving something to the larger world, O/S isn't exactly a cakewalk.

but for once K makes me go "oh shit!" and not because of the craziness of its syntax... rather, if that syntax is so precise/terse, and even if to the outsider it looks fucking insane, does it have a wicked lower TCO because you literally have insanely less lines of liability in your code, libraries, dependencies, etc?

Is that the reason finance uses it, they maybe empirically know what the rest of us don't seem to grasp? Or just because it's so domain specific? Or... something else entirely?

Yes. q (a language and distribution of k4) has a built-in webserver with websockets, messaging, all types can be serialised over IPC (except pointers to C code) and most data structures have an efficient on-disk representation.

You might've recalled a few days ago someone announced a specialised database for doing "have-i-been-pwned" lookups in 49µsec. I wrote[1] one line of q that is about 10x faster:

[1]: https://news.ycombinator.com/item?id=22467866

It's just a single operation, and it's built-in, so maybe you think Python could just have mmap-efficient variables and a bin operator and you'd be there, but those things don't exist in Python. If I had the same problem as the author, it certainly wouldn't materialise as "custom database" of any kind, because I'd already be done.

In fact, one of the biggest challenges with "big data" is choosing an efficient representation. The author spent a lot of time on this, but my language just had one built-in. How much time? What's the target? I mean, if I've already paid for q it's clearly a no-brainer, but q isn't free either (even if you get it for no cost: you still need to learn it!)

> Is that the reason finance uses it, they maybe empirically know what the rest of us don't seem to grasp? Or just because it's so domain specific? Or... something else entirely?

Back in the day, k was the only thing you could put tick data into. Now there's lots of systems, and computers are a lot faster, so quants can use Python. Their code doesn't need to be "fast", but "fast enough".

My code still needs to be fast though.

There are non-terse APLs out there; the most obvious one floating around is Nial[1]. Culturally though, once you've figure out how to read and write in the compressed ascii runes, it's kind of hard to go back to the other way. J is actually denser because of the hooks, forks[2], rank and full tacit aspect of things. Personally whenever I've composed an algorithm in J I feel like I really understand it in a way that pretty much nothing else gives me with all the extra typing.

[1] https://github.com/danlm/qnial7

[2] https://crypto.stanford.edu/~blynn/c/apl.html

Have you looked at q?

numpy has a lot of k/APLish primitives built in.

fml was also interesting but never made it past the spec stage:

https://www.reddit.com/r/apljk/comments/etpbbf/fml_an_optimi...

I personally find the given example harder to read than the equivalent in python/matlab/javascript with lodash/probably many other languages

sum(range(10))

sum(1:10)

With "K", you have to know a few new conventions. In the second case you just have to read, and the data flow is more obvious.

http://www.kparc.com/z/comp.k

https://web.archive.org/web/http://shootout.alioth.debian.or...

Note that the relevant k program here is contained entirely on the last line of code and denoted by a comment telling you what it does:

http://www.kparc.com/z/fun.k

http://c2.com/cgi-bin/wiki?WardNumberInManyProgrammingLangua...

Most Joy people are also k people, and Joy appears to be the only one similar, although still very large.

The C++ program is lost to link rot now, but by the author's own admission, was substantially larger than even the substantially large examples within that page.

      1    2   29 k
     23   70  498 scheme
     25  111  768 otherhaskell
     44  135 1096 erlang
     53  125 1043 rb
     67  190 1419 ocaml
     74  346 1693 haskell
     80  357 2586 java

These are just some basic examples, though.

The Scheme and Haskell versions are actually shorter than you imply since they include the pairings in the count (when the pairings in the code are examples). Additionally, the k version is one line longer than you suggest. The definition of a is on its own line is used in the solution. Still only 2 lines, but if you want to persuade you should probably have correct numbers.

k mandelbrot line count: 1

c++ mandelbrot line count: 148

Java line count: 157

Modern PL example (found on the modern shootout page, which I just found out they changed the link on):

Rust mandelbrot line count: 116

Ada mandelbrot line count: 532

And that's not even the one where k comes off the best.

Even the word “product” would introduce certain ambiguity in context of real world codebase: is it “math” product or some “domain” product?

It’s even worse with most other words.

So if in any given codebase you still have to know the specific meaning of every word, how is it different from having to know specific meaning of every K symbol?

So you never encountered namespaces then? Because that's why you use namespaces. The number of symbols is limited, which is the reason why the whole "a symbol is more explicit"-argument simply doesn't hold in general. It breaks apart pretty quickly. That's why you'll find the following sentence pretty much verbatim in most maths and cs publications: "Throughout this paper, unless otherwise specified, we use the following notations," followed by half a page of definitions. Plot twist: different authors often use different notation even within the same field of research.

Sum types, checksums, …

https://norvig.com/python-lisp.html

https://people.csail.mit.edu/gregs/ll1-discuss-archive-html/...

So by that estimate, if you switch to Python you're another halfway to Lisp (three quarters of the way from C++).

Python, maybe.

What makes Java nearer to Lisp than C++?

Mostly the Java runtime, the JVM. C++ basically comes with some kind of manual memory management and raw data layout.

Lisp comes with Garbage Collection and managed memory. In Lisp one passes references to arrays and other objects around. The runtime tracks which objects are of what dynamic type and of what size. The garbage collection may be a simple mark&sweep or a more complex, say, generational GC. The first garbage collection was developed for Lisp around 1960 and has then slowly spread to other languages. C++ was object-oriented, but with a more traditional non-GC approach. Java changed that.

the closest i got to finding some pattern is this:

  >>> xs
  '34210'
  >>> grade(xs)
  [4, 3, 2, 0, 1]
  >>> grade(grade(xs))
  [3, 4, 2, 1, 0] # :O

  grade = lambda xs: (
     [i for (i,_) in sorted(enumerate(xs), key=swap)]
  )
  swap = lambda p: (p[1], p[0])

(i guess "ordinal" really is too ambiguous...)

EDIT

alright, i see it now - `<<xs` is "for each item x of xs, where does x land when you sort xs?

    x: 10?A:"abcdefghijklmnopqrstuvwxyz"
     x
    "ysyselacwl"
     +`x`y`z!(x;<x;<<x)
    x   y z
    --- - -
    "y" 6 8
    "s" 7 5
    "y" 4 9
    "s" 5 6
    "e" 9 2
    "l" 1 3
    "a" 3 0
    "c" 8 1
    "w" 0 7
    "l" 2 4

    x:“baced“
    <x
  1 0 2 4 3
    <<x
  1 0 2 4 3

What does column y mean? If I sort the original with it, I get “wllaysysce”.

Has this changed between versions of K?

  xs = "cab"

  <xs = [1,2,0] =
  {
    0 → 1
    1 → 2
    2 → 0
  }
  
  <<xs = [2,0,1] =
  {
    0 → 2
    1 → 0
    2 → 1
  }

One interesting philosophical difference, at least among some APL programmers, is that building abstractions should be avoided. TFA has a hint of that philosophy, in its suggestion that perhaps naming (the root of abstraction) hinders clarity. I think this is definitely worth considering, and it doesn't really have anything to do with the (supposedly) cryptic syntax.

One reason I don't like K/APL-ish vector programming is performance. I once spent some time working with others on a GPU-targeting APL compiler, and I found that some common programming patterns are quite antithetical to good performance. In particular, the use of "nested arrays" (not the same as multidimensional arrays) induces pointer structures that are difficult to do anything with. Such nested arrays are typically necessary when you want to do the equivalent of a "map" operation that does not apply to each of the scalars at the bottom, but perhaps merely the rows of a matrix. Thus, control flow and data are conflated. This is fine conceptually, but makes it difficult to generate high-performance code.

Another concern is that encoding control flow as data requires a lot of memory traffic (unless you have a Sufficiently Smart Compiler; much smarter than I have ever seen). Consider computing the Mandelbrot set, which is essentially a 'while' for each of a bunch of points. An idiomatic APL implementation will often put the while loop on the outside of the loop over the point array, which means it will be written to memory for every iteration. In other languages, it would be more idiomatic to apply the 'while' loop to each point individually, which will then be able to run entirely in registers. You can also do that in APL, but it is normally not idiomatic (and sometimes awkward) to apply complex scalar functions to array elements.

Just look at this Mandelbrot implementation from Dyalog; they do it in exactly the way I described: https://www.dyalog.com/blog/2014/08/isolated-mandelbrot-set-...

Specifically, the conceptual 'while' loop has been turned into an outer 'for' loop (this is OK because the 'while' loop is always bounded anyway):

       :For cnt :In 1↓⍳256                      ⍝ loop up to 255 times (the size of our color palette)
           escaped←4<coord×+coord               ⍝ mark those that have escaped the Mandelbrot set
           r[escaped/inds]←cnt                  ⍝ set their index in the color palette
           (inds coord)←(~escaped)∘/¨inds coord ⍝ keep those that have not yet escaped
           :If 0∊⍴inds ⋄ :Leave ⋄ :EndIf        ⍝ anything left to do?
           coord←set[inds]+×⍨coord              ⍝ the core Mandelbrot computation... z←(z*2)+c
       :EndFor

https://github.com/indiscible/qmandel

And Arthur's version in k (also below):

http://www.kparc.com/z/comp.k

I should convert it to k9 (Shakti)...

    +/~^+/b*b:49{c+(-/x*x;2**/x)}/c:-1.5 -1+(2*!2#w)%w:10

Humans need abstractions to think. What programmers need are abstractions that are not black boxes. I.e. can be opened.

> the use of "nested arrays" (not the same as multidimensional arrays) induces pointer structures that are difficult to do anything with

J doesn't allow nested arrays by default; if you want to create such a structure, you have to use boxes (and then unbox the values inside of them to get at them), which makes it explicit and reduces its usage. AFAIK, k doesn't have nested arrays at all.

Most likely available through your package manager.

Free software K interpreter:

https://bitbucket.org/ngn/k/

Arthur's K interpreter isn't actually that expensive for casual use anymore (it's free for most non-commercial use and you can download it off the Shakti website pretty easily), though. Not that I'd recommend it: Arthur's a genius, but nothing's worth using proprietary software for.

What I usually recommend for learning the paradigm is J, largely because it has far better resources than anything else in the space. J's labs are wonderful, and the books Ken Iverson wrote before he died (all available for free) are great both for learning the language and also for learning many other things.

GNU APL is nice, and by no means am I trying to diss it: it's just not the best for learning, in my opinion. Its info pages are really nice though, like most GNU info pages.