Use the command line to get the URLs for each chapter?
Go to conqueringthecommandline.com/book
Save the page source
Insert the filename into the script below
Alternatively, omit filename and pipe the source of the page to the script
Works with BSD sed as well as GNU sed; first lesson: portability
a=$(printf '\004')
sed 's/s3_url/'"$a"'&/' \
|tr '\004' '\012' \
|exec sed '
s/\\u0026/\&/g;
s/.*s3_url.:.//g;
s/\".*//g;
/https:.*Amz/!d;
' filename
I do not recommend this "book". Look at the hoops they make readers jump through. Try grymoire.com; much better and no Javascript required
I guess journalists read HN looking for stories? I was suprised to spitbol mentioned on HN and now there is a story on Vice news?
Alas, the interviewer asked no technical, interesting questions. There is a portable assembly language called MINIMAL that is used in spitbol. This could be a topic for an entire story itself.
IIRC there have been prior submissions of Vice stories where the author has jumped into the discussion here to clarify things, so at least one of them is familiar with the site. It doesn't seem too far fetched that HN submissions may generate the occasional interest in a story if the author happens to check in occasionally (or more).
> In a 2012 blog post describing his role as the sole contributor to the SPITBOL GitHub repository—which recently got some attention on Hacker News—Shields calls himself SPITBOL's only user as well.
My recollection is this was published on the author's site years before being added to the "handbook" on the FreeBSD website. I could be wrong. Perhaps check archive.org.
I love this software, along with k/q. I admire the work Mr. Shields has put into this project. I especially like the use of musl and provision of static binaries.
I do not use Perl, Java, Python, Javascript, Go, Rust, Closure, etc., etc. Whatever the majority of people are recommending, that is generally not what I use. It just does not appeal to me.
I guess I am stubborn and stupid: I like assembly languages, SPITBOL, k/q, and stuff written by djb. Keep it terse. Ignore the critics.
Yet this is now on the front page of HN. Maybe because it is the weekend? I really doubt that the software I like with ever become popular. But who knows? Maybe 10 years from now I will look at this post and marvel at how things turned out.
There is no "structured programming" with spitbol. No curly braces. Gotos get the job done. Personally, I do not mind gotos. It feels closer to the reality of how a computer operates.
Would be nice if spitbol was ported to BSD in additon to Linux and OSX. As with k/q I settle for Linux emulation under BSD.
Fascinating, I'm curious, what do you do with such languages and tools? I'm not trolling or trying to start a flame war, I'm genuinely curious as to the best use of these tools. Also - are you doing it as a hobbit, or commercially (or both)? How did you discover this tech?
> It feels closer to the reality of how a computer operates.
In an alternate reality, high-level languages would be wired directly into our "hardware", via microcode or FPGA's or what have you. Software systems would be designed first, then the circuitry. In this alternate reality, Intel did not monopolize decades doubling down on clock speed so that we wouldn't have time to notice the von Neumann bottleneck. Apologies to Alan Kay. [0]
We should look at the "bloat" needed to implement higher-level languages as a downside of the architecture, not of the languages. The model of computing that we've inherited is just one model, and while it may be conceptually close to an abstract Turing machine, it's very far from most things that we actually do. We should not romanticize instruction sets; they are an implementation detail.
I'm with you in the spirit of minimalism. But that's the point: if hardware vendors were not so monomaniacally focused on their way of doing things, we might not need so many adapter layers, and the pain that goes with them.
Don't we have cases of this alternate reality in our own reality? Quoting from the Wikipedia article on the Alpha processor:
> Another study was started to see if a new RISC architecture could be defined that could directly support the VMS operating system. The new design used most of the basic PRISM concepts, but was re-tuned to allow VMS and VMS programs to run at reasonable speed with no conversion at all.
That sounds like designing the software system first, then the circuitry.
Further, I remember reading an article about how the Alpha was also tuned to make C (or was it C++?) code faster, using a large, existing code base.
It's not on-the-fly optimization, via microcode or FPGA, but it is a 'or what have you', no?
In general, and I know little about hardware design, isn't your proposed method worse than software/hardware codesign, which has been around for decades? That is, a feature of a high-level language might be very expensive to implement in hardware, while a slightly different language, with equal expressive power, be much easier. Using your method, there's no way for that feedback to influence the high-level design.
I just wanted to thank you (belatedly) for a thoughtful reply. The truth is, I don't know anything about hardware and have just been on an Alan Kay binge. But Alan Kay is a researcher and doesn't seem to care as much about commodity hardware, which I do. So I don't mean to propose that an entire high-level language (even Lisp) be baked into the hardware. But I do think that we could use some higher-level primitives -- the kind that tend to get implemented by nearly all languages. Or even something like "worlds" [0], which as David Nolen notes [1, 2] is closely related to persistent data structures.
Basically (again, knowing nothing about this), I assume that there's a better balance to be struck between the things that hardware vendors have already mastered (viz, pipelines and caches) and the things that compilers and runtimes work strenuously to simulate on those platforms (garbage collection, abstractions of any kind, etc).
My naive take is that this whole "pivot" from clock speed to more cores is just a way of buying time. This quad-core laptop rarely uses more than one core. It's very noticeable when a program is actually parallelized (because I track the CPU usage obsessively). So there's obviously a huge gap between the concurrency primitives afforded by the hardware and those used by the software. Still, I think that they will meet in the middle, and it'll be something less "incremental" than multicore, which is just more-of-the-same.
Exactly. If the world had standardised on something like the Reduceron [1] instead, what we currently consider "low-level" languages would probably look rather alien.
> There is no "structured programming" with spitbol. No curly braces. Gotos get the job done. Personally, I do not mind gotos. It feels closer to the reality of how a computer operates.
That's because it is closer, as I'm sure you know, since you stated your fondness for assembly languages. I even like them for specific, limited tasks (advanced loop control). That said, I think preferring them over more "modern" constructs such as if/while/for is sort of like disparaging all those new gas powered carriages, because you can get around just fine with your horse to power your carriage, thankyouverymuch. There are very good reasons to approach most uses of goto with skepticism.
I don't think that's a good metaphor. There are no actual horses inside any gasoline (or electric) engine. But there are gotos behind many (though not all) of these modern constructs...
There's actually a lot more implied by the metaphor than just goto and constructs built upon it. It's about the art and science of programming, and advancements in the field. I specifically didn't say car or automobile because I wanted to evoke the feeling that the "new" thing being shunned is actually itself far behind the current state of the art. For loops and if blocks aren't very new and shiny either. You know what is (for some relative value of "new" that includes coming back into prominence or finally gaining some traction)? Static code analysis. Typing concepts beyond what C offered. IDEs and tooling infrastructures to assist development. Languages that support formal proofs.
Goto is essential, it's the glue that holds the instruction set together. That said, we must not fetishize it, just as we must not fetishize items of the past that are largely superseded by what they helped create. To do so slows us down, and we fail to achieve what we otherwise could. We must not forget them either, they have their places, and to do so would also slow us down.
> But there are gotos behind many (though not all) of these modern constructs...
I'd argue that e.g. an x86 LOOP instruction is far more equivalent to a do/while loop than a goto. Most of the jump instructions I see in my disassembly aren't unconditional like goto is - if anything, car engines are closer to horses in what they accomplish than, say, jnz is to goto! Even jmp certainly doesn't used named labels, as any goto worth it's salt will use - instead you'll see absolute or relative address offsets.
>> Personally, I do not mind gotos. It feels closer to the reality of how a computer operates.
There's a time and place to get close to the hardware, but I've never felt that goto got me meaningfully closer. Of course, my first and primary exposure to GOTO was in BASIC - where it was interpreted.
You want to get close to the hardware? Play with intrinsics. Open up the disassembly and poke at it with a profiler. Find out if your algorithm manages to execute in mostly L1 cache, or if it's spilling all the way out into RAM fetches. Figure out where you need to prefetch, where you're stalling, where your cycles are being spent. Diagnose and fix some false sharing. Write your own (dis)assembler - did you know there's typically no actual nop instruction? You simply emit e.g. xchg eax, eax, which happens to do nothing of note, and label it "nop" for clarity.
IMO, you'll have more time to do these things when embracing the advantages that structured programming can provide. Of course, I may be speaking to the choir, at least on that last point.
NOP is most certainly a NOP on modern x86 CPUs. Yes, the encoding matches what would be XCHG EAX,EAX (or AX,AX or RAX,RAX) but it hasn't been that for quite some time as it could create a pipeline stall waiting for [RE]AX to be ready for the following instruction.
As for JNE not being a GOTO, it most certainly is. It just so happens to only happen under certain circumstances (along with the other conditional jumps, and yes, that's how they are described). Compare:
IF X <= 45 GOTO THERE
with
CMP EAX,45
JLE THERE
Not much of a difference if you ask me. Also, the LOOP instruction is more of a FOR loop than a DO/WHILE, as the ECX register is decremented as part of the instruction.
And let me assure you, when writing assembly, you almost always use labels. A disassembly will show you the absolute/relative address because that's all it has to go by.
I have a question: In his analogy, what represents the hardware and what represents the software? Wouldn't the change from horses to combustion engine be a change in hardware? And software might be represented by something like the reins or a gas pedal?
On both sides it's technology and advancement of the status quo. More explicitly, it's programming and personal transportation.
Oh, and my rant wasn't aimed at you, per-se, but the statement about goto which I expanded in isolation to a fictional point of view. That point of view may or may not have any relation to how you feel about programming and goto, I have no idea.
Gotos were shot down more than 40 years ago and they have never really made a comeback since. They are still used for error handling in the kernel, I've seen.
Beautiful response to a recurrent misunderstanding.
"Shell languages are defined by their terseness."
It is common to see people put a layer of verbosity on top of the Bourne shell (or system(3)) to make "a new shell".
It is also very common to see people put a layer of abstraction on top of a large, verbose scripting language and claim the result to be a new, "terse" language (with all the power of the shell).
But what I like about the Bourne shell is that it is built from only C. And it does not add too much verbosity. It is, as you say, defined by its terseness.
There is also terseness in the roff-like typesetting languages, assembly languages, FORTH, k/q, etc.
Compared to k/q, sh is verbose.
For me, verbosity means loss of power and loss of time.
I am glad there are terse languages.
They may never again be popular but I think they will always exist.
"The worst parts of the shell are the flow control structures..."
For some reason I dislike if/then/else. Instead I make heavy use return values, ||, test(1) and case/esac.
I always wished Bourne's shell (cf. Joy's C shell) made use of more C operators. There's a file called arith_lex.l in the Almquist sh source but these operators are not used in the sh language.
I would switch to the Plan 9 shell but Bourne sh remains more useful due to its ubiquity.
No joke: I strongly believe that "real programmers" do indeed write in assembler. Not x86 assembler, mind you. Rather, "real programmers" first compose an abstract machine semantics for the solution domain, create an instantiation of those semantics in the form of a virtual machine on some host architecture, and then express the solution in an assembler language targeting the abstract machine semantics. (See, for example, the Prolog abstract machine semantics to represent the solution domain of expert systems.)
They might, if their problem is really hairy and requires large amounts of exploratory programming, create a high-level programming language (usually a macro-language) that allows easy access to the same abstract machine semantics, and then code in that. For well-defined "closed" problems (e.g. in-kernel packet filtering), this added baggage is usually unnecessary, but for more "engineering-oriented" problems, you might end up with something more like an Erlang than a Prolog.
Note that C itself follows this rule; the "C abstract machine" is a pretty good fit (though perhaps not perfect) to the solution domain of low-level systems programming.
Mostly that it's the simplest and oldest (and, in my opinion, the least "sandwiched in between five layers of indirection") version of a very common idea, that I'm not quite sure the name of. It's the idea behind DSLs, and Lisp-like languages. Most of Chuck Moore's writings on the potential behind Forth-like languages are also about this idea. The most modern instantiation of the idea, I believe, is "Behavior-Driven Development" with Cucumber et al. But it's all the same thing, in the end.
Generally, the idea is: first, you should define a language to directly express statements about your problem domain. Then, in one place, you can define the solution in terms of the problem domain. Completely separately to that, you can implement the mechanism by which a machine interprets and acts upon statements about the problem domain. Putting the two together results in a program that solves your problem. But either may be maintained separately.
The term "separation of concerns" has effectively always been about making sure that demands for change that come from separate "departments" don't end up touching the same line of code. This model excels in that: maintaining the specification of the solution is solely the province of the business, while maintaining the interpreter of that specification is solely the province of some engineers somewhere. As long as the interface between the two is stable—a stable ABI—improvements can be made to one or the other without the other group ever having to be aware.
A great example of this is the Inform 7 language. One group of people writes text adventures in this language, or enhancements and plugins for text-adventure development. A separate group of people maintains interactive-fiction interpreters that parse a given ISA. Neither group has to think about the other. The ecosystems of the IDE and the runtime are, in fact, completely divorced.
However I would be willing to bet you are using a bootloader and a kernel written by people who are comfortable with assembly language. Call them real programmers or something else.
I wrote my own. I can change it faster when YouTube changes something than waiting for someone else. YouTube makes downloading very straightforward. No need for Perl, Python or any interpreter except sh.
There is a way to do the xxd step using original netcat.
Included with nc was a short program called nc-data.
It converts from btoa and atob for shoveling data to nc.
Slower than xxd but still useful. It also shows octal, decimal and byte number.
Another exercise might be a one-liner for the TAICLOCK protocol: http://cr.yp.to/proto/taiclock.txt (Alternative to SNTP with smaller packets and support for leap seconds.)
For me there are generally 3 steps to the process of watching a youtube video.
1. Get the video id. Retrieve HTML containing youtube /watch?v= urls or other urls that contain the video id. Extract the urls from the HTML or other markup garbage.
2. Retrieve the video. Feed the /watch?v= url to a script that does some "find and replace" on the absurdly long googlevideo urls. Below I have given an example of such a script. Complaints welcome. It takes a /watch?v= url on stdin and retrieves the video in the format specified on the command line.
3. Play the video. ffmpeg libraries, mplayer, etc.
Whatever it is Flash does in the process of watching youtube videos (I am quite sure it is not step 3), I do not need it.
Thus even if by not using Flash or a complex "modern" web browser to watch youtube videos somehow were to reduce my exposure to vulnerabilities that routinely occur in such software, I would not care. Because the reason I do not use Flash is.... because I do not need it.
# proof of concept: video retrieval
# requirements:
# sh, sed, tr, openssl, ftp
# Adobe Flash not required
# HTML5 not required
# Python not required
# Awk not required
# web browser not required
curl=ftp
file=1.mp4 # default outfile
url=www.youtube.com # example
# itag #s are on the wikipedia page for youtube
f061(){
sed '
s,%3D,=,g;
s,%3A,:,g;
s,%2F,/,g;
s,%3F,?,g;
s/
