If you know C at the level of K&R, a good start would be learning from Bryant and O'Hallaron's book: http://csapp.cs.cmu.edu/

From there, you would be well served by working through Hennessy and Patterson's Computer Architecture and Computer Organization and Design.

In parallel, you could work through a book on Operating Systems, maybe even playing with the code for a toy system (e.g., http://pages.cs.wisc.edu/~remzi/OSTEP/ or https://pdos.csail.mit.edu/6.828/2016/xv6.html).

An alternative or perhaps parallel approach would be to play the assembly games put out by Zachtronics.

A simple thing is to figure out if you can restrict the runtime environment during development (available memory, clock speed, network speed, etc), and develop to that. The nice side effect is there's more of your computer available for IDE's and browser tabs.

Secondly, abstraction doesn't necessarily track with efficiency/inefficiency.

Efficient programming is all about knowing how to use CPU features and memory to your advantage. If you don't already know C, I suggest "Learn C the Hard Way"[0]. After that, here's two great places to start: "What Every Programmer Should Know About Memory"[1] and "What's new in CPUs since the 80s and how does it affect programmers?"[2]

[0]: https://learncodethehardway.org/c/

[1]: https://people.freebsd.org/~lstewart/articles/cpumemory.pdf

[2]: https://danluu.com/new-cpu-features/

Go learn yourself some embedded, if you want to deal with extreme resource constraints, or some gamedev, if you want to know how to write performant code with enough tasty high-level architecture decisions to satisfy your abstraction needs.

"Arduino" and other small-system programming. Or get an emulator and try to write an Atari 2600 game: you have 128 bytes of RAM, don't use it all at once.

I should have been clearer: I'm mostly interested in books and articles. I like to approach things theoretically before approaching them practically.

(Nonetheless, thanks for your suggestion!)

Try "Racing the Beam"; a neat book describing the programming challenges with writing software for the Atari 2600.

https://www.amazon.com/Racing-Beam-Computer-Platform-Studies...

Though, instead of reading, you really should just get over your complex and try writing some efficient code.

>get over your complex

It's not a complex. I'm always very surprised by this attitude. Theory serves practice.

Case and point: the OP mentions that an understanding of cache architecture enables you to reason about which data-structures exhibit good cache locality. Discovering that hashtables have poor cache locality by trial-and-error seems like a waste of time compared to gaining theoretical insights.

I'll second "Racing the Beam", although it's a history rather than a theoretical approach.

I think you may have hit a linguistic snag here; there's lots of "theory" in the classical CS sense dealing with performance from an O(n) point of view, such as Knuth's work. But for producing fast results on actual hardware you end up having to take into account lots of ugly details of the platform. Learning about cache behaviour doesn't really fit into CS so it's not the first thing people think of when you ask for theory.

"Knowledge transmitted by practitioners through written and oral culture outside of the academy": what's the word for this?

Part of the problem is that some of this stuff has a pretty short shelf life. There's a lot of optimization knowledge that was useful at one time but is now flat out wrong.

Oral culture is especially prone to this -- at least a blog post or a physical book has a date on it. You have no idea when the your co-worker's suggested optimization technique was developed.

I think 'theory' is just fine, you can't fix all communication failures by throwing moar words at it. At some point you have to make peace with the fact that not everybody uses words the same way you do.

"what's the word for this?"

I think the word (or phrase, rather) is "tribal knowledge".

tools of the trade

* from the book

secrets of the cult

I think there's a miscommunication here. By "write some efficient code" GP presumably meant the "pull" approach, in which you identify the code that's causing you performance problems and then try to make it faster, learning everything you need on the way. Rinse and repeat. It's a very efficient way to learn, and also good at teaching you those intutions that are hard to put down in writing.

That said, if anyone knows some kind of collected guide to writing efficient software, I'd be happy to learn about it, 'cause I haven't seen anything like this published.

I suppose the legendary "PC Game Programmers Encyclopedia" counts, although it is very old: http://qzx.com/pc-gpe/

There are various analyses of Doom and Quake source around which look at the techniques used there.

I am like you. Every big performance win i ever got was from changing algorithms. Sometimes i didn't write either the old or the new, but just swapped out a different implementation. A good handle on that stuff can get you a long way. And can certainly be picked up from reading. it's a lot closer to just math.

That said, spend a week with C and valgrind/cachegrind. There's a lot of theoretical stuff that is hard to get at (or is for me) without a little exposure to how the system works. a coupe hours here and there will extend your mental model to include the various layers of cache. That'll make the more esoteric stuff more accessible.

You could do worse than Steve McConnell's Code Complete - https://www.amazon.com/dp/0735619670.

I think Peter Norvig's article 'Teach Yourself Programming in Ten Years' [0], might be useful to you. One of his points is to 'Remember that there is a "computer" in "computer science"', along with examples of things to investigate.

[0] http://norvig.com/21-days.html