In my work I have the opposite experience, that latency is favoured to throughput: People write messages and expect an immediate answer, for things that are not time-pressing. A lot of things are destroying both though: Starting several things at once, so you can't concentrate on either. We spend a lot of time planning sprints, when half of the work is not planned for but arises spontaneously.

I think the problem is that nobody really thinks about what they need and how they can achieve it. If you need latency are you ready to give up throughput? If you need a high throughput, do you manage to shield the team from distractions?

As the article said:

> How much does delay cost?

Hum, nothing. Those people expect a quick answer just because they want it. If it had real costs, you wouldn't be complaining.

> How likely are we to learn something that will change our approach?

If you learned something useful about the thing you are working on from the interaction, you wouldn't be complaining.

> How likely are external forces to change our approach?

If the conversation changed the entire thing you were working on (aka: we are canceling your project, you are supposed to do some Y now), you wouldn't be complaining.

If those messages didn't fail all the tests for when latency is important, you wouldn't be complaining about them. That validates the article, instead of detracting form it.

The low latency approach is pretty much already encapsulated by the MVP approach to product design and incremental release schedule. Actually, considering the lower overhead of testing & implementation for small incremental changes vs. monolithic releases that are much harder to reverse course if there's a problem, the low-latency approach could also be the higher throughput approach.

Anyway, I am literally going to make myself one single cup of coffee now. No one in my house drinks it, so I have tried just about every method of making a single good cup of coffee. Right now, that balances towards very low latency but very low quality: instant coffee, because my tolerance for coffee latency is usually very low. (also low tolerance for cleaning up the equipment needed to make coffee-- garbage collection!) At some point the needle will swing back towards quality and I'll take the extra 5 minutes to do a pour over & deal with cleaning more stuff.

Lol, I alternate between aero-press and pour over depending on the grounds I currently have. My tolerance for latency for good coffee is high regardless of how urgent other folk are for my time! Not that I'm a snob about coffee, I have instant on hand for convenience as well, and I'll happily drink a poorly brewed cup too. It's just a nice ritual and something to optimize for fun.

If you're not very cost-conscious, you can get excellent instant coffee (e.g. Sightglass), but it'll cost you >$2.50/cup, much more than equivalent quality beans.

Yet another instance of fast/cheap/good, pick two.

I'll have to give it a try. I'm in no way willing to spend that much for coffee on a regular basis, but having a little on hand would be nice.

Right now I'm using Mount Hagen, which is a decent compromise on price/quality: it takes most of the bitter edge off compared to something like Folgers.

As a side note, instant coffee is an excellent way to make something coffee flavored, like in baking. I've even used it in small quantities to get a deeper, richer flavor in pumpernickel bread... I don't know how real bakeries do it, but it's the only way I've come close in taste to what a bakery produces.

Instant coffee is also pretty nice for making iced coffee at home, which is what I usually do with it. I prefer a coffee-flavored beverage versus actual coffee most of the time, so Nescafe Clasico and a decent creamer to take the bitter edge off works well for me. Also a lot cheaper than a daily Dunkin' run. :)

I'll have to try Mount Hagen for a change of pace sometime, though.

I also drink Mount Hagen as my midgrade instant coffee-- I tried several varieties a while back and decided it's my favorite of them. It's not mind-blowing "wow I can't believe it's instant" like the Sightglass, but it's very good.

For quick turnaround, the Keurigs and Nespressos have always been the best low volume quick turn around solution for me, but at the price point, the quality just isn't satisfactory for me. Obviously not the most eco-friendly (last I checked).

I no longer care about quick. I like good quality coffee and tend to take the time and machinery to make a cup to suit my tastes, usually espresso based, sometimes a French press. It likely borders on snobbery to an outside observer, but it's just me being picky and personal preference. I don't really care what someone else drinks or doesn't.

Yeah, keurigs seem to only choose 1 of the fast/cheap/quality "choose two" options.

It's too expensive and for my tastes, entirely too weak. We have one at work, and I tried a reusable pod with double the coffee and still wasn't a fan. That goes for most coffee though-- When I buy at Dunkin Donuts I have them add an extra shot if espresso. With instant I can actually control strength more easily than with other methods... Aeropress I would usually run the water through twice. And with french presses I can never seem to avoid getting a bunch of coffee grounds mixed it so the last few sips are extremely gritty.

As sibling said aeropress makes delicious single servings. Probably one of my favorite purchases in the 15-40 USD range.

Thanks! I've heard of matcha as some sort of tea-like thing, but never really looked into it. I'll give it a try!

Have is very much not caffeine, but in appearance matcha reminds me of kava powder. I'd recommend it as an interesting additive (and mild anti-anxiety) when I make coffee smoothies with iced instant coffee and cashew milk (I much prefer regular milk but even skim has too many calories for me, and cashew is a little creamier to my taste than almond) The kava part is a bit of an acquired tasted, but adds a nice tingly feeling once you get used to it. It just needs a lot of shaking in a shaker smoothie cup, or a blender, to mix nicely otherwise it gets clumpy. Definitely a very high-latency option.

If you like coffee with kava (never tried but read about the effects) you might also be interested in yerba mate. Specifically chimarrão, the south brazilian green non-smoked variant. It is in the same tier as coffee caffeine-wise but to me it is more multipurpose as there is a sort of mild euphoria that goes along with it. I find I can quite effectively code and learn with it, but it is also an enjoyable drink for my leisure/relaxation time.

It is a higher latency option if using the traditional container along with a thermos as one has to wait for the water to cool under 80C. It is also higher throughput in the sense that one continues to refill their container with water from the thermos a number of times until the plant matter has given its all.

This analogy maybe works if this is your first time ever making coffee, but anyone who drinks several cups of coffee a day has a process that looks very different from the sketches and doesn’t need feedback from the first cup. Ironically, I think too many problems in software development come from treating things that we do every day and on every project as though they’re unpredictable shots in the dark.

Most software projects aren't successful after spending ten minutes on them. Don't get hung up on the analogy.

> We bias these decisions towards throughput (I blame Taylorism).

I think he's ignoring another tradeoff that may be hiding which leads to a bias towards throughput.

One of my favorite lessons with regard to video is something that old school Amiga demo scene taught years ago (slightly before my time) is that standard deviation trumps frames per second in video (I'm starting to suspect this is also true for audio). This has ramifications for video pipelines. Naughty dog has blogged about how they were able to greatly increase their throughput by adding latency (especially on the cell processor). In the process they were also able to make guarantees about variance, and their 30fps was a consistent 30 vs the 60fps that they would occasionally miss. This allowed them to have lower frame rates while still looking better than most gamers would expect from such a low frame rate.

So, I would say in the context of making coffee, in real life, boiling enough water for 2 cups is definitely going to have lower variance in the finished time of the second cup.

Does it matter for a cup of coffee? Not unless you're at barista scale. But it's an important tradeoff in certain contexts. And an important one to at least be aware of as an engineer.

You make a good point about variance.

Eg. I could take the car, which is usually the fastest, but if I am unlucky there's a traffic jam and I have to wait.

Or I take the bike, which is slower on average, but there is no risk of a traffic jam.

Cute analogy, but kind of begs the question that you know how much you are going to do/want. In many situations, that 2 is unknown.

Even the example of a bus begs the question that you know how many people will take it.

Instead, I like the view that you maximize your options. If you know you will definitely want two cups of coffee, get something that makes that more automatic. If you may decide that the one cup is enough, go with what lets you change your mind easily.

For the bus, you provide the option at a regular interval so that some folks can weigh that against the cost of a car.

Throughput and latency, then, fall off in absolute significance to optionality. Note, they don't disappear. Such is the nature of all trade offs playing against each other.

> Cute analogy, but

It's like you're dismissing the article but then everything you go on to mention as your alternative view is something the article directly mentions. Maybe try reading it again with an open mind?

Optionality on even making the second cup is covered under "Optimize for latency when preferences are likely to change." Of course optionality just means that you prefer having the choice, but latency is also about when you have the information to make the choice. So the article both includes and goes beyond the idea in your comment.

I'm really mystified by your dismissive tone. Have you heard of Kent Beck? He's not coming to these ideas for the first time. Why not try to engage with and build on the ideas instead?

Apologies if it sounded dismissive. It really is a cute analogy. (Where I mean that as good.)

I agree he touched on optionality. I am just elevating it to a first class concept in the story. Latency and throughout are not hallowed facets.

Edit: I can see how my post looks like a counterpoint. I should have framed it more that I don't think he is fully capturing the important facets of his own metaphors. If you are a city planner, transit is expensive, but can allow people to live with a lower footprint at a personal level. It is not primarily a throughput versus latency decision, but a cost option. Similarly, most people will boil however much water their kettle holds, and they picked that thinking on how many people they make coffee for.

Kent Beck wrote a post that represents exactly how I feel about software development: "By emphasizing latency [of process] we get feedback sooner. Learning and adapting to external changes lead to less waste and therefore greater efficiency. Each piece is inefficient (compared to some theoretical maximum), but the whole is efficient. In my world, latency dominates. Mostly. But it depends." (4 minutes read.)

Another way to look at emphasizing latency/feedback is to "increase learning". There's a great book, Lean Software Development: An Agile Toolkit, that covers this.

The idea is that there are so many unknowns within a project, that organizations should aim to move quickly enough to uncover them sooner. A lot of these unknowns lead to constraints that have profound effects on designs, so if you delay learning them, you are more likely to have to re-do designs deeper in the process, when it is more costly. The sooner you discover them, the more likely you will have the flexibility to account for them in the design.

I strongly feel that productivity is proportional to the frequency that a feedback loop is performed.

More meetings!

You don't always necessarily need to choose one or the other.

Frequently there exists perfectly good compromise that has almost the benefit of best latency and almost the best throughput.

For example, you have a batch process to process 1 million elements to return a response.

You can either process all 1 million elements together and then return response (best throughput at the cost of waiting until everything is processed) or stream results of each one item as they are being processed (best latency but you pay additional cost per item).

But there is possibility you can process them in batches of 100? 1000? This means all the additional costs of processing items individually are amortized to something like 1/100th or 1/1000th of the original cost but you still are getting first piece of response very soon (1/1000th or 1/10000th of what you would have to wait if you processed everything in one go).

I am currently building large scale trade processing systems using those rules to stream what would traditionally be a batch process, and so far this works beautifully and applies neatly to wide range of problems.

As a worker one deals best with large companies by maximizing thru put. I always keep more than one project going and when one is blocked I do something to save state, and pick the next most urgent one to work on. (Where urgent can also mean “fun, no deps and a clear improvement towards the long term goals,” even if relatively unimportant). However for development process, latency is a total killer. Your growth equations vary smoothly between “development work returns simple interest” and “development work pays off as expinentially with time” as the interval between a change and it being fully known to be good in production drops to zero.

Put it this way. I am going to have that second cup of coffee. Making the whole pot at once pays off consistently day over day. Will this slightly different way of doing the load balancers pay off tho is less certain. It needs experimentation. And if I can turn the dial and watch the effect in real time, I learn quickly. If I wait one week for each data point, I learn slowly.

Also the happy path for my save state is just flick away from the given labeled shell/emacs in tmux. Some things need a note jotted down but then I have failed to do something earlier. Sometimes he thing to save state is just put in a syntax error where I am working; that way I can afford to forgot the task altogether.

All these pending tasks bear only a light resemblance to the project tracking entities. For that project stuff, I mostly fight to reserve bandwidth among the team to work on long term projects that are experimental (and so not always plannable in advance In detail) but with a highly desirable end state.

Like now I am working to make good terraform modules that make the common use cases of change to be simpler and more concise than without the modules. We find a good use case, do a module, iterate a few times based on how it works out, then find a other use case. I refuse to “write stories” for the future tasks in this extremely valuable work because it’s a back and forth between solution space and actual work for actual customers. I don’t need giant modules that are as complex as the raw terraform, I need to solve specific problems in the specific business.

I know the coffee thing is just an analogy to make his point, so there is not much point in arguing about it. But since I face this question every morning, I can't help but say what I think about, and it is not latency versus throughput. It's latency versus risk.

For me it is tea, and not coffee. I have an automatic kettle that will bring the water to a desired temperature and then holds it there. I also have a small ceramic tea pot I brew tea in. I often rew 2 or 3 batches (I know which when I start). I heat the entire amount of water by default, even though that takes longer to initially heat.

Sometimes I am in a hurry and want the tea sooner. Then I consider if I should heat a smaller amount of water to get started faster. I don't think I have ever done that though. The reason is risk. (1) Will it heat the water in time. The answer is yes it should, but that would be very bad if it didn't. Then my tea pot would cool down as I waited. I think that might be bad for the tea quality. To me this is an assymetric bet. (2) Will I even remember to put more water in the kettle. I am such a creature of habit. That would really mess up the tea pot temperature. (3) Will I even put the right amount of water in for the initial pot. That takes 2+ portions of water. One to preheat the tea pot, a small amount to rinse the tea, and then the portion to actually brew the tea. That would also be bad if I didn't have enough water.

Granted, a number of those items are just because of habit/momentum. If I normally heated enough only for he first batch of tea, then I might be hesitant to heat all the water ahead of time.

So for me it comes down to latency versus risk. I suspect this is a factor in "real world" decisions to.

For what it is worth, when I write software, I would say I dive in sooner and start writing something like an MVP, along the lines ofwhat the author is saying. Admittedly I plan more features than I plan on writing for the first version, kind of like giving yourself a good leve when playing pool.

Other aspects of latency (for delivering software/features):

(1) Psychological. Being able to see early signs of progress can build momentum. Momentum and morale matter. If your efforts don't feel connected with a reward, you can get discouraged. (Obviously, perseverance is a good thing, but realistically it doesn't make this a total non-issue.) Nobody wants to spend 40 years in the wilderness before entering the promised land.

(2) Political. People like to see visible results. It's good internal PR. Even if delayed delivery is more efficient, telling others in your organization "just be patient" is a hard sell, can be risky, and may require backbone. And it may require someone to stick their neck out for you. (Like a manager telling their manager, "Yeah, it does look like nothing is happening, but my team has it under control.")

(3) Competition. Lower latency can mean gaining an advantage by being first to market. Sometimes the race is close and reductions in latency would be valuable, and sometimes not. Competitors' plans are usually a big unknown, so there's lots of guessing. Excessive fear is a hazard that can lead to overemphasis on reducing latency.

I'm currently taking a small Hacker News break from my Coursera course on project management (the one made by Google!). This is oddly relevant.

When compacting timelines because of a schedule deviation, you typically decide to crash or fast track. A crash is your typical nightmare of achieving more throughput (like the author mentions). A fast track is performing previously sequential tasks (high latency) in parallel (more throughput).

I realize this isn't groundbreaking information but I wanted to share what I learned of project management lingo :-)

https://www.simplilearn.com/fast-tracking-vs-crashing-articl...

PS - The course is very good. Coming from a world of consulting most of my life I thought I had a good grasp on terms, ideas, etc - but I've picked up quite a bit here, Google did a great job with the course.

If the two cups of coffee are for one person, I don’t think you want them at the same time, one might get cold.

I think you can start to heat again sometime during the drip phase. Start to enjoy your first cup while the 2nd cup is heating. Then the time between 2 cups being ready is shorter than option 1 and not at the same time as in option 2.

Do people really make individual cups of drip coffee? Drip coffee is perfect to make a pot with as many cups as you want.

And while the coffee is dripping, you can go do something else!

I do, yes.

For ultimately path-dependent reasons. In my storage unit, I have a Chemex, burr grinder, and thermos, so when I want drip I tend to make a 'pot'.

I was nomadic for a couple of years, and brought along a hand grinder and a Hario V60 O1. Despite being "sedentary" for almost a year and a half, this is still how I make coffee: I've been reluctant to duplicate my entire kitchen, and never got around to shipping everything I own across the ocean.

Which turns out to be a good thing, since I'm heading back to the mainland. But yeah, definitely make individual cups of drip!

I make one mug of drip coffee at a time. Know exactly how much water to put in to avoid any waste.

Is anyone else bothered by the fact that the coffee example as drawn has been contrived to make the throughput of each approach different.

In the “low latency” example, heating and dripping are sequential, when you can obviously be heating water for the second cup, while waiting for the first cup to drip. Increasing the total time by two units.

In the “high throughput” example, heating double the water only takes one third more time rather than double the time. Decreasing the total time by one time unit.

So if you eliminate these contrivances both approaches take the same amount of time. Meaning they have the same throughput, but one has lower latency to first cup. So obviously the better approach.

I appreciate the article isn’t really about the best way to make coffee, but using a broken example does detract from his point a little.

In the "optimal latency" case you can start heating the 2nd cup immediately after the first, then it's ready after 7 time units, for average of 3 time units per cup, which is better throughput than the "optimal throughput" case (3.25 t/c).

Interesting that he didn't consider the speed of his consumers. Presumably a single consumer won't drink 2 cups of coffee at the same time, and the 2nd cup will grow cold. I'm sure there's something to his software analogy there too.

Interesting. As a single consumer,I know I'm going to consume two or more cups. So I make it all at once and keep it on a warmer so it's not cold when I pour my later cups. In the past when releases contained several features it was overwhelming for our users to use them all and consume at once. It led to support needing to also know and support all the new things right away and led to shallower knowledge, support, and adoption. We found it was better to focus on one feature at a time and per release. It also smoothed out our demand curves and resource needs.

I have:

An espresso machine

A bonavita temperature controlled kettle

A baratza vario

V60 and chemex

I have the espresso machine on a timer for 30 minutes before I wake up. The bonavita has temperature hold.

Depending on the day and my wife, I’ll do pour over or espresso. In both cases we are only talking about ~2 minutes of actual time because the setup is optimized and I don’t need it the second I wake up (If I did, I’d do espresso). If it’s pour over, I push two buttons then do a few other things in the morning.

A double boiler espresso machine would be the king for optimizing for both, having the most stable temps for any task.

Whet it comes to networks, this is a thought tradeoff because we can have the same latency even though the throughput (mbit/s) is high. Also on modern consumer CPU's it will take some time to throttle up. Also JIT compilers can do different things depending on load. It seems that every step in the hardware/software stack is optimized for throughput.

I found it very difficult to read this as it started out with the whole 'I made a bad analogy about coffee making that nobody understood, and actually is probably not realistic but anyway now let me tell you what my point is referring back to this bad analogy periodically'

The coffee-analogy was such a perfect example for wasting time on irrelevant optimization that I didn’t read on.

Even if the authors’ perspective in the rest the article may have been more useful, spending my time with other things felt like an optimization.

Where I work often latency is prioritized at the expense of throughput. People tend to amplify minor issues which left untouched resolve themselves. Just by leaving the respective task untouched, it leads to increased throughput.

Isn’t the latency for two cups in parallel shorter than the latency for two cups serially?

Seems like a too simplistic analogy to me.

Probably. But you’d need two burners and two pots. Not exactly equivalent.

I'm referring to the example in the article. It puts double the amount of water in the kettle, and uses two drips and pouring and dripping are interleaved. The pouring stage is parallelized.

Some of the same ideas, like cost of delay, are presented in The Principles of Product Development Flow.

What’s wrong with brewing twice as much and then pouring it into two cups?

related: "Theory of constraints"

https://en.wikipedia.org/wiki/Theory_of_constraints

Kind of ignores context switching overhead.

Medium is blocked in Egypt, no idea why.

Parallelism of software effort is very very expensive not just in terms of people but in terms of communication. Unless you mean non-brute force, where you split the problem into complete awesome abstractions so team a need not talk to team b and developer a need not worry about the big picture, but just this nice little problem. But that takes a bit of higher latency where you have to walk around and think in a relaxed and open fashion.

10 coffee pots aren't cheap either. It's not a free lunch, just the answer to the question "how do I get both low latency and high throughput".

Author is not aware most home drip coffee makers have a spring valve under the filter basket. You can put in two cups, pour one cup when it's ready, replace the carafe and continue making the second cup.

The problem is coffee brew is not linear. The first cup will be a much stronger coffee than intended and the last cup will be naturally brown tinted water.

Been there, tried that, failed miserably.

French press coffee and drip are so different though!

French press has a rich mouth feel, but it can be kind of muddy. Including literally, the last sip generally must be left in the mug. It's good for a medium-grind French roast; a nice French press Kona can be sublime.

Drip is brighter, cleaner, and the flavors of the coffee really come through. You can get even sharper separation of notes with an Aeropress "Americano" style, but that is decidedly thin.

Which can be nice, but to my taste, most medium and light roast coffees really shine from a Chemex. You can get good results with Hario filters but the filters themselves aren't as good and you kind of have to fiddle with it. Fine ground, but not espresso fine.

Sure, I wouldn't use a machine, my goodness. But that's mostly because I'm a snob, a coffee which would be better in a Chemex will also be better from a (clean) drip machine than from a French press.

Something that other people haven't mentioned yet either is just how much more oily French press coffee is, I think for many people trying to drinking multiple French press coffees per day will result in heartburn and an unsettled stomach.

Personally filter tastes much cleaner. I still like French press, I just disagree that it's somehow better than well made filter coffee.

The article about latency vs throughout and is only incidentally about coffee and even less so about the method of making it.

Pain in the ass to clean a French press. I prefer my Bunn.

That, or buy a nicer coffee maker.