Once I rented a car in Seattle one way to Vancouver. Apparently there is some shitburg nobody has ever heard of called Vancouver, WA but I was going to the more famous Canadian city. The flunkie at the counter booked the domestic trip and I drove to the proper Vancouver. When I got there they tried to charge me over $8000 in fees. I think there is no practical upper bound to what they'll try to charge you for such things.
Please keep regional slurs off this site. Imagine if you were one of HN's readers in Vancouver WA and read this comment. Not nice, and quite unnecessary for your point. Ditto for "the flunkie at the counter". We don't need comments here that think they're that much better than other people.
You shouldn't project your obliviousness on everyone else.
Vancouver, WA is a suburb of and part of the greater Portland area. Also referred to as the Portland / Vancouver metropolitan area.
It's completely understandable that the "flunkie" at the counter assumed someone leaving from the largest city in Washington would be be referring to the 4th largest city in Washington (Vancouver).
When you consider that's 12,000 for the least energy intensive part, just pumping the water, yes. A hell of a lot.
How many extra windmills for desalination and freezing to produce snow? How many acres or square miles of desalination plant? How to dispose of the few(!) tonnes of salt? How much of the world's shipping, construction and engineering requires commandeering, WW2 style to ensure it's a ten year not fifty year project?
The problem is that we need it to just barely break even and then a little bit more. Having a tremendously energetic fusion device is both useless and a solved problem: we already know how to light one off, but it only lasts a millisecond and it destroys everything for miles around.
For what it is worth, fission also suffers this problem. A promptly critical fission reaction is possible, indeed quite easy to achieve, and very undesirable. All fission technology is dedicated to keeping the reaction just barely critical.
Conservative GC is pretty much: look at all reachable memory, whatever looks like a pointer to a range which is allocated keeps that range allocated.
Pros: you don't have to care about type information and precise stack/structure walking.
Cons: If you have range 0x8000-0x8010 allocated, and have a variable with integer value 0x8001 somewhere in the memory, it will keep that range allocated. It doesn't matter that it's an int, not a pointer. Floats and 32-bit pointers have quite a lot of accidental collisions that way.
Conservative GCs don't know which bytes in memory are actually pointers, so they treat every word in memory as being a pointer if it looks like one. This means if you have some other value that happens to look like a pointer — in this case a float — the GC will think it's pointing to some other memory and keep that memory around even though it isn't used.
Thank you. That makes sense. I appreciate the answer. I camp outside that area sometime -- I'll try to get a good look with my binoculars the next time I'm there.
Oh really? Why? Is it to keep things at a nominal operating temperature in prep for the next day, or is it to continue generating electricity once the sun has gone down (or other reason)?
>Gas is burned at night to keep the system primed and to heat water used in the tower boilers. This allows electricity production to start up more quickly when the sun comes up each morning, plant operators said. Gas also is burned during periods of intermittent cloud cover.
The steam cycle has to be up and running before sunrise so they can meet their generating obligations, so they have to ramp up with natural gas while it is still dark. The facility is also capable of operating as a dispatchable "peaker" power plant.
There's a huge amount of right-wing disinformation about solar power so if you just Google for "Ivanpah natural gas" you'll get an outpouring of stupid blogs, but reading them will get you close to the truth.
Don't really agree that this list could have come about through discussions with engineers at Google, Facebook, etc. The more computers you have the less important it becomes to monitor junk like CPU and memory utilization of individual machines. Host-level CPU usage alerting can't possibly be a "must-have" if there are extremely large distributed systems operating without it.
If you've designed software where the whole service can degrade based on the CPU consumption of a single machine, that right there is your problem and no amount of alerting can help you.
I work at a FAANG and host level cpu is most definitely an alert we page on. Though a single host hitting 100% CPU isn't really a problem in and of itself (our SOP is just to replace the host), its an important sign to watch for other hosts becoming unhealthy. It might be overkill but hey theres mission critical stuff at hand.
For example: if you have a fleet of hosts handling jobs with retries, a bad job could end up being passed host to host killing each host / locking up each one as it gets passed along. And that could happen in minutes while replacing and deploying and bootstrapping a new host takes longer. So by the time your automated system detects, removes, and spins up a new host everything is on fire.
Could you mention which FAANG so I can avoid applying or a job there? Large-scale software systems _must_ be designed to serve through local resource exhaustion. If you are paging on resource exhaustion of single host you are just paying the interest on your technical debt by drawing down your SREs' quality of life.
I stand by my beef with this article. The statement that "I've talked with engineers at Google [and concluded that a thing Google wouldn't tolerate is a must-have]" doesn't make sense. What I get from this article is you can talk with engineers at Google without learning anything.
Im not at liberty right now to name my employer but our systems are definitely designed to serve through local resource exhaustion. But we aren't talking about cheap hosts here. We generally run high compute optimized or high memory optimized hosts depending on the use case and if these generally powerful hosts hit 100% CPU or full memory utiliziation theres usually more going on than something random or simple so its important to have someone check it out.
A single host stuck at 100% CPU also has a nasty effect on your tail latency, in a system with wide fanout. If a request hits 100 backend systems, and 1 of them is slow, your 99th percentile latency is going to go in the toilet.
This very much depends on the kind of software system. If there is parallel orchestration going on, such as join operators in a scale-out database, the performance of a single machine in the cluster can impact the performance of the entire cluster. In fact, the software will often monitor this itself so that it knows when and where to automatically shed load.
I'd say you should always have CPU monitored, but I get that you might night care to aggressively alert on it. It can be invaluable for hunting down root-causes after the fact: nothing's perfect from the first deployment. I single bad host is best if it crashes, but is a lot more dangerous if it's just wonky.
Things like CPU hopefully shouldn't be your key/gold service-up metric, but paradoxically, the more mature your system the more CPU can tell you; you can catch problems before they happen. It can help notice things like bad CPUs.
Memory stays pretty important in my experience; even more than CPU.
And in addition to all the other responses there are also different levels of pages: Some are page me at 5am, some can wait till morning, and some can wait till Monday. FAANG is more likely to have their own hardware so you actually get deeper/more diverse monitoring needs than a shop on AWS or something.
Outliers are where the interesting stuff happens, and outliers happen to individual instances. Aggregates are useful but can be very misleading. You can have milliseconds 99 percentile latency with ~1% of requests timing out.
I wouldn’t alert on a single machine having CPU issues, but I’m definitely interested in a small collection of individual machines all having CPU issues at the same time.
This is an incorrect statement. CPU utilization and memory matters because it limits how many other containers you can load on the same host, and means that it becomes more and more expensive to run that particular service.
> If you've designed software where the whole service can degrade based on the CPU consumption of a single machine, that right there is your problem and no amount of alerting can help you.
5G is too new for that but there was a decade-long study in which they blasted mice and rats with ridiculous EM fields similar to 2G/3G technology but at much higher power, and the results were equivocal. https://www.realclearscience.com/articles/2019/04/30/is_5g_w...
Just FYI the completion results in the omnibox have little to do with the search engine results. Clearly the search engine produces millions of hits for “Hillary Clinton emails”. The completions are a completely separate system based on what people type in the box, not what’s in the index, and it’s laser-focused on producing interactive results.
Sure, however they are not directly tied to their bread and butter - pay per click. They could over time divest and move data centers and jobs elsewhere without directly affecting revenues. Would be painful and risky but feasible.
