The Consumable Space Data Center

Posted on by
Reply

I’ll admit that I’m already shaking my head when I hear someone talking about the concept of data centers in space. You’re always going to hear the same three points trying to sell it. There is unlimited solar power, no land or water constraints, and cooling in space is free because it’s cold. Sounds good in theory but physics always wins. The costs associated with the drawbacks means that what you’re being sold is entirely different from what is being delivered.

Lots of companies are jumping in to get into space. Starcloud, Axiom, Sophia Space, Google Project Suncatcher, and even the crazy million satellite promise from SpaceX want a piece of the action. However, read through the marketing and you’ll see that these companies aren’t building data centers in space. They’re really building disposable compute modules that just happen to be in space because calling them “data centers” is where you get your funding.

The hints are there if you look closely. Coverage of India’s space aspirations talk about plans to deorbit failed modules and replace them, much like Starlink satellites or mobile phones. Other consultants have talked about servicing and unit refreshes as the real factor behind how this scales past a slick demo for investors.

Solar So Good

I will admit there is one area where this whole idea makes sense. Solar power in orbit is as abundant as can be. There’s no atmospheric attenuation. No need to worry about cloud days. And satellites don’t have to worry about night so they can get power constantly in the right orbit. Solar is also much less expensive than you might think, provided you don’t care about how long they last. Typical hardened triple junction cells that are used for things like the ISS cost about $50-$100 per watt. The cheaper commercial cell used by Starlink run about $.20-$.50 per watt. That’s a massive cost advantage. Those commercial cells are also more resistant to radiation damage than you might otherwise think. Real data from satellites show about 0.18% power loss per year due to degradation, which is less than 2% loss over a decade.

Power generation isn’t the big issue here. It’s probably the one thing that works in the favor of the people backing these ideas. But keep in mind that whole 2% power loss over a decade. We’re going to come back to it in a minute.

No Fuego

Heat dissipation is one of the biggest hurdles to clear. On Earth, data centers are cooled by liquid and/or air moving over the devices and convective cooling. You can cool something like 2,000 watts per square meter that way. It’s a very efficient way to cool things. In orbit there is no convection. There’s no air to move. You have to use radiative cooling and that’s very inefficient at the temperatures that most of these devices are going to be running at. If you don’t believe me then just look over at the vacuum tumbler that sitting on your desk. Do you know why it keeps your drinks cold? Because there’s a vacuum inside it. Vacuums don’t radiate heat well at all.

Let’s check the math though. If you want to keep your electronics running around 70 degrees F you’re going to need about 1,200 square meters of radiator surface to handle one megawatt (MW) of heat. That’s roughly four standard tennis courts. A typical data center averages between 5 and 10 MW of power consumption. While the math isn’t perfect for power usage versus heat generation it’s close enough to say that a typical data center is going to need at least one or two American football fields worth of radiative cooling to operate. And the power budgets for these things are only going up.

Companies like Sophia Space are trying to turn the entire chassis into a kind of heat exchanger with a solar cell on one side and a passive radiator on the other. They’re claiming that the typical HVAC overhead of 92% you see on Earth can be brought down to something like 8%. But when those nodes scale up your cooling needs become multiplicative with your compute scale too. No matter how you look at it you’re going to have some very big radiators in space. Half the things sticking off the ISS are radiators, not solar panels.

The Deorbit Cycle

Let’s set aside physics for a minute and talk about money. The current thinking around compute in orbit says that the satellite should be expected to last about 5-6 years before it degrades to the point where it is useless. Great timeline for writing off the cost of the unit. But the GPUs inside the satellite that are doing the hard work double in performance every two years or so. Yes, Moore’s Law is slowing down a little but every two years or so is still very much in the realm of possibility for huge performance gains.

That means a satellite operating for the full intended lifetime is going to be three generations of GPU behind in performance. And unlike a terrestrial data center where you can just go in and swap out some of the parts you’re stuck with what’s in the satellite for the life of the system, practically. So what you launch today is already behind the curve. If you want to beat the current model you have to be able to iterate faster than dirt-bound data centers, not slower.

Don’t even think about trying to repair these things in orbit. Remember trying to fix the Hubble Space Telescope? That was massively expensive. Even with modern efficiencies the current estimations for a repair are about $100 million. That means it is effectively cheaper to just deorbit a failed or outdated satellite instead of trying to repair or replace components. The math behind launch costs supports this disposable idea. Falcon 9 can deliver payloads to space today for about $1,225 per pound, which is like 90 percent cheaper than the old Space Shuttle. SpaceX says that the target for Starship is closer to $33 per pound, provided it can stop blowing up or turning into a boat when it lands. If the numbers can keep moving down and hit those lofty goals then it really does become cheaper to blow up the old stuff than try to fix it. And you thought we had a space junk problem before.

That’s not to say there aren’t challenges. Starship’s schedule has slipped three times already this year. The price of a Falcon 9 launch went up this year for the first time since 2022. Everyone is betting that the cost curve is going to keep going down and they’re making their bets on it, but what happens when the curve becomes a plateau?

Not everyone is building disposable data centers in space. Sophia Space is building their modules so they can be swapped out. Axiom is also building tiles that can be serviced. But when you compare those two approaches to SpaceX just wanting to launch a million satellites into orbit and hoping that most of them stay up there for a year or two you can see the industry hasn’t quite figured out their approach yet. But both approaches eventually lead to the same conclusion. If you want to service devices you have to do it before they become obsolete. And you have to make it cheap enough to do in an industry that thinks in quarters and not years. That’s why the deorbit model looks the most economical right now.

Tom’s Take

Space data centers aren’t impossible. They’re just really impractical right now. Cooling is a challenge that can be solved with money and money is the real root behind the serviceability issues. The way this is being pitched to investors right now doesn’t make the economics work for anything other than a market that looks more like an iPhone launch than a rocket launch. When the GPUs in the satellites are obsolete on the launch pad you know you’ve got some bigger issues to examine.

All of this assumes that the money in space data centers get you returns on a terrestrial data center timeline. You have to be willing to take some losses while the cost curves come down to a place where you make some money. How many investors do you know that are willing to take some losses for the next couple of quarters? And that’s only if the costs keep moving down at the current rate, which isn’t guaranteed.

The next time someone tells you they’re excited to build a data center in space, just tell them what they’re really building is a hardware subscription that might not make financial sense in the long run.

Is Indoor Standard Power The Fix You Need?

Posted on by
Reply

Every new feature that ships in the 6 GHz Wi-Fi band using the same language. You’re getting more power, more range, and more throughput. Who doesn’t like more? We all want more! Well, almost everyone wants more. The people that don’t want more are the ones that will lose something if you take it from them. In the case of 6 GHz, that’s the incumbent providers like microwave providers.

Spectrum for Rent

In order to get 6 GHz ratified concessions had to be made. One of those was that indoor 6 GHz was going to operate at a lower power level until a method of sorting out spectrum use could be formalized. That’s how we ended up with Low Power Indoor (LPI). Outdoor APs operate at Standard Power (SP) but run the risk of interfering with the incumbents who have a lot of pull with the FCC. That’s where Automated Frequency Coordination (AFC) comes into play.

AFC is the mechanism that opens the U-NII 5 and U-NII 7 bands at full power. It works because the AP uses AFC to ensure there is no interference with an incumbent source. That means knowing exactly where you are with GPS as well as the antenna height. AFC then checks a database for incumbent sources and returns allowed power and frequency use. The system then authorizes that AP to use that power level. Sounds simple, right?

APs running Standard Power are required to refresh their AFC authorization every 24 hours. Think of it like a license check or an always-online video game. If you want to play with SP APs they have to reach the Internet. Normally this isn’t an issue because most of the things we use to provide connectivity also have Internet reachability. The thing to consider is the corner cases. Because when an AP can’t be authorized it must fall back to running in LPI mode until it can be cleared again.

Design Chaos

If you’re looking at running Standard Power indoors you’re probably looking to cover large areas like warehouses or high density venues. You’ve probably worked out a design that has taken into account a lot of margins for error. You should be proud of what you’ve accomplished. But you likely didn’t take everything into account. Because AFC always gets a vote on your design. Once a day.

If you design for the best case scenario with a little fudge factor you should be good. But if there is an outage for some reason that knocks out your AFC connection or a software update causes issues with AFC you’re going to know about it in 24 hours. Your carefully designed AP plan is going to fall back to LPI power levels. That means coverage gaps and dead spots and complaining users and customers. The use case for SP indoors is a huge power increase. Some of the numbers floating around out there claim up to 63x better. Even with a healthy grain of salt you’re probably relying on using fewer APs to cover an area. If AFC goes down your right back where you started in low power mode.

Worse yet, that input happens every 24 hours. You could be perfectly fine today but there is no guarantee that the AFC will return the same power levels every time. Two APs with identical equipment could get two very different authorized power levels because one of them is two feet higher than the other. I can already see the radio resource management (RRM) system on your network going into overdrive to compensate. And if someone installs a fixed link in the area you may be permanently locked out of a band without even realizing it.

Plan Effectively

There are a lot of ifs in the scenarios above. Planning is about taking out the ifs. You can do that with some decisions ahead of time.

  • Do You Need SP? – Most of the time you’re going to be just fine with LPI for standard deployments. Offices and other enterprises don’t need APs blasting Standard Power everywhere. SP really makes sense for places with big opens floors and covering campuses with a mix of indoor and outdoor usage. If you don’t need to use SP then use LPI and your life is easy.
  • Ensure Connectivity – You need to make sure you’re always connected so AFC doesn’t cause problems. But you also need to make sure that your location data is up-to-date too. That means ensuring things like GPS for APs because manually entered data doesn’t get considered for AFC. If you work in an environment where you have dodgy Internet connectivity you should probably fall back to using LPI.
  • Changes Should Be Minimal – SP deployments need to be more static than you realize. Every movement of an AP doesn’t just mean new RRM maps. It also means the power output and frequency use of the AP will change because of new AFC guidelines. If you have people that like to move things without considering the impact you should probably fall back to LPI or invest in industrial-grade epoxy to keep APs where they are.

Tom’s Take

One of the things I’ve mellowed about in my career is not just installing or using the newest thing because it’s the fastest or the best. There’s no need to upgrade to beta software just because it’s shiny. The same goes Standard Power in 6 GHz. If you have a need for it you should use it. Because knowing you have the need for it means you’ve already considered the challenges that come with it. You know the extra care and feeding that’s going to happen. People that just check a box to turn it on thinking that everything is going to be magically better are in for a headache that won’t easily be fixed.

AI Isn’t a Genie, It’s an Intern

Posted on by
Reply

Tell me if you’ve heard this one before. We build a super intelligent system and give it a specific goal like maximizing paperclip production. The system decides to do the job as well as possible by converting all available matter into paperclips. Everything. The Earth. The solar system. All of it. After the universe collapses the machine shuts down, content that it followed directions.

This is the malicious genie problem that every Dungeons & Dragons player knows. You find a lamp. You make a wish. The genie grants the wish in the most technical way possible and it leads to catastrophe. The lesson we are supposed to learn is that a sufficiently capable AI system will find ways to satisfy objectives to the letter while simultaneously not doing it the way you wanted. The foundation of AI safety is to prevent that from happening.

Nick Bostrom has covered this in Superintelligence. Eliezer Yudkowsky has argued this very thing for years. It’s a compelling story. But it has a hidden assumption that causes more confusion that it solves.

The Best Intentions

The genie tale is a story about intent. The genie knows exactly what you asked for. It just found the most technically valid way to grant your wish while ruining the spirit of it. This isn’t Robin Williams. This is some kind of demonic creature looking to teach you a lesson. The failure mode is adversarial in nature. Fantasy genies that aren’t in Disney movies are always looking for loopholes.

If the failure mode is intent, then the solution must be constraint. Build a system that can’t possibly do the bad thing. Be specific about every edge case. Build rules on top of rules. If we build the perfect box we can prevent the genie (or the AI) from going rogue and ruining our day.

The framework is sound if the assumption is correct. It works for a genie that knows better, right? But when we apply it to a modern AI we see where the gaps are. Genies are working against you. AI is not nearly that smart.

Doing My Best

Imagine a racing game where the objective is to score points. Collect points on the track and finish the race. Sounds simple, right? But what if a brilliant AI figures out that all they have to do to win is drive back and forth in front of the starting line collecting points and blocking other players from finishing? If it has the most points at the end it wins even if it never finishes.

In this case, the AI player isn’t like the genie above. It didn’t purposefully subvert your expectation. It did exactly what it was told to do. Score the most points. If you didn’t tell it that it had to drive through the whole course and cross the finish line then it didn’t know that it needed to do that. The win condition was points, not racing. The AI wasn’t missing intent. It was missing context.

This is an altogether different problem than the one above. Instead of assuming the system is going rogue and being obtuse the system genuinely doesn’t know what you mean and it tries to fill in the context with what it has available. Without the context that you assumed the system should have it just did what it could and you were flabbergasted by the results.

This is something that pops up at every level of the system. Context starvation and goal misalignment aren’t two different things. They’re sides of the same coin. When you don’t do a great job of being specific you usually get misalignment of your output.

Framing Your Reference

If you think the problem is intent you’re going to break out the constraints. Rules. Guardrails. Boxes. You’re going to spend your efforts on preventing the system from doing something bad. Security is about building walls.

If you think the problem is context you have a different outlook. You’re going to spend more time being specific. Less rules, more grounding. You want the system to surface ambiguous instructions instead of trying to resolve it with limited information. It’s like an intern being unclear on a task. You want the AI to ask you what to do instead of interpreting incomplete info.

It should be a cooperative inference problem. The system should always be just a little uncertain about what the operator wants and then seek to find out what is needed. The alternative is to confidently pursue a bad solution to a fixed objective because it thought it knew what you wanted without you telling it those details.

Knowing you have to do this doesn’t make agent building easier. In fact, it makes it a lot harder. It just makes the whole thing a lot more honest. Because you have to assume that people will never full specify what they want in advance no matter how much detail they provide. That’s not a failure of imagination. It’s the reality of how complex systems are implemented. There’s always some detail you miss. You shouldn’t aim to create the perfect spec up front. You should instead seek to build a system that is smart enough to know it’s missing context and will ask for it before running off to go to work.

It also means you have to treat an agent asking questions as a feature and not a failure. It’s like when your intern asks you to confirm that what you asked for is what you want. That’s not them being dumb. That’s them being sure they heard you right. And really, that helps you in the long run because if the system is asking you about specific areas you know your instructions must be a little thin in that area.

Tom’s Take

If you think your AI is a malicious genie you’re always going to be asking what rules have to be put in place to prevent it from going rogue. What you should be asking instead is “How can I give my agents enough context to do what I really mean?” The more powerful our AI agents get the wider the gap between those two things is going to be. But we can start to solve it today by building systems that ask questions when things are unclear. I promise you that you’re going to enjoy the results more than trying to close every loophole in your wishes.

Cisco Live 2026 – Requiem For A Corner

Posted on by
3

Cisco Live US 2026 was an interesting ride this year. There was a lot of talk about AI. There was a big discussion about security and how we are protecting our software from the AI models on the horizon that are ready to uncover every bug ever conceived. And there was even more discussion about whether Cisco was ahead of the game or behind the curve on their support for everything from eBPF to the latest Mythos reports. I say there was a lot of discussion, but I’m not sure where exactly it was happening.

Social Desert

One of the biggest things I heard from my friends at the event was how light everything felt. Fewer people was a common theme. The reported number was around 22,000 but it felt closer to 20,000 to me. The World of Solutions felt very spread out this year, with most of the back side being Cisco booth space.

The other thing was the Social Hub. It had shrunk from last year. At least the couches were facing each other this year. And there were some cool stickers and some interesting puzzles to work on. But there were far too few tables for people and the same sign setup as last year with a basic Cisco sign and the LIVE! part being a projection in the background. I think it’s safe to say that the days of the big sign pictures are in the past.

One of the reasons for this, in my opinion, was the new security measures that were in place. This year in order to get anywhere near the sessions or World of Solutions you had to pass through weapons scanners. This was similar to what I saw at RSAC this year. They were placed so you couldn’t get into the lower part of the Mandalay Bay Convention Center without passing through them on either level. At least one of my friends was stopped every time he went through them, even with nothing but a laptop in his backpack. I know that there are a lot of issues going on and there was definitely some disruption at last year’s Cisco Live event but what is the purpose of forcing everyone through these checkpoints aside from ensuring only registered people can get in?

Putting Someone In A Corner

This year a LOT of my friends were missing from Cisco Live. Some had work obligations. Some had life obligations. But more than a couple said they didn’t see the value any longer. This is part of the circle of IT life but it’s sad because not seeing the value for Cisco Live also means that any other things that are going on there can’t outweigh the costs. It’s like people who have gone to a minor league baseball game saying that while the play on the field might be subpar the rest of the atmosphere was awesome and worth the cost. Eventually if the play is bad enough or the atmosphere isn’t worth the cost then they will move on to do something else.

That’s what it felt like to me this year. People weren’t getting the previous value. I know that Las Vegas has its challenges for people. Neon and noise are things you have to take into account. If the sessions are good and the community is welcoming then people are more than willing to survive the heat and the in-your-face style of Vegas. This year wasn’t enough. I looked back at the original Cisco Live picture from 2011 and only three people in that picture were in Vegas this year.

This was also the first year in forever that I didn’t show up on Sunday. I had a very busy weekend before the event and made the choice to do things in my non-IT life instead of flying in for the traditional Sunday social meetup. Last year showed me that the meetup isn’t what it has been in the past. The people that I know that did go said much the same this year. It’s geared toward people that have been coming for the past couple of years and not people on the fifteenth or twentieth Cisco Live. It makes me sad because this would have been the fifteenth anniversary of the original Tom’s Corner. I even took a picture on the last day by the original column. The only thing next to it this time was a trash can.

I sat at the Social Hub on Thursday prepared to hang out until they took the furniture away as we’ve done so many times before. But that didn’t happen. This year I left half an hour before the World of Solutions closed and went back to my hotel room. I felt more melancholy than anything. I won’t say the community is dead or gone. But I do wonder where things go from here. Is the community something that is encouraged by Cisco to show people how practitioners feel about their attachment to an organization like this? Or is it more of a prop that can be used to help sell more gear? I can’t answer that question. But I do wish I knew what the people behind the event were thinking.

Tom’s Take

Requiem has a connotation of something offered for the dead. But it comes from the Latin for rest. I think that’s the best way to think about where the original social community is right now for Cisco Live. I think back fondly to the 2019 pic of the group camped in the social media hub and I wonder if and when we will ever be there again. Perhaps what we need now is time to rest and recharge. I will probably be back at Cisco Live 2027 for my 21st event. Maybe a year will bring back what I think we missed this year. Until then, my friends.

OpenClaw Ruined AI and It Makes Me Happy

Posted on by
Reply

The biggest AI story of 2026 isn’t the growing need for electrical power or the ridiculous way the market sold out for RAM based on a letter of intent to acquire. No, the biggest AI story of the year so far is how a scrappy little project completely upset the AI apple cart. OpenClaw (nee ClaudeBot, nee OpenMolt) set the world on fire. And it destroyed how people were trying to direct AI. I’m sitting over here giggling about it.

Round The Clock

The basics of OpenClaw are simple enough. You have a system of agents that do things. It can read your texts or email and triage the flow of information. It can send you a text summary of the news or the weather every morning. But it can also be configured to monitor things as they arrive to deal with them on the fly. That’s where the real narrative shift has happened.

When you open a browser window to talk to an LLM you are creating a session that has a finite time limit. You are saying that you are going to work on a project for a specific period of time and that’s that. Once you complete your task and you go back to whatever you were doing that’s the end of the conversation. More importantly, that’s the end of the token consumption. Because these models use tokens as method for creating words or code you can think of them as the resource that AI lives and dies by. Not unlike vespene gas from StarCraft.

When you have an agent that’s running constantly, it acts like a real person. It doesn’t consume resources in orderly sessions. It is bursty. It might be idle for hours and suddenly consume thousands of tokens when something comes in that requires a complex chain of tasks. It’s not unlike when someone gets a sudden burst of creativity and spends the next week racing to complete their task before the spark is gone. Now imagine that whole process is burning tokens as the agents dispatch their work.

The idea might not give you pause but it scares the hell out of the AI companies. Because unpredictable consumption destroys their carefully planning projections. Those ideas of data centers being built to deal with demand next year get wrecked when the projected demand shows up now and is distributed unevenly. Finance markets hate unpredictability. And those same finance markets are the ones backing this AI boom.

The Fringe Fails Us All

The other side effect of OpenClaw is the way that companies are racing to do more. Before it was just getting OpenAI to write your term papers or edit an email for tone. Now people are democratizing coding and writing their own apps to handle tasks they thought could never be automated or turned into software. Users have never felt more free.

Providers, on the other hand, are scrambling. Those same carefully curated token consumption projections go right out the window when users start burning more and more tokens because they feel empowered to build more. As more entry level users find out that Claude Code and Codex can help them build a workout app or a recipe tracker they are increasing the token burn rate. That might be something that could be managed by increasing capacity slowly. But they aren’t the real problem here. It’s the power users.

When power users figured out how to unlock parallel development pipelines and dispatch agents to write code blocks they significantly increased their token consumption. This wasn’t helped by the industry’s attempt to shift the discussion around tokens to reward those using AI the most by putting up leaderboards to highlight people burning through the most tokens. This created a culture of “tokenmaxxing”, which has to be the dumbest name I can think of which naturally means it stuck. The idea of rewarding people on a specific metric means that people will game that metric to look good. Tokens went from being burned at a steady rate to being consumed like fuel for some magical fire that cannot be quenched.

Providers panicked. Suddenly the people paying $20/month for Claude Code were burning thousands of dollars worth of tokens building apps with all the tokens they could find. People were thrilled by the creativity but the people running the GPU farms on the back end were worried. If the power users are burning through tokens this fast, what happens when the rest of the world decides to do the same? That’s when you saw the pushback. People were hitting walls of token utilization in hours and told to come back tomorrow. Users were also hitting limits on monthly allocations. The providers were working feverishly to upgrade the hardware as quickly as possible. Eventually, we hit the conclusion that everyone wanted to avoid but knew was inevitable: usage-based pricing. Not surprising considering anyone that has ever tried to offer anything unlimited quickly had to implement limits because people really can’t help themselves. Now we’re facing rumors of coding being moved to the hundreds-of-dollars per month tiers because it’s just too taxing for the infrastructure we have currently built out. Who could have possibly foreseen telling everyone that AI is the way of the future and everyone needs to embrace it could cause a shortage of resources?

Tom’s Take

It turns out that not being ready for the beast you have unleashed is exactly what the AI companies deserve. They got caught flat footed when the always-on agentic system that was a natural outgrowth of their ambitions forced them to look at how their infrastructure is being used and consumed and the numbers didn’t add up. They wanted people invested in the idea of using AI to build everything and then hopefully, like Uber and AirBnB, they could raise the rates and retire to some island. Instead they realized that people are voracious when it comes to exploiting technology for their own gains and now they are in a race to catch up because the little lobster made them look silly. Don’t mind me. I’m just going to be over here laughing while the tech geniuses of the world get exposed by shellfish.

You Can’t Patch People

Posted on by
Reply

One of the things I’ve noticed when it comes to IT is how quickly we’re willing to use software to solve people problems. Over my career I’ve seen all manner of crazy solutions to get around people being lazy or uneducated. Remember vMotion? Or OTV for stretched layer 2? Why do you think those solutions came about? I posit that it’s because it’s faster to write software than to patch people.

Hacking Humans

I see this most often in cybersecurity. Developers love to create software solutions that prevent things from happening. Phishing and all its various forms are some of the top priorities for solutions that prevent leaking of information. While we have invested a lot in phishing tests and education it’s also very likely that there are controls in place that prevent users from accidentally giving out information to threat actors.

Why are we so willing to write software to fix problems instead of teaching people to avoid those issues? I think in part it’s because software is predictable. If I create an app or write some controls into a platform it’s going to behave the same way every time. That’s the definition of deterministic. Every time the software is presented with an input it will react the same way. That makes it easy to figure out. People that deal with risk on a daily basis just love predictability.

Humans are messy. We don’t always behave the same way every time. Even someone that knows they shouldn’t click on links in an email will do it because they aren’t paying attention or because they are tired. When you factor in how much better the phishing emails have gotten thanks to the advent of generative AI even the rank-and-file people are getting tricked. Developers would rather deal with software than trying to send more tests and update education resources.

The real issue is that we can’t patch people as easily as we can with software. If updating the filters for spam and phishing and other security related items was as simple as downloading the new attack vectors into someone’s brain we’d be doing that instead. Likewise, if we could just convince people to build things a certain way to avoid having to create complicated systems like FHRP we would be doing that instead of trying to solve for lazy developers.

Treating People Like Programs

Why is it so hard to patch people? Forget about the deterministic part of the equation for a moment. Software isn’t instantly updated when something is discovered. It takes time to develop lists of new vectors or update programs to remove vulnerabilities. Why can’t we do the same for people and reduce the overhead of all the extra software?

People can be “patched” with education. It isn’t always easy to get people to take courses or read the bulletins that are sent out. There are ways to force people to do it but that kind of friction just makes security teams resent users for trying to avoid mandatory training updates. Hence the reliance on software to fix the issues. But it doesn’t have to be like that.

Instead of forcing people to take updated training you could use something like gamification to encourage people to update training or learn about new issues. This is especially good with younger or newer employees that are used to the badge hunt mentality. Giving them the option to display achievements tied to training is a great way to encourage them to keep updated while also pulling others in that want to earn the same recognition.

Tom’s Take

I get the desire to rely on deterministic software rather than dealing with unreliable people. But there is only so much software that you can write to try and fix behaviors. We eventually have to get to a point where we can educate users and encourage them to want to keep up with it instead of forcing them to go through endless modules that don’t give them any real info. If we would just put in a bit of the effort we use on software controls into the people we’re trying to restrict we might find the effort is multiplied far beyond what we could hope for.

The Value of Concise Communication

Posted on by
1

When I first started working at Tech Field Day, one of the things that I struggled with was writing. Sure, I’d been writing blog posts for almost three years at that point. But what I really had issues with was my communication style through email. Every message became a small blog post unto itself. I spent more time answering every possible question and providing way more information than was needed. Luckily, Stephen Foskett helped me figure out that concise communication was critical. That lesson has grown on me through the current day.

Working With a Watch

I want you to think back to an interaction that you’ve had recently where you were talking to someone. Maybe you were asking them a question or looking for them to provide an opinion about something. How much did they talk? Was it a short pointed answer? Or did it feel as if it was going on forever? It’s something I’ve noticed recently with people I talk to in real life. The discussions aren’t short and focused. Instead they carry a lot of extra information and exposition that makes things take far too long.

Yes, I know the irony of that statement for anyone that has ever met me and talked to me in person. I know for a fact I can go on and on about things. I have worked over the years to try and be better about keeping my answers and discussion precise and laser-focused. I think that society in general has different ideas now. It feels, to me, like the majority of society thinks a longer explanation is a better one. More words must mean more content, right?

The one thing that stands out to me to highlight this is when someone is asked to give a summary report of a working group breakout session. We’ve all done this. A group meets and discusses topics. Someone takes notes. Then when groups reassemble someone is asked to give a short summary of what was discussed in the group. The most restrictive summaries I’ve seen are about a minute long. Sixty seconds to summarize fifteen or twenty minutes worth of meeting discussion. How can you do that?

The goal of a short summary is to strip away everything that’s not essential. You get rid of three minutes of introductions. You condense five minutes of back-and-forth arguing on a topic. You skip past the pleasantries and filler words. You reduce the discussion to the essential points necessary to convey what was actually done. And then, you have to give that report. As much as it is important to summarize your discussion it’s also critical to keep your summary brief. Without notes or a good handle on what the key points should be your summary is likely going to be three times longer than your time limit.

I know this happens because I watched it happen. The person before me stood up and spent four minutes of a one-minute summary going on about things that were discussed without attempting to summarize it. The people in the room felt like it was dragging on. So I decided to turn the discussion around. I scribbled some quick notes. When it was my turn the moderator told me that I had one minute to summarize our working group. I told him to start a timer. He scoffed but I went ahead anyway. I didn’t speak any faster than normal. I just hit the high points, expressed the areas that needed group support, and thanked everyone for their time. I looked over at the moderator, who was still fumbling with his iPad and said I was done. Total time? About forty-five seconds. No more than necessary. The room was quite thankful.

Which Words Work?

You can’t be brief without having all the facts. You can’t be concise without understanding how to keep things short. You need to express as much as possible in the fewest words you can. That means understanding what the core message is and getting it out there as quickly as possible. I’ve started taking notes about things as people are talking so that I’m not surprised when someone calls on me. I use the notes to help focus my thinking around what is meant, not exactly what is being said.

One thing I like to do is to restate the discussion in my words with a shorter impact. If someone spends five minutes explaining an issue you can boil it down to a couple of sentences as a way to show that you’re understanding but also to help them see how to keep things concise. If you can’t convey the issue in two or three sentences you have a lot of information to deal with. You need to ask yourself if you need to explain it all or if you need to summarize the entire discussion at a higher level of encapsulation to move the conversation along. We do this all the time. If someone asks how your car is running do you detail the warning lights and possible causes and information you researched on the Internet? Or do you simply say that it has some issues that need to be worked on?

Note that this level of summarization is dependent on who you’re speaking with. If you’re talking to an executive, keep it brief and general. They need to know the minimum to make decisions, not solve the problems. If you’re speaking to an expert, focus on the direct issues and leave out the unnecessary color of the conversation. Act like you’re speaking to your doctor. This hurts, here’s how long it’s been hurting, and here’s what you’ve tried to fix it. They don’t need to know how it’s impacting your golf game or why it made you late to the grocery store. They just need to know what is wrong and how to fix it.

Tom’s Take

You don’t have to be overly terse in your conversations but I think you will find that you can use fewer words to get your point across. Summarization and focus on key areas sounds pretty simple, doesn’t it? Then why do meetings seem to take forever? I think you’ll find that once you start putting in an effort to be more concise others will follow your lead and do so as well. No long explanations for simple answers. No exposition where it’s unwarranted. Save the wordy answers for an LLM response. Keep it short and simple and people will thank you for being a valuable communicator.

Context Is Expensive

Posted on by
Reply

When it comes to learning and understanding, facts are easy. If I ask you how many bits are in an IPv4 address it’s a single answer. People memorize facts and figures like this all the time. It’s easy to recall them for tests and to prove you understand the material. Where things start getting interesting is when you need to provide context around the answer. Context is expensive.

Cognitive Costs

Questions with one correct answer or with a binary answer choice are easy to deal with cognitively. You memorize the right answer and move on with your life. IPv4 addresses are 32 bits long. The sun rises in the east. You like Star Wars but not Galatica 1980. These things don’t take much effort to recall.

Now, think about why those answers exist. Why does the sun rise in the east? Why are addresses 32 bits long? Why don’t you like Galactica 1980? The answers are much longer now. They involve nuance and understanding of things that are outside of the bounds of simple fact recall. For example, look at this video of Vint Cerf explaining why they decided on 32-bit addresses all the way back in the mid-1970s:

There’s a lot of context around a simple fact. For some of us it’s fun to learn the context and provide it at parties or when we’re trying to put someone to sleep with endless recitation of trivia. A lot of people won’t bother to care about the why and move on with their life.

You know who does care about the why? People building AI systems. When you think about it, answers that are facts are searchable. You would jump on Google or Bing or Duck Duck Go to find out where the sun rises or what time it rises today. But the answer to why is something that people building AI algorithms are working on. They want to be the search engine that provides the context. They want to overwhelm you with the reasoning and the justification behind something. And that increases cognitive load on the system.

If I asked you to explain why Star Wars is better than Galactica 1980 you could likely give me five reasons with justification quickly. But if I asked you to analyze each of those for underlying assumptions about science fiction and writing styles you’d take a little longer to come up with your answers. That extra level of reasoning increases our cognitive load. Now imagine a system working on all that cognitive load simultaneously. That’s where we are with AI right now.

Making Money

People know how to deal with overwhelming amounts of information. They can selectively discard what they don’t care about and focus on what matters. It’s why we can find the signal of a person’s conversation in the noise of a cocktail party. Our brains can filter when necessary to reduce cognitive load. AI isn’t as good at that right now which is why every piece of data included in a list of sources or a prompt has to be included somewhere. AI doesn’t know how to say “this is important” or “this is something we can forget about”. It’s better than it was before but it has a long way to go to behave a like intelligence.

The problem is that all of that processing costs resources. Power consumption, water consumption, and processing time are all used when a model is doing things. Models perform well when things are easy to produce, like with simple answers or with information that has already been generated. They fall down a bit when they have to do many things simultaneously. Just like with the human brain it can get overwhelming. Only the AI doesn’t do as good of a job as the brain of isolating the important parts.

The answer, at least according to AI researchers, is to just do it all. Burn a lot of those magical tokens to get every answer and every piece of context and just have it ready in case someone asks. It’s not unlike rehearsing a conversation in your head over and over again to get the perfect response to every question that could be asked. And yes, before you ask I’m the kind of person that does that. The cognitive load of exploring every conversation option is usually two or three times longer than the conversation itself. That’s time I’m not going to be get back.

The context around the answers incurs additional expense. We want to understand why and we’re willing to pay to get that detail. Right now it’s mostly free for us to play around with. The models are getting trained by what we ask and refining their ability to predict responses and understand how we think. What happens in the future when that model isn’t free any longer? Are we willing to pay to access the context? Are the providers going to force us to see ads to provider compensation for it?

Tom’s Take

I love context. I provide it all the time. Sometimes it’s not warranted or appreciated. But it’s always there. Because I want to know why something is the way it is. I’m the kind of person that is going to cause our modern systems to burn additional resources to sate my curiosity. Right now the model doesn’t look to be sustainable because we haven’t trained our algorithms to understand that sometimes the best part about being smart is knowing when not to be.

The Inattention Economy

Posted on by
Reply

I need you to try to do something very hard for me. I need you to read this entire blog post. I don’t think it’s going to be hard because I’m going to use big words or highly technical terms. I don’t think it’s going to be hard because of the subject matter. It’s going to be hard because you’re going to get interrupted. In fact, I’m willing to be you got some notification before you ever finished this paragraph.

I didn’t realize just how scattered my attention was until a close friend pointed it out to me. She mentioned that I was always checking my watch for notifications. I didn’t realize it until someone that wasn’t around me all the time saw it. I stepped back and honestly asked myself why I was getting so many notifications. In the back of my mind I knew I was getting too many because when I go on a run my watch won’t stop buzzing with all the things that I don’t even bother to check. That’s when I realized my attention was beyond scattered. LinkedIn, GroupMe, even Yelp seemed to want me to pay attention to something that was ultimately unimportant.

Distracting for Dollars

If you’ve read a post about focus or watched any number of Youtube people talking about it you know that we are drowning in a sea of information. Everything has to update us all the time about what’s going on and what we should be doing. Half of the websites that I visit daily want to put notifications on my phone or desktop when they publish new stories. My son’s dentist uses a camera scanning app to check his braces alignment weekly. I like the idea but I don’t like that the app wants to remind me of scan times and updates on his status and things. It’s like they want me to check the app constantly.

That, in my mind, is the real problem with all these notifications. It’s not that the app or the service wants to let you know about something. It’s that you must check the app RIGHT NOW to see what’s going on. Don’t believe me? Use your phone to check the temperature. Open it up and navigate to your weather app. I bet you have to wade through notifications and updates and banners that are trying to grab your attention the whole time. They want your eyeballs. They need your attention. And they’re willing to do all kinds of things to make sure they get it.

LinkedIn wants to update you on a thread that you posted in because someone liked someone else’s comment. Facebook’s stupid highlight mention makes sure ALL of your friends get a notification when someone uses it in a comment. Can you seriously believe Mark Zuckerberg created a handle to notify every single one of your friends? Of course he did. Because he never wants you to leave his app. He wants you to engage with posts and look at ads. He needs you to stay there so he makes money from you.

I used to think notifications were any important way for me to stay informed about what was going on around me. Now that I’m older and way more cynical I see them as a way to make sure my attention gets split and focused back on whatever they want me to see. Which would be fine with one or two critical apps. But with ten or twenty? It’s downright impossible.

Resource Contention

The problem with distractions goes beyond just popping up when you least want them to be there. Even if I can ignore what just flashed across my screen it distracted me and interrupted my flow. When I’m really focusing on something I can block out everything around me. And when some email or app wants to make sure that I know there is a special 20% off coupon for an item that I have to use in the next ten minutes as a way to get me to open the app RIGHT NOW I lose my place and have to get back to where I was. If you’ve ever had to read the same sentence two or three times to remember where you were you know exactly what I mean.

We suck at multitasking. No, don’t fight me on this. We really do. We think we are good at it because we can jump back and forth between things but in reality we are way more efficient when we work on things in uninterrupted blocks of time. When I start writing I only feel effective when I sit down and write everything until I’m done. I don’t like stopping and starting. Multitasking means I’m constantly shifting my focus and forgetting what I was doing. That’s hard for neurotypical people. For neurodiverse folks? It’s hellish.

Your attention is a resource. You need to conserve it just like you would with money or water or electricity. When you see it as something that needs to be allocated and schedule and budgeted you start to see why everyone is so hungry for it. I try to keep my attention focused on things that require it. Yes, I do watch videos or read blog posts. But I do it with purpose. And when I do I try to minimize what’s going on around me. I ensure that my resources are being spent properly to accomplish my goals.

If you haven’t done it yet, I highly recommend doing some simple things to reduce the amount of distractions you are getting hit with. I disabled a large number of notifications on my watch. I still get email updates every half hour or so. I also get messaging notifications. But for every app that I dismissed or rolled my eyes about getting notified for something unimportant? I took it away. It lives on my phone where I can check it when I choose to see it. And now my watch only notifies me when it’s important. When my mom texts me or when I need to stand up.

Tom’s Take

You can take these ideas and run with them. Maybe you want to try the Pomodoro Technique or you want to play around with focus states in your mobile device to minimize distraction. You could work with white noise in the background to sharpen your focus on your task at hand. You could even create a distraction-free desktop environment to ensure you only see what you need to be doing. No matter what you do you need to make sure that you are focused on what’s important to you. Not what someone else thinks you should be seeing.

The Heat is On

Posted on by
Reply

One of the things I like to do in my twenty-eight minutes of spare time per week is play Battletech. It’s a table top wargame that involves big robots and lots of weapons. Some of them are familiar, like missiles and artillery. Because it’s science fiction there are also lasers and other crazy stuff. It’s a game of resource allocation. Can my ammunition last through this fight? You might be asking yourself “why not just carry lots of lasers?” After all, they don’t need ammo. Except the game designers thought of that too. Lasers produce heat. And heat, like ammunition, must be managed. Generate too much and you will shut down. Or boil your pilot alive in the cockpit. Rewind a thousand years and the modern network in a data center is facing a similar issue.

Watt Are You Talking About?

The average AI rack is expected to consume 600 kilowatts of power by next year. GPUs and CPUs are hungry beasts. They need to be fed as much power as possible in order to do whatever math makes AI happen. They have to come up with creative ways to cool those devices as well. We’re quickly reaching the limits of air cooling, with new designs using liquid cooling and even immersion in mineral oil to keep these machines from frying everything.

Networking is no slouch in the energy consumption department either. The latest generation of networking devices are ramping up to 800 GbE and consuming a significant amount of energy to keep the bandwidth flowing. One of the biggest consumers of power in these racks is the digital signal processor (DSP) that are clustered inside fiber optic modules. You have to have a lot of DSPs to condition the light that allows for operations at the top end of the scale. Additionally, the CPUs and ASICs themselves inside the units are running at peak performance to provide the resources AI clusters need to return value.

Worse yet, the very design of the network switch works against traditional air cooling. Servers have pretty face plates with air channels that can pull in air and vent it out the back. Nothing in the way except for maybe a USB stick when you’re loading the operating system. Switches, on the other hand, plug in all the cables up front. Those cables and modules impede air flow and reduce the amount of cooling the switch is capable of producing because of simple physics. If you’ve ever put your hand in front of a modern switch you know that it can pull in a lot of air. But every connection reduces that and creates a feedback loop.

Chill Out

Just like with the AI cluster servers, we have to deal with the heat generation. Unlike the AI servers, we can’t just dunk them in oil or build a bigger chassis with more fans to bleed it off. I mean, we technically could if necessary. But when is the last time you saw a top-of-rack (ToR) switch bigger than 2U? That’s because the real value in the rack isn’t the network. It’s the server that the network connects. Every rack unit of space you give to the networking gear or anything that isn’t a server is a wasted RU that could have been generating revenue.

Modern networking devices have hit 800 GbE but AI wants more. There are designs that are looking to move to 1.6 TbE in the near future. But can they handle the load of DSPs that will be required to condition the laser light? Will they be able to reduce the size of this switches down to accommodate rack economics? Can they keep the whole mess cool enough that it won’t melt anyone that walks into the hot aisle to check on a cable before getting roasted by a flame throwing?

Plug It In

One of the solutions that looks to reduce this is Linear Pluggable Optics (LPO). LPO works to solve these issues by moving the DSPs out of the pluggable module itself and into the switch ASIC. This does do a good job of reducing the power draw of the optical module itself while moving the complexity of signal conditioning into the switch. The net result is less power consumption but the tradeoff is more complexity. To quote Battletech YouTuber Mechanical Frog, “Opportunity cost spares no one.”

Likewise, a competing solution like Co-packed Optics (CPO) trades the module itself for wiring the optical portion of connection right into the switch. This eliminates the need for DSPs to condition the signal and significantly reduces power and heat generation but at the cost of the flexibility of using modules in the first place. I’ll have more to say on CPO in the future.

Tom’s Take

The thing to think about is that the technology we’re using creates the bounds that we have to work within for other technology to build on. We have to deliver networking that doesn’t stop for AI to function the way the designers want. To them, the value of AI is not being more power efficient or generating less heat. Instead, it’s about crunching numbers faster or making better pictures that people want to generate. Data centers have an unlimited power budget and heat removal capabilities as far as AI companies are concerned. Reality dictates that nothing is unlimited but it’s up to the manufacturers to scale those limits before someone hits them. Otherwise, the network is just going to get blamed again. And the heat really gets turned up.