Notes

Over three decades of watching the tech industry and watching big companies grow from tiny teams to global powers, I’ve observed the same pattern: Ethics don’t scale up. Tech companies like to start with a mission. Google wanted to connect the world’s information; Microsoft wanted to put a computer on every desktop; Twitter wanted to give all people a platform to publish their thoughts. These are good ideas — the stuff of TED Talks. But users show up with their own beliefs and ideas, by the millions. As a tech founder, you end up putting enormous work into making users behave (and stopping them from breaking the law). Lawsuits pour in, saying you did wrong, some because you’re a convenient target.

All the while, money keeps gushing in. You start out transparent, sharing your journey, but then before an initial public offering of shares, you must honor the S.E.C.-mandated quiet period and restrict promotional communications. After that, the transparency never quite returns. The market demands a rising stock price. Your company still makes a lot of software, but a huge amount of time goes to tax strategy and compliance.

Maximizing shareholder value doesn't scale.

But regulation is absolutely in the interests of both America and the big A.I. companies themselves. Let me add two more terms people should know: “Google zero” and “model collapse.” Google zero (coined by Nilay Patel, the editor in chief of The Verge) is when Google stops sending traffic to websites and just provides an A.I. answer instead. When that happens, websites get less traffic, sell fewer ads and make less money. As a result, they may not be able to produce as much content. Model collapse is related: It’s when the A.I. models run out of knowledge to digest. What then? Do they excrete their own prose to redigest? Do they just give up?

Silicon Valley types like to say that data is the new oil. I think that’s right in two ways: Data is valuable, but it’s also a commodity, and these new A.I. tools are infrastructure. We regulate the electric grid, so why not these?

The soundtrack of the past week or so has been the boos of graduating college students as out-of-touch adults try to tell them that they need to embrace AI or else. Perhaps most prominent were the boos of University of Arizona graduates as ex-Google CEO Eric Schmidt told them, “The question is not whether AI will shape the world. It will. The question is whether you will help shape artificial intelligence.”

These grads, according to Schmidt, have no agency, which was confirmed by this comment a few minutes later: “When someone offers you a seat on the rocket ship, you do not ask which seat. You just get on, Graduates, the rocket ship is here.” What Schmidt doesn’t get is that these young people have already been forced onto the ship and there aren’t enough seats.

As a developer, it's not much different for us senior devs, but juniors definitely seem to be taking the brunt of the disruption. I'm also amazed at how these speakers can't read the room.

What’s driving me crazy, then, is not the idea that AI exists or that people are using AI. It’s that I have a finite time on this earth that I mostly want to spend interacting with other human beings. I don’t want to be the person arguing with a robot, or wasting my time reading something that a real person couldn’t be bothered to write.

Given all the bullshit I have to wade through, I'm not looking forward to the slop and misinformation that I'm sure is coming.

Even developers can get ensnared by poor code generated by the tools. Someone who is not nearly familiar with all the syntax, performance and security pitfalls would stand no chance.

For a senior UX designer to become a senior-level coder is like asking a master chef to also be a master plumber because “they both work in the kitchen.” You might get the water running, but you won’t know why the pipes are rattling.

Once a model is available on your device, per the specification, any website you visit will be able to send prompts to that model without requesting permission to do so, then do whatever it wants with the responses. And again, Gemini Nano is on your device if you’re using Chrome, and it will be again if you remove it, unless you start tearing out wires in ways that the average user of the web can’t. So, in short: you now have an LLM running on your machine, and any website you visit can make use of it, and whatever processing resources it requires. Google — a company that has paid billions of dollars in settlements for lawsuits related to privacy violations and deceptive practices in data collection — has said not to worry about it.

Add another reason to avoid Google Chrome like the plague. If you value your privacy at all, I’d highly suggest never using Chrome, and if possible, remove it from your computer all together. It will provide another easy way to fingerprint you.

There’s also the question of what the browser is even for. A browser is a user agent – it’s supposed to act on behalf of the user, not the vendor. Silently downloading 4 GB of AI model to your machine, re-downloading it if you remove it, and then letting any website access it without your permission – that’s Chrome acting as Google’s agent, not yours. And it’s hard not to notice how many Google products stand to benefit from having an LLM pre-installed on every Chrome user’s machine. With opt-out, of course.

Ugh...

What makes this particular instance so frustrating is that the contrast is right there in front of us, in the same set of release notes. There’s lazy loading for audio and video: carefully proposed, properly standardized, welcomed by all implementers. A true gift to the web community. And then, there’s the Prompt API: rammed through despite broad opposition. One feature shows how the standards process can and should work. The other shows what happens when a company with dominant browser market share decides that the process doesn’t apply to them.

You don’t just switch the lights back on for these things after Trump is gone and expect to find everything just as it was. From NIH, FDA, SAMHSA, CMS, and CDC, to NSF, NOAA, EPA, and NASA, the harm is so significant that massive amounts of resources will be needed just to bring us back to baseline, let alone prepare ourselves for the challenges of the 21st century and beyond.

Make America Scientific Again?

None of this is to say public health and scientific research in the US were perfect before Trump took power again. But that’s also the point: Recovering from the devastation gives us the chance to build the system we’ve needed for so long. Yet confronting the sheer scale of what needs to happen is the first step toward recovery.

I am sympathetic to the view that space exploration is a luxury. I don't disagree, even. But NASA's budget is not the reason gas costs $6 a gallon, or why we don't have universal healthcare or pre-K. We don't have those because those in charge, and the people who voted for them, have chosen for us not to have those. It is a false binary that we even have to choose at all. The U.S. is the richest polity that has ever existed; there is more than enough money to go around to satisfy basic human services while still funding spaceflight. The people denying us those basic services would very much like for you to identify NASA as the culprit for its $24.4 billion budget, which represents 0.35 percent of all government spending, at the same time a pointless and purposeless war costs us a billion dollars a day, and the government seeks a $1.5 trillion defense budget.

And look at all of what NASA is able to accomplish with that little bit of the federal budget.

It was an extraordinarily human moment. Humanity feels a little more tangible when surrounded by nothing but machines and darkness; Wiseman's is a soft beating heart in a big black void that has carried everything, even its grief, farther than any heart has ever been. And now when he and his daughters look up to the Moon, he'll be able to point to Carroll Wiseman's memorial. When we explore, we bring our humanity with us, and leave it wherever we go.

In some ways I feel like a carpenter whose job is now to write the blueprints for the IKEA factory. Of course there is still artistry in designing the blueprints, but you don’t care if the factory spits out one or two tables with splinters in the legs. The point is to produce enough furniture fast enough that the little imperfections don’t matter. Taste and judgment still count, but they’re at the level of the overseer on the assembly line, not the master carpenter working a chisel.

One worry I have for my entire generation of programmers is that many of us have been getting our artistic “fix” from coding: taking craftsmanship seriously, reviewing others’ code with the eye of a literary critic, trying to elevate the profession. Now the profession has been turned into an assembly line, and many of us are eagerly jumping into our new jobs as blueprint-designers without questioning what this will do to our souls. I believe art is necessary for a rich and full human life, so this isn’t an idle concern.

Everything has trade-offs, a set of attributes optimized and balanced towards a particular outcome.

You get X, but you lose Y.

Life is full of trade-offs. Anyone who says otherwise is trying to sell you something.

Unlike technical debt, which announces itself through mounting friction - slow builds, tangled dependencies, the creeping dread every time you touch that one module - comprehension debt breeds false confidence. The codebase looks clean. The tests are green. The reckoning arrives quietly, usually at the worst possible moment.

Margaret-Anne Storey’s describes a student team that hit this wall in week seven: they could no longer make simple changes without breaking something unexpected. The real problem wasn’t messy code. It was that no one on the team could explain why design decisions had been made or how different parts of the system were supposed to work together. The theory of the system had evaporated.

Knowing the code you are introducing to your codebase is so important. As soon as you offload that blindly to AI, the timebomb starts ticking.

I read one engineer say that the bottleneck has always been a competent developer understanding the project. AI doesn’t change that constraint. It creates the illusion you’ve escaped it.

And the inversion is sharper than it looks. When code was expensive to produce, senior engineers could review faster than junior engineers could write. AI flips this: a junior engineer can now generate code faster than a senior engineer can critically audit it. The rate-limiting factor that kept review meaningful has been removed. What used to be a quality gate is now a throughput problem.

The nightmare is when the AIs create such large PRs that make it so easy to miss wrong turns in code.

There’s also a specific failure mode worth naming. When an AI changes implementation behavior and updates hundreds of test cases to match the new behavior, the question shifts from “is this code correct?” to “were all those test changes necessary, and do I have enough coverage to catch what I’m not thinking about?” Tests cannot answer that question. Only comprehension can.

The tests pass...the code must work...right? Right?

You will pay for comprehension sooner or later. The debt accrues interest rapidly.

People, the context-bearers, have experience and capabilities that machines might never understand encoded in our muscles and memory. I’m on record saying I despise nuance –and I do– but it’s more important than ever to be able to connect to our fellow humans over this nuance so our world is not paved over by contextless opinions from ill-informed robots. Empower and believe people over machines.