Thoughts on Software & Systems

 For years, one of the easiest architectural rules to remember about AWS Lambda was this: If the job might run longer than 15 minutes, Lambda isn't the right place for it. That rule influenced countless architectures. Long-running work often moved to ECS, Fargate, Batch, Step Functions or another compute environment. AWS has now complicated that rule—in a good way. Lambda Managed Instances can support asynchronous and event-source-mapping executions lasting as long as 90 minutes . That's six times the traditional Lambda maximum. And it changes an architectural boundary we've relied on for years. Constraints Shape Architecture Platform limits matter because they force decisions. The original Lambda timeout encouraged developers to decompose long-running operations. Instead of one enormous process, applications often became pipelines: Upload file. Place event on queue. Process chunk. Store result. Trigger next step. Those patterns created resilient architectu...

  AI is quickly changing the software-development workflow. Agents can investigate issues, modify files, run tests, prepare pull requests and increasingly operate development tools themselves. That's exciting. It also creates an architectural problem we shouldn't ignore: The more capable our automation becomes, the more important its permissions become. GitHub recently made workflow execution protections generally available for GitHub Actions Enterprise environments. That might sound like another CI/CD feature. I think it represents something considerably larger. We're moving from pipelines that execute predictable scripts toward development environments containing software capable of making decisions. Our security model needs to evolve with them. Traditional Automation Was Mostly Deterministic A conventional CI pipeline might: Check out the repository. Install dependencies. Compile the application. Run tests. Build an artifact. Deploy it. The work...

One of the signs of good software architecture is something users will never notice: The infrastructure underneath the application can change dramatically while the application itself barely notices. OpenAI recently published an engineering article describing the evolution of the storage platform supporting ChatGPT at a scale exceeding one billion users. The scale is obviously interesting. But the architectural lesson is much more useful. A well-designed system doesn't simply handle today's workload. It creates boundaries that allow tomorrow's system to look very different without requiring everything above it to change. Architecture Is Really About Change Architecture diagrams often emphasize components. There is an API. There is a database. There is a cache. There is a queue. There are services. Those things matter, but I've increasingly come to think that architecture is less about arranging components and more about controlling the cost of change. Cons...

For the last several years, most applications of generative AI have followed a fairly familiar pattern. A user submits something. An application sends a prompt to a model. The model returns a response. The application displays or processes that response. That model isn't disappearing, but I think we're beginning to see something considerably more important emerge. AI agents are becoming an architectural layer of their own. OpenAI's recently announced Agents API is a good example of how quickly this transition is occurring. Rather than asking developers to assemble every component required for an autonomous workflow, the platform provides infrastructure for long-running sessions, tool use, context management, sandboxed execution and multiple cooperating agents. That distinction matters. From API Call to Digital Worker Traditional APIs are largely transactional. Send input. Perform an operation. Receive output. An agent can potentially operate very differently. Give it an obj...

 Good product engineering is not simply about writing clean code, choosing the right framework, or shipping features quickly. It is about making technical decisions and product decisions reinforce each other. The best engineering teams understand that architecture, deployment, user experience, maintainability, and business priorities are not separate concerns. They are parts of the same system. Decisions made in one area eventually affect all of the others. That sounds obvious in theory. In practice, it is surprisingly easy to forget. Engineering Decisions Are Product Decisions Consider something as ordinary as adding a new feature. From a product perspective, the question might be: What should the user be able to do? From an engineering perspective, the questions quickly become: Where should this functionality live? What data does it need? Does it require a new service or API? How will it affect existing workflows? What happens when it fails? How difficult will it be to change lat...

Shipping quickly is usually treated as a sign of a healthy software team. More releases. More experiments. More customer feedback incorporated into the product. More visible momentum. But there is another side to velocity that gets less attention: operational churn . A team can be shipping constantly while also creating a steady stream of deployment noise, configuration changes, broken automation, support questions, emergency fixes, and operational uncertainty. The problem is rarely that the team is moving too fast. The problem is that the system around the team is not designed to absorb change gracefully. The goal should not simply be to deploy more often. The goal should be to make frequent deployment feel routine. A Good Release Should Be Boring One of the best signs of a mature engineering organization is that production deployments are uneventful. That does not mean the product itself is boring. It means the mechanics of shipping are predictable. A developer merges a change. Autom...