I was interested in business before I was interested in tech but when I fell in love with computers and taught myself how to code in high school it seemed like an obvious marriage between my skillset and interests. At the time it felt as though tech was the highest leverage skill one could have when building or operating a business and it made sense that the darling companies of the day were all high growth tech giants like Google, Facebook, and before them Microsoft. These companies stood above most others because of the clear advantages and leverage that tech provided them.

The biggest advantage for software companies was zero marginal cost for increasing inventory. Consider a traditional automobile company that spends $10m to build 1,000 cars. If they successfully sell all those units they can only create another 1,000 cars if they spend another $10m and that’s not to mention the time and cost to get assembly lines and manufacturing in place as well as the risk of recalls and other manufacturing defects that might eat into the profits. This is different from software companies where once the product is built the marginal cost of producing 1 copy of it versus 10 billion copies is simply the cost of creating a new CD. Once the consumer internet became fast enough and companies moved off of CDs in favor of downloading software the marginal cost actually reached zero. Microsoft might have spent $1 billion to build Windows (random estimate) but the cost to them for selling 100m copies of it versus 500m was the same. This allowed tech to scale far better than traditional business models. Sure, car companies might see savings from economies of scale but that’s negligible compared to the scalability of software.

You might think the tradeoff for such scalability is higher upfront costs but upfront capex was almost always lower for tech than physical products and the margins much higher. It felt like every other billion dollar tech company was started by a couple friends in a garage or hobbyists and tinkerers plugging away at their code. There was not such a need for credentials or permission from investors and bankers just to get the funds needed to try a new venture. You could even build and distribute your software anonymously which opened the playing field to many people who were traditionally excluded from many avenues of business due to age, race, or sexuality.

Software didn’t just have better scalability and capex but also better logistics. You didn’t have to spend time talking to suppliers, getting factories set up, dealing with regulators and governments, etc. You could just start building. And if you were good at building and you built something useful then you could create products that were used by millions with just a few people. And even once it was it built changing it or upgrading it was far faster and easier than traditional products where you needed to update factory processes and deal with the whole cycle around that. Software engineers could move massive building blocks with a few strokes of the keyboard and make sweeping changes to mature products faster than ever before.

The leverage provided by software that was once the obvious differentiator between tech companies and the rest is no longer so apparent to me these days. Whether it was interning at Fortune-50 “legacy” companies, working at Microsoft, or hearing stories from friends at their tech startups the power of software to deliver business outcomes seems to be diminishing rapidly. Entire teams of engineers are employed to maintain simple features and new features are implemented at a snails pace. Adding new endpoints to an api gateway to enable a new product line at Microsoft was a multi-quarter endeavor. Changing simple UI components, fixing a deploy pipeline, adding a new api endpoint, fixing a bug and many other small tasks take weeks instead of minutes, even at tech companies an order of magnitude smaller than Microsoft. I personally have experienced this frustration countless times across every job I’ve had and every friend who works in tech has recounted similar frustrations even at some of the highest growth startups that are supposed to be agile and lean.

The experience I’m generally describing is work that Google calls toil : fighting with the tooling, debugging dev environment problems, fixing CI/test flakiness, and everything else that isn’t easily automatable but also isn’t novel work nor related to the core value prop that the software is trying to provide. It sometimes feels like upwards of 70% of an average engineer’s job is just toil these days except at a select few startups and I believe it’s why so many companies slow their innovation and feature improvements to a crawl. Why do Notion, Airtable, the social medias, GitHub, Zapier, etc. all basically feel like the same products they were 3-4 years ago? Maybe this is what happens at scale: your core value prop is baked in and there’s not much reason to rock the boat but perhaps I’m naive in doubting that many of these tech unicorns can justify their valuations in their current states. Airtable and Notion are two big examples here because they are solid products but in the enterprise space which they both seem to be committing their focus on they are competing with giants like Microsoft and Google. I don’t see how a smooth but narrow product like they currently have can outcompete Office. And just because you have a solid core product offering doesn’t mean you should take your foot off the gas in expanding its capabilities. Notion should want powerful spreadsheets embeddable in its documents and it should want notion pages that can become presentation slides and yet they don’t seem to be able to build these features and even if they were to, say, buy Airtable is anyone confident they could actually integrate the two products in a way that’s synergistic? Maybe I’m wrong to conclude that the reasons they haven’t done these things is because of all the time they’re wasting on toil and other tech debt but for whatever reason the challenge must feel too big for them and when you spend the majority of your time working on toil then it is too big.

My instincts tell me that as an industry we have taken two steps forward and four steps back in many regards. For starters, you can throw away the zero marginal cost of inventory. Not because you can’t distribute your product freely anymore but because it’s quite rare for a company to actually do so. Instead the dominant business model is to distribute a license to use the product which is running on hardware in the cloud owned by the seller. This means that to increase units sold i.e. increase the active users you need more compute and the seller is paying for that compute. The cost to Figma for selling its product to 1,000 users is not the same as selling its product to 1 million users. Software as a Service has changed the scalability equations for software companies.

SaaS comes with many benefits and there’s a reason why it’s become the dominant business model but it does mean that many companies have switched from product companies to quite literally service companies. How many of those benefits are actually software related versus business related? Investors and boards like recurring revenue because it’s predictable and can be planned around. It also means you can potentially earn more lifetime revenue for a single customer even if the product doesn’t change much. Whereas in the past you’d need to provide a new version of the product that is good enough to convince an existing user to upgrade, with SaaS they will continue to pay as long as they need their problem solved. Renting is more profitable than selling. But does it make it easier to build and distribute software?

The second problem that SaaS introduces is that products are never finished. In theory this is good: products should keep getting better over time but in practice this means that products are actually just sold before they are ready. This is not just a side-effect but an encouraged practice for startups and if you’re a founder or investor this is great! It lowers the upfront capex requirements for building software products even more and gives you an opportunity to pivot earlier if you do not have product-market fit. The downside is you become beholden to paying users before you are ready. This can distort the incentives of product teams and companies as a whole which leads to feature-bloat and tech debt that is carried throughout the lifetime of the company. It’s essentially a greedy algorithm for company building which disincentivizes proper software methodologies and patterns. Advocates for this playbook will correctly argue that code is not the product its just a means to an end but how many medium-size and growth startups are drowning under the weight of their tech debt? Would many tech unicorns be growing faster today without that early tech debt? Maybe scale is to blame here and there’s no avoiding the slowdown that comes with success but I’m doubtful that explains everything. Is the ship-fast-at-all-costs model of company building a local maxima?

If we ignore the change in business models around software I feel that the adoption of agile methodology in many instances has hurt rather than helped productivity for the same reasons. Anecdotally speaking the enterprise organizations I’ve worked for that touted their agile work environment have used it as an excuse for shitty planning and irresponsible creation of tech debt. Of course, leadership will never actually allocate the time to pay down that debt but they love to listen to the troops so be sure to point out the cracks in the system during your next retro. If you’re enterprising enough as an IC they’ll encourage you to fix those issues on your own time (you won’t have the resources to make the required cross-cutting changes ofc). I’ve even experienced the discouragement of any planning or technical specifications because it “wasn’t agile.” You combine this with micro-services and you’re guaranteed to reduce the pace of development by an order of magnitude and make the defining contributions of smart engineers the reordering of some buttons and the deployment of one or two new functions during their 3 year tenure at your company. This development style leads to a weird situation where so much engineering time is spent on random minutiae of these giant systems while at the same time the end-product feels so disorganized and unpolished. The rough edges show everywhere and the product improves at a snails pace because no individual or team is capable of moving large enough building blocks across the codebase.

Specifically relating to the tech we use to build I believe the unification around the web as the dominant runtime set us back in many ways. We had gotten pretty good at creating rich experiences using native apps and moving to the web nullified a lot of that tech tree. We became constrained by static hyper-text documents and over the past decade we’ve been reinventing all the tooling required to build those rich app experiences in the browser. The web also made us lose so much of the offline capabilities we used to build applications around. Maybe this helps for DRM and deploying updates but it means that any sufficiently powerful app needs to run on a server hosted by the company that built it and storage that used to come from a user’s disk now needs to be paid for in “the cloud” and the surface area for security vulnerabilities becomes a non-trivial problem. It did allow us to unify around Javascript (although unification is a strong word) and avoid cross-compatibility problems of different operating systems but I’m not convinced that’s a worthy tradeoff for what we have now. We are still hamstrung by supporting different browsers and mobile platforms while our best Javascript frameworks are only starting to catch up to some of the capabilities that we had a decade ago and the economic scalability of a software product is now worse than before.

I strongly believe moving to the web also changed the way dependencies are used in software development to focus on APIs instead of libraries. APIs are rarely conformant to a spec like OpenAPI and are difficult to customize when necessary. How much time is wasted because your startup’s subscriptions are handled in Stripe and your auth is handled in Auth0 and your secrets are managed in Doppler and half your application is spread across random AWS services so now a simple change to rename or add a payment tier to your offering is accompanied by the rewiring of all these disparate pieces tied into different places in the codebase. And if, God-forbid, you want to make a change or add something that is slightly out of the scope of what these services might provide you’ll have to hack around their API interfaces instead of just modifying or forking a library.

It’s not clear to me if this loss of leverage is even a real phenomenon or if I’m naively misremembering what once was. I wasn’t working professionally back before the dominance of web development and Javascript frameworks and everything else that has now become the standard. It’s also not lost on me how requirements and expectations of tech and its applications today have grown immensely even from 15 years ago. Security, redundancy, usability, speed of development, support expectations, feature-sets, etc have all become much more complex and harder to fully implement. The demands from customers and the problems that tech is being asked to solve continue to grow and it’s possible the change in productivity I am describing is a result of that.

After all, the cloud is ridiculously complex but it almost certainly lowers the barrier to entry by removing the need to manage physical servers in a fledgeling company. And for all the complexity of SaaS it does theoretically make software more profitable. Unifying around the web simplifies target runtime environments that need to be supported, deployments and updates can happen on the schedule that best suits the developer, and has lowered the barrier to entry even further by making Javascript a somewhat universal language.

These were necessary advances in the tech landscape because it means that many more people can build software to supply the exploding need of it. And maybe I have it completely backwards: tech has more leverage than ever because there’s almost no such thing as a tech vs non-tech company if you look at it through the lens of using software in daily operations. Every company uses software to solve some problem and an increasing number of companies need engineers to build custom software even “legacy” brick and mortar companies like Starbucks and Dominoes. Maybe a bunch of the inefficiencies and problems I see today with software is a result of this influx of inexperienced and often young cohort of engineers in the workforce and the fact that these legacy companies can build software at all should be considered a raving success. It’s obvious that a lot of the progress in the industry has been necessary and beneficial and maybe my disillusionment is the result of seeing how the sausage is made so to speak. I don’t think that’s the full picture here, though.

As I write this I realize I’m just describing the Software Crisis and maybe that’s why I’m so frustrated by these experiences. It feels like every new language, framework, cloud provider, and random dev tool is claiming to change the way we develop and provide the solution to this development problem we find ourselves mired in but none seem to deliver on this promise. Not only does it feel like this problem is worse than before it feels like it’s getting progressively worse. Surely as an industry we should be able to figure out why it often feels like making even small changes to codebases to add a feature or solve an underlying problem comes with so much baggage and effort for such little gain. As engineers we are failing to use tech to drive solutions for a business when we cannot implement necessary changes to products in an appropriate timeframe. Projects fail and companies die because of this failing.

I do not have a clear idea of what the solution looks like because it’s not clear to me if there is a single cause to this problem of increasing toil. We can start by creating better tooling to handle all the common problems that every SaaS product faces: pricing/account tiers, emails, common models like workspaces, organizations, users, etc, a variety of auth methods, trials, user invites, auditing, admin panels, OpenAPI-conformant endpoints, etc. There are some products in this space such as Ship SaaS, Go SaaS and SaaS Starter Kit as well as others, but none of these sorts of solutions seem to be mature and well thought out enough to really solve these problems at scale. Another thing we can do is create new frameworks that handle many of these problems far better than they do today. Rails is the best web framework I’ve ever used and does basically everything right but it simply relies on too much magic with untyped Ruby and meta-programming. Combine that with Ruby’s Object-Oriented design and the intense coupling to the DB with ActiveRecord and most Rails projects become incomprehensible at scale (I find this unfortunate because Rails is awesome and Ruby is far more beautiful than Javascript). At the risk of introducing yet another JS framework into the world it would be nice to see a Rails-esque framework written in Typescript with first-class support for reactive frontends and built-in standard library for solving most of the common problems web apps need to solve.

Either way I do think this problem is solvable because many startups seemed to have avoided it. From the outside companies like Stripe (a few years ago), Replit, Linear, Tailscale, Ramp, Clickup and some others seem to innovate at a breakneck pace. It’s hard to know if that’s because they’ve actually solved a lot of the complexity and toil internally or if they just brute-forced this pace of delivery but from the outside they are releasing features at a pace that most companies simply aren’t. It’ll also be interesting to see how the culture of these companies changes over time as they grow and scale and if most of their shipping velocity is behind them. Can any of their lessons and philosophies be systematized and scaled to larger organizations? To more organizations? Oftentimes these companies are successful because they have a unique composition of 10x engineers that drive the core growth in the early years when the team is small but that’s not a scalable strategy. As the need for software grows we need methodologies and tools that allow even 1x engineers writing software for your local DMV to ship quality consistently.

These days when I talk to my friends outside of tech I’m consistently surprised by how much better many of those opportunities appear to be. A friend working for a boxed wine startup that’s only been around for 6 months is seeing higher growth and revenue than many startups I know that have raised tens of millions — hundreds of millions even — of dollars. And between AWS and all the other SaaS subscriptions modern tech teams use the margins on the boxed wine is far superior! Maybe the low hanging fruit in the enterprise software space is simply gone or maybe the requirements and expectations have outgrown our abilities. For one reason or another it feels like tech has lost its leverage.