Devconf.cz This year, in addition to the usual in-depth technical presentations on Linux at Red Hat’s Devconf.cz developer conference, there were also several people on hand to promote AI-related projects and the techies’ previous favorites: blockchain projects.
The promise of these projects is that they will change the world and free us from burdensome financial regulations, government oversight and taxes, and so on – leading to the undefined and empty (but undoubtedly hugely lucrative) web3.
While this stuff is now everywhere, its connection to open source Unix is not obvious, and it is not directly related to FOSS. Of course, the algorithms commonly used to generate and update blockchains are largely open source. So are many of the tools that generate the massive statistical text prediction models that these marketers desperately want you to believe are functioning artificial intelligence. (It’s not, and the proof is that they’ve shifted the goal by coining the new term “AGI” – artificial general intelligence – for it.)
But while the tools are FOSS, what they do with them is certainly not. Moreover, it is clear that the companies building large language models do not own the data used to generate the models.
The presence of some of the most predatory companies in the industry is an unfortunate, but almost inevitable, consequence of the market forces that have developed open source software into the multi-billion dollar industry it is today.
We thought it would be interesting to try to figure out how we got here, to see where some people want to take the software industry and dig deeper into why that is. That in turn leads us to how it is done and whether there are alternative directions for the software industry.
The commercialization of software
The software industry, which began roughly in the 1970s, has undergone a similar development pattern to that of other manufacturing industries since World War II. Market expansion leads to commoditization, which reduces scarcity and drives down prices and perceived value. Market growth involves simplification, reduced choice, and the elimination of less competitive suppliers.
Today, market forces very similar to those that drove the development of FOSS are driving AI into a kind of race to the bottom.
Are there realistic alternatives? Yes, there are, and they’ve worked before… but they’ll require changes in methods and attitudes. There’s a silver lining, though: the tech bros will hate it.
First came the commoditization of hardware
The necessary condition for the commoditization of software was the earlier commoditization of hardware. Programmable digital electronic computers have been around for about 75 years, but for the first third of that time they were rare and expensive. It was only with the invention of microprocessors in the 1970s that they began to become cheap and plentiful.
Microprocessors began a pattern of hardware development that is being repeated today. In the early days, there were only a few early microprocessors and a few companies that sold the components to build your own computers from those parts.
Then in the 80s there was an explosion of varieties: Hundreds of manufacturers, thousands of models, mostly incompatible with each other. Each successive model was faster, with more storage and better capabilities.
More and better computers led to more and better software. When programmers had to manually type bits into the front panel, everyone kept things as short as humanly possible. When professionals could afford a decent editor and compiler, and floppy disks to store things on, programs got more ambitious. When developers had GUIs and hard drives, software started to get fancier, and when it could safely be assumed that customers had those things too, applications started to compete on style as much as functionality: Simple and, most importantly, attractive UIs; not just ease of use, but ease of customization and integration with other tools.
Then the marketing of advertisement software
Sales, however, breed competition, and competition breeds the elimination of weaker rivals. The 1990s saw an era of consolidation, as the majority of manufacturers of both components and entire computer lines folded. Only the strongest survived: x86-compatible processors (and a few RISC chips, which themselves consolidated into Arm); multiple ports and interfaces disappeared, replaced by USB and SATA; and so on.
The same thing happened with software.
Early commercial programs – all different types, from applications to games to OSes and tools – were written by hand, often by individuals. As specifications grew, and the software grew to fill them, this led to ever-larger teams of developers, and specialist roles, and entire management practices.
Dozens of weird and wonderful operating systems and their apps, and the programming languages and entire toolchains that built them, quietly died out. What remained were a few flavors of Unix and Microsoft Windows.
Commercial software for the masses followed the same pattern. While dozens of word processors and spreadsheets once competed on their unique text-only UIs, features, performance, and speed, standardized GUIs lost much of their distinctiveness. Few apps and few app vendors survived the transition from console apps to GUI apps. As apps grew larger and more complex, they became, almost paradoxically, cheaper—partly because operating systems provided more and more supporting components, partly because development tools provided more and more assistance to developers, and partly because of the ever-decreasing need for efficiency.
Just as hardware standardization on the x86 PC drove multiple hardware vendors out of business, software standardization on Windows drove multiple software vendors out of business. Operating systems gradually absorbed third-party tools, killing off those vendors as well. Software vendors began bundling, acquiring each other, and combining separate products into suites that sold for little more than the price of the originally independent products.
When there was only a limited choice of very similar packages with very similar features, other steps became necessary to maintain sales, such as vendor lock-in: changing file formats so that even satisfied customers had to pay to upgrade so they could still read other people’s documents.
Worse still, computers that are online everywhere now have even more severe restrictions, such as subscriptions that require you to keep paying to access your own files, even though you don’t get any useful upgrades for your subscription fee.
Also the marketing of free software
Some of the pressures that drive down the prices of commercial end-user software do not apply so much to server software. When tools work naturally by communicating over the network using standardized protocols, vendor lock-in becomes more difficult. Not impossible, but more difficult.
Software efficiency is lower than ever and bloat activity is higher than ever, but sales are held up by the constant customer demand for security patches.
We think of the great Douglas Adams:
“It is very easy to be blinded to their essential uselessness by the sense of satisfaction one gets from getting them to work at all. In other words – and this is the rock-solid principle upon which the entire success of the Corporation throughout the Galaxy is based – their fundamental design defects are completely hidden by their superficial design defects.” – Goodbye and Thanks for All the Fish
This only works for so long. Software can only get bigger and slower as long as computers get faster, and the rate of improvement has slowed dramatically there and shows no signs of recovering.
With FOSS, however, the pattern is a bit different because of the nature of the consumers. Open source OSs and server software have not outdone each other in terms of usability, for example.
Desktop FOSS OSes and their associated apps have been good enough for over a decade. They’re faster and cheaper than their commercial rivals, just as secure, and just as powerful. But they’re not identical, and many people don’t want to relearn or take the time to switch. Just as one T-shirt looks a lot like another, some people are happy to overpay for brands they like or want others to know they can afford.
Free software that works well enough but requires a little more effort from the user appeals to a certain kind of people, and these are often the people who work with computers.
FOSS is not successful because it made hard things easy. It wasn’t. And it still isn’t easy. It is successful because it made hard things cheap.
(One proprietary computing vendor successfully retained its market share because it discovered it could use lots of FOSS to build shiny, easy-to-use consumer products with beautiful apps, without the significant expense of building the entire software stack from scratch.)
On the server side, however, the price race to the bottom has yet to go on. There is little customer loyalty because the kind of techies who work on this stuff enjoy learning it. With current open source licenses, if desperate developers or vendors try to dictate prices, users can simply fork the latest, completely free version and continue using it for free.
The result is that the price can drop all the way to zero.
Commoditization has gone all the way to network protocols. Today, almost everything runs not just on TCP/IP, the lowest common denominator of network protocols, but specifically on web protocols.
In a market of free tools on free operating systems, this offers one last source of revenue: the lure of Software-as-a-Service, so tempting to many a slightly technophobic financial director. No need for all that annoying fiddling with infrastructure, expensive hardware and complicated software. That all becomes someone else’s problem, leaving you with trivial implementation details such as where your data is, who else can access it and whether there is a way to get it back.
As capacity in the big data centers climbs toward effective infinity, and storage costs shrink to near-zero, and software costs shrink to near-zero, executives must look elsewhere for ways to cut costs, or increase margins, or gain some other advantage. That’s what we’ll talk about next. ®