Think of this as Volume 15, Number 9 of A-Clue.com, the online newsletter I've written since 1997. Enjoy.
There is no Moore's Law of Software
Just the opposite.
Ironic that in this awesomely complex world of the early 21st century our most important product remains hand made.
There have been breakthroughs since Grace Hopper pulled the first moth from inside the relays of ENIAC and wrote in her log that she had debugged the computer, but very few of them.
My dear wife, for instance, who began her career in the 1980s writing in Assembly Language, then moved on to Cobol, is now learning C++. Each evolution makes her more productive, but only arithmetically so. Meanwhile the complexity of hardware, and its capability, keeps expanding geometrically.
We keep falling further behind.
Writing software is like building a house of cards. I learned this in trying to learn programming myself. Nothing works until suddenly everything does. It's incredibly frustrating. Dear Wife compares the process to doing crossword puzzles, but what if you felt no accomplishment until the whole puzzle were done, done precisely, done correctly?
It's hard work.
As programs become more complex, the chances of making a mistake, of having an error, rises toward certainty, then blows right past it. Because there are other types of errors besides those which shut things down:
- Hidden errors, instructions that allow for mistakes, aren't tripped until the right (or wrong) piece of data comes along.
- Security holes aren't errors at all. They're just places in the program where a clever programmer can insert some nefarious bit of code, where its function might conceivably be hijacked.
- Intellectual property errors aren't errors, either. They're often found by lawyers, who are constantly searching programs (or having others search them) for sections that might have been used before, thus a copyright violation, or that do something their clients have protected through a patent, which should not be granted to software but are, and are often written so broadly that everything you might do is covered.
Each of these types of errors have created whole industries, which create new software that tries to deal with the errors. Software is audited for mistakes, it's checked constantly for security holes, and it's audited again for intellectual property violations.
And almost each time this is done some new bug is found. Some new problem arises.
The industry has come up with creative ways to deal with this over my career.
One way is through the "stack." You layer programs by purpose, treat each separately, leave it to specialists. The LAMP stack so beloved of open source programmers consists of four different layers:
- Linux, the operating system.
- Apache, the Web server.
- mySQL, the database.
- PHP, Perl or Python, the language in which new programs are written.
All this has to work before you write anything that actually does anything.
Dear Wife prefers that kind of program. Real time application programs are her thing.
She likes it when software does something useful, when it can "make money" each time it runs. (Even if it's just a penny or two it does add up.) Unfortunately her programs, too, become increasingly complex with time. New customers use them, there are new (or old) types of hardware, there are new conditions that must be met for industries, for sets of customers, or for regulators. My point is that even a process that seems simple and uniform, over time becomes anything but.
It's hard work.
There's another way the industry simplifies things. Every once in a while they blow everything up and start over.
Throughout my career I've watched new industries emerge, which all manage to use the same types of applications, which are then rewritten to fit. Take a simple game like Solitaire. There were solitaire games around when I got my first PC almost 30 years ago.
- They were rewritten for DOS.
- They were rewritten for Windows and the Mac.
- They were rewritten for new graphics and then put on CDs.
- They were rewritten for the Web.
- They were rewritten for phones.
- They're being rewritten again, as we speak, for the iPhone and iPad.
- I'm sure they're being rewritten as we speak for "the cloud."
It's not that Steve Jobs is perverse. But as I said before an operating system is like a giant house of cards. The foundation doesn't really gain stability as it grows more complex. Just the opposite. So like a 19th century cotton farmer, or a 20th century industrialist, every once in a while he sets off for the territory and creates a "green field" to grow on. Which makes everything old new again. It all has to be redone.
Sometimes, of course, you can't do this. You can't just pick everything up and move down the road. That's where open source enters the picture.
Open source lets you build the software equivalent of cities.
With open source all the work is shared, because the code is visible to everyone. (That's what the open part means — it's not just a business model, it's also a development model.) Lots of businesses of many types can co-exist along the same road, sharing the same sewer and water systems, the same electrical and telephone wires.
It gets hairy at times. Deep in the stack, in the operating system, where Linus Torvalds (left) works near Portland, it gets incredibly complex. So many changes which have to be considered and approved, so many bugs which have to be fixed and tracked. I'm glad he and his team like the work. It would drive me crazy.
It is possible to build complex systems without open source. But as they're built they inevitably become monopolies. We only have one phone company at my home, one electrical company, one gas company and one cable. We sometimes pretend to competition in these areas — some places have "overbuilt" cable, the phone and cable pretend to compete, there are multiple "gas marketers" in my area. But even this limited amount of "competition" can't work without a lot of regulation to keep someone from using their control as a choke point and holding up the whole town.
Software is the same way. Microsoft works. Oracle works. But over decades, as the code bases have become ginormous, both have become choke points, and both have sought monopoly rents from customers. Microsoft's monopoly was eventually broken by open source. Oracle's control remains, as it buys out companies that might grow into competitors the way an old lion drives other males out from the pride.
So this is where we are. There are, in software, two worlds. One is proprietary, leading eventually to monopoly. The other is open source, shared infrastructure many can build on. The two compete with one another, the monopolist and his ecosystem on the one hand, open source and its ecosystem on the other. There is balance, competition.
Without open source, the whole software house of cards would collapse into monopoly. The democracy and capitalism we so pride in our tech industry would, without open source, become dictatorship and a command economy. One nation under Steve. Or Bill. Or Larry.
Open source prevents monopoly rents. It lets us build giant software cities with shared infrastructure, even while allowing giant enterprises to build their own cities, with infrastructure controlled from the top.
Take your choice. Windows or Linux, iPhone or Android. They work because the competition of open source makes it possible. The competition of open source keeps even the largest companies on their toes. And it gives small ones a leg-up.
