To Open Source or Not
Open Source is an idea that a lot of people talk about these days. To some it’s a solution that allows a company to harness the power of thousands of individuals globally to help build a product. A development or test team for free! To others it’s an evil, value destroying phenomenon, aimed at nothing less than the destruction of intellectual property as a business. A communist plot!
What is the truth, and should you use open source in your products, or participate in open source projects?
A few weeks ago I was challenged to think about the use of open source in product development as a way to accelerate the development of a product, or platform. Specifically, the question was whether or not you should endorse or start an open source project to gain share in the market. My gut reaction was to say no, because I’m a strong believer in the valued of owned intellectual property. My natural reaction is to want to own the IP developed, and build a business around that IP.
Obviously, I have biases. I like to think, however, that those biases are rooted in facts and lots of experience with the developer community. The following opinions are drawn from:
1) Extensive reading on open source, both on the net, and from the shelf full of books I have on open source and the people who started the movement. In particular, Eric Raymond’s famous article, the Cathedral and the Bazaar, is worth reading as are some of the essays in the book titled “OpenSources, Voices from the Open Source Revolution” published by O’Reilly. In particular, Bruce Perens and Eric Raymonds (from the Open Source Initiative), Brian Behlendorf (Apache), and Michael Tiemmann (Cygnus, now Red Hat) are worth reading.
2) My experience at QNX where we made extensive use of open source componentry in the product, open sourced several portions of the product, and participated in open source projects initiated by others.
3) My experience working with the group at IBM/OTI who set up the Eclipse open source project, which was launched while I was at QNX, which we participated in, and which I was asked to think about the business strategies involved in participating.
4) My experience at Microsoft where we fought Linux, participated extensively in standards organizations (and standards bodies, I will argue, are attempting to achieve the same goals as open source) and actively considered open sourcing some components of some products. In fact, since I have left, much of the Windows CE code is available under a “community source” license, as are significant portions of .NET.
5) My experience launching the UPnP.ORG effort, which was an open development effort to get device to device interactions created, and documented.
For most companies, my recommendation would be:
1) Participate in standards bodies as appropriate.
2) Hold off on open sourcing any of your code until there is a need, and a series of other conditions (documented later) are met.
3) Comb the open source world for any components that you can add to your offerings to enhance them at low / no cost.
The genesis of open source
To understand the open source movement, you need to understand the history, because it explains the mindset of the individuals who participate. In the late 1970′s and early 1980′s, AT&T’s Unix was freely available to anyone who asked for it. Before breakup, AT&T was not allowed to participate in the business of being a software company, so they freely gave away the OS that they were developing. UC Berkeley was one of the recipients of this largesse, and a whole culture of hackers sprang up, modifiying and working on the OS to suit their own needs. It was ported to PDP11, VAX-11, and a variety of other architectures. The code was widely regarded as being like scientific knowledge — intended to be shared.
During the AT&T breakup, that source code base was identified as an asset, and taken back into the corporate world. The hacker worlds response was embodied in Richard Stallman, who started the GNU project and the free software foundation to recreate AT&T Unix without using AT&T code. The BSD releases of Unix were born as a result, and the tradition of trading code amongst individuals, commenting on that code, and returning the code to the community continued. Stallman created the GNU Public License (GPL), which is one of the most widely used licenses today, which explicitly forbids the distribution of code without a source license, in order to preserve this tradition.
People who participate in open source projects do it for a variety of reasons, but it’s not generally to get paid. They participate in order to earn the respect of their peers, in order to be able to work on neat stuff (not everyone gets to hack on the OS kernel), and in order to be able to discover new ways of doing things. They have an “itch to scratch”, as Stallman says, and scratch it they do.
During the 1990′s, one of the later open source projects – GNU/Linux – became powerful enough that a movement of business people in places like IBM, Sun, and Netscape determined that this code base, and the GPL license, might prove to be a significant threat to their biggest nemesis at the time which was Microsoft. The hacker tradition was already old by this time, though. Contrary to popular belief, it didn’t spring up as a result of Linus Torvalds deciding he wanted to bring down the biggest software company in the world. Linus was a student in Helsinki, motivated by something else entirely. In 1991, Windows was a clunky imitation of the Mac, not quite altogether functional. “Real programmers” used Unix, and Linus simply wanted a Unix implementation on his PC. He liked the idea of putting a Unix-like OS onto a 386, and wanted to prove that a monolithic kernel was more powerful than the micro-kernel that was being touted in academic circles (the CMU Mach architecture), and by folks like Andrew Tanenbaum of Minix fame.
It was the power of marketing in the late 1990′s and the endorsement by companies like IBM nearly 20 years after the movement began, that gave the open source movement strategic legitimacy, and made it attractive to some companies.
Why choose to participate in open source?
Today, a company generally chooses to participate in open source / open standards projects for one of two reasons:
1) The company is facing a large industry incumbent, with a proprietary technology that gives the incumbent market power. Banding together with other second tier players provides scale that can potentially match that incumbent. Agreeing on common standards, or a common source base, gives customers for those components the comfort of knowing that they are replaceable in the event of a business failure, and spreads the workload of developing and maintaining those components out over a larger group of individuals than any one of the participants has alone. Commoditizing a component of the value chain can introduce pricing pressure on the incumbent. Open source / open standards initiatives are a common strategy for a number 2 player to use when combatting a strong, well funded and dominant player.
2) The company is seeking to accelerate the development of its own product by tapping into the pool of talent that exists in the open source community, rather than hiring and building software components from scratch. Tapping open source gives access to, often (but not always) high quality components, that can be readily integrated with the company’s own products.
For most companies, there are three ways to look at using open source software in a project:
1) Participating in standards organizations around the API’s and protocols they intend to use in their products. The objective of this participation is to ensure int
eroperability between the company’s products and other products in the marketplace, giving the customer the benefits of a higher degree of interoperability, and replaceability should the company’s business fail.
2) Starting an open source project using a portion or all of their software. This is much like choosing to participate in a standards effort, except that much of the standard is already created — it’s defined by the code. The objective of this participation is to accelerate the development of the company’s code by making the source freely available to any programmer on the net, and then incorporating worthwhile changes into that code.
3) Incorporating open source code the company’s products as a way to accelerate time to market, and reduce costs. There are many high quality components, ranging from databases to operating systems and GUI’s that could be used in this fashion.
Standards body participation
Participating in standards organizations is beneficial to most companies, whether they participate at an executive level, a contributor level, or an observer level. In today’s market, no company can afford to ignore industry standards. Standards body participation gives you the necessary optics into how a standard is evolving to make informed decisions about how your product should implement those standards. Participation may also give you the ability to influence the outcome of the standards effort if it is going in a direction not conducive to your business.
Few companies will be able to set defacto standards as Microsoft did in the late 1980′s and early 1990′s.
Open sourcing code
The case for starting an open source project is more complex than simply participating in a standards organization. In general, successful open source projects have the following characteristics:
- a body of quality donated source to begin working from. For IBM’s Eclipse donation, 80 man years of code were delivered to the consortium as a way to get started. In the case of GNU/Linux, the GNU project had already built all the libraries for a free Unix in the 1980′s, and were working on the last component — the HURD kernel — when Linus Torvalds kernel was introduced to them in 1991. Netscape donated their source base to Mozilla.org to get that effort off the ground.
- a dedicated core of individuals to move the project forward. A common misconception is that millions of coders contribute to an open source project. This is marketing. The Linux Kernel takes contributions from a vast number of individuals, but Linus freely admits that himself, Alan Cox, and one or two others are the prime contributors. Their biggest headache is wading through the mountains of stuff that gets contributed, and figuring out what should make it into the final distribution and what should not. Similarly, mozilla.org, while taking contributions from huge numbers of people, employs 14 full time staff in order to keep the project running. Brian Behlendorf, at Apache, estimates that 3 full time people in 5 different roles are required to run an open source project — and not surprisingly, they’re in roles such as development, debug, test, build, etc. Eclipse.ORG has several dedicated IBM employees, plus source code maintainers for each of the major areas from the major companies.
- a platform nature. Successful open source projects typically involve a requirement for interoperability. Linux, Apache, Sendmail, Build are all great examples of successful open source businesses. Open source efforts to build clone applications, like word processors and graphics programs, have been much less successful. If the requirement for interoperability / modifiability is missing from a project, then open source vs proprietary simply boils down to a matter of pricing or personal preference (aka religion).
- readily available inexpensive hardware. The free software movement was born in the 1980′s after AT&T pulled the licensing rights to Unix back. This so offended the hacker community that had developed around Unix, that they decided to build their own version. However, these early free Unixes still required expensive hardware to operate. By 1991, BSD was being shipped to “several hundred” individuals per release. It wasn’t until Bill Jolitz did a port to the 386, and released 386BSD later that year that the BSD release of Unix started to take off. Similarly, the genesis of Linux sprang from the following around Andrew Tanenbaum’s Minix in the late 1980′s, on PC hardware, and the fact that Tanenbaum’s licensing terms prevented the hacker community from porting Minix to the more powerful 386 processor. Linus Torvald wrote his own Unix-like kernel on the 386, and Minix was relegated to being not much more than an interesting historical footnote.
- relationship to other open source projects. There are few Windows-based open source projects out there. The hacker community tends to prefer free OS’s, eschewing the commercial OS’s as “not open enough”. Linux is the preferred platform.
If your project doesn’t have these characteristics, or you don’t have the resources to devote to managing the project, then open source is likely not for you. If you do fit the profile, then there are other considerations as well.
1) Which open source license to use? The GPL is totally IP unfriendly. There are many companies which will never by code based on the GPL. It’s an obvious non-starter. But a license based on the IBM Community Source License, or the Mozilla.ORG license could work. One strategy which has been successfully employed by TrollTech in Denmark, is to have two licenses for their software. Qt (their graphics package) has both a commercial and a GPL version. Those who want to just create cool stuff using Qt, and distribute it “not for profit” may do so. Those who want a commercial license may also have one. The GPL license acts as a gatekeeper here, making it unattractive to ship Qt commercially without a commercial license, but giving free access to those who want to do non-commercial work.
2) Who will be part of the organization, and how will it be run? Successful open source projects have defined rules for who can make a contribution, who not, etc. And, successful open source projects have more than people from just one company governing them.
3) Making the source code ready for public consumption — this involves stripping out proprietary components, cleaning up documentation and comments, and (in some cases) restructuring badly written code. The last thing you want is for the hacker community you’re trying to attract to look at the code and say “it’s terrible code”. In Netscape’s case, this took most of the development team there a period of 60 days from mid-January to Mid-March 1998 to do, before they could switch on Mozilla.org.
From a competitive perspective, an important question to ask is whether you need to open source any of your code. Is there an incumbent that has an applications lock that you must unseat? If the answer is yes, are there other means that you can employ to do so without giving away your intellectual property? Are developers already looking for an alternative, and willing to talk with you? Open source is an all or nothing proposition, so try the other tactics first.
What about open sourcing new development? Perhaps you’re looking to create a ground swell of support in the open source community for your product, and get key pieces written more quickly and more cheaply than doing them in house. Can an open source project achieve this? The answer is not likely. An open source project is likely to fail here. Why? The whole concept of open source is that freely available code creates a feedback loop which causes that code to continue to be improved and redistributed to the communit
y. Historically this has happened around an existing codebase. Will developers want to write new code, and then give that code back to their competition? Highly unlikely. The developer community that would most likely respond to an open source initiative would be the hacker community, assuming that they have an interest in creating your kinds of applications.
To successfully open source new development, you need to have developed quality v1′s first. That’s the hard nut to crack. Without a code base to work from, the community won’t do it on it’s own – the effort is no different than a standards effort at that point. In the QNX world, no new projects started as a result of the free download / open source effort. Many pieces of code were ported from the Linux world, but that’s all. In the UPnP world, reference implementations from Intel (on Linux, coincidentally) resulted in the first UPnP implementations. The lower quality Microsoft UPnP implementation was ignored by the community. Quality and completeness of the source base matters.
Using open source code
Perhaps the most promising aspect of open source in any business would be to simply leverage some of the code that’s out there today. Using open source in your products could help you get to market more quickly, and potentially reduce the cost of our product. So, for example,
Open source operating systems. Carrier Grade Linux, for instance, is a project to get a redundant, carrier grade version of Linux onto the market, with high availability features built in, some of which are the same or similar to the features built into commercial OS’s. Choosing to use CGL could drive down the cost of new telecom systems.
mySQL, or PostGres – SQL based database servers. You may need to add database support to your project. Both of these products are stable, use industry standard SQL, and could be replaced by a more expensive Oracle, SQL Server, or other license if the customer needed to do so.
Eclipse - a big problem for tools vendors is the support of a solid integrated development environment. Eclipse is an IDE framework that frees tools developers to focus on the tools, and not the wrapping around them.
And there are many many more open source projects you could tap into.
There are risks associated with open source, however. Some customers will demand to know the pedigree of every piece of code in the system. Some will demand that certain license types (such as the GPL) be excluded from products we sell to them. Some licenses leave us vulnerable to intellectual property infringement claims, should they arise. At QNX we managed this risk by building a system to tag each module with information that revealed the provenance and pedigree of each and every piece of code, and, for a fee, we would rewrite open source code if the customer required it.
Open source is neither the demon that some believe, nor is it the panacea that others portray. Depending on your business model, open source development can be a useful model to exploit in some parts of a project. However, one shouldn’t have any illusions about either the productivity of the project, or the management costs associated with it. Open source is just one way to get a product built, and the evidence suggests that it is neither more, nor less, effective than traditional methods.
Selected writings from Eric Raymond – including the Cathedral and the Bazaar. http://www.tuxedo.org/~esr/writings/
A link to the Microsoft “Halloween Memo”, leaked onto the net, and annotated by Eric Raymond. Discusses Hacker culture, and the implications of the rise of open source to MSFT. Vinod Vallopillil was Jim Allchin’s technical assistant at the time. http://opensource.org/halloween/halloween1.php