Eric Raymond once wrote:
Every good work of software starts by scratching a developer's personal itch.
(from catb.org/~esr/writings/cathedral-bazaar/ )
And that's pretty much the gist of it. Almost all successful FOSS projects began because they solved a specific problem, whether that's an individual developer as ESR wrote, or an individual organisation. From there, these projects grew and became big because many people shared that specific problem, and it's easier to solve that problem together than starting out alone.
This is how Linux grew out of humble origins. Recall the very first announcement of Linux:
Hello everybody out there using minix -
I'm doing a (free) operating system (just a hobby, won't be big and
professional like gnu) for 386(486) AT clones. This has been brewing
since april, and is starting to get ready. I'd like any feedback on
things people like/dislike in minix, as my OS resembles it somewhat
(same physical layout of the file-system (due to practical reasons)
among other things).
I've currently ported bash(1.08) and gcc(1.40), and things seem to work.
This implies that I'll get something practical within a few months, and
I'd like to know what features most people would want. Any suggestions
are welcome, but I won't promise I'll implement them :-)
PS. Yes - it's free of any minix code, and it has a multi-threaded fs.
It is NOT protable (uses 386 task switching etc), and it probably never
will support anything other than AT-harddisks, as that's all I have :-(.
Despite having severe shortcomings (i.e. only usable on 386, can only use AT hard disks, it was already minimally useful. Having bash and GCC is significant; it means people can tinker with it via the commandline, and compile and run real programs on it already. Soon enough a handful of contributors added things like more hardware support, improved modules and so on. Distributions like Slackware and Debian sprung up making it even easier for people to use Linux. Meanwhile folks in the Free Software world, notably GNU, developed a symbiotic relationship with Linux: free software made Linux more useful, and Linux was a free kernel that could run free software without having to purchase obscenely expensive UNIX licenses.
So really the success of Linux could not be distilled down to one factor; it was a confluence of things that went right with it:
- Writing an OS from scratch is hard. At the time, perhaps less than 100 people in the world were capable of doing what Linus did. Linus was brilliant and obsessed, and had loads of spare time being a University student, but it still took him months of hard work to get to something useful. Anyone wanting to work on a free OS would have to replicate all that hard work. It was much easier to help Linus with Linux than to start from scratch.
- Linux came at a time with just the right infrastructure. Linus credits the internet and available GNU utilities like GCC as crucial for the early success of Linux. Without these, he wouldn't have had easy access to contributors to test and write new code.
- Perhaps most importantly, Linux scratched the itches of many many people. At the beginning, Linus wrote Linux due to being dissatisfied with MINIX. It just so happens that MINIX had many dedicated users (the comp.os.minix newsgroup had >40,000 users worldwide), who had similar dissatisfactions with it. The first version of Linux was released into a large and receptive userbase. As more people contributed patches to Linux, it became more useful and to more people, so the whole thing snowballed. People who wanted to use a free UNIX could use Linux, whereas other variants were either super expensive or legally ambiguous. Many people in the GNU community, who were working on a free OS for a long time, flocked to Linux as GNU's own effort went nowhere.
So now you can see why Linux is one-of-a-kind; its success would be extremely hard to replicate.
A lot of FOSS projects don't grow beyond its founder for a few reasons:
- It doesn't solve a problem. An obvious way to prevent this is to follow ESR's advice; make sure the project at least solves your problem; this guarantees that the project will have at least one dedicated user: you. But even so, the project could fail because it doesn't solve that problem well, because the problem is hard. Linux definitely fits this; writing an OS from scratch is hard, but fortunately Linus had the right knowledge and motivation.
- It doesn't solve other people's problems. This is probably by far the biggest reason why projects fail; there is simply no demand for it. Perhaps you're the only user who has the problem, or there's so many competing solutions that no one needs to use your solution. Linux was extremely lucky in this regard; its competitors were either expensive or useless, and it inherited a userbase with voracious demand for it.
- It solves the wrong problem than you think. There's a lot of unsuccessful, minimally-useful FOSS projects out there. I suspect many were started for the wrong reasons: often these are vanity projects; the author wants their own version just so they can say they wrote it, or the alternatives have the wrong colour or shape or whatever. If the problem is vanity, by definition, these projects can't solve other people's problems.
- It solves the problem too well. If the problem is small and self-contained enough, there's a point where the problem becomes solved and there is no need for further development. For FOSS, this means people can just take the project's code, incorporate it into their codebase and call it a day. Think about small parsers, algorithms and so on. Many of them existed as separate projects at one point.
It's ok to have lots of unsuccessful projects. In most cases, it's not because they did something wrong (in which case, someone could just fork it, do it right, and become successful). The success of projects depends so much on factors outside their control (market demand, timing, the nature of the problem).