The Software That Links It All

Not long ago, when you kicked off a new software project, perceived advances in technologies and methodologies made it seem desirable to start with almost EVERYTHING new. Each project selected from among the latest frameworks, toolsets, and languages, and data modeling, tiering, and presentation approaches. With so many unknowns and so many decisions to make the resulting projects started on shaky ground, and many fairly promptly resulted in failure.

Hope was on the horizon, but it's taken some time getting here. Back in 1996 Ann Winblad spotted the way out. Brad Cox's ideas of a software factory had some truth to them, and Winblad spoke of a future when she wrote:

Small developers have become specialty parts suppliers to the growing population of software assembly-line workers.

She was singing the praises of "object oriented" programming, and with time the great truth here has only gotten truer. The key was not "object-oriented programming" over its predecessor "procedural programming", but a design doctrine that is far more profound: Convention over Configuration.

The right conventions get things rolling sooner and more surely, and let development teams be more starfish than spiders. As approaches and toolkits become conventions there are far fewer niggling decisions that need to be made, and everything can move a lot faster.

A couple of years ago, hiring a good team, getting set-up, writing a decent spec and delivering salable code for a modestly-scoped project might be done in 6 months. Today the same tasks, with still-greater use of conventions, might be done in half the time. This for projects that might have taken years back before the Internet era.

There are more tools and conventions than I can write about in a single post, but to give you some idea of the basic toolkit -- all tools that can be chosen conventionally and adopted -- just among the FOSS toolsets we have

• SCM tools (Subversion, Git...)
• Cloud providers (Amazon, RackSpace, Google...)
• Virtualization environments (VMWare, Xen...)
• Conventional Relational databases (mySQL, PostgresSQL...)
• "NoSQL" databases (MongoDB, CouchDB, SimpleDB, Cassandra...)
• Mid-tier frameworks (Rails, Django, Sinatra, Merb (now merging into Rails)
• Languages (Ruby, php, perl...)
• Presentation frameworks (JQuery, Prototype...)
• Testing frameworks (RSpec, Cucumber, Selenium...)

There are lots more, but the remarkable thing about the design of modern frameworks is that each layer is self-contained and with the progress of the codebases it's possible to swap components in and out while maintaining the core of a working application. In each of the major projects I've worked in over the past three years, we've swapped framework components without affecting the overall schedule or milestones.

Rich frameworks and solid conventions count for more than you might think. In 2010 the question is no longer "can we build it?" but rather "what is most worth building?"

The software that links it all is ushering us all into a new age....