Spreadsheets for the New Millennium -- Part 3
Monday, May 23, 2011 at 4:33PM 
So here's what comes next:
When I write about "spreadsheets," I'm thinking about technology bringing a real innovation to market. Spreadsheets were a breakthrough in modern business because they took new technologies - low-cost PCs, high-resolution displays and comparatively large amounts of RAM - and combined them into a facile metaphor that fit a rich set of problems. Hadoop and MapReduce are terrific but they are elemental -- they provide a rich, parallel, functional-programming approach, but they remain basically metaphor-free. They are to Big Data what Quicksort is elementary computer science -- a nice step beyond Bubblesort, but in themselves just tools. The Killer App lies elsewhere.
For that reason I think Datameer and Factual are a step forward in the routinization of big data, but I don't think they've got it yet either. The metaphor is still wrong. Visicalc and Lotus 1-2-3 were a big step forward because they gave a hands-on way for non-IT people to grasp the rows-and-columns world of financial analysis. The impedance barrier went away because you could make financial models in a visual domain-specific language (DSL) that mirrored the world you were modeling.
The DSL has to match the world you're modeling, thus I expect that jamming big data into a spreadsheet today will be like jamming financial calculations into Wordstar would have been back then. It's a step forward (maybe a big one) but the gestalt will arrive elsewhere. When I wrote "big data needs a spreadsheet" in the past Spreadsheets for the New Millennium what I meant was that big data needs a metaphor and a DSL -- a way to put big data understanding into the hands of everyday users. Putting big data in a spreadsheet is a start, but these aren't rows-and-columns problem domains and stuffing them into rows and columns might provide some facility, but at a cost of richness and understanding. Big Data deserves its own metaphor and a DSL ... somebody's incubating it ... even as I type this ... now, where is it??? In my next post I'll lay out a few steps to the epiphany.
