Beyond Scrum - The Evolution of Software Development

It might have seemed unkind to sack Scrum and Agile development methodologies in my previous post.  After all, they are an improvement on the time-honored "Waterfall" method. Few know that their predecessor Waterfall was itself based on a mistake, a mistake so appalling that even it's inventor, Winston Royce, wrote in the paper that delivered Waterfall to the world:

“I believe in this concept, but the implementation described above is risky and invites failure.”

The bar was low, and Scrum and Agile methods easily cleared it.  We can't forget, though, that they were the products of their time, and many of the things they took for granted may have applied only during that tiny sliver of Internet-bubble time.    Let's take another look at the implied software development environment of the approach that I wrote up last time (paraphrased):

• We’re all co-located...
• We don’t know how to write requirements...
• Our leader doesn't work here / is a consultant...
• Our entire team are newbies, so we'll
  • let everyone own all the code
  • track performance daily with check-ins and velocity
  • eliminate any concept of seniority
• We’ll code in sprints that last from 2-4 weeks each
  • it’s only payroll
  • if its something big we’ll just have the sprints go on forever
  • everything can be fit into a 2-4 week sprint
• We'll deliver software at the end of each sprint.
  • the software doesn’t have to be great, just delivered

These assumptions might have been been fine for Internet-bubble days, but in 2015 and beyond we can do better.  We won't throw the (Scrum-)baby out with bathwater, but that implied Scrum environment is no longer valid for software development teams today.

"The real voyage of discovery consists not in seeking new landscapes, but in having new eyes." ~ Proust "Form follows function" ~ Ludwig Mies van der Rohe

Let's start with some new assumptions and our development approaches will follow:

1. The development team is distributed.   EVERY team is distributed, even if they are co-located!   Work-from-home vs. work in the office.  "Morning people" vs. Evening (or 2:00AM) people, "flu season."    The first rule of new development is to have everyone on the team work in the best environment for them. The second rule of software is to enough eyes, all bugs are shallow. We will need great collaboration tools for software review and pair programming.
2. Timely delivery is nice, but compelling delivery captures markets.    The Apple App Store is a microcosm of the software industry overall: your software might not have 1.5 million applications competing with it, but you surely have competitors and (as Bill Gates and Mark Zuckerberg can confirm), in large and small winner takes all.   If you write software you better be ready to put a ding in the universe, otherwise you and your "snowflake" will disappear completely.
3. Different components have different delivery schedules.   The transition to microservices is well underway, but until that transition is complete we will be writing software with widely varying levels of granularity.   Form and schedule follow function -- let the software determine it's own best delivery schedule.
4. Timing?  Long sprints and short sprints are false timings -- in 2015 we deliver software continuously (with Jenkins, and increasingly with Docker) and take advantage of the heartbeat that the regular old workweek gives our teams.
5. Fast-Track everything that can be fast-tracked.   The Project Management Institute has the right idea here - serialization is the enemy of compelling, continuous delivery.
6. "Pull" development beats "push".   Pull has revolutionized manufactured components, why shouldn't it also revolutionize conceptual ones?   If you want compelling delivery (per point #2), then you need a buy/sell market where the customer has the right of refusal.

These are the rules -- software has come a long way since the days of Chrysler Payroll.  Let's move forward...

"Scrum programmers are destined to code Scrum sprints for the rest of their lives, and thereafter." ~ John Repko (after Bertrand Meyer)