Advanced outsourcing: What is spiral development and how can it help me with my software project?

(Note: This article was updated on July 19th.  Before, the model below was nebulously described as "iterative  development".  Now it is more accurately described as "spiral development". Additional detailed information has also been added on the waterfall method as well.)

The traditional way of developing software results in a shockingly high failure rate of 75%.   Agile development methods (such as the sprial method) were created to solve these problems and slash project failure rates dramatically.  They also allow you to create your software much faster, cheaper, more accurately and with far less risk.

This article explains the differences between traditional and sprial  development.  It also talks about the most effective ways to achieve spiral benefits using vWorker.com.

(Note: if you're an employer who wants to take advantage of agile methods, but don't want to bother learning all the details, then consider hiring a Tech Sherpa.  Sherpas are experts in advanced methodologies like this one, and will allow you to do this without lifting a finger).

• The traditional method (waterfall model):
  Traditionally, most software projects are done according to the "waterfall model". It's called this because the steps look like a waterfall:
  
  
  
  

To give an example: let's say you are creating the next Facebook. In the "requirements" stage, you create a complete document of the thousands of features you want the site to have. After taking a few weeks to do this, you move to the next stage in the waterfall: "design". When this happens, the requirements are locked and you cannot make any changes to them. This lock-down makes it much easier and quicker for the technical workers to their work (but as we'll see later is one of the main reasons this model results in software that is not acceptable to the end user). Theoretically, at the end, you should receive exactly what you wanted and are delighted. However in practice this rarely happens. A 2004 study found that 75% of projects following this method failed or were never used.

Entire books have been written on the many reasons why this model performs so poorly. Some of the biggest reasons are:

1. The myth that you can document all the true requirements in advance:
   
   The waterfall model requires you to document your entire software project 100% accurately in advance. If you don't, then you will end up not getting what you want. This sounds reasonable in theory, but in practice it only works very small software projects. On anything larger, you'll find that:
• You will simply forget to include some requirements. (No human being can design complex software 100% perfectly in advance).
• Others you will remember to include, but will be misunderstood by the programmer due to miscommunication and will end up wrong.
• Others will be perfectly communicated and perfectly implemented by the programmer. But when you see them live in the product, you'll realize they are wrong (should be done differently). Again, no human can visualize complex software 100% perfectly in advance.
• Others will be perfectly communicated, implemented and still be correct...at the time you created the requirements. But due to business needs changing during the time of the development (which can weeks, months or longer), the needs changed and they are now wrong. (Here's a real-life example.)
2. The myth that your programmer can estimate 100% accurately:
   
   Even if you create the requirements 100% correctly (which you can't) the programmer still will not be able to create an accurate estimate from it. Studies have found that time estimates (from otherwise excellent programmers) are too small by (on average) two to five times. The larger the project the larger the chance of underestimation. There are many reasons for this (which you can read more about here).
   
   The important thing to understand is that your programmer will probably seriously underestimate how much work the project will take. This leads to many of the missed deadlines and slow project delivery problems of the water-fall method.

The above are just a few of the many reasons that waterfall method projects fail so often and consistently.

 

• The spiral method:

  To deal with the unacceptable failure rate of the waterfall method, newer methods have been created. These are called "agile" because they are nimble and quick. They have a much higher success rate and allow software to be developed much faster, cheaper and more accurately. A very effective one is called the "spiral method". The spiral method, with an example of four iterations. This method results in much faster, cheaper, more accurate software development with substantially less risk of project failure. In the waterfall method, you wait months before you see the product live for the first time. With the spiral method you see live releases of your software much more frequently (often every week). And each release builds on and is an improvement on the last. Also, if your worker is not doing a good job, you'll know right away and can immediately switch to a better one (rather than having to wait until the very end). This method is called the "spiral method" because your software evolves larger and larger (like a spiral) with each release. Here's how it works. You start by identifying the smallest possible core portion of your final product. Your programmer designs it and develops it. At the end of the week, you get to view and try out the live software and see how it works. At that point, you'e reach the end of the first "swirl" (which is caused an iteration). You may find some things that are wrong. If so, you tell the programmer and they go back to work. The next week, you receive your next release. If it's perfect, you now tell the programm to add on the next most important core features, and repeat the process. It will take several iterations to get everything right. But they happen so quickly, that this doesn't take a long time. And every release, you see more and more of your software, until it's finished. Why is this a better way to create software? There are several reasons. Extremely accurate final product: The end result will actually be exactly the way you want it to be, instead of being incomplete or buggy. Ability measure and accurately manage the project: The spiral model lets you view/use the software after every iteration. Every week you will know: If your worker did a good job or bad. If your worker was fast or slow. If the project was on schedule or late. If the software was high quality or buggy. If the software met your requirements or didn't. If they are not doing a good job, you have the ability to recognize it early and switch to someone who will; instead of only knowing after they've wasted considerable amounts of your time and effort. Less arguing; more working: Workers in the waterfall method are in a mutually-opposed relationship with you. Every change to the software (whether caused by your missing requirements, or their misunderstanding of them or their under-estimation) costs them money. This often results in arguments over requirements, including needed changes. This can severely reduce worker productivity which results in slower progress. It can also cause you to have to constantly cycle through new programmers, which results in high turnover costs for you. With the spiral method, changes become mutually-beneficial. Now, instead of getting into fights over changes, you can harness them for competitive advantage. This results in higher programmer productivity which increases the speed of your project and reduces your time to market. And by creating long-term relationships, you also greatly reduce your turnover costs. Release to the public earlier: Unlike the waterfall-method, you don't have to wait until the very end of the project to release to the public. Since you're forced to work on the most important features first, you'll find you get a feature-complete product for your target market much earlier in the process. Once you do, you can release it to the public and tweak it with further improvements. This lets you get to market sooner and may mean the difference between having the first-mover advantage and losing it. To give a concrete example: If you are creating the next Facebook, the entire site will ultimately be hundreds of pages in size and encompass thousands of features. With the waterfall method you might take several weeks to lay all that out on paper. But with the spiral method, you don't. Instead, you identify the smallest possible core of the product. And after thinking about it, you realize it's the profile page with two things: the wall tab and the info tab. So you start with just that. Now you need some workers. You post your project and hire a programmer and a user interface (UI) designer (unless you are already an experienced UI designer). You describe what you want the profile page and tabs to do to the UI designer and they create paper mockups. After a few attempts they look good to you. So the UI designer passes them on to your programmer, who codes them into a live site that you can then visit in your browser. Unlike the waterfall method, all of this is very quick. In about a week, you're actually viewing your live site with the core working! Of course, once you see the working site, you'll realize it's not what you wanted. You'll see that you forgot some things, made some things too difficult and could do other things better. If you were using the waterfall method you'd be in trouble. But instead, you just tell the UI designer the changes you want and the process repeats itself. This time you might get a live release from the programmer in 3 days. It might be perfect this time; but if not you repeat the process until it is. Then you take on the next most important feature, in the same way. Incrementally, but very quickly, your site takes shape. You are able to build your site faster, cheaper, more accurately and quicker than using the waterfall method.

  
  

What payment methods work best with agile/spiral development?

The standard pay-for-deliverables payment method requires the worker to place a fixed-price bid on your project at the very beginning. In order to do this, they need to know everything the project entails. This requires you to define your entire project up front, which is the main feature (and shortcoming) of the waterfall method. So this way of working is incompatible with the spiral / agile method. Pay-for-time, on the other hand, requires no up-front requirements document and pairs up well with spiral / agile development.

Some may be tempted to try to use a modified version of pay-for-deliverables on an spiral/agile project by doing a new pay-for-deliverables project for each iteration. This is possible to do. However, unless your project is very small, you will find that the increased overhead required to define each iteration completely in advance will negate the main benefits of spiral/agile development: which are speed, and reduced costs.

A much better idea is to use one of the hybrid methods ("Hybrid Payment Model" and "Hybrid Payment Model + Sherpa"). These methods give you the protection of pay-for-deliverables for the first few iterations when the worker is new and untested. Then once they've proven themselves, you switch to pay-for-time. This gives you and them more flexibility, reduces your vWorker.com costs by 40% and allows you to enjoy all of the previously mentioned benefits of spiral / agile development.