My company has a large software project that is failing right now

Yep, that's right. The project is about 3 years old. This development project was launched about 1 year before I joined the firm. I was assigned to work on it very briefly (meaning two months) during which time the two men working on the system were trying to close it out. This was last year. I rarely had anything to do during that time. My plate was empty during those two months. I think I wrote 3 classes for them. That was the end of my contribution to this project.

I know something about this project, but not everything. I think I know enough to comment intelligently about just what went wrong. So what are the characteristics of this project?

1. The project was started by a (then) 51 year old programmer with 3 years of VB experience (at Microsoft). I shall call him Agent 51.
2. This programmer executed a midlife career change from medical nurse (or maybe an orderly) to VB programmer approximately 6 years ago
3. The project is written in VB.NET 8.0
4. Option Strict Off, Option Explicit Off, Option Compare Text.
5. It is a classic WinForm application (not WPF).
6. It is just CRUD forms over database tables.
7. Forms are bound directly to the tables for the most part.
8. Interactive searches are accomplished via stored procedures.
9. It does make use of the Microsoft Enterprise Data block.
10. There are very few classes in the project.
11. There is no inheritance from abstract classes.
12. There is not one original Interface declaration or implementation in the code.
13. There is no Dependency Injection. How can we have DI? There are very few classes.
14. Parameter passing is never done by abstract class or interface type.
15. There are lots of global variables in several global BAS files.
16. There are lots of global functions in several global module files.
17. The application's actions are stitched together and coordinated by these global, mutable flags.
18. When compiled, the compiler issues many warnings that not all function code paths return values.
19. Most of the important business logic is tucked into event handlers.
20. There are side effects in all of the event handlers
21. Event handlers can be 100 lines or longer
22. There is no IoC.
23. There is no test project, no NUnit, no Microsoft Test, no MBUnit. We do not have automatic testability
24. There was no source code control for the first two years on the project
25. The two programmers working on the system frequently collided, stepping on each others changes.
26. Code collisions were unintentional. One man was working on one bug, another was working on another. This strongly implies a lack of modularity and isolation. This also implies problems with global flags in the global bas files.
27. We have no formal QA system. There is no QA department. There is no QA officer who is responsible for testing and validation.
28. This was a skyscraper project without blue-print. The objective was to build the "Everything" application, but they didn't do much design up-front. The notion was that this was a simple project. All Agent 51 needs to do is make some CRUD forms.
29. There are two main managers in charge of the two major departments who will use this app, if it is ever finished. Neither manager wants this application to launch. It is difficult to determine whether they are afraid of the potential efficiency gains and subsequent layoffs this app will create, or whether they have no confidence in the original programmer who started this project. It may be some of both
30. These two main managers are basically sabotaging the launch of this application by issuing endless demands for extension prior to launch. They constantly say "X is vital. We cannot launch version 1.0 of this app with X." The X function changes each week.

The two managers may not know it, but their strategy for defeating the launch of the app is utterly perfect. No matter how good an architecture is, you can't beat an ever-expanding spec. If San Francisco moves 1 mile away from Oakland every week, the Bay Bridge can never be completed. Worse still, this app is an example of everything that has always been wrong with common Visual Basic 'architecture' since day number 1 of its launch. Give such a brittle and tightly coupled application architecture, you cannot extend this app without breaking it in critical ways. Each time the X function is added, new bugs emerge all over the place. This is pretty much a consequence of global shared mutable state flags, used to coordinate the action of every WinForm in the system. It is also a consequence of trying to add more code to those long nasty event handlers. The insertion point for code is frequently the breakage point.

So right about now, every man woman and child with a CS degree is laughing their collective ass off. In short, all of the industry's worst software practices have been rolled into one failed project. This project has taken 3 years to collapse. Every half-arsed approach to cheap software development has been utilized here. We have recapitulated the finding that cheap software is very expensive.

I would say that a minimum of $450,000 has been wasted on this project. The formerly 51 year old dude (now 54) has been paid about 95,000 per year to do this project. The more senior (but younger) programmer, probably gets more, but he has only been working on the system for about 1 year. The rest of the org has put some effort into the project, but not much. They would resent this comment, but they have no clue how much effort companies like Transamerica, Wellpoint Pharmacy Management, and 21st Century Insurance (firms I have worked for) put into their custom software projects. All hands are on deck. Organizational effort is extreme. Non-Software managers have their work cutout for them. If they don't deliver good accurate test results, they get sacked. My firm's managers believe they have been working hard on this app. Not true. They have not.

Probably the most amazing aspect of this project is the old fellow originally assigned to do this project. Agent 51 has told me about his history as a programmer. It is clear that he got his big break in programmer from Microsoft itself. Microsoft was working hard on the OLAP and business intelligence components of Microsoft SQL Server at the time they selected Agent 51 for a test group. A team of Russian programmers were responsible for doing these mighty extensions to the SQL Server Engine. Microsoft needed to create developer tools for OLAP/Analysis Services/Data Cubes that any Visual Basic programmer could use. To accomplish this mission they needed a test group of developers who were given assignments to write simple and moderate business apps with Visual Basic using experimental prototypes of client tools developed by the VB Language team. Agent 51 was a member of the test group. Agent 51 was a Guinea Pig for the VB Language Team. If Agent 51 could develop a client app with a proposed client tool, the VB Language team knew they had a potentially useful component. They tested their new ideas for client tools on Agent 51, and about 20 other guys like him.

Eventually, Microsoft finished their client tools project for OLAP, and Agent 51 was given his walking papers. Still, Agent 51 was able to honestly write on his resume that he had 3 years of professional Visual Basic consulting experience for Microsoft in Redmond Washington. Furthermore, he could honestly write that he worked on the VB Team's client tools for Multi-Dimensional OLAP. That looks pretty fucking impressive to managers like ours. We are pretty damn close to being an OLAP-only firm. 90% of what we do for money is pure OLAP. The bosses thought they were acquiring a pretty good man in Agent 51. Naturally, they didn't give him an OLAP job, they gave him a CRUD forms project. What's the difference anyway?

Agent 51 has asked me for help on this projects many times. Often times my recommendations fall on deaf ears.

I have told him many times that you cannot rope together an application with global variables. With the most disturbed and disconcerted look on his face, he has asked me how the hell you can write an application any other way. This is a sincere question, backed by real confusion. Agent 51 does not know how else it might be done.

I have told Agent 51 many times that he needs to set Option Explicit On, Option Strict On, and Option Compare Binary. He stresses severely when I tell him this, and he replies "It's too late to do that now. You have no idea how much code I will break if I do that. I can't do that now." The real issue is that Agent 51 does not understand binary data types. He will have to type cast in certain situations, especially to Marshall across to the SQL Server type system, and he does not know how to do that. Automatic Data coercion is his life-line. He can't loose that.

I have told Agent 51 many times that you cannot write entire business operations into a single event handler. Rather, you write a set of objects that collaborate to get the job done. These objects do not look to global flags for data and facts regarding what they are supposed to do. Rather, theses objects have constructors which accept and require the specific params they need to accomplish their function. They store these params in immutable ReadOnly form. You sum up the final business transaction in one class that takes a minimal number of params. You make an instance of this class in an event handler, and then you dispose of the object. Doing it this way makes the business objects are both testable and reusable. Agent 51 is completely lost when I tell him this. He also looks irritated and vexed. The expression on his face seems to say "You are trying to change everything about the way I program. I don't need to do that. I can program just fine."

Well, if that is the case, then why are you asking me for help? Why is this project 3 years late? Why are you having difficulty adding functions without breaking the system?

Recently, Agent 51 got a side assignment to fix a problem with another piece of software. He was stumped by one mystery in the code he could not figure out. He called me over to help him. It turns out he did not understand what an Interface was. I tried to explain to him what an interface was. He looked utterly confused. "Why do you keep using the term interface? This is just a weird empty piece of code! This isn't a user interface".

I asked him to get up an let me drive. I tacked the IReturnValues interface onto the object he had to write. I implemented the single method IReturnValues.ReturnValues() as List(Of String, Object) and passed all values from the object to the calling function as a generic dictionary of key string and Value object. The code compiled and ran fine. Agent 51 had no idea what I did. He said repeatedly "I don't understand that? Why did they do that? It doesn't need to be like that! That isn't the way you do things in VB."

Scott Hanselman likes to quote Quentin Tarantino who once wrote "He who is least likely to make declarative statements is least likely to be called a fool in retrospect."

Unfortunately, in terms of normative statistics, I think Agent 51 is right. You don't normal use interfaces that pass generic dictionaries in VB. Tragically, that isn't the way VB programmers do things.

I think Agent 51 is going to get fired soon. The $450K loss on this failed project is going to be hung on his head. The other guy working on the project is safe. He is arguably the most senior guy in the shop, and he has delivered the goods on many occasions in the past.

If everybody on the team stood up for him, Agent 51 might survive. It will never happen. As it appears to me now, none of the 7 guys on our team are going to defend him. Why? Some folks should not be programmers. At this particular company, our programmers tend to be a good and merciful lot. If he were teachable and coach-able, most of the guys would defend Agent 51. This is not the case. Agent 51 isn't coach-able. Worse still, Agent 51 wants to hold all of us back. Agent 51 wants everybody to program at his level. He wants us to work at the pace of the slowest learner. If you do anything even moderately sophisticated his comment is "This is so fucked up! It doesn't have to be this complex. This is not the way a good VB app is written."

God forbid that you should write anything in C#. Then he feels intensely personally threatened.

This piece has gone fairly long. Tomorrow, I will write a development plan that will explain how I would go about the process of starting a new fresh project such that the mission can be accomplished.