Visual Basic is destined to die

Visual Basic is going to die because there are five specific things that kill parallelism:

1. Shared, global, mutable state.

2. Side-effects all over the place

3. Mutable data in just about any scope.

4. Synchronous communication between threads

5. A highly imperative approach to coding.

All of these things are the bread and butter, the day and night, the warp and woof of life for a Visual Basic programmer. I challenge you to show me any sizable Visual Basic App currently in use in any department in any corporation that does not manifest all five of these elements. Element 4 is the only one that might possibly be missing. If so, it is only because the application is absolutely single threaded, with no split between interface and worker threads at all.

For these reasons, I think Visual Basic is going go the way of dBase and FoxPro. I am not alone. There are many within Microsoft who do not believe that Visual Basic can be saved. Better minds, and less biased minds, know the culture of Visual Basic will have to be violently altered to permit a wide deployment of parallel processing through this language community.

Bill himself, does not like notion dead VB in the EOL category. Bill may personally see to it that the system lives a little longer than it should.

How would Bill save Basic and do a really good job of it at the same time? The only way to achieve this is for Microsoft to (once again) make violent changes to the language and overall software pattern of the Basic language. This happened once before. When VB.Net hit the market, VB programmers almost became violent over the loss of COM objects (such as DAO and ADO) and the kind of performance degradation their favorite sloppy language constructs produced. They screamed their lungs out.

So Bill could order the construction of PB--Parallel Basic--that would not allow Global.bas files, enforce class encapsulation, push immutable data, and statelessness, but... Can you imagine how much more upset VB programmers will be if and when they were to discover than Microsoft had removed the Module, the Global.BAS, and the ability to float global variables, the ability to do functions without classes in PB? Can you image what would occur if every DIM statement produced a value identifier that immutable by default?

I believe the typical VB programmer would be livid to the point of heart-attack and stroke.

It might just be easier for Microsoft to place Visual Basic in the EOL category, and offer no further upgrades to this language. EOL means End of Life. They did it to FoxPro. They did it with their Fortran product (which was excellent). They did it to VBScript & ASP.

This brings us to the subject of Oslo and M. Already there is a theory that Oslo, also known as the M language, is being groomed by Microsoft as the declarative and thread-safe replacement for Visual Basic. According to the poop sheet, M is going to be an extremely parallel language system. Lots and lots of parallelism is going to happen under the hood whether you know it or not (as is the case in SQL). More will be availible if you simply learn & implement a few elementary patterns of development. It has to be seen whether a recalcitrant and stubbornly lazy VB community will even be willing to learn this new programming system.

At this point the Visual Basic programmer is probably screaming his head off "AS IF ANY OF THIS IS REALLY NECESSARY?? WHAT THE FUCK IS THE BIG DEAL ABOUT THREADING?" I have stood nose to nose with 54 year old VB programmers paid $95K+ as they insisted that that sort of programming is categorically unnecessary in an LOB departmental programmer's tool kit. "Yeah, but we'll never have to do that kind of thing around here! Why would it ever be necessary? That just isn't necessary."

Make no mistake about it: we are all going to have to program in parallel. We are going to have to use threads and PLinq and everything else that PCP throws at us. This is the only way our apps will be able to handle the terabytes of data we will be required to process in the next decade. The world changed in June of 2005, and most programmers have still not accepted this fact. Processors are not going to be getting that much faster in single-thread execution mode. Processors are going to become massively more parallel. We are only going to get faster by exploiting multiple cores at a time. We can only process increasing volumes of data by abandoning the single threaded and dual threaded application architecture. This means you will not be able to continue making a living programming in single threaded or dual threaded mode. Parallelism is the new God and maximum imperative of programming.

I find that most programmers in most languages are in a vehement state of denial about the gravity of our current CPU architecture. No where is it more stridently expressed than in the VB world. VB programmers are strident because they have great reason to fear. Many of the better VB programmers have tried their hand at threading. Most of them discovered threading caused all manner of problems in their applications. Basically, threading broke their existing application architectures. This is because of shared, global, mutable state, uncontrolled side-effects all over the place, synchronous communication between threads, and a highly imperative approach to coding?

So what about C# and Java? Java and C# programmers have a tendency to thread more in their apps, but these languages may not survive either. C# is a bit better off than Java. C# has absorbed many features of functional programming, although it is much more difficult than it should be to do immutable data. Also, F# is not a very good challenger for C# on the .NET side. Many question whether it was ever indented to be. Java, on the other hand, has not really absorbed anything of the gospel of functional programming. Java still preaches the old-style of lock-based, synchronous communications thread model. Java is also faced with a serious threat of replacement by Scala, which is surging in popularity all over the world. There is no real doubt that Martin Odersky intends Scala to be the general-purpose replacement for Java on the JEE platform. He has explicitly testified to this in interviews.

Neither Java nor C# is guaranteed survival in the coming marketplace, but I firmly believe Visual Basic is dead.

I'll give you a buck for every example of VB.NET code in this month's MSDN magazine

So, I just got my copy of the June 2009 MSDN Magazine. This is Vol.24 No.6 of MSDN magazine. Not one fucking scrap of VB.NET code anywhere to be seen in any code example in any article. They did some Cobra code... what the hell is Cobra? They did some IronPython code. This is reasonable, especially in connection with test projects. Overwhelmingly, the content articles are illustrated with code examples in C#. Not one fucking shard of VB code anywhere.

I dare you to find one example of VB code. I have promised my co-workers $1 for each example of VB they find. If they don't find any soon, I will broaden the offer to the entire world. They are rummaging through the mag right now with disconcerted looks on their faces.

I could be wrong, but I am pretty sure I am right about this. I flipped through this mag 3 or 4 times because I could hardly believe my eyes. Microsoft used to have an official bilingual policy... just like Canada. Everybody had to speak both languages. Everybody had to print code examples in both languages in any MSDN piece. Well, this may not be true anymore.

Based on this issue, it would sure seem that everything is ALL-C# ALL THE TIME in Redmond Washington.

Whilst I like XML litterals in VB, I have to say that I am pretty well pleased by the representation of C# in MSDN magazine. I pretty much love it. All it well. It is as it should be.

My company has a large software project that is failing right now

Yep, that's right. The project is about 3 years old. This development project was launched about 1 year before I joined the firm. I was assigned to work on it very briefly (meaning two months) during which time the two men working on the system were trying to close it out. This was last year. I rarely had anything to do during that time. My plate was empty during those two months. I think I wrote 3 classes for them. That was the end of my contribution to this project.

I know something about this project, but not everything. I think I know enough to comment intelligently about just what went wrong. So what are the characteristics of this project?

1. The project was started by a (then) 51 year old programmer with 3 years of VB experience (at Microsoft). I shall call him Agent 51.
2. This programmer executed a midlife career change from medical nurse (or maybe an orderly) to VB programmer approximately 6 years ago
3. The project is written in VB.NET 8.0
4. Option Strict Off, Option Explicit Off, Option Compare Text.
5. It is a classic WinForm application (not WPF).
6. It is just CRUD forms over database tables.
7. Forms are bound directly to the tables for the most part.
8. Interactive searches are accomplished via stored procedures.
9. It does make use of the Microsoft Enterprise Data block.
10. There are very few classes in the project.
11. There is no inheritance from abstract classes.
12. There is not one original Interface declaration or implementation in the code.
13. There is no Dependency Injection. How can we have DI? There are very few classes.
14. Parameter passing is never done by abstract class or interface type.
15. There are lots of global variables in several global BAS files.
16. There are lots of global functions in several global module files.
17. The application's actions are stitched together and coordinated by these global, mutable flags.
18. When compiled, the compiler issues many warnings that not all function code paths return values.
19. Most of the important business logic is tucked into event handlers.
20. There are side effects in all of the event handlers
21. Event handlers can be 100 lines or longer
22. There is no IoC.
23. There is no test project, no NUnit, no Microsoft Test, no MBUnit. We do not have automatic testability
24. There was no source code control for the first two years on the project
25. The two programmers working on the system frequently collided, stepping on each others changes.
26. Code collisions were unintentional. One man was working on one bug, another was working on another. This strongly implies a lack of modularity and isolation. This also implies problems with global flags in the global bas files.
27. We have no formal QA system. There is no QA department. There is no QA officer who is responsible for testing and validation.
28. This was a skyscraper project without blue-print. The objective was to build the "Everything" application, but they didn't do much design up-front. The notion was that this was a simple project. All Agent 51 needs to do is make some CRUD forms.
29. There are two main managers in charge of the two major departments who will use this app, if it is ever finished. Neither manager wants this application to launch. It is difficult to determine whether they are afraid of the potential efficiency gains and subsequent layoffs this app will create, or whether they have no confidence in the original programmer who started this project. It may be some of both
30. These two main managers are basically sabotaging the launch of this application by issuing endless demands for extension prior to launch. They constantly say "X is vital. We cannot launch version 1.0 of this app with X." The X function changes each week.

The two managers may not know it, but their strategy for defeating the launch of the app is utterly perfect. No matter how good an architecture is, you can't beat an ever-expanding spec. If San Francisco moves 1 mile away from Oakland every week, the Bay Bridge can never be completed. Worse still, this app is an example of everything that has always been wrong with common Visual Basic 'architecture' since day number 1 of its launch. Give such a brittle and tightly coupled application architecture, you cannot extend this app without breaking it in critical ways. Each time the X function is added, new bugs emerge all over the place. This is pretty much a consequence of global shared mutable state flags, used to coordinate the action of every WinForm in the system. It is also a consequence of trying to add more code to those long nasty event handlers. The insertion point for code is frequently the breakage point.

So right about now, every man woman and child with a CS degree is laughing their collective ass off. In short, all of the industry's worst software practices have been rolled into one failed project. This project has taken 3 years to collapse. Every half-arsed approach to cheap software development has been utilized here. We have recapitulated the finding that cheap software is very expensive.

I would say that a minimum of $450,000 has been wasted on this project. The formerly 51 year old dude (now 54) has been paid about 95,000 per year to do this project. The more senior (but younger) programmer, probably gets more, but he has only been working on the system for about 1 year. The rest of the org has put some effort into the project, but not much. They would resent this comment, but they have no clue how much effort companies like Transamerica, Wellpoint Pharmacy Management, and 21st Century Insurance (firms I have worked for) put into their custom software projects. All hands are on deck. Organizational effort is extreme. Non-Software managers have their work cutout for them. If they don't deliver good accurate test results, they get sacked. My firm's managers believe they have been working hard on this app. Not true. They have not.

Probably the most amazing aspect of this project is the old fellow originally assigned to do this project. Agent 51 has told me about his history as a programmer. It is clear that he got his big break in programmer from Microsoft itself. Microsoft was working hard on the OLAP and business intelligence components of Microsoft SQL Server at the time they selected Agent 51 for a test group. A team of Russian programmers were responsible for doing these mighty extensions to the SQL Server Engine. Microsoft needed to create developer tools for OLAP/Analysis Services/Data Cubes that any Visual Basic programmer could use. To accomplish this mission they needed a test group of developers who were given assignments to write simple and moderate business apps with Visual Basic using experimental prototypes of client tools developed by the VB Language team. Agent 51 was a member of the test group. Agent 51 was a Guinea Pig for the VB Language Team. If Agent 51 could develop a client app with a proposed client tool, the VB Language team knew they had a potentially useful component. They tested their new ideas for client tools on Agent 51, and about 20 other guys like him.

Eventually, Microsoft finished their client tools project for OLAP, and Agent 51 was given his walking papers. Still, Agent 51 was able to honestly write on his resume that he had 3 years of professional Visual Basic consulting experience for Microsoft in Redmond Washington. Furthermore, he could honestly write that he worked on the VB Team's client tools for Multi-Dimensional OLAP. That looks pretty fucking impressive to managers like ours. We are pretty damn close to being an OLAP-only firm. 90% of what we do for money is pure OLAP. The bosses thought they were acquiring a pretty good man in Agent 51. Naturally, they didn't give him an OLAP job, they gave him a CRUD forms project. What's the difference anyway?

Agent 51 has asked me for help on this projects many times. Often times my recommendations fall on deaf ears.

I have told him many times that you cannot rope together an application with global variables. With the most disturbed and disconcerted look on his face, he has asked me how the hell you can write an application any other way. This is a sincere question, backed by real confusion. Agent 51 does not know how else it might be done.

I have told Agent 51 many times that he needs to set Option Explicit On, Option Strict On, and Option Compare Binary. He stresses severely when I tell him this, and he replies "It's too late to do that now. You have no idea how much code I will break if I do that. I can't do that now." The real issue is that Agent 51 does not understand binary data types. He will have to type cast in certain situations, especially to Marshall across to the SQL Server type system, and he does not know how to do that. Automatic Data coercion is his life-line. He can't loose that.

I have told Agent 51 many times that you cannot write entire business operations into a single event handler. Rather, you write a set of objects that collaborate to get the job done. These objects do not look to global flags for data and facts regarding what they are supposed to do. Rather, theses objects have constructors which accept and require the specific params they need to accomplish their function. They store these params in immutable ReadOnly form. You sum up the final business transaction in one class that takes a minimal number of params. You make an instance of this class in an event handler, and then you dispose of the object. Doing it this way makes the business objects are both testable and reusable. Agent 51 is completely lost when I tell him this. He also looks irritated and vexed. The expression on his face seems to say "You are trying to change everything about the way I program. I don't need to do that. I can program just fine."

Well, if that is the case, then why are you asking me for help? Why is this project 3 years late? Why are you having difficulty adding functions without breaking the system?

Recently, Agent 51 got a side assignment to fix a problem with another piece of software. He was stumped by one mystery in the code he could not figure out. He called me over to help him. It turns out he did not understand what an Interface was. I tried to explain to him what an interface was. He looked utterly confused. "Why do you keep using the term interface? This is just a weird empty piece of code! This isn't a user interface".

I asked him to get up an let me drive. I tacked the IReturnValues interface onto the object he had to write. I implemented the single method IReturnValues.ReturnValues() as List(Of String, Object) and passed all values from the object to the calling function as a generic dictionary of key string and Value object. The code compiled and ran fine. Agent 51 had no idea what I did. He said repeatedly "I don't understand that? Why did they do that? It doesn't need to be like that! That isn't the way you do things in VB."

Scott Hanselman likes to quote Quentin Tarantino who once wrote "He who is least likely to make declarative statements is least likely to be called a fool in retrospect."

Unfortunately, in terms of normative statistics, I think Agent 51 is right. You don't normal use interfaces that pass generic dictionaries in VB. Tragically, that isn't the way VB programmers do things.

I think Agent 51 is going to get fired soon. The $450K loss on this failed project is going to be hung on his head. The other guy working on the project is safe. He is arguably the most senior guy in the shop, and he has delivered the goods on many occasions in the past.

If everybody on the team stood up for him, Agent 51 might survive. It will never happen. As it appears to me now, none of the 7 guys on our team are going to defend him. Why? Some folks should not be programmers. At this particular company, our programmers tend to be a good and merciful lot. If he were teachable and coach-able, most of the guys would defend Agent 51. This is not the case. Agent 51 isn't coach-able. Worse still, Agent 51 wants to hold all of us back. Agent 51 wants everybody to program at his level. He wants us to work at the pace of the slowest learner. If you do anything even moderately sophisticated his comment is "This is so fucked up! It doesn't have to be this complex. This is not the way a good VB app is written."

God forbid that you should write anything in C#. Then he feels intensely personally threatened.

This piece has gone fairly long. Tomorrow, I will write a development plan that will explain how I would go about the process of starting a new fresh project such that the mission can be accomplished.