One of the classic bug reports of all time

So, later on in the afternoon yesterday, I got a bug report. It came from Marina here at work. Something like 2.5 years ago, I wrote a piece of software for our media partners which delivers advertisement campaign reports, according billing information, and facilitates requests for "advertising cap increases".

Our advertisement program works on a pre-paid basis. It is one of the key reasons we are still afloat in these dark economic times. Mortgage companies need to pay us upfront before we begin advertising their 'financial products'. They give us a corporate credit card, we bill them, the money hits our bank account, we begin metering out advertisement rotations to them. We never get cheated. That is why we still live.

This brings us to the subject of "advertising cap increases". I constructed a very simple .ASPX page which allows an authenticated, permission rich user to request a fresh billing of those corporate credit cards. Of course, this uses https super-secure sockets, and 128 bit encryption.

I was a little flummoxed that a problem had arisen. The app is deadly simple, not rocket science by any means, and it had been functioning flawlessly for 2.5 years. Nobody every complained. More than 6,000 billing requests had been made (according to the log) without error over the past 2+ years.

Of course, my thinking was typical of most programmers in this situation. What happened? What changed? Who changed some server or system security setting(s). My code has not been altered in 2.5 years so some externalities must be impacting my system.

I went over to talk to Marina and we went through the steps of recreating the error. She spotted the problem before I did. The almighty user who has the authority to order cap increases needs to provide us with an email address. This is partially for authentication, and partially for notification. We notify the almighty one when his or her corporate card has been charged, we notify of the billing results, we notify when the advertisement rotations increase, begin again, or cease.

Guess what the almighty one typed in for his (and it was a him) email address? He put in http://www.bgnobgno.com. The URL has been changed to protect the guilty, in this case. As you can see, this is not an email address. This is a web address. There is a difference between a eMail address and a URL, just as surely as there is a difference between a mail server and an http server.

Marina quickly changed the web URL to the proper eMail address. She had the eMail address because the almighty one had been sending her complaints about my web work. Marina is a Capricorn (yeah!) and she is good at handling the organizational politics of an embarrassing user error like this one. As a tactless, blunt and factual Virgo, I would have simple sent him a short mail telling him he put his website were his email should have gone. You complained about me too! Embarrassing aye?

She will find a nice way to smooth this over without alienating or embarrassing a paying client.

Anyway, this event served as a reminder of why I am so damn bored around here all the time, and yet still employed. I wrote some pretty good software in the first 18 active months I was here. I wrote at least 3 small apps that are continuously pumping money into the corporate coffers. That is not exceptional for a corporate programmer. That is the way it should be. I am not boasting about that.

What I am boasting about is that my software has been clicking away flawlessly for some 30 months without error, and it took a nice dumb error on the part of a user to produce a failure. I would say that this is an example of the "Do it once and do it right" mentality I love, except for one little nasty little fly in the ointment.

It would appear that my input validation failed in this case. I used simple JavaScript to validate the User's inputs in this case. I chose a routine that was recommended by Google Codeplex some 4 years ago. I did not write the JS myself. It would appear that this smart little piece of code is just too smart. It does more than I originally thought it would. It will parse FTP addresses, email addresses, and URLs. It will tell you if your string conforms to any of these specs.

Of course, this is not what I had in mind. I wanted to validate eMail and just eMail. Only this and nothing more. I don't want a boolean true value when a URL is entered. I am going to have to fix that code.

Dynamic Languages, Revisited

Hey, shock of shocks! A few programmers are reading my blog! These are guys I don't even know. How wonderful for me! But there was a bit of discord on the definition of just what is a dynamic language and what isn't.

So, for the record, I will quote the wikipedia.org article on Dynamic Languages:

The definition of a dynamic language is ambiguous because it attempts to make distinctions between code and data as well as between compilation and runtime which are not universal. Virtual machines, just-in-time compilation, and the ability of many programming languages on some systems to directly modify machine code make the distinction abstract. In general, the assertion that a language is dynamic is more an assertion about the ease of use of dynamic features than it is a clear statement of the capabilities of the language.

There you go! Wikipedia clearly acknowledges that there is big old nasty mess where a nice precise definition aught to be. Everybody (except me) would like to lay claim to the Dynamic tag because it is considered a hot buzzword right now... especially for the resume. In the process the real meaning of the term is absolutely lost.

The long list of languages at the bottom of the Wikipedia article include everything under the sun, including such preposterous inclusions as Forth, D, C#, Lisp, VB9 and REALBasic. HOLY SHIT! I want to make it clear that Spanish, Hungarian and Mandarin also qualify as a dynamic computer language under the catch-all basket definition presented in this wikipedia article.

Alas, such is the case with all hot buzzwords. They all become polluted and corrupted until the very last shred of meaning is lost. Such was the case with the term Light-Weight. Everything is now Light-Weight. Every last group claims to have a light-weight application stack or frame work. Such was the case with RISC. Everyone not manufacturing an explicit x86 chip tried to lay claim to the RISC title. When IBM claimed the PowerPC was a RISC design, the very last vestige of the original meaning of RISC died.

Therefore, allow me to clean up the fucking mess:

A Dynamic language is an object oriented programming language which provides a metaprogramming model that allow a runtime engine of some sort to construct new and modify existing classes in any application. These systems are capable of both creating and modifying classes without intervention or explicit programmatic effort by the original developer.

The emphasis is on the terms object oriented and classes. This eliminates languages such as JavaScript and VBScript. Why? Because they are not object oriented, that is why. I have heard protests on this subject before. Some think the existence of JSON implies that JavaScript is an example of OOPS. Not so. JSON is basically open data container format, mailable into many formats. It is no more object oriented than XML is. You cannot encapsulate public or private methods in JSON, nor can you redistrict visibility or access to data elements.

Weak typing has nothing to do with the subject of dynamics, although it is a necessary step along the way. There have been a vast array of languages which implement weak typing. The overwhelming majority of them are not dynamic. As I mentioned briefly in my prior piece, this is the greatest point of confusion for the masses. Many know that their favorite language is weakly typed, and believe (ergo) a dynamic language also. Perish the thought! Not even close to being true.

So what if I am a great lover of JavaScript and I desperately covet the buzzword Dynamic? JavaScript still has an undeserved bad rep in many places. I am trying to rehab my favorite language's image in the court of public opinion. I want people to think highly of JavaScript. I want to lay claim to the title Dynamic to help forward this end goal. Can't you just bend the rules a bit to let my baby in the club?

Nope. The end.

On the question of just how fast IronPython is, you might be surprised. Have a look at this little piece right here. That article is essentially a soft retraction piece, where a good benchmark artist discovered something very interesting about IronPython vs. CPython performance. The future looks very nice for IronPython.

As to whether Python is a static language or dynamic... Well, fortunately more ink than blood has been shed over this subject. Python types are never stated in variable declaration. Data types are inferred from arguments passed or assignments made. Variable types are immutable, within scope. There is nothing that would prevent one well written Python function from handling three different binary types in three consecutive calls. This is very dynamic, my friends. Finally, and most importantly, you can mutate a class definition at runtime. This is the absolutely crucial key to dynamics.

Anyhow, I appreciate the readership! Thanks for bumping up my ad campaign. :D Hope I can help clarify somethings. One famous wag said "It is better to insulted than ignored... but only in passing." Not that I was insulted, mind you.

So just what is a Dynamic Language anyway?

It's remarkable to me how many friends I have in the business who aren't really sure just what the heck a dynamic language is anyway. Many things they say to me indicate that they don't know what a dynamic language really is. For the record, let's clear it up.

Let's start with a quartet of distinctions.

1. Object Oriented Language
2. Interpreted language
3. Weakly typed language
4. Dynamic language

• An OOPS language is not necessarily interpreted, weakly typed, or dynamic.
• An interpreted language is not necessarily OOPish, weakly typed, or dynamic.
• A weakly typed language is not necessarily OOPish, interpreted, and not necessarily dynamic. This is one of the most frequent bits of confusion. Many believe Weak = Dynamic.
• A Dynamic language isn't necessarily interpreted, but it is always OOPSy and weakly typed.
• With that said, most dynamic languages are interpreted and weakly typed.

Here is a list of languages that are not dynamic, although some think they are:

1. JavaScript
2. VBScript
3. Perl
4. PHP
5. Lua

Here are a list of languages that are truly dynamic:

1. Smalltalk-80
2. Python
3. Ruby
4. Groovy

So, you will notice that most of the popular and well known interpreted scripting languages are not dynamic. More obscure choices, selected by the ubergeeks, are the dynamic languages. Being weakly typed is necessary but not sufficient to be classified as a dynamic language. I would have said that being interpreted was necessary also, but then along came IronPython.NET and blew my world appart.

So what is the key? What distinguishes the dynamic language? The answer is in the meta-programming model. In a dynamic language, no class is ever final, not even at run time. It is not only possible, but common for a dynamic interpreter to mock-up or attach methods to a class or object at runtime. I can also tack on additional properties at runtime. I can also write code that edits its own source code, and dynamically reloads it on the fly. I can create self-mutating code. This is the key feature which permits research into genetic algorithms.

Can I attach new methods and properties to a class at run-time in Java and C#. No, not per se. There are ways to try to achieve a similar effect, but these belong in the category of dirty hacks. This approach is absolutely not supported by either of these languages. How about VB.NET? Nope. How about VBScript? Please! This isn't even an object oriented language! How can you attach new properties and methods to something that is not a class in the first place? Same thing goes for JavaScript.

So why the hell would I want to dynamically dispatch new methods I never intended to be a part of my class when I designed a wrote it? Why would want to dynamically attach additional properties to the object at runtime? Isn't this unsafe and unsound? Aren't I allowing the interpreter to alter my architecture in ways I did not intend?

Maybe, if you don't know the language and don't know how to use it.

What if I do know the language and do know how to use it, doesn't the mere existence of such a feature open the door to entirely new categories of unintended consequences and unpredictable behavior?

That is the rub now, isn't it? We have put out finger on the exact point of critique which serious computer scientists have been arguing about for several years now. Decades really. Ever since Smalltalk-80, this debate has been running among serious men of great learning and skill.

The majority have chosen not to use dynamic languages. The majority have chosen statically typed OOPs. A rather small minority have chosen dynamic languages.

So, why have those who have chosen dynamic languages chosen dynamic languages?

• Very loose coupling. You can pass anything to anything. If it works it works. Very large systems can be built this way.
• Interpreter safety: If it doesn't it throws a soft error and you make a correction. The OS does not crash.
• Auto-Mocking. These days, we have come to recognize that we need automatic testability for all code. We call this Unit Testing. We aim for 100% code coverage. This means we want every line of code automatically tested in a unit test project. To facilitate this process, you need something called mock objects. These are objects that doppelgangers for dependencies your test unit needs to function, but they do not produce the real world side-effects. Real files do not appear and disappear. Real records are not created, read, updated, and deleted.
• Mocking in a static language like C# and Java is a real pain. Python and Ruby just construct Mocks for you, dynamically attaching empty properties and methods for your. You never need to write Mock object when doing TDD in Python or Ruby.

Isn't there a substantial performance penalty for this approach?

Yep, Ruby and Groovy are the two slowest languages currently accepted in the world. Although Smalltalk and Python are faster, they are not that fast. We can pretty well destroy them with C#.

With that said, a fully compiled variant of Python, called IronPython.Net is now making some waves in the world. IronPython's compiler is written in C#, and it cranks out CIL like all the languages on the .NET platform. This CIL assembly is locked and loaded by the CLR and handed over to the JIT for transformation to x86 or AMD64 code. That code is greatly fattened and slowed by the dynamic approach to things, but it is real machine code. IronPython is now something like 8x faster than standard C Python. It is still not as fast as something like Java or C#, but it is still far better than it's brother and cousins in terms of performance.

I personally am very interested in IronPython.Net. However, I am more interested in Scala.

My religion is generally opposed to Dynamic languages. I believe in static typing. I believe in a strong compiler. I believe in things like checked exceptions. I love design my contract. I love annotating requirements on code. C# does not give me as much static verification as I would like. I hope Scala is better. Python means busting my version of the 10 commandments. This is risky, and I dislike the notion. However, I am trying to keep an open mind, and determine whether IronPython can provide some wonderful new service to my .NET applications.