The "Right Tool for the Job" Myth

There is a recurring theme that i see in forums and mailing lists. Some guy posts a message like: "what's better, Flex or AJAX?" or "What's the best tool for making a pdf: iText, Jasper or FOP?" or "What language is more productive, Groovy or Scala?". A number of people chime in with their favorites. Then some bozo chimes in: "Dude, its really a dumb question. You just need to choose the right tool for the job"

Let me tell you why i think this is mostly crap. This "best tool for the job" logic works great for some things, like a shovel. I can reasonably afford a snow shovel for my driveway and a different shovel (a spade) for my yard. Shovels are cheap and the learning curve is low. But compare that with cars. I want horsepower. I want planet-friendly. I want safety. I need to tote the kids around. But I can't afford a Porshe, a Prius, a Volvo and a mini-van. I need one vehicle that (comes close) to meeting all my needs. And then i will live with the decision for the next 5 years.

I view software tools (like frameworks, IDE's or languages) more like cars. Some of these tools take years to learn. The cost of learning a new language, migrating your old code, ditching your favorite IDE, etc. can be huge.

I once knew a guy who used MS Excel as his spreadsheet, database, word processor, desktop publishing and html tool. I tried to convince him to stick with Excel for number crunching but use more specialized tools for those other functions. But he was happy with his universal tool.

I kind of feel that way with programming languages. The other night, at our Java user group meeting we were trying to "position" Groovy versus Java or Groovy versus Scala. Some suggested that Groovy was good for web/database programming. While Scala was good for systems programming. Or some said Scala was suited towards mission critical tasks, where errors are too costly. But here is why I think this is mostly horse shit: Java, Groovy and Scala are all general purpose programing languages. 90% of their target use-cases is overlap. There might be a few things one is more suited to than the other. But all are powerful, advanced, general purpose programming languages - with steep learning curves.

I don't really want to know which of these tools is right for which job. I want to know which is best for most jobs. Having a different tool for every project (or sub-project) is really not appropriate when the tool takes 5 years to learn. Here are 2 extreme examples:

1. XSLT. XSLT is a special purpose language for transforming one XML document into another, usually a raw-data xml into a presentation (i.e. HTML) xml. The problem is this. The book I learned XSLT from is 1000 pages long. XSLT is HUGE! It's an interesting and powerful language. And I learned a lot about functional programming and pattern-matching thru XSLT, so I have no regrets. But 90% of XSLT has nothing specifically to do with transforming XML. 90% of XSLT's constructs would be (and are) useful in any application, xml-transforming or not. And transforming XML is a pretty small part of what I need to do on any given day. Anything you can do with XSLT, you can do (not quite as elegantly) using a general purpose language plus a library. For most application developers who need to transform XML, I think a general purpose language plus a library is the smartest approach.

2. Another example is JavaFX. JavaFX is a really cool new language for the JVM. It is marketed as a special-purpose programming language (aka external DSL) for rich clients (anything you would have used Swing for in the past, you could now use JavaFX). But here is the rub. JavaFX is very feature-rich. In many ways JavaFX is a superset of Java. So if it takes you two years to master Java, it might take 4 years to master JavaFX. And like XSLT, 90% of the new programming constructs you will learn in JavaFX are not specific to creating Swing apps. Most of the cool, new features introduced in JavaFX would be useful for any application.

I don't think the world needs any more languages or tools with the surface area of a general purpose language but only 5% of the applicability. At least not today - with so many general purpose languages offering rich internal DSL capabilities. If you are a person who is really pigeon-holed, and all you do is xml transforms all-day-every-day and you don't already know a general purpose language like Java, then perhaps things like XSLT (or JavaFX) have a place. Or if you can somehow organize your team into super specialized niches, then maybe this will work.

But I am a general purpose programmer. I might code a mobile app one week, a Web/Database app the next week and a parser the next. JavaScript, ActionScript, Java, Ruby, Python, Groovy, Scala and C# are all general purpose languages. 90% of the constructs provided by these languages apply to systems programming, transforms, web apps and database apps. I have never had a client that choses a new programming language for every job. It just isn't feasible. Even Google has standardized on only 4. And I don't really care that Groovy is a few hairs better for scripting. And Scala is a few hairs better for system programming (although, if Scala was 10 times faster for system's programming I might want to know that).

I have been searching for a Java replacement (as my default, general purpose language) for some time, watching new languages like Ruby, JavaFX, Boo, C# 4, Groovy, etc. I am trying to learn which language is most productive for the 90% crap that *all* programming languages do - and doesn't screw me over completely if I need something from the other 10%.

I think with the "internal DSL" functionality of most newer general purpose languages, it is perfectly reasonable to ask the question, "which is the best general purpose programming language?".

So if I start posting questions comparing various languages or frameworks please don't respond with the "right tool for the right job" lecture.

I do want to know the strengths and weaknesses of the various languages and platforms. But I have no intention of using a different general purpose programming language for every project or sub project I do.

Also, please don't get offended if I ask questions that may put your favorite tool in a bad light. It is nothing personal against you. I am just trying to learn.

Side note: One huge limiting factor in all of this is runtime environment. At present, there is no powerful, general purpose language that runs on every runtime platform. In fact, I was just about to jump head first into Python, in spite of the fact that it is 90% redundant (I am guessing) with languages I already know. Why? Because it runs on the Google App Engine. Luckily I procrastinated long enough, and the Google App Engine now runs Java.