Leaky Web Abstractions

As a consultant and trainer, I have worked with web developers of every variety. Two things that I observe over and over again are:

1. Many web developers are so entrenched in the stacks upon stacks of frameworks needed (or not needed) to build web apps that (to their detriment) they loose track of the fundamental concepts that underlie all web apps.

2. The frameworks we use to build web apps are not good enough to allow us to learn only the abstractions provided by the framework while remaining blissfully ignorant of the web underpinnings. I believe this is called a leaky abstraction. This is not the case in more mature technologies. For example, I have been coding in Java for 10 years and not once have I needed to drop down to the byte-code level to solve a problem. I have been using Oracle for years and not once have I needed to know what goes on inside of Oracle once it receives my SQL. Nor have I ever needed to know the file format that Oracle uses behind the scenes.

But the abstractions we use to build web apps are much more leaky than Java and SQL. It's a very small set of problems that can be solved without understanding the web. True, I have seen successful apps created using slick .NET tools by people who know nothing about HTTP or client/server architecture. But just as often I see million dollar disasters that could have been prevented had the architects understood the web at a more fundamental level.

Summary:

1. As of 2009, you still need to understand the foundational technologies (HTTP, client-server, HTML, CSS, JavaScript, etc) of the web if you want to effectively architect web applications.
2. Our Web frameworks must, by necessity, not obscure what is going on under the covers too greatly.
3. Our Web frameworks must allow us to, on occasion, drop-down to the lower levels simply and elegantly. And then go back up to the high-level abstraction without missing a beat.

An example of number #3 is GWT. In GWT you can drop (from Java) down to the underlying JavaScript using GWT's JSNI (JavaScript Native Interface) mechanism. But unlike JNI (after which JSNI was probably named), JSNI is freakishly easy. So easy, that I now effortlessly blend Java and JavaScript in a single app all the time.

As a side note, this GWT feature has altered my perspective on the whole Static versus Dynamic language debate. If GWT can do it, I am convinced that the inventor of the NBL (Next Big Language) will able to seamlessly blend static and dynamic typing in single language.

Member Literals

I am aware of the pro and cons of static and dynamic languages. I lean toward static because of my error-prone typing, IntelliJ addiction, identifier indecisiveness and unit-test laziness. All things being equal, I would rather have the compiler find my errors than write more tests (which will probably have errors and then I'll need to write a test for the test).

This is why I was excited about Scala. What I was hoping for from Scala's type system:

#1. Less intrusive than Java's type system (Scala succeeds here)

#2. Find more of my errors than Java does

For #2, Scala let me down in one area that is important to me (and i think many others). Can you find the typo in the code snippet below? If you can, then you are smarter than scala.

1  class Person{

2 var firstName:String;

3 var lastName:String;

4 //etc.

5  }

6

7  val rs = statement.executeQuery(

8  "select firstName,lastName from Person")

9

10 p.firstName = rs.getString("firstName")

11 p.lastName = rs.getString("lastName")

12

13 p.doSomeStuff();

14

15 val e = new XmlElement()

16 e.addTextElement("firtName",p.firstName)

17 e.addTextElement("lastName",p.lastName)

18

19 p.addPropertyChangeListener("firstName"){

20 txtFirstName.setText(p.firstName)

21 }

22

23 p.addPropertyChangeListener("lastName"){

24 txtLastName.setText(p.lastName)

25 }

I probably made 7 typos, but the one i was referring to was the "firtName" typo on line 16.

Q: Have you ever been burned by this type of error?

Q: Have you ever had the feeling that the compiler should be able help?

Q: Is it that hard for a compiler vendor to give a member literal syntax? Perhaps something like this:

e.addTextElement(Person#lastName,p.lastName)

I would actually pay money for a compiler that did that.

The only reasons i can think of that this feature does not seem to exist in Java and Scala:

1. I am the only one who wants the feature

2. It must be really hard to implement

By the way, I am aware of constants (static final), but it seems a bit silly to have code like this:

class Person{

String firstName;

public static final String FIRST_NAME = "firstName";

String lastName;

public static final String LAST_NAME = "lastName";

The "Right Tool for the Job" Myth

There is a recurring theme that i see in forums and mailing lists. Some guy posts a message like: "what's better, Flex or AJAX?" or "What's the best tool for making a pdf: iText, Jasper or FOP?" or "What language is more productive, Groovy or Scala?". A number of people chime in with their favorites. Then some bozo chimes in: "Dude, its really a dumb question. You just need to choose the right tool for the job"

Let me tell you why i think this is mostly crap. This "best tool for the job" logic works great for some things, like a shovel. I can reasonably afford a snow shovel for my driveway and a different shovel (a spade) for my yard. Shovels are cheap and the learning curve is low. But compare that with cars. I want horsepower. I want planet-friendly. I want safety. I need to tote the kids around. But I can't afford a Porshe, a Prius, a Volvo and a mini-van. I need one vehicle that (comes close) to meeting all my needs. And then i will live with the decision for the next 5 years.

I view software tools (like frameworks, IDE's or languages) more like cars. Some of these tools take years to learn. The cost of learning a new language, migrating your old code, ditching your favorite IDE, etc. can be huge.

I once knew a guy who used MS Excel as his spreadsheet, database, word processor, desktop publishing and html tool. I tried to convince him to stick with Excel for number crunching but use more specialized tools for those other functions. But he was happy with his universal tool.

I kind of feel that way with programming languages. The other night, at our Java user group meeting we were trying to "position" Groovy versus Java or Groovy versus Scala. Some suggested that Groovy was good for web/database programming. While Scala was good for systems programming. Or some said Scala was suited towards mission critical tasks, where errors are too costly. But here is why I think this is mostly horse shit: Java, Groovy and Scala are all general purpose programing languages. 90% of their target use-cases is overlap. There might be a few things one is more suited to than the other. But all are powerful, advanced, general purpose programming languages - with steep learning curves.

I don't really want to know which of these tools is right for which job. I want to know which is best for most jobs. Having a different tool for every project (or sub-project) is really not appropriate when the tool takes 5 years to learn. Here are 2 extreme examples:

1. XSLT. XSLT is a special purpose language for transforming one XML document into another, usually a raw-data xml into a presentation (i.e. HTML) xml. The problem is this. The book I learned XSLT from is 1000 pages long. XSLT is HUGE! It's an interesting and powerful language. And I learned a lot about functional programming and pattern-matching thru XSLT, so I have no regrets. But 90% of XSLT has nothing specifically to do with transforming XML. 90% of XSLT's constructs would be (and are) useful in any application, xml-transforming or not. And transforming XML is a pretty small part of what I need to do on any given day. Anything you can do with XSLT, you can do (not quite as elegantly) using a general purpose language plus a library. For most application developers who need to transform XML, I think a general purpose language plus a library is the smartest approach.

2. Another example is JavaFX. JavaFX is a really cool new language for the JVM. It is marketed as a special-purpose programming language (aka external DSL) for rich clients (anything you would have used Swing for in the past, you could now use JavaFX). But here is the rub. JavaFX is very feature-rich. In many ways JavaFX is a superset of Java. So if it takes you two years to master Java, it might take 4 years to master JavaFX. And like XSLT, 90% of the new programming constructs you will learn in JavaFX are not specific to creating Swing apps. Most of the cool, new features introduced in JavaFX would be useful for any application.

I don't think the world needs any more languages or tools with the surface area of a general purpose language but only 5% of the applicability. At least not today - with so many general purpose languages offering rich internal DSL capabilities. If you are a person who is really pigeon-holed, and all you do is xml transforms all-day-every-day and you don't already know a general purpose language like Java, then perhaps things like XSLT (or JavaFX) have a place. Or if you can somehow organize your team into super specialized niches, then maybe this will work.

But I am a general purpose programmer. I might code a mobile app one week, a Web/Database app the next week and a parser the next. JavaScript, ActionScript, Java, Ruby, Python, Groovy, Scala and C# are all general purpose languages. 90% of the constructs provided by these languages apply to systems programming, transforms, web apps and database apps. I have never had a client that choses a new programming language for every job. It just isn't feasible. Even Google has standardized on only 4. And I don't really care that Groovy is a few hairs better for scripting. And Scala is a few hairs better for system programming (although, if Scala was 10 times faster for system's programming I might want to know that).

I have been searching for a Java replacement (as my default, general purpose language) for some time, watching new languages like Ruby, JavaFX, Boo, C# 4, Groovy, etc. I am trying to learn which language is most productive for the 90% crap that *all* programming languages do - and doesn't screw me over completely if I need something from the other 10%.

I think with the "internal DSL" functionality of most newer general purpose languages, it is perfectly reasonable to ask the question, "which is the best general purpose programming language?".

So if I start posting questions comparing various languages or frameworks please don't respond with the "right tool for the right job" lecture.

I do want to know the strengths and weaknesses of the various languages and platforms. But I have no intention of using a different general purpose programming language for every project or sub project I do.

Also, please don't get offended if I ask questions that may put your favorite tool in a bad light. It is nothing personal against you. I am just trying to learn.

Side note: One huge limiting factor in all of this is runtime environment. At present, there is no powerful, general purpose language that runs on every runtime platform. In fact, I was just about to jump head first into Python, in spite of the fact that it is 90% redundant (I am guessing) with languages I already know. Why? Because it runs on the Google App Engine. Luckily I procrastinated long enough, and the Google App Engine now runs Java.

Fail Fast Fail Nice

You may be familiar with the term "fail fast" as it applies to software development. It basically states that errors should be reported sooner rather than later. So, for example, suppose an error occurs on line 30 (x has a null value when it shouldn't) but the error doesn't rear its head until line 246. This kind of thing can take hours to debug. A solution would be to place an assertion on line 31 like this:

if(x == null) throw new IllegalStateException("x should not be null");

A few corollaries to the fail-fast rule might be that compile errors are preferable to runtime errors. And errors caught in unit tests are better than errors caught in customer tests.

I would like to take the fail fast principal and expand it to "fail fast, fail nice" (FFFN). By nice, I mean this: the system has to give me a nice error message with enough information to quickly pinpoint and correct the problem. To me, FFFN is important. And not just in the code we write. But in the code we *use*: languages, libraries, frameworks, compilers, runtimes, IDE's, etc.

Let me give a few examples. A year or so ago, I decided to learn JavaFX. Being new, I made lots of mistakes. Every time I screwed up, JavaFX gave me the exact line number and column number of the error. And it gave a nice message telling me what i did wrong. In other words, right out of the gate, JavaFX seemed to succeed at FFFN. It was surprising to me because at this time JavaFX was brand new, not released yet, and mostly the brainchild of a single guy named Chris.

Consider JSF as another example. I had been using and teaching JSF for 3 years. In class, a student makes a mistake on his lab exercise. His app does not work. He just gets a blank screen. No stack trace. No error message. We turn up logging to its most verbose setting. Still nothing. In other words, the JSF implementation did not give us a *bad* error message. It gave us *no* error message. No clue. This kind of thing can takes hours to fix.

An other example would be GWT. If you make a mistake, GWT almost always tells you the exact line number with a friendly description. Compare this to JavaScript programming (without GWT). You often get the wrong line number. And a completely unhelpful error message (although Firebug can help a little).

Another example is Groovy. Every 6 months or so I get excited about Groovy. The clean, elegant syntax. Less need to repeat stuff (the DRY principal), closures, etc. I love Groovy. But then I loose my enthusiasm after a few mistakes take me into java stack trace hell. As of my last go around, Groovy scores high in most areas that are important to me. But it stills scores low in FFFN.

When developing with a low-FFFN tool (like JavaScript) you must drastically decrease your mean-time-between-testing.

What are your experiences? What technologies (languages, libraries, frameworks, compilers, runtimes, IDE's, etc) have you used that are particularly high-FFFN? Which tools have you used that are particularly low-FFFN? Is FFFN an important factor for you?

Is Spring Useful?

I friend recently asked my take on Spring. I thought others might benefit from my response.

> What's your take on the pros and cons of Spring?

You know me, i always seem to have non-standard opinions on things. I can't help it :) I have been using Spring and IOC for a long time. I actually wrote my own (crappy) dependency injection framework before i knew about spring. I have also used used Pico. And lately Guice.

Regarding spring, I use it a lot (it's in the classpath of most projects i do). This is because there are a number of little gems in there. The jdbc stuff, the reflection/bean-util stuff, etc. But, on the whole, i think the *core* of what Spring does is useless (for me). Here is what i have learned that led me to this conclusion:

1. I realized that the IOC pattern is a great pattern, but its a *pattern*, more than a framework. You don't need a framework to do ioc. If you really think of it, the core premise of ioc (or dependeny injection as its also called) is this: Don't lookup dependencies from some kind of global scope. Instead, pass them in as parameters. By "global scope" what i really mean is: (registry | jndi | serviceHashMap | global-variable/singleton). If you really think of it, the ioc pattern is just a re-phrasing of what we have always been taught: prefer explicit argument passing over globals.

Also, in good oo design, one should think about things like "object life cycle" and things like: some objects are users of a service (aka object) while other objects (or object) is in charge of creating the service (or object). This stuff needs to be planned out thoughtfully as part of a good oo design.

So my point here is that 80% of the ioc value comes from the ioc pattern (which is really just good oo practices) and not the ioc framework.

2. We learned that "autowire" has issues. The one area where an ioc framework can actually clean up your code is the "autowire" feature. But, the autowire feature, as it turns out, kills the dave fail-fast/fail-nice principle. In fact, the spring docs caution against the autowire feature.

3. You already know I am against random creation of crappy, unneeded "external" DSL's  (i.e. spring's xml file). There are a million reasons for this. The one good thing about java (the strong type checking) you loose. I kind of view the Spring/Java combo as the absolute worst of all worlds. You don't get all of the great benefits of a dynamic language like Ruby. And you don't get the benefits of type safety.

4. For most things that people use Spring for, I can show them cleaner better ways. Things like transactions, hibernate, dev/staging/production config, swappable impls, factories, etc.

So to summarize, Spring has a place in my toolset, but these days, it is more in the role of an apache commons type of thing. The core idea of what spring is, is mostly useless to me. There are very few use-cases that (for me) warrant the use of Spring (in its central role of ioc/aop container). One use-case would be if i needed a transaction layer that worked seamlessly with JTA (i.e. 2 phase commit) and non-jta environments. But i can't imagine needing that. You either need 2-phase commit or you don't.

If I ever decide to take up ioc frameworks again, it will probably be juice.