I am a big fan and practioner of automated unit-testing, but throughout the years I took my lessons. Starting with “everything has to be automated tested” throughout years I experienced situations where doing unit-testing is not optimum approach.

The presented sections go along with my favorite test-smells:

1. Brittle tests: Though functionality hasn’t been changed the test fails. Test should show green but in fact shows red (false positive).
2. Inefficient tests: The effort of writing automated tests doesn’t pay out at all. The benefit/cost ratio (short + long term) is extremely low.

Unit-Test little scripts/tools

There is often no sense to write unit-tests for little scripts or tools, which are one or two-liners. The script content is already so “declaritive”, short and compact that the code is too simple to break. Further more often stubbing or mocking the dependencies is tough (e.g. writing to stdout/file, shutdown machine, doing an HTTP call). You can end up writing a external system emulator which is overkill in this situation. Surely testing is important but for that I go the manual way (executing script, and smoke-test sanity check the outcome).

Unit-Test high level orchestration services

Orchestration services have many dependencies and chain-call lower services. The effort of writing such unit-tests is very high: Stubbing/Mocking all these outgoing dependencies is tough, test setup logic can get very complex and make your test-code hard to read and understand. Further more these tests tend to be very brittle, e.g. minor refactoring changes to production code will break them. Main reason is that inside test-code you have to put a lot of implementation detail knowledge to make stubbing/mocking work. You can argue having many fan-out/outgoing dependencies is a bad smell and you should refactor from start on. This is true in some cases but higher order service often have the nature to orchestrate lower ones, so refactoring won’t change/simplify much and make design even more complicated. In the end for such high level services I much prefer to let test-cover them by automated or non-automated acceptance tests.

Test-first during unclear Macro-Design

When implementing a feature or something from scratch often the macro-design is blurry, I like to call this “diving-in”. For diving-in development or quick prototyping you get a feeling which design fits or not. During this phase class structures/interactions change a lot, sometimes even big chunks of code are thrown away and you restart again. Such wide code changes/deletions often will break your tests and you have to adapt or even delete them. In these situations test-first approach doesn’t work for me, writing test-code even distracts me and slows me down. Yes, unit-tests and test-first approach can and should guide your design but I experienced this counts more when the bigger design decisions have been settled.

100% Code-Coverage

I can’t overstate this: Code-Coverage != Test-Coverage. The Code-Coverage of unit-tests is a nice metric to see untested spots, but it is by far not enough. It just tells you that the code has been executed and simply misses the assert part of your test. Without proper asserts, which check the side-effect of your production code and expected behaviour, the test gives zero value. You can reach 100% code-coverage without having tested anything at all. In the end this wrong feeling of security is much worse as having no test at all! Further more 100% code-coverage is inefficient because you will test a lot of code, which is “too simple to break” (e.g. getters/setters, simple constructors + factory-methods).

Summary

Above points shouldn’t give you the impression that I do speak against automated unit-tests, I think they are great: They can guide you to write incremental changes and help you to focus, you get affinity for green colors ;), they are cheap to execute and regression testing gives you security of not breaking things and more courage to refactor. Still going with the attitude that you have to go for 100% code-coverage and to test every code snippet will kill testing culture and end up in Red-Green color blindness.

In recent years dynamic typed languages like Ruby, Groovy, JavaScript or Python rightly gained more popularity. Some even said they will soon replace their static typed counterparts. Though I am a big fan of dynamic typed and intepreted languages (for smaller tools/tasks they make life so much easier) my current bet is that the language-future it is not a question of either/or but a gain of diversity of your toolset. Static typed languages just have this big diverse plus when it comes to maintainability of big codebases.

Overview of differences

Because there still is some confusion about dynamic vs. static typing here a short overview of my perception:

1. Dynamic typed programming: The types or variables are bound during runtime. This happens if you use languages which have dynamic typing in their core design (PHP, Ruby, Python). Be aware that you can still do dynamic typing with languages focusing more on static typing (like C# or Java): Simply have a look at the reflection APIs… Further more type saftey is deferred to runtime, if a programmer introduces typing errors they will pop up during runtime (e.g. ClassCastException, MethodNotFoundException, TypeError).
2. Static typed programming: The types are resolved/bound during compile time. For this check to be possible your code needs to have typing information (e.g. when defining a variable or method signature). If you have obvious type errors in your code the compilation result will catch it and inform you.

Pros dynamic typing

If designed correctly into the language, dynamic typing offers a lot of advantages:

• Code is much more compact. Small tools and scripts are a charm to write and some nice syntactic sugar constructs are possible. Programmers need less time to understand and extend code. Further more the proportion of functionality vs. lines-of-code is greater, you need to produce less code to solve a problem. Smaller codebases are a plus for maintenance.
• Sometimes dynamic typing is a necessity of solving a problem (e.g. the calculated return value is dependant on the parameter’s type, which can differ for every caller).
• Development feedback cycle tends to be shorter. Most languages offering dynamic typing are intepreted and a simple ‘Save file’ will do. In most cases you instantly will see the effect. For bigger server applications less restarts of the server are necessary.

Pros static typing: Maintainability

Above I mentioned compactness of code as a maintenance plus for dynamic typing. Still, on the same topic static typing scores significantly.

Analyzation of codebases

Typing information bloats code but also serves documentation. In many cases explicit typing helps a lot in reasoning the code’s semantics. Also the IDE can help to easily jump throughout the codebase. It doesn’t have to reason or guess but can directly guide you to the target code. Explicit typing information also does help tools like Structure101 or JDepend to automatically show you relationships, which can point to design or layering flaws.

Feedback of errors

You get faster feedback when you did a typing related coding-error, e.g. wrongly related a function/method or did a typo. The compliation phase does this for granted. Some argue that you anyway should do cover anything by automated tests and therefore can instantly catch typing related programming errors. This theoretically is true but in practice doesn’t hold, I’ve never seen convincing test suites which execute every code-path instantly uncovering such errors. Often you get such errors later by accident, in testing phase or worse during production. Compile time type safety is nothing to be neglected…

Safe refactorings/tooling

It is the “nature” of dynamic typing that you can’t find out typing just looking at the structure of the code. The opposite is true for static typing, therefore you can create tools for safe and automated refactorings (e.g. Rename method/function, Add/Remove parameters, general restructuring of classes/modules). If many refactorings wouldn’t be automated I would lose a lot(!) of time. Being able to refactor codebases in an efficient sensible time is a must have for me. The greater the codebase the greater the cost for unsupported automated/unsafe refactorings. Some people again argue that 100% test-coverage safety-net would suffice, for my point of view see above ;)

I purposely discarded the old performance comparison. It states that compiled binary-code is faster as interpreted code. It may have been true in older times, but nowadays, also with all the JIT compiling features, I haven’t seen convincing benchmarks that static typed languages binaries are better in performance in general. This doesn’t imply that you won’t hit scaling problems which are indeed bound to a language (see also podcast about facebook getting problems with PHP at some point)!

Summary

Dynamic typed languages are great but for bigger codebases I still would go for languages like Java or C# (haven’t tried out yet, but Scala seems to be a new alternative here). For very specific modules solving highly generic/dynamic problems you still can code in the “right” language. Modern runtime environments (like JVM or CLR) give you possibility to deploy common-libraries/APIs, which are written in dynamic typed language.

If you have case studies or other resources investigating dynamic vs. static typing and overall productivity or providing different opinions please comment.

Many people ask me why I prefer IntelliJ over any other IDE. I switched to IntelliJ about 3 years ago so I cannot compare current IntelliJ 10 vs. other current IDEs Eclipse 3.6 or Netbeans 6.9. Still pair programming with colleagues who use different IDE and sometimes having to skip back to Eclipse I feel confirmed that IntelliJ is best IDE on market (my opinion is primarily based on Java-Apps). IntelliJ has recently released version 10 and improved a lot of things.

Performance

IntelliJ strikes in performance. It keeps all the files in an index. Access to and searching through all files is extremely fast, also compilation is instant (you don’t even sense it). Only the initial indexing process sometimes feels a bit slow, but version 10 shows big performance improvements on the initial index-process. I subjectively think that version 10 feels faster and UI is better responding.

Automatic in-memory compilation

Whereas for Eclipse you have to do a manual for saving a file, IntelliJ is doing it for you. Some people say, that this is “just” another shortcut to press, but I remember that it was a big relief not to do it. It really makes your fingers faster to go on with another task.

Auto-Completion + Intentions

The autocompletions and intentions are very clever (general code improvement, refactorings, type-completion, varables names, refactorings etc.). I sometimes feel that they read my mind. Since version 10 another major improvement got live: Instant auto-completion, you get suggestions as you type. I like this very much on Microsoft Visual-Studio IDEs, now finally there for IntelliJ.

Refactoring Support

Codebases should be continously improved. Structural improvements impose a risk that you break code, therefore automatic safe refactorings are extremely important. Here IntelliJ has the best toolset. It also plays nice with SCM support, when moving around files or renaming packages.

Prepackaged tool support

On other IDEs you have to install many plugins manually. On IntelliJ the most important ones are already there (maven, nearly alls SCMs) and are integrated well. I can’t remember one case where plugins got conflicted with each other.

The small things…

IntelliJ shows many little gimmicks, which put together make “the” big difference. Here an excerpt:

• Run Unit-Tests on package basis (package focus in Project Window and <Shift>+<Ctrl>+F10)
• Instant code execution on breakpoint (Debug-Window <Alt>+F8)
• Instant copy file path so you can quickly jump to path on command-line (focus file/directory and <Ctrl>+C)
• File comparisons: Excellent Diff-View. Compare with clipboard. Compare with other branch.
• Coloring of files on tab and lines inside editor if they where changed by you without having committed. Instant notification inside editor, when file got out of synch with SCM repository (shows the commit-time and author of change).
• Show SCM history of selection/marked codelines.
• Working with resource-bundles. Inside code hover over a message-key and it shows you the translation instantly (like foo.bar.logout would give you little text-box “Logout”). Also refactoring the message-keys is safe (messages.properties gets upated).
• Quick jump to Run/Debug settings (<Alt>+<Shift>+F10).
• Automatic Code quality checks + report before SCM commit.
• Intention ‘Create Test-Class’
• Automatic files refresh. When switching to command line and doing SCM or maven actions, switching to IntelliJ back all files are refreshed automatically. No danger of stale data inside IDE.
• General search for plain text or structural search.
• Auto collapsing tool windows on losing focus. Very convenient on smaller notebook screens or generally increasing editor space.
• Stable editor, even for very large files, e.g. it can show 5MB large XML-docs and even diffs between them (Eclipse always crashed here).
• etc. (list goes on forever) ….

Minor annoyances

Of course with the praise from above there are still some drawbacks. I often had problems with SCM merging facilities (especially subversion), I now always do merging on command-line. When upgrading or changing plugins restart has to be done manually (at least on Linux version). Also some intentions could be added (when adding @Override above a method <Ctrl>+Enter, Pull-Up method, Extract Superclass or Introduce Interface should be suggested). For other IDE converts the different types of autocompletions are a bit confusing (<Ctrl>+Space, <Ctrl>+<Shift>+Space, <Ctrl>+<Shift>+<Alt>+Space).

Price considerations

The Community Edition is free. For Ultimate Edition which I use you have to pay some money, but regarding the productivity boosts this is simply peanuts. Budget people, please do the math: Depending New User vs. Upgrade (~1.50EUR vs. ~0.75EUR per day) how much does a developer cost an hour? Apart from making the developer happier you will also save money if only a fraction of the IDE related idle/waiting time is reduced.