A lightweight language such as Python or JavaScript fits a lot of program design into a small amount of source text. A heavyweight language such as C++ or Java uses more tokens to express the same design.
Many people (Guido van Rossum, Bruce Eckels , Steve Ferg) report that working in a lightweight language is about five times more productive than working in a heavyweight language. This matches my personal experience. Lutz Prechelt1 reproduces this difference within a research setting.
It’s easy to make up reasons why it’s easier to work with shorter sources. There’s less data entry, fewer places to make mistakes, and it’s easier to read the programs. A program in a heavyweight language has a lot of boilerplate that doesn’t add information about the program’s design or intent. Eliminating the boilerplate seems like a clear win.
But what about source text that does add information about the program — in particular, what about types? Removing type declarations from a program also makes it shorter, but it removes information that the compiler and runtime can use to verify the program. Doesn’t this make the program less reliable? Or, given a reliability goal, doesn’t it make it take longer to reach that goal?
From the anecdotes and studies above, it looks like this isn’t actually the case. I’d like to look at why.
1 An empirical comparison of C, C++, Java, Perl, Python, Rexx, and Tcl for a search/string-processing program, Lutz Prechelt. See especially figure 16.
Two Types of Types
An Explicity-Typed Language (ETL) is one in which functions and variables must be declared with type declarations. (These type declarations are conventionally called manifest types.) Java and C++ are ETLs.
An Implicitly-Typed Language (ITL) is one in which type declarations are not necessary. Some ITLs, such as JavaScript 1.5 and Python, lack type declarations altogether. In other ITLs, such as Dylan, Haskell, JavaScript 2.0, and JScript.NET, type declarations are optional, and can be added at any point during program development.
An ITL isn’t necessarily a weakly typed language. (In a weakly typed language, type mismatches are never detected. C++ is weakly typed where it uses with static casts.) Python is an example of latent typing: types are always checked, just not at compile time.
Neither is an ITL necessarily a dynamically typed language. Haskell and ML are examples of statically typed languages where the type declarations are often optional, but the types are known at compile time. The compiler infers variable and function types from the source, just as a Java or C++ compiler infers the types of individual subexpressions.
(This discussion is condensed from the appendix to Steve Ferg’s excellent article comparing Python to Java.)
No Pain No Gain?
It’s common knowledge that it takes longer to get a program in an Explicitly-Typed Language to first clean compile, because it takes time to get the type declarations right. Initial program development consists of editing type declarations as well as non-declaration text (the expressions, control flow, and program structure), followed by a block of time to get the program to first clean compile. The white area below represents work on the non-declaration portion of a program; the gray stripes represent work on type declarations.
In order to show the division between time spent on type declarations and time spent on other program text more clearly, I’m going to slide the type declarations to the right:
This picture appears to show that it takes more time to develop a program that contains type declarations. The received wisdom among ETL fans is that what you get for this extra time is error checking. ETL development takes longer, but gives you a more robust product. The equivalent ITL development effort (which isn’t shown here) would be even longer, because it would include debugging time to catch the problems that the compiler didn’t. In fact, some of these problems might not show up until the product is in the field.
Tests versus Types
Test-Driven Development (TDD) is a development methodology where the test cases are written first, and code is added to the program proper only as necessary to implement correct behavior for test cases.
TDD advocates have noted that if you write your test case first, you also get error checking. In fact, you get better error checking than you do with explicit types, because the tests are tailored to the code paths, the edge cases, and the specific problem domain. After all, the only kinds of errors that explicit typing catches are those of the kind where you used a string in integer context. Programmers who have used implicitly-typed languages can attest that these aren’t the kinds of errors that are hard to find. In fact, a lot of the work to make an ETL program compile is work that’s only necessary to get around how little to compiler knows about the runtime types.
The comparison of an ETL, where the extra work goes into typing explicit types, and an ITL with TDD, where the extra work goes into test cases, looks like this:
Let’s say you could start with a program without type declarations. Would you rather add type declarations, which catch a certain class of problems at compile time? Or would you rather add test cases, which in a strongly typed language catch this same class of problems at runtime, and catch problems that the type system can’t express as well?
But why not use TDD with an ETL?
Tests Plus Types
Here’s what happens when you use TDD with an ETL. You’re taking that first picture, and multiplying it by two: once for the test cases, and again for the program itself.
Robert Martin compares this TDD to dual-entry bookkeeping. Explicit types are another form a dual-entry bookkeeping. Using both explicit type declarations and test cases to validate is triple-entry bookkeeping. (And the time spent writing explicit types could have been spent on more test cases, or more of the program.)
But it’s actually worse than triple-entry bookkeeping, because an ETL requires that you use explicit type declarations in your test code too. Not only does the program itself cost more to write, but so do its test cases. It’s quadruple-entry bookkeeping: one of the three entries has two entries itself.
This is why even software written in an ETL is often tested with an ITL test harness. If you can afford to work in two languages, and if the test harness and the debugger are well-enough integrated that you can write the test cases first and use them for TDD, this works. (It brings the cost of using an ETL back down to triple-entry bookkeeping.) But there’s one more gotcha.
The Type Tax
The problem is that once type declarations are part of the source text, they’ve got to be maintained along with the rest of the code. The earlier the type declarations are added, the greater their contribution to the maintenance cost.
In a waterfall model of code construction, where all the code is written, brought to clean compile, and works the first time, this doesn’t matter. In incremental development, where compilable program is debugged or extended to handle additional features or test cases, the cost can be high.
Types are similar to comments, test cases, and end-user documentation in this way, except that types are coupled more closely to each line of source text. Factor out a variable from an expression, and the method comment and end-user documentation can stay the same, but a new type declaration is required. This tight coupling makes the maintenance tax on types even higher than it is for other program annotations, and the tradeoff for when to start adding them, if at all, is different.
Since the cost/benefit ratio of type declarations can be different at different points in program maturity, one style would be to add them late in program development (for performance reasons, or for their documentation benefits to future maintainers). This style of development is possible in a language with optional type declarations, such as Dylan or Haskell. That’s what the first picture on this page shows. ETLs such as C++ and Java simply don’t allow this.
Updated 9/25/2003 to rename “write types” to “edit tpes”, and to add a picture showing type declarations interleaved on a finer grain.
Comments
Very true.
Maybe you want to look on my workshop paper for oopsla 2004:
(Workshop on Revival of Dynamic Languages)
"One-Method Commands: Linking Methods and Their Tests"
http://pico.vub.ac.be/~wdmeuter/RDL04/papers/Gaelli.pdf
Joe wrote: "Python's list comprehensions, for example, are a huge timesaver--there's no real reason stuff like that would be incompatible with an ETL. I wonder why someone doesn't write a productivity-focused ETL..."
Slightly surprised that Oliver didn't mention Haskell's list comprehensions, Clean's array comprehensions...
"The received wisdom among ETL fans is that what you get for this extra time is error checking"
So are there really explicitly-typed-language fans or are they statically-checked-language fans? (The latter.)
Strangely you've fixated on whether or not types are part of the language syntax - most people fixate on when the language type checks: is the language statically-checked or dynamically-checked.
Pretty page!
"*latent typing" "a weakly typed* language"
Latent type is a synonym for implicit type.
Manifest type is a synonym for explicit type.
Strongly-typed / Weakly-typed don't have much meaning. Where you say "strongly typed" you probably mean "type safe".
http://www.cs.brown.edu/~sk/Publications/Books/ProgLangs/ page 205
http://citeseer.nj.nec.com/cardelli97type.html
Niccceee pagee
When we talk about development speed and correctness we cannot forget about the rule played by language's syntax itself. How many lines of STL code would we write to do something similar to Python's:
list = [1,2,3]
newlist = [i * i + 2 * i for i in list if i * i > 3]
Python has almost all features I would like to see in a programming language, and IMHO syntax is at least as important as implicitly-typed features.
Dynamic vs. static typing
Dynamic vs. static typing is a hot topic in computer programming lately. James Robertson points to an article by Oliver Steele on the subject. Although it is biased towards dynamic typing, it's an interesting read and explains the different kinds...
Typing is no substitute for testing, and it can help.
Often missed is the technology of Type Inference. Here I do not manifestly state types, but the compiler will ensure that my Type implements the appropriate Interface.
I get the best of all worlds!
"I have been experimenting with Python for the past serveral months and am becoming convinced that it is much faster to get a program which works correctly than in C++. IMHO its part the type issue and part just better language design."
I've come to similar conclusions with Ruby vs. Java (although the difference is much less dramatic if you consider IntelliJ part of the equation). I personally benefitted much less from the dynamic typing and much more due to Ruby's far easier to use data structures, blocks, and regular expressions. Python's list comprehensions, for example, are a huge timesaver--there's no real reason stuff like that would be incompatible with an ETL. I wonder why someone doesn't write a productivity-focused ETL...
"After all, the only kinds of errors that explicit typing catches are those of the kind where you used a string in integer context."
That's true, if you're just making an academic distinction between ETL and ITL. However, I think it's more interesting to compare today's mainstream static/compiled and dynamic/interpreted languages (C++/Java/C# vs. Python/Perl/Ruby). I often hear advocates of the latter group make the argument that compilers only catch type-related errors.
Modern compilers for the former group detect far more interesting programming errors like misspelled (or otherwise undeclared) identifiers, methods that have code paths that don't return values, unreachable code (due to invariant conditions), calls to unknown classes/methods, using an uninitialized variable as an rvalue... it goes on and on. Also, you're guaranteed that every inch of your source has been syntactically and semantically checked.
Of course, no tool is going to automatically catch all of your mistakes, just the easy ones. But I personally still think this brings a lot of value to large codebases--I feel a lot better knowing that even the code that is least likely to be touched, such as handling unlikely exceptions (like OutOfDiskSpaceException--how many TDD teams actually test for these kinds of conditions?), has at least gone under this cursory level of inspection; so if my exception handler has a misspelled method call it will be caught immediately.
That said, I believe there are tools for dynamic languages, like PyChecker, that do this to a cetain degree. If I were writing production code in a dynamic language, I would definitely grab one of those. Also, I would like to add that I'm not a static typing bigot; I use Ruby for writing one-off utilities and prototypes.
Test versus Type
(SOURCE: Test versus Type )- Nice explanation of the "type tax" QUOTE A lightweight language such as Python or JavaScript fits a lot of program design into a small amount of source text.
Typing
A nice discussion of Test versus Type, found via Ted Leung.
I have been experimenting with Python for the past serveral months and am becoming convinced that it is much faster to get a program which works correctly than in C++. IMHO its part the type issue and part just better language design.
The comments about testing are interesting but I am not sure what kinds of tests to write in some applications. Last night I wrote a program to extract information from our MySQL based bug database and make a chart of the number of new bugs per day. What kind of test program would i write to make sure my charting program is working?
About Peter comment about Intellij IDEA:
I have to say that one of the thing that I miss in the Python world is a decent tool like IDEA. Most of the IDEs I have see are just glorified text editors. With IDEA a Java class can be written SINGLEHANDELY without significantly slowing you down (I did it, typing with one hand while eating pizza on a deadline lunch), mostly because all the wonderful keyboard shorcuts and it's near-magical, telepaty-like interface.
The lack of a good, stable refactoring tool is also a showstopper to me... I like python, and I'm somehow proficient with it, but there is no way my productivity with it is going to be better than in Java as long as there isn't and IDEA-like tool for python (well... as long as there isn't a lot of String processing :))
Peter writes: "Implicitly typed language people don't have experience with a tool like IntelliJ IDEA whre the 'coding/refactoring' process is actualy driven by types which makes it even faster then without writing types at all. Tools like IDEA are not possible without explicit typing." Actually, the first refactoring browser was for Smalltalk, an ITL. (See here, here, and here). This obviously doesn't depend on types.
But the general point, that IDEs can use type information to provide additional services, such as parameter lists and method name completion, is an excellent one.
The other two uses of types that I didn't touch on is their use as program documentation, and to improve runtime performance.
The general point is that the fact that a runtime for a dynamically typed language can do "poor man's type inference" doesn't help the other readers of the program source: development tools, and humans.
I wrote about one problem with using manifest types for these purposes is their maintenance costs. That's just a cost/benefit tradeoff, and when you change the tools, the performance requirements, or how much of the design of the program is expressed through the types of its variables, the tradeoff can change.
Dave writes: "The fact is that without static typing you HAVE to write tests to make sure you aren't mis-using variables IN ADDITION to the normal tests you would write to test functionality." But if your functional tests don't test for correct usage of these variables, you've either got some unused variables in your source or you aren't testing much of your program's functionality. And most ways of mis-using a variable don't have to do with treating it as the wrong type.
Dave also suggests writing two programs in Python, one with and one without type definitions. I haven't run this exact experiment, but I did write both a Java and a Python interface to Wordnet, and tools that used each of these. The difference was striking. You can chalk some of this up to other differences between Java and Python, but I've done enough programming in languages that vary from each other in different ways (C++, Haskell, Common Lisp, and Smalltalk, to pick three that are very different from each other) to have something of a feel for what slows me down in each of them.
This is the most retarded thing I've ever read. You act like writing in a statically typed language is a process of writing it without types and then adding them back in. "Write Types" my ass! Have you ever written a line of code in Java or C or C++? Your "workflow diagrams" are a complete joke.
It doesn't take much longer to declare variables. The fact is that most of the reasons why there is a percieved increase in productivity for PERL or Python has nothing to do with typing. If you ad types and variable declarations to a perl program, you are not slowed down that much.
Testing is not just an all or nothing type of thing. The fact is that without static typing you HAVE to write tests to make sure you aren't mis-using variables IN ADDITION to the normal tests you would write to test functionality.
It is WAY easier to just use the compliler to do static type checks than it is to write test cases for those types of checks. The fact is that your idiotic notion of "write types" amounts to little or no time when coding.
Just do what I suggested, and write two programs in Python, one without your scary type definitions and one with them and see how long it takes you. The difference is probably negligable.
Dumbass.
Test vs. Type
Another article in the long, ongoing debate (30 years? :) about static and dynamic typing. Oliver Steele's Test versus Type is probably the most accessible and believable of the arguments I've seen. One variable that he does not account for...
Implicitly typed language people don't have experience with a tool like IntelliJ IDEA whre the "coding/refactoring" process is actualy driven by types which makes it even faster then without writing types at all. Tools like IDEA are not possible without explicit typing.
You didn't mention the third contender: design by contract. Pretending for a moment that its another extreme, contracts are written before the code (like tests), but perform checks at runtime (like types).
The upside: they are more comprehensive than types; the downside: they don't probe for failures at build time, and because they are invoked at runtime, aren't suitable for time-consuming tests.
The entire argument has been played out as contracts vs types and contracts vs tests as well, in the past; see
http://c2.com/cgi/wiki?DesignByContractAssertionsVsUnitTestsVsTypes
On the language front...
Oliver Steele, the Chief Software Architect at Lazlo Systems, has a
great explanation of the argument for implicitly typed languages in the presence of a rigid unit test discipline. He's also got some screenshots from Apple Dylan, showing the IDE. N