In ‘Cosmiblog’, the previous incarnation of my blog (2001-2005), I had written several times about how the Computing industry, as manifested through the development of operating systems, applications, hardware, is stuck in a vicious circle of pointless-reinvention very occasionally disturbed by massive paradigm shifts originating either from brilliant startups or industry leaders aiming to maximise profits. There may be several reasons as to why we’re stuck in 1970s user interfaces, 1960s software reliability and so on, but I think development has lots to do with it, both from a technical, but also from a cultural point of view.
In this article I’ll write a bit about the development process, as I see it after about 20 years of programming living with computers.
1980s to today
I started programming when I was 8, about a year and a half after my first close encounter with a computer. First programming languages were Logo and BASIC; the turtle was my friend and, for a brief period, so was goto 🙂
Then came Pascal, followed by C and x86 Assembly. While all those languages varied considerably, they were more or less all similar in their approach to computing: imperative, procedural, rigid, statically typed. During my first two years at university I was exposed to a number of programming environments: from the low-level assembly and machine code of the 6502, 68000, PowerPC and MIPS architectures, to the elegance and fluidity of Haskell and Prolog, with more or less 25 languages and paradigms in between.
Twenty years. Then, like now, it was common to do much of the work involved with computer science — and the industry in general — in procedural (or OO) languages: C, Java, C++ and so on.
The commoditisation of developers
I am not a developer. And that’s not because I do not develop software.
I am not a developer because I do not identify with the definition of the role. I probably do not identify with the majority of the group possessing the role. So what the hell is this article about? Well, a developer used to convey a composite meaning to me: a person with the ability to transfer considerably complex mathematical, logical reasoning to computer code in a personal, almost artistic manner in order to achieve some desired outcome. The developer meant the Architect. The Designer. The value lied in the confluence of art and science, discipline and freedom. Think of what Richard Stallman defines as a hacker. Smart. Playful. Free.
The freeware and shareware market is full of them. True developers. The Open Source movement has some. Programming computers for a few years should make even the most average of programmers realise that the most important part of programming is thinking about the program, designing it, sketching it out. Hey! New job description: A Software Architect.
In the world of real-life economics, corporations etc. good programmers become software architects if they like computers or project managers if they don’t.
Sometimes there just isn’t enough money to go around, so the software architect is also the project manager. Maybe she’s the developer, too! 🙂
Someone designs software and others help in implementing it. Sometimes they don’t even know why they are implementing some specific functionality. Sure, it pays, but I think the forced classification and the low (oh so low) standards might have something to do with the reason software so often sucks.
Looking back at several high-level software projects of the past twenty years, you’ll probably see that most successful of them involved a relatively small number of extremely capable people, instead of hordes of programming monkeys.
Programming monkeys are bad, yet the job market seems to support them. People that come out of a technical school or university proudly exclaiming:
I love {C,C++,Java,C#,Prolog,FORTRAN} (ok the last two were a joke, no one does that!)
People with no imagination, no brains. Just a million synapses full of the JDK or STL reference manuals.
Smart people learn ideas, not languages or technologies. Smart people love ideas, not languages or technologies. They see those ideas where dumb people cannot, and use them. Yet, the industry drives those people today to becoming ‘architects’ or ‘managers’. The dumb — or young — end up writing the code and injecting the bugs. I am not generalising here, I’m trying to make a point about corporate economics in the computing sector and how things work (or, rather, don’t). Development is largely (and increasingly) considered as a lowly activity carried out by the lowest members of a complex and long hierarchy. And that’s bad. Not just for software written today, but also for the technologies: where good universities and smart people will learn the concepts, ideas, and theory behind computer science, the lesser institutions and students, will focus on the vocational skills that will get them a job; which almost always translates to some computer language rapidly becoming obsolete any time now.
The supply, and uptake by the industry, of a large number of cheap ultra-specialised-I-know-what-a-threadlocal-variable- is-in-java-but-I-don’t-understand-anything-else ‘developers’ is, I feel, responsible to a large extent to the state of development today. Oh, that and the developer that expects everything to be magically provided by a library or API of some sort.
Back in 1998 I was told by a professor at university that it would be less than 10 years before 90% of newly written code will be API calls. Well, excluding the pointless rewrites of existing functionality of course…That goes some way in explaining why we need the monkeys..but don’t let me digress.
Technology
During my first two years at university I worked for 2 or 3 companies either as a contractor or as an intern. In one of them, a number of medium to large sized applications were developed in strict, yet primitive programming environments: new development was taking place in J2EE and older applications were maintained and extended in C and C++ programs, with an assortment of utility and shell script supporting functions written in Perl or Korn/Bash shell scripts. In every dev. group (7-8 people) one or two would know the software and the technology well: they’d either adopt the ‘old-wise man role’, giving advice to the rest, but other than that having a minimal contribution (and a very fat paycheck) or they’d be working hteir arse off trying to cover for the newbie developer that can’t find his way around the repository.
This was the era of J2EE, and as such, many of the new projects were adopting the Java enterprise spaghetti that’s become as scary as win32 APIs. During my short time there, I got to know one of their C++ programmers and we both shared an interest (passion) in programming so we had some interesting discussions on ‘advanced’ C++ topics. Yet, the more I spent talking about C++ concepts such as multimethods, patterns, traits, and the generic programming principles so often found in books by Alexandrescu, Sutter, Meyers,
Just before I left London, about a year ago, a colleague and friend at Imperial switched from using C++ to using Lua for the mainline code for his software simulator. He was also a (computer language) polyglot and he too was familiar with the advanced and often ignored aspects of C++ programming. A few months after he started using Lua, he witnessed an impressive boost in productivity. At the same time he retained fast, ‘tight’ C++ code for some of the more computationally intensive parts of the simulator.
It may come as a surprise to many developers, esp. those stuck in corporate environments, but languages such as Lua, Python, Ruby and others are increasingly gaining momentum among those that keep an open mind to new technology. Nope, I’m not referring to Web Development. Besides academia, games are an excellent example: perhaps the majority of large commercial games use (and have been using) scripting languages for several years: from classic adventures such as Grim Fandango and Escape from Monkey Island (Lua in 1998! Wow!), to fragkings such as Unreal, FarCry, strategy games like Civilization 4 to name just a very few. Adobe’s recently announced Lightroom application contains a significant portion (40%) of Lua code.
Technically, there’s a multitude of technologies we have today, that can make a developer’s life easier, software more robust, faster etc. Academia has produced numerous ideas that are just gathering dust. We know the properties of dynamically typed languages and those of statically typed ones. We know the advantages of OO vs, say, functional languages. We understand patterns. We also have pretty good ideas about self-modifying code (a paradigm that has eclipsed in almost every single domain of programming since the 80s). A few weeks ago I was reading about Scala, a very promising language by the Ecole Polytechnique Fédérale De Lausanne in Switzerland. Similar in some ways to OCaML, Scala is an interesting pure-OO programming language with several interesting features including (but not limited to) functional-language-like pattern matching, higher-order functions, case classes whilst remaining firmly within the statically typed realm. It seemed interesting enough to merit some attention. Even if that meant reading up on some introduction and some examples.
The Parrot VM has been in development for several years. How many developers of dynamically typed languages such as those it supports are familiar with it? I doubt there’ll be many. Same thing goes for Nemerle, a language that marries functional and OO features under the .NET platform and supports meta-programming.
There’s so much more we can do with computers, yet we’re imprisoning ourselves in the confines of arcane languages and ideas. Worse, we worship those same arcane languages and ideas. In a way, machine code/assembly programmers of the ’50s,’60s and ’70s were free. Sure, they had a task of infinitely larger difficulty, but they were free to perform as their imagination and intelligence allowed. Frustratingly, modern developers cannot.
But then again the meaning of the word has changed, hasn’t it?
Closing words..
This is not meant to be a comprehensive treatise on development. I hope it’s as clear to you as it is to me. I don’t dislike Java, any of the above mentioned languages that start with the letter C, or any of the languages mentioned above for that matter. I have and still use most of them, both professionally and for fun, and they are all good for some things.
Developers today are often engaged in utterly boring tasks involving excruciatingly long API call sequences (see J2EE or win32) and an ever dumber job description that can fry your brain and make you a programming monkey. You don’t want that to happen. If you’re a professional developer, engaged in the stressful routine of writing software in a single platform/language, but you still love technology and computers, try giving yourself a break by reading up on some new exotic language instead. Write some Open Source software with it. Open your mind! It helps! You’ll write code faster! You’ll be fitter! You’ll enjoy life more!