Saturday, September 26, 2009

Let the Agile Fad Flow By

The blogosphere is ablaze with"duct tapers" vs "agilists". As usual, there isn't much logic behind all the fiery arguments.

This is what happened. Peter Seibel interviewed Jamie Zawinski about how he built the original Mozilla browser for his *great* book Coders at Work. In it, Jamie was scathing about design pattern cultists who kept waving the "Gang of Four" book around and having intense discussions about the "right" pattern and so on, but couldn't write good code to save their lives (Sounds like some people we encounter today? Read on!). Joel Spolsky, writing a review of the book took a dig at what he saw as the successors of the design patterns cultists - people who write books and attend conferences etc and wave different books around.

Robert Martin, one of the chaps who signed the Agile "Manifesto" wrote a very logic lite, but moderate(for him), article juxtaposing "Duct Tape programmers" against what he calls "Clean Code" programmers. A few more bloggers took their lead from him and blew this artificial dichotomy between "shipping" coders and "clean coders" into a full fledged blog war about (as usual) nothing much.

The first thing to note is that this is a storm in a teacup. For all the fury on blogs and twitter, essentially people who really write great code and ship continue to do that, not paying any attention to these passing fads.

Software has, from the time it became important enough to build businesses round, always had flaky methodologies and people who try to sell them.

Just in my experience, first there were the "OO will save the world" hypesters, including the design patterns evangelists - this was the crowd Jamie encountered. Then there was RUP, then XP, then agile, now "lean software".

The common pattern in all these fads (and a quick way to recognize these things as fads) is to realize that the set of people who try to sell methodologies and the set of people who are brilliant programmers, product designers and so on are completely disjoint.

Do a thought experiment. Think about some brilliant programmers you know, the "change the world with code they write" type folks. For me these are people like Peter Norvig and John Carmack and Paul Graham and Linus Torvalds. You may have other heroes.

Can you imagine these people peddling agile? Just try envisioning it. Linus Torvalds trying to become a Scrum Master?

Your mind will melt down.

Think about people who've designed world changing products. (Jonathan Ives who designed the IPod for example) Do these people peddle methodologies or act as methodology consultants? Does Steve Jobs do this? Would Warren Buffet do this?

Here is the secret. Nobody who is really good at programming/product design/marketing/business (or anything) will demean himself by trying to make money telling other people how to do what they are good at. They'd rather do whatever it is they are best in the world at. World Champions may become coaches when they are old and tired and broken down, not when they can still play and win.

The standard defence is "but, but, but we are coaches. Even Tiger Woods has a coach, Coaches aren't necessarily the best players".

Yes in some fields coaches have value. The interesting thing is that it is Tiger Woods chooses a coach, and validates his coaches' skill by becoming the best golf player in the world, not the other way round.

When Linus Torvalds hires Uncle Bob to educate him about how to write Kernel Code better, or Tim Sweeney hires Ron Jeffries to help him design the next Unreal Engine, then then yeah they are "coaches". The fact is, sports doesn't map to software all that well in this aspect.

The best programmers don't hire "agilists" or "RUP Experts" or "Six Sigma Black belts" or whatever as coaches. If they have design issues to thrash out they talk to other brilliant developers. Brilliant product designers might talk to other brilliant product designers.

Fact of the matter is, "agile coaches"(and other methodology vendors) are hired by some combination of mediocre corporations, mediocre managers, and mediocre teams. Agile in particular rose in the enterprise world and that is where its focus (rightly) remains. So yes, if you work in the enterprise world, you sometimes get stuck with such "coaches" and get caught up in the latest methodology fads. It is just something that goes with the job. You might even want to become such a "coach" figure than muck around with some 10 year old accounting module from hell. It is an easier life. More power to you.

Now I'll tell you how to shut up an agile (or any other methodology evangelist). From my comment on Luke Halliwell's blog post, "the Agile Disease"),

The problem with agile is not “consultants” per se but “consultants” who have no credibility building and delivering great software.

For Games, when John Carmack or Tim Sweeney advocate agile for game studios, I’ll listen.

When its someone who has no product we can judge, advocates a methodology that failed in its very first project, where it originated (spare me the redefinitions of “failure” If the client kicked the team out that is “failure” enough for me) that is a different kettle of fish entirely.

Next time an “agile consultant” pitches for your money, ask him to show you code he has written that made a substantial change in your domain.

“I wrote/ led-the-team-that created Doom/Unreal/World of Warcraft” is acceptable.

“Oh yeah I created this uber cool enterprise system which you can’t look at and I don’t have any open source code, but I know how to write clean code and to “lead” teams, cross my heart” isn’t."


The key sentence is this,

"Next time an “agile consultant” pitches for your money, ask him to show you code he has written that made a substantial change in your domain."

The fact is no agile consultant today has written any world changing code,nor have they designed any world changing products (Imagine the guy who designed the IPod peddling a product design methodology for a living) or built or run any world changing companies.

The "coach" argument is dismissed easily too. If they are coaches, are their clients brilliant programmers? (this is where the Tiger Woods analogy falls down) It is always some El Crapola enterprise project *manager* who hires a methodology coach, never a great programmer, or team of great programmers (Do you think when Paul Graham, Robert T Morris and Trevor Blackwell were working on ViaWeb, they needed an "agile coach"?) .

Coming back to the Jwz-Spolsky-Uncle Bob tangle, just think of who among these three is trying to sell you something. Jamie doesn't sell anything (well I think he sells drinks and entertainment at his club/bar/whatever but he doesn't sell anything programming related). Spolsky sells a Bug Database. Uncle Bob sells Training Courses (via his company Object Mentor).

Now there is nothing wrong with selling anything in a capitalist world. But there is a motto every buyer should live by, Caveat Emptor- Buyer Beware. It is upto you to be on the defense against snake oil salesmen (of whatever category) and it is up to you to ensure you don't spend money on being "coached" to become a good developer by people who aren't themselves great developers.

TDD vs Duct Tape is a false dichotomy. There can be great design and code without TDD (or even unit tests) and rigorous TDD can result in terrible, ultra brittle codebases. (Private joke for ThoughtWorkers of a specific era - anyone seen "Class FieldedBusinessObject " recently?! Those were the days!).

TDD as a "design methodology" has no legs (See an agile "guru" struggle to write a Sudoku Solver). Having unit tests helps in regression testing and maintenance. That's it. If you need unit tests to "design" you aren't a great designer of software. Sorry. Sad but true.

Every fad eventually dissipates and sometimes they leave behind (very small) nuggets of value. The "everything should be modelled with Objects" fad passed (remember all the books from Booch and Rumbaugh and Iverson about how to do Object design?). Design Patterns are almost dead as a movement. Agile is facing a backlash (and will die soon enough).

Will there be other as yet unimagined fads (and associated methodology consultant/certification types) who'll replace agile? Of course. As long as fools with money are willing to be "coached" into improving their programming ability, there will be demand for such fraudsters and where there is demand, there will be supply. Whether you should be part of this supply/demand equation, and at what cost to your soul, you have to decide.

Let all these distractions and squabbles flow by. Focus on doing good work (however you define good work). Learn from people who are really good in your chosen field. Shut out the babbling of the methodology vendors and consultants.

So if you are a programmer trying to get better, just write the best code you know how. Then get better. Listen to people who are better programmers than you (Read "Coders At Work"!). Ignore the fads.

Update: jwz's reaction to this whole fiasco "It's such a strange article, in that it's mostly favorable to my point of view but with such a breathless amazement to it, like he's just discovered an actual unicorn or something. "Look, everybody! Here's a hacker who actually accomplished things and yet he doesn't fetishize the latest fads that I and all of my friends make our living writing about!"

As Luke Gorrie tweeted, "The takeaway from jwz C@W interview isn't "write crappy code,"but "quit your stupid enterprise job & have an impact on the world""


[1] What makes the book "Coders At Work" awesome and a book every programmer should read, in my opinion, is that the author interviews great programmers, not random fad vendors. There is a lot of wisdom in the book.

If he ever writes "Methodology Vendors at Work" how many people will buy a copy? ;-)


Vladimir Levin said...

I do wonder about this 'duct tape' programmer metaphor. It seems to me that the kind of programmer in 'coders at work' and in general the ones you refer to in this entry is more the 'lone genius'. Such people are great at developing unique, game-changing software that is well ahead of the curve, often before anyone else sees value in it (for e.g. the fellow who wrote that torrent system), but I'm not sure of the applicability of their experience to a broader context... I think it was Feynman, when asked how he approached a problem, who said something like he wrote the problem down, thought really hard about it, and wrote down the solution.

Ravi said...

1) There is no "duct tape" programmer in the meaning you have given it, anywhere in the *original* discussion. The netscape browser was built by a team of people (including Jamie Zawinski) not one guy hacking in a garage.

The "Duct Tape" (vs "Clean Code") juxtaposition was brought in by SPolsky and "Uncle" Bob, not by jwz or Seibel,.(which ws one of my points)

(2)"All the ones you refer to in this entry is more the 'lone genius'"

- not at all I refer to Linus, Norvig and Carmack. They are all geniuses certainly (and therefore rare) but hardly "lone". All of them have worked/do work in teams. They might have "lead" status sometimes because of their talent, but to equate that to "but I'm not sure of the applicability of their experience to a broader context." is not very meaningful. By this logic, since a large number of people are idiots, we should look at how to have more idiots in programming.

This (ducttape loner vs a "team player") is a red herring, just as much is "duct tape" vs "clean code".

. I've worked with many brilliant programmers in my limited experience,a nd to a man, they were all much more humble and modest than your garden variety patterns/agile fanatic who is more certain of his orthodoxies and so insufferable.

Linus for example is said to be arrogant and harsh on fools. I would rather work with him any day than some "nurturing" moron like Ron Jeffries.

Madhav said...

Nice post, Ravi. Would be good to catch up sometime. Will ping you over the weekend perhaps.

sriram said...

Insightful. Thanks. I had never asked myself the question "Why don't world class programmers have coaches?" I'll have to forward THE Agile Sudoku Solver's resume to jwz, Norvig and the rest of the guys in coders at work.

robin2 said...

I don't think it is adequate just to think: "This is just a fad, so as long as I personally avoid falling for it then I'll be OK."

Here's my reading of Zawinski's story: work that he'd put his heart and soul into got stomped on by people less able than him, people who most likely had no idea that this is what they were doing; this left him hacked off enough to quit the industry entirely. This isn't a victory, a heart-warming story of an honest craftsman winning out over a bunch of gobshites. And this is jwz - what chance to the rest of us have got?

Another thing: in your post about Peter Norvig you criticized Ron Jeffries for not knowing the fundamentals of his craft. I mention this because in the Norvig interview in Coders at Work he says that programming is best characterised as being a craft. In fact, "craft" is becoming a bit of a buzzword. This is no bad thing, if it ranks the abilities we develop, over the years, through our own work or the observation of others, over a tick-box view of skills. I'd take the publication of Coders at Work and Masters of Programming as being part of this trend.

But wouldn't it undermine the effort to take the craft of programming seriously if "craftmanship" itself became another fad? I've got bad news here: there already is a Software Craftsmanship manifesto, and - well, well - Ron Jeffries and Bob Martin are signatories.

Going back to the Agile Manifesto: the first value - individuals and interactions over processes and tools - looks reasonable at first sight. Isn't this just saying that we can't expect methods, frameworks, and what-have-you to do our thinking for us, or to relieve us of the burden of developing our own judgement? So how does it end up with people saying that, no, if we just use this method and this framework (e.g. TDD and xUnit) then everything else will take care of itself?

Unless we figure out what is really going on here then this will happen over and over, like a re-occurring nightmare, like a bug we keep observing but never fix.

Tord.F said...

Nice post, really puts the "become a 'certified' whatever master if you do this course" on the agenda. But a lot of these Methodologies doesn't really speak about if you follow this Methodology you will become a great programmer, it's just a way to work. And at the end of the day you just got to do what works. Great post.

John said...

You mention that Agile arose in the enterprise, and there is should stay. You also mention that the Agile peddlers are only good for mediocre corporations/managers/developers.

Well you're completely right. It seems Agile was developed in response to the the much less useful waterfall process in use in most enterprise environments. So that's what it's for. I'd argue that it's exactly suited for that environment, a mediocre corp, run by mediocre managers full of mediocre developers who may or may not want to get better, but who, by the law of averages, are as good as Linus Torvalds.

So what's wrong with mediocre developers posing as evangelists selling Agile to mediocre corps and, probably, making the developers lives better because their manager suddenly has been trained to listen to them and remove obstacles ? The enterprise environment practically requires expensive consultants to advocate change before they'll chance it, so I don't see how this is a problem.

Agile isn't for Norvig or the others, it's not for world-changing products. It's for Joe #2774 out of 5k developers as CompuGlobalHyperMegaCo who needs to write another accounting report.

Anonymous said...

You asked to mention famous programmers who use methodologies:

Donald Knuth advocates literate programming and used it to write TEX.

Dijkstra wants us to prove programs correct, although I don't know if he himself has authored any famous software.

You mention Warren Buffet, well he is a proponent of value investing.

Ravi said...

Ah well yet another anonymous idiot.

let's see,

When Warren Buffet makes his money by being a "value investment coach" vs being a value investor, then you can use him as an example.

No wonder you preferred to be "anonymous", with such muddy thinking.

All the examples you use are successful *as* programmers (or as an investor). Sure they have their prefered ways of working but they remain programmers or researchers or investors, not methodology vendors.

Agile "coaches" have done nothing creditworthy in terms of products or programming and can do nothing but "coach" those who can't see through their spin.

Just read the damn blog post already!