Monday, September 28, 2009

Funding vs Hiring

From Hacker News,

"Candidate A is a great programmer. He is unbelievably fast, very creative, has a lot of knowledge. He sees a problem and has brilliant solutions at hand very quickly. He just gets it. But he is not a team player. He'll go off and do stuff his way without telling anyone. Nobody sees his code before he's done. He hates discussions. He might just ignore any decisions that were made. And you never know if he'll quit tomorrow.

Candidate B is not the most creative type. His code is acceptable but you can see his limited experience. He is still good and he certainly gets the job done but there's not much you can learn from him. But he's extremely reliable and very loyal. He's happy with any work that comes along. He's a very good team player. He will give his input but accept decisions once they're made. People enjoy working with him.

What would you do? Pass on both of them? Hire A? Hire B? Wait for somebody who is a programmer as A but as reliable as B (and risk waiting forever)?"

Paul Graham responds

"Incidentally, if you're funding rather than hiring, you generally want A. That's one reason I'd rather be an investor than a boss."

(Full Discussion here)

Paul makes a subtle point (beyond the hypothetical candidate selection).

Funders and bosses think, decide, and act differently. As do "BigCo" developers vs startup founders.

Bosses want to control (or "boss around" ) their subordinates. Funders want to maximize chances of success. These are not necessarily identical, or even congruent forces.

I was thinking of a "startup within BigCo" effort I was (briefly) part of and one of the fundamental mistakes made was that no one really acted like a founder or (more importantly) a funder.

Talented (if not particularly aggressive) people, lots of money. One Year. Lots of meetings (Oh God were there meetings! We had meetings about meetings. Meetings to report results of *other* meetings. Meetings to discuss the agenda of forthcoming meetings...).

No product. No revenue. Lots of PowerPoint.

Lots of hope.

My prediction? This effort will meander along for another year or so, produce some tiny revenue streams (eventually) and then die a quiet death. Which is a tragedy. The people involved (with a couple of exceptions) were all great and I would work with any of them again in a heartbeat. The system was self defeating.

PG nails it (again).

Saturday, September 26, 2009

Let the Agile Fad Flow By

The blogosphere is ablaze with"duct tapers" vs "agilists". As usual, there isn't much logic behind all the fiery arguments.

This is what happened. Peter Seibel interviewed Jamie Zawinski about how he built the original Mozilla browser for his *great* book Coders at Work. In it, Jamie was scathing about design pattern cultists who kept waving the "Gang of Four" book around and having intense discussions about the "right" pattern and so on, but couldn't write good code to save their lives (Sounds like some people we encounter today? Read on!). Joel Spolsky, writing a review of the book took a dig at what he saw as the successors of the design patterns cultists - people who write books and attend conferences etc and wave different books around.

Robert Martin, one of the chaps who signed the Agile "Manifesto" wrote a very logic lite, but moderate(for him), article juxtaposing "Duct Tape programmers" against what he calls "Clean Code" programmers. A few more bloggers took their lead from him and blew this artificial dichotomy between "shipping" coders and "clean coders" into a full fledged blog war about (as usual) nothing much.

The first thing to note is that this is a storm in a teacup. For all the fury on blogs and twitter, essentially people who really write great code and ship continue to do that, not paying any attention to these passing fads.

Software has, from the time it became important enough to build businesses round, always had flaky methodologies and people who try to sell them.

Just in my experience, first there were the "OO will save the world" hypesters, including the design patterns evangelists - this was the crowd Jamie encountered. Then there was RUP, then XP, then agile, now "lean software".

The common pattern in all these fads (and a quick way to recognize these things as fads) is to realize that the set of people who try to sell methodologies and the set of people who are brilliant programmers, product designers and so on are completely disjoint.

Do a thought experiment. Think about some brilliant programmers you know, the "change the world with code they write" type folks. For me these are people like Peter Norvig and John Carmack and Paul Graham and Linus Torvalds. You may have other heroes.

Can you imagine these people peddling agile? Just try envisioning it. Linus Torvalds trying to become a Scrum Master?

Your mind will melt down.

Think about people who've designed world changing products. (Jonathan Ives who designed the IPod for example) Do these people peddle methodologies or act as methodology consultants? Does Steve Jobs do this? Would Warren Buffet do this?

Here is the secret. Nobody who is really good at programming/product design/marketing/business (or anything) will demean himself by trying to make money telling other people how to do what they are good at. They'd rather do whatever it is they are best in the world at. World Champions may become coaches when they are old and tired and broken down, not when they can still play and win.

The standard defence is "but, but, but we are coaches. Even Tiger Woods has a coach, Coaches aren't necessarily the best players".

Yes in some fields coaches have value. The interesting thing is that it is Tiger Woods chooses a coach, and validates his coaches' skill by becoming the best golf player in the world, not the other way round.

When Linus Torvalds hires Uncle Bob to educate him about how to write Kernel Code better, or Tim Sweeney hires Ron Jeffries to help him design the next Unreal Engine, then then yeah they are "coaches". The fact is, sports doesn't map to software all that well in this aspect.

The best programmers don't hire "agilists" or "RUP Experts" or "Six Sigma Black belts" or whatever as coaches. If they have design issues to thrash out they talk to other brilliant developers. Brilliant product designers might talk to other brilliant product designers.

Fact of the matter is, "agile coaches"(and other methodology vendors) are hired by some combination of mediocre corporations, mediocre managers, and mediocre teams. Agile in particular rose in the enterprise world and that is where its focus (rightly) remains. So yes, if you work in the enterprise world, you sometimes get stuck with such "coaches" and get caught up in the latest methodology fads. It is just something that goes with the job. You might even want to become such a "coach" figure than muck around with some 10 year old accounting module from hell. It is an easier life. More power to you.

Now I'll tell you how to shut up an agile (or any other methodology evangelist). From my comment on Luke Halliwell's blog post, "the Agile Disease"),

The problem with agile is not “consultants” per se but “consultants” who have no credibility building and delivering great software.

For Games, when John Carmack or Tim Sweeney advocate agile for game studios, I’ll listen.

When its someone who has no product we can judge, advocates a methodology that failed in its very first project, where it originated (spare me the redefinitions of “failure” If the client kicked the team out that is “failure” enough for me) that is a different kettle of fish entirely.

Next time an “agile consultant” pitches for your money, ask him to show you code he has written that made a substantial change in your domain.

“I wrote/ led-the-team-that created Doom/Unreal/World of Warcraft” is acceptable.

“Oh yeah I created this uber cool enterprise system which you can’t look at and I don’t have any open source code, but I know how to write clean code and to “lead” teams, cross my heart” isn’t."


The key sentence is this,

"Next time an “agile consultant” pitches for your money, ask him to show you code he has written that made a substantial change in your domain."

The fact is no agile consultant today has written any world changing code,nor have they designed any world changing products (Imagine the guy who designed the IPod peddling a product design methodology for a living) or built or run any world changing companies.

The "coach" argument is dismissed easily too. If they are coaches, are their clients brilliant programmers? (this is where the Tiger Woods analogy falls down) It is always some El Crapola enterprise project *manager* who hires a methodology coach, never a great programmer, or team of great programmers (Do you think when Paul Graham, Robert T Morris and Trevor Blackwell were working on ViaWeb, they needed an "agile coach"?) .

Coming back to the Jwz-Spolsky-Uncle Bob tangle, just think of who among these three is trying to sell you something. Jamie doesn't sell anything (well I think he sells drinks and entertainment at his club/bar/whatever but he doesn't sell anything programming related). Spolsky sells a Bug Database. Uncle Bob sells Training Courses (via his company Object Mentor).

Now there is nothing wrong with selling anything in a capitalist world. But there is a motto every buyer should live by, Caveat Emptor- Buyer Beware. It is upto you to be on the defense against snake oil salesmen (of whatever category) and it is up to you to ensure you don't spend money on being "coached" to become a good developer by people who aren't themselves great developers.

TDD vs Duct Tape is a false dichotomy. There can be great design and code without TDD (or even unit tests) and rigorous TDD can result in terrible, ultra brittle codebases. (Private joke for ThoughtWorkers of a specific era - anyone seen "Class FieldedBusinessObject " recently?! Those were the days!).

TDD as a "design methodology" has no legs (See an agile "guru" struggle to write a Sudoku Solver). Having unit tests helps in regression testing and maintenance. That's it. If you need unit tests to "design" you aren't a great designer of software. Sorry. Sad but true.

Every fad eventually dissipates and sometimes they leave behind (very small) nuggets of value. The "everything should be modelled with Objects" fad passed (remember all the books from Booch and Rumbaugh and Iverson about how to do Object design?). Design Patterns are almost dead as a movement. Agile is facing a backlash (and will die soon enough).

Will there be other as yet unimagined fads (and associated methodology consultant/certification types) who'll replace agile? Of course. As long as fools with money are willing to be "coached" into improving their programming ability, there will be demand for such fraudsters and where there is demand, there will be supply. Whether you should be part of this supply/demand equation, and at what cost to your soul, you have to decide.

Let all these distractions and squabbles flow by. Focus on doing good work (however you define good work). Learn from people who are really good in your chosen field. Shut out the babbling of the methodology vendors and consultants.

So if you are a programmer trying to get better, just write the best code you know how. Then get better. Listen to people who are better programmers than you (Read "Coders At Work"!). Ignore the fads.

Update: jwz's reaction to this whole fiasco "It's such a strange article, in that it's mostly favorable to my point of view but with such a breathless amazement to it, like he's just discovered an actual unicorn or something. "Look, everybody! Here's a hacker who actually accomplished things and yet he doesn't fetishize the latest fads that I and all of my friends make our living writing about!"

As Luke Gorrie tweeted, "The takeaway from jwz C@W interview isn't "write crappy code,"but "quit your stupid enterprise job & have an impact on the world""


[1] What makes the book "Coders At Work" awesome and a book every programmer should read, in my opinion, is that the author interviews great programmers, not random fad vendors. There is a lot of wisdom in the book.

If he ever writes "Methodology Vendors at Work" how many people will buy a copy? ;-)

Friday, September 18, 2009

Owen Rogers on "Releasing To Production Every Week"

My friend and ex-colleague (from my days at ThoughtWorks) was in India recently to speak at the CodeChef conderence, an invite only geek conference organized by Naresh Jain (another friend and ex-colleague from TW ! Did I mention I have awesome friends?) and has put his slides on "Releasing to Production Every Week" online.

Unfortunately I wasn't around when he was in Bangalore, but since he has put his slides online, (imo,) everyone in the business of producing software should take a look. (That is deploying *to production* every week btw). Owen is a brilliant, thoughtful developer, and it shows (e.g, see this blog entry).

So what are you waiting for? Get the slides already!

Sunday, September 13, 2009

Intuit acquires for 170 million!

And my reaction is ....

Bwaaaaa ha ha ha ha ha ha ha ha!!!!!

People I worked with in Intuit will understand why I am rolling on the ground laughing and wiping tears of mirth. (especially those who attended the very first "generate ideas for GBD" meeting).

And there was this one manager who smirked when I said that was a big threat to Intuit and that Intuit would eventually have to pay big money to acquire them, if the "startup within the company" efforts proceeded on their existing trajectory.

Now who's the chump boyo? :-P

Don't get me wrong. Intuit had (and has) a lot of very nice people working for it, but it is hardly the most congenial environment for "startups".

My (politically incorrect now doubt) opinion is that this is a *good* thing. Large companies with entrenched bureaucracies can be many things but the one thing they can't be is innovative. They *should* innovate by acquiring startups who do. In that sense this is a great deal. Intuit has the money. has the people with the tech chops. Win Win.

*If* handled correctly (big if) an Intuit backed (however it is branded tomorrow) could be fearsome. But, as I said, it is a big if.

From the WallStreet Journal.

"Intuit plans to rewrite its Quicken Online software to be powered by Mint's technology."

We'll check back next year to see how that goes!

Impressive Language Implementors

(expanded from a tweet)

Slava Pestov just added Advanced Floating Point support to Factor. A very impressive piece of work. As someone who works with large datasets and machine learning algorithms, I've been bitten by weird floating point "features" of many mainstream languages (For those who don't know how important FP is, take a look at "How java Floating Point Hurts Everyone" (warning pdf) - an old paper but a good one, to get some background).

Anyway, this got me to thinking of who (among language implementors) I am most impressed by and I came up with the people working on GHC (I think mostly Simon Peyton Jones, I could be wrong), Slava Pestov (Factor) and Rich Hickey (Clojure). All these implementations have this mind bending quality that if you read the source for a while, you'll *always* learn something new. The Lua implementation has this quality too.

In the second bucket (the "average") come the various Ruby implementations (and Python). yes you can learn stuff by reading the code, but it is mostly "ho hum"Ruby is a weird case. The JRuby and Rubinius implementations in particular, have some clever ideas and have very bright people leading their efforts (as Slava pointed out on Twitter), but the original MRI implementation is so, for lack of beter words, brain dead, that *on the average*, Ruby implementation quality is only so so. (This is probably an unfair characterization but hey I was *tweeting*! You expect a reasoned thesis?!)

If you want to see an undead zombie of a language (in terms of the suckiness of the implementation), look at PHP (the language sucks too but that is another rant).

NB: I am talking of the quality of the *implementation* of languages here, not the design of the language, though the two are often correlated.

Friday, September 11, 2009

Sudoku in Coders At work

Once upon a time, I wrote this blog entry. Peter Norvig references it (indirectly) in his interview in Peter Seibel's great new book "Coders At Work".

"Seibel: Though your job now doesn’t entail a lot of programming you still write programs for the essays on your web site. When you’re writing these little programs, how do you approach it?

Norvig: I think one of the most important things is being able to keep everything in your head at once. If you can do that you have a much better chance of being successful. That makes a small program easier. For a bigger program, you need extra tools to be able to handle that. It’s also important to know what you’re doing. When I wrote my Sudoku Solver, some bloggers commented on that.

They said, “Look at the contrast—here’s Norvig’s Sudoku thing and then there’s this other guy, whose name I’ve forgotten, one of these test-driven design gurus. He starts off and he says, “Well, I’m going to do Sudoku and I’m going to have this class and first thing I’m going to do is write a bunch of tests.” But then he
never got anywhere. He had five different blog posts and in each one he wrote a little bit more and wrote lots of tests but he never got anything working because he didn’t know how to solve the problem. I actually knew—from AI—that, well, there’s this field of constraint propagation—I know how that works. There’s this field of recursive search—I know how that works. And I could see, right from the start, you put these two together, and you could solve this Sudoku thing. He didn’t know that so he was sort of blundering in the dark even though all his code “worked” because he had all these test cases.

Then bloggers were arguing back and forth about what this means. I don’t think it means much of anything—I think test-driven design is great. I do that a lot more than I used to do. But you can test all you want and if you don’t know how to approach the problem, you’re not going to get a solution.

Seibel: So then the question is, how should he have known that? Should he have gone and gotten a PhD and specialized in artificial intelligence? You can’t know every algorithm. These days you have Google, but finding the right approach to a problem is a little different than finding a web framework.

Norvig: How do you know what you don’t know?
Seibel: Exactly.

Norvig: So I guess it’s two parts. One is to recognize that maybe there is a known solution to this. You could say, “Well, nobody could possibly know how to do this, so just exploring randomly is as good as everything else.” That’s one possibility. The other possibility is, “Well, probably somebody does know how to do this. I just don’t know what the words are for it, so I
have to discover those.” I guess that’s partly just intuition and saying, “It seems like the kind of thing that should be in the body of knowledge from AI.” And then you have to figure out, how do I find it? And probably he could’ve done a search on Sudoku and found it that way. Maybe he thought that was cheating. I don’t know

Seibel: So let’s say that is cheating—say you were the first person ever to try and solve Sudoku. The techniques that you ended up using would still have been out there waiting to be applied.

Norvig: Let’s say I wanted to solve some problem in biology. I wouldn’t know what the best algorithms were for doing gene sequencing or whatever. But I’d have a pretty good idea that there were such algorithms. Then I could start looking around. At another level, some of these things are pretty fundamental—if you don’t know what dynamic programming is, then you’re at a severe disadvantage. It’s going to come up time and time
again. If you don’t know this idea of search in general—that you can make a choice and backtrack when you don’t need it. These are all ideas from the ’60s. It was only a few years into programming that people discovered these things. It seems like that’s the type of thing that everyone should know. Some things that were discovered last year, not everybody should know.

Seibel: So should programmers go back and read all the old papers?
Norvig: No, because there are lots of false starts and lots of mergers where two different fields develop completely different technology and terminology, and then they discover they were really doing the same thing. I think you’d rather have a story from the modern point of view rather than have to follow all the steps. But you should have them. I don’t know what the best books are for that since I picked it up the hard way, piecemeal."

Dr. Norvig is a much nicer (and wiser) person than I am. I (completely) agree with him that bloggers' blather (including mine) doesn't mean much and TDD is great as one (vs the only) tool in one's arsenal.

That said, I still think many so called "agile" "gurus" can't program for nuts, as Jeffries so amply demonstrates. It is hypocrisy of the first order that people who don't know how to program well advise the rest of us how to. The "agile" movement brimmeth over with such pseudo programmer snake oil salesmen.

"These are all ideas from the ’60s. It was only a few years into programming that people discovered these things. It seems like that’s the type of thing that everyone should know."

"Should know" is correct. It is also true that most "developers" wouldn't know Dynamic Programming if it showed up at the front door and slapped them silly.

One reason for this is that most "enterprise" programming uses essentially two data structures, the List and the Hashtable and most language implementations provide default implementations of these. And a lot of "programmers" do this all their lives and think they are cool because they drank the latest/greatest shiny methodology Kool Aid. And these folks get the "gurus" they deserve!

"Seibel: What about the idea of using tests to drive design?

Norvig: I see tests more as a way of correcting errors rather than as a way of design. This extreme approach of saying, “Well, the first thing you do is write a test that says I get the right answer at the end,” and then you run it and see that it fails, and then you say, “What do I need next?”—that doesn’t seem like the right way to design something to me."

heh! heh!

Anyway, it is great that something I wrote got reflected, however indirectly, in this great book.

Coders at Work is a great book with legendary programmers talking in great detail about many aspects of programming. It is very interesting how Peter Seibel (the author) has captured each "voice" so perfectly. Every programmer should read this book, *especially* if you are stuck in a sucky job writing leasing systems (or accounting systems or whatever).

"What power would Hell have if those imprisoned there were not able to dream of Heaven?" as the Dream King says in Neil Gaiman's "The Sandman", but even at the cost of granting Hell some power, do dream of Heaven, say I, for that way lies eventual redemption.

This book provides a glimpse of what programming looks like for people who are already "in heaven".

"Coders At Work" is Awesome

An extract from Chapter 1

"Seibel: What about books? Are there particular computer-science or programming books that everyone should read?

Zawinski: I actually haven’t read very many of those. The one I
always recommend is Structure and Interpretation of Computer Programs, which a lot of people are afraid of because it’s Lispy, but I think does a really good job of teaching programming without teaching a language. I think a lot of introductory-level stuff focuses on syntax and I definitely saw that in the classes I had in high school and in the intro classes at Carnegie-Mellon during my brief time there.This is not teaching people to program; this is teaching people where the semicolon goes. That seems like the kind of thing that’s going to scare people away from it more than anything, because that’s not the interesting part. Not even to someone who knows what they’re doing.

There was another book—what was it called?—about debugging, written by someone from Microsoft. It was about how to use asserts effectively. I remember thinking that was a really good book, not because I learned anything from it, but because it was the book you wish your idiot coworker had read.

Then there was another book that everybody thought was the greatest thing ever in that same period—Design Patterns—which I just thought was crap. It was just like, programming via cut and paste. Rather than thinking through your task you looked through the recipe
book and found something that maybe, kinda, sorta felt like it, and then just aped it. That’s not programming; that’s a coloring book. But a lot of people seemed to love it. Then in meetings they’d be tossing around all this terminology they got out of that book. Like, the inverse, reverse, double-back-flip pattern—whatever. Oh, you mean a loop? OK."

heh heh!

This book is really good and I am sitting here and reading this instead of working (and I have a LOT of work to do). I'll do a full review later, but if you are any kind of programmer just go buy it already.

@Peter Seibel Great Work.