Wednesday, October 24, 2012

On the "Do you want to be a programmer at fifty?" thing.

Once upon a time, James Hague asked an interesting  question on his blog.

"When I was still a professional programmer, my office-mate once asked out of the blue, "Do you really want to be doing this kind of work when you're fifty?"

James went on to identify two kinds of programming

Type A "work(ing) out the solutions to difficult problems. That takes careful thought, but it's the same kind of thought a novelist uses to organize a story or to write dialog that rings true. That kind of problem-solving is satisfying, even fun."

Type B "what most programming is about - trying to come up with a working solution in a problem domain that you don't fully understand and don't have time to understand... skimming great oceans of APIs that you could spend years studying and learning, but the market will have moved on by then ... reading between the lines of documentation and guessing at how edge cases are handled and whether or not your assumptions will still hold true two months or two years from now.. the constant evolutionary changes that occur in the language definition, the compiler, the libraries, the application framework, and the underlying operating system, that all snowball together and keep you in maintenance mode instead of making real improvements."

He went on to state that while he'll continue doing Type A programming, he isn't particularly interested in Type B, (presumably at fifty).

I was looking forward to some good discussion on this, but HackerNews (which, in spite of its flaws still has no competition) went off into some  tangents primarily about ageism in the software industry and there was surprisingly little discussion about what James actually said.

Now, is ageism a problem? Yes, it is. As people grow older, they are expected to do anything but programming. It is a cultural thing and not necessarily logical. I know someone who is a good programmer, but left Bangalore for a decade (programming all the while) and now can't get an interview (let alone a job)because "oh you have 18 years experience, we are looking for people with two years of experience.Sorry".

 So, yes ageism is a problem, even in Outsourcing Land, and there is plenty to be discussed, and action to be taken, with respect to ageism.But that is a topic for another day and  isn't quite the problem addressed by James Hague in his blog post.

In this post, I'll try to explain what I think (and it is just that, my opinion ymmv etc etc) about "Do you really want to be doing this at fifty?"

The essence of the question is "Do you want to be doing this(at  a future time point)". The question addresses the (evolution of) motivation to program, and James goes on to state that his motivation to do a  certain type of programming (unfortunately this is the more dominant type  of programming worldwide)  decreases with increasing age.

The question of motivation with respect to career activities has been discussed by a wide variety of people and a lot of research has been conducted. One  interesting insight has been articulated by Dan Pink - in his book "Drive - The Surprising Truth About What Motivates Us" he identifies 3 factors  that motivate us  (or demotivate us) to undertake and pursue any activity.

(1) Mastery - getting better at what you are doing 

(2) Autonomy - the degree to which you can direct your activity.

(3) Purpose (or meaning) - doing something that really matters.

If you get high scores in any of the above, ideally all three, and more importantly get more and more of all the above as your programming career progresses, of *course* you'll be programming at 50. Or 60. Why wouldn't you?

The problem of course, is that in most programming jobs you either hit a declining slope or at best, plateau, with respect to one or more of the above as you age. If you are on a team of 50 people, maintaining some legacy leasing system written in Java, with business analysts doing the business thinking and you are converting their thoughts into code you are being a scribe for other people's ideas in a rigid and ageing language, in a context where you are an expensive 'resource'.

In general , even at many product (vs services) companies,  a 'line programmer' has low levels of autonomy - other people - product managers, business analysts etc etc - tell him what to do. Legacy codebases constrain technology choices. His 'mastery', while not non existent, is of a shallow and frothy kind (hey I use Rails today instead of J2EE yesterday! Node.js vs rails blah) and writing the n-th business app pulling data off a database and putting it on a web page for some corporate drone to use to update his TPS reports crushes 'being part of a higher purpose'. Little autonomy, modest mastery, non existent  purpose. No wonder few people  want to be doing this kind of programming at fifty.

Thankfully, other kinds of programming do exist. John Carmack of Id Software is still programming in his forties because programming (and till recently, being a majority shareholder of a cutting edge games company!) helps him in maximizing all three  attributes.

Programming is a skill, like writing. Unlike writing, we live in a society where most people are code illiterate. And coding ability has (some) economic value. "Software is eating the world" etc, and so anyone who is comfortable with coding can exchange that skill for money. The deeper question is whether you can trade increasing experience in the skill of programming for increasing amounts of money (and mastery and autonomy and purpose) as time passes. For most people that function plateaus and then stays steady or declines.

If you were someone who knew how to write, but lived in an illiterate society you could exchange that skill for money, by being a scribe at so many cents per word. You write people's letters and wills etc and you get paid for it by word count. But if you did it for thirty years, and you are still writing letters for people when you are fifty, would you be satisified with your career? What about when your customers move to that desperate youngster who offers a lesser rate per written word?

A novelist uses writing in a different manner than someone who sets up as a letter writer for illiterate people. A novelist is trying to do something that uses writing grammatically correct sentences as a base skill, but the core of his work, plotting, characterization, dialogue, world building, etc lie on a plane well above deciding whether to put an i  before an e, or vice versa. And you don't even need much base skill.  Many people are pretty bad at grammar and still write best selling or world changing books.

Generalizing, the (conceptually) shortest step to getting away from the 'path to ageist irrelevancy' for programmers is to find a way to make money by transcribing your own ideas to  code. This might involve, for example,  stepping away from time and materials services  types of programming to product development. If not by yourself, then as a part of a small team. Even if you are still technically an employee, your are much more autonomous in small teams and companies (and codebases).

A second way out of a programming career deadpool track is to move to something related where programming skill actually helps in a major way, but it isn't the core of your job.

 If you are a Computer Science researcher who is also an excellent programmer, your primary job is the creation of new knowledge (aka research, embodied in published papers) but your programming skills will help.

If you are a (technical) startup founder using cutting edge languages and algorithms to build a superior product, your primary job is to satisfy users and pull your company ahead of the competition, then superior programming ability can help.

If you are a finance expert who can also code, you probably have a significant edge over your competitors who have to depend on the software people to come in after the weekend to prototype your idea.

Programming skill amplifies effectiveness in almost everything you can do.

Of course you could find yourself in the same situation in your new career. If you still lack money, autonomy, mastery and purpose, you are back at square one.  That said, being "an excellent programmer and a good  X" seems like a decent plan.

The idea that a programmer always has to work in a half understood domain transforming some one else's ideas into code is just that, an idea. It is a dominant idea, but nothing really stops anyone from mastering an interesting domain or acquiring a complementary  skill  in addition to programming.

That gets me to what I think is the right way to go about 'career planning'.

Decide what increased levels of autonomy, mastery and purpose mean to you.  Figure out what you need to do to get to that point. Then do whatever it takes.

If increased programming skill will move you towards increasing one or more of the three attributes, work on it. If something else (like writing skill, or knowing a domain, or getting good at sales, or going to medical school) looks more promising, work on that.  Assembly  line programming inside 'the industry', converting other people's thoughts into code in stone age languages, is a beginning. It need not  be the end.

To conclude, will I be programming  at fifty? I think so (these days I do as much maths and stats as programming, and everything feeds very nicely into everything else), but at fifty,  I'll be writing novels, not scribing letters.

Sunday, October 14, 2012

Why I (need to) write (more)

For some context on what sparked this, see this excellent post and discussion on Hacker News, (for those so inclined)

The reason I used to blog somewhat regularly is very simple - I found that I could take some half formed thought and flesh it out by writing about it.  Sometimes this would result in something actually useful to  people. Most often, I would just end up 'emptying my head' and think new thoughts.

And then something happened. I got an audience. And comments. And emails.  And controversies, and flames.

All good, because I didn't (and don't) *mostly* care what people thought of anything I wrote (or what people think of me for that matter) .  But there *is* a small element of reactivity and  friction when you know persons X, Y and Z will be reading what you write.

"Hmm is this too harshly worded for friend X. After all he is a big Ruby fan and if I say Ruby is a particularly  brain dead language, would this ruin his day?". And then I have to write stuff twice. First write down what I really want to write, and then go through it with and delete stuff or add more explanatory stuff and cautionary qualifiers and so forth.

I am not the only person facing this. People who are much better at writing than me apparently face this too.

Someone asked Paul Graham (he of the glowing essays fame) on Hacker News

"what's it like to have your every written (or spoken!) word analyzed by a bunch of people? Esp. people that you end up having some form of contact with.

It seems like it would be difficult to just have a public conversation about a topic. Do you think about that much when you write?"

and PG replied to say

"It's pretty grim. I think that's one of the reasons I write fewer essays now.
After I wrote this one, I had to go back and armor it by pre-empting anything I could imagine anyone willfully misunderstanding to use as a weapon in comment threads. The whole of footnote 1 is such armor for example. I essentially anticipated all the "No, what I said was" type comments I'd have had to make on HN and just included them in the essay."

If pg can't escape this fate, I sure can't.

but otoh I am less concerned than pg about whether someone mis-understands me etc because, I am (comparitively) not famous,  and I am not writing essays just spewing out (comparitively) unpolished *blog posts*. I could deal with misunderstandings just fine.

What I found harder to deal with was -- Twitter.

Once I started tweeting regularly, I found I could just try to distill whatever I was thinking about and just tweet about it. 140 characters is pretty good as a constraint. And since I regularly purge my twitter following of idots and nutcases, I am fairly sure I can convey exactly what I want to, and most people following me would understand (and if not clarifications are just  1 < n <   3  * 140 character tweets away.

But as good as twitter is, 140 character tweets aren't as good as multi paragraph blog posts to *explore ideas* (vs expressing their seed forms concisely). My writing has suffered, though like riding a bicycle it should come back pretty fast, and then I'll go around improving it.

The goal of my writing remains the same. I write to explore thoughts and ideas and 'empty me head'. No more. No less. 

One thing I am doing differently this time is to pay even less attention than usual to comments and reactions, and not bother clarifying what precise shade of  meaning I intended to convey and so forth. This is just me writing a letter  to a friend every  other week or so. It just takes the form of a blog post other people can read.

And so here goes.

I'm back.