The Secret of Professional Happiness

I was talking to Prakash Swaminathan the other day and he said something that I thought encapsulated the essence of having a great professional life.

(a) Work with people you admire, (b) on interesting projects and (c) work from home as much as possible.

I could imagine dropping (c) if the other two criteria were met (though it does make a lot of sense in today's networked world) but whenever I've compromised on (a) or (b) life has sucked, without exception.

So children,learn from your elders. Always work with great people on great projects and avoid the corporate politics bullshit and you'll be happy professionally.

Of course this assumes you are skilled enough (or are willing to work to get there) that awesome people want you on awesome projects but that is a different post altogether.

Who (and what) I would like to see at DevCamp

Comments, requests and suggestions on my last post are pouring in (re: my last post). Thanks everyone. One of the folks who sent me email asked "Who would *you* like to see speaking at DevCamp, assuming they are in India and willing to deliver a talk, and on what?"

Hmm. Interesting question. I haven't really thought about this very deeply but here is a quick response(very busy day, no time to edit, link to home pages etc, sorry)

In no particular order,

(1) Debashish Ghosh on deep Scala programming. This guy is really good.

(2) Baishampayan Ghose on the technical aspects of paisa.com

(3) Bhasker Kode on Erlang at Hover.in

(4) Peter Thomas - guru on things Wicket-ey, speaking of things wicket-ey. (Due disclosure , old friend of mine)

(5) Narayan Raman on *the evolution* of Sahi (and on running a company based around an open source tool he wrote. How cool is that?)

(6) Anyone from c42 or ActiveSphere on the challenges of setting an n-man (n < 7) consultancy and competing with the big boys (due disclosure both companies built by ex tw -ers. I know a few of them)

(7) Anyone (technical) from FlipKart. They seem to be doing good things (I am a satisfied customer) and I am interested in how they tackle the huge challenges in building (for e.g) reccomendation systems.

(8) Anyone at all in India doing serious work in Haskell (Scala would do).

(9) Anyone building/working in a *technically* challenging startup (Notion Ink, say) on their *technical* challenges.

(10) ThoughtWorkers hacking on stuff, on what they are hacking on. TW ers in general have all kinds of side projects going. The two Viveks (Prahalad and Singh) would be a good start.

Speaking at DevCamp 2010

I'll be speaking at DevCamp 2010. As in 2008, I have a "menu" of topics that people can vote on and will select the topic at the very last minute. Since I don't use slides(in general) this isn't very hard to pull off. More on this below.

Dev Camp is interesting because in India, there aren't any "developer to developer" conferences. Most are either company sponsored events (e.g Sun/Oracle/Adobe Tech days) or are overrun by "evangelists" hired by MegaCorps to sell their crapware to developers. DevCamp attempts to head these people off by stating "DevCamp is an annual BarCamp style unconference for hackers, by hackers and of hackers that began in Bangalore in 2008 with code and hacking as its core themes" Some of these "evangelists" are shameless enough to crash the conf anyway, but the Law of Two Feet often takes care of that.

So what do you talk about at DevCamp? (everything that follows is *my* opinion. I have nothing to do with the organizing of DevCamp)

If you are any kind of hacker, you have a pet project running on the side. You are learning or doing something that might be of interest to other developers. So in the last devcamp I attended (in 2008) someone was trying to replace JMeter with an equivalent Erlang tool and he gave a very interesting talk on the advantages and challenges of this approach.

Bring your laptop and show us what you are working on. *Don't* make one of those slide heavy "Introduction to Blah" type talks that are prevalent at most Indian conferences (last year's PyConf India was a good example of this iirc. Hopefully this year is better). Your audience consists of professional developers who are quite comfortable with looking up stuff on the Internet.

As the Dev Camp page puts it, "Assume a high level of exposure and knowledge on the part of your audience and tailor your sessions to suit. Avoid 'Hello World' and how-to sessions which can be trivially found on the net. First hand war stories, in-depth analysis of topics and live demos are best. ". Again some folks so try to sneak in "Introduction to Blah" where X is the latest "hot" topic (Clojure or Android would fit the bill these days for e.g), but again "The Law of Two Feet" (mostly) takes care of them.

If you want to talk about Clojure don't do "An Introduction to Clojure". In the days of YouTube, Rich Hickey can do that much better than you could. Talk about "How I built a Text Processing/WebcCrawler library in Clojure" or "My startup runs on Clojure" (and show us the code). Tell us what *you* know that few others do ("in-depth analysis") and/or show us interesting code you wrote ("live demos"). If someone were to do a talk on (for e.g) how the Clojure *compiler* works and the tradeoffs in its design, that would be interesting to me. If you are recycling "Clojure has macros, woot!" I don't care.

The other interesting aspect about DevCamps is how lightweight it is. There is none of the stuffiness associated with the usual company conferences. It is an *un* conference, like Barcamp, but without the legion of SEO marketing people, "bloggers", non-tech "founders" trawling for naive developers who'll work for free on their latest "killer idea"s etc who swarm Barcamp. BarCamp (imo) attracts fringe lunatics. DevCamp attracts (or should attract when it works well) competent developers.

So, these are the things I could talk about at DevCamp. Since I work on Machine Learning and Compilers, the topics reflect that experience. I could talk about how to build a Leasing System in Java but I doubt I'd have anything interesting to say ;-).

Send me email if you have a preference (or leave a comment here). I'll talk about whatever has the highest number of votes on Sep 4. "Customer Development" for sessions woot? Email > comments here > twitter but any and all forms of media are acceptable.

The topics from highest to lowest number of votes registered at the time of writing are

(1)An In Depth Look at the Category Theory Bits in Haskell (expanded version of the old Monad tutorial)

At DevCamp 2008, I presented a talk on "Understanding Monads" where the idea was that someone who knew nothing about Monads should come to the talk and walk out knowing how they work and when to use them. Instead of giving vague analogies("monads are space stations/containers/elephants.." you build monads from the ground up using first class functions. The talk included, in its first iteration, the List, Maybe and State monads. Later versions (over the years I have given the talk a few times) broke down the Category Theory behind monads and how it helps in structuring programs.

The latest version encompasses all the hairy Category Theory related bits and pieces(Applicatives, Monoids, Functors , Monad Transfomers...) which impede programmers trying to learn Haskell/Scala/ML etc. I don't assume any theory/math background from the audience and introduce required formalisms. The good news is that this is a very polished and popular topic (and is trending highest in the number of "votes") . The bad news is that I am bored of this talk (but will still use it if it scores the highest number of votes).


(2) Building a Type Inferencer in 45 minutes


Static Type Systems, especially those more powerful than the Java/C# variety are a mystery to most programmers. This can be seen for example in how developers with a Java background write "Java in Scala" than idiomatic Scala. The best way (and the Hacker's way) to understand how a Type Inferencer works is to build one. This session builds a Hindley Milner type checker with a couple of extensions.

(3) WarStory: How I escaped Enterprise SW and became a Machine Learning Dev

Self explanatory ;-)


(4) Proof Technique for Programmers - A Developer's gateway to Mathematics (and Machine Learning)

This comes out of something I observed in the Bangalore Dev community. A lot of people read "Programming Collective Intelligence" (a terrible book - read my HN "review" here - I am "plinkplonk". See also comments by brent) and fancy themselves "Machine Learning" people ("we aren't experts but we know the basics". Ummm . No, you don't :-P. )

The sad truth is, you can't do any serious machine learning (or Computer Vision, or Robotics, or NLP or Algorithm heavy) development without high levels of mathematics. "Pop" AI books like PCI are terrible in teaching you anything useful.

To quote Peter Norvig from his review of Chris Bishop's Neural network book (emphasis mine)

"To the reviewer who said "I was looking forward to a detailed insight into neural networks in this book. Instead, almost every page is plastered up with sigma notation", that's like saying about a book on music theory "Instead, almost every page is plastered with black-and-white ovals (some with sticks on the edge)." Or to the reviewer who complains this book is limited to the mathematical side of neural nets, that's like complaining about a cookbook on beef being limited to the carnivore side. If you want a non-technical overview, you can get that elsewhere (e.g. Michael Arbib's Handbook of Brain Theory and Neural Networks or Andy Clark's Connectionism in Context or Fausett's Fundamentals of Neural Networks), but if you want understanding of the techniques, you have to understand the math. Otherwise, there's no beef. "

The "if you want understanding of the techniques, you have to understand the math" bit is true for all areas of ML, not just Neural networks. The biggest stumbling block (there are many ;-)) for most developers attempting to grok the underlying mathematics is the proof based learning method most higher level Math/Machine Learning books assume.

E.g here is the *first* exercise of the *second* chapter of "Elements of Statistical Learning", a which (unlike PCI) book you *should* read if you plan to do Machine Learning-ey things

"Suppose each of K-classes has an associated target tk , which is a
vector of all zeros, except a one in the kth position. Show that classifying to
the largest element of y amounts to choosing the closest target, mink ||tk − y ||, if the elements of y sum to one."


This "Given X, Prove Y" structure is how almost all books in the field teach things. Sure you should code up the algorithms, but doing such problems is how you get *insight* into the field. And algorithms have their own problems (pun intended). Open Cormen et al's "Introduction to Algorithms" and you'll find questions like (randomly opening the third edition)

Problem 20.1 (e) Prove that, under the assumption of simple uniform hashing, your RS-vEB-TREE-INSERT (Note vEB tree == van Emde Boas tree) and RS-vEB-TREE-SUCCESSOR run in O(lg lg u) expected time.

Thus it turns out that for getting into many areas of interest, a knowledge of how to prove things is critical. You will make very slow or zero progress without that understanding. That is the bad news. The good news is, proofs are (relatively) easy for programmers to understand when presented the right way (acquiring skill takes a while). I wasted many years learning this stuff in inefficient ways. Don't make the same mistake.

Zero math background required. Just bring some paper to write on.


(5) Trika - A Hierarchical Reinforcement Learning framework in Scala

A demo and discussion on an RL framework I built. I haven't yet cleared the paperwork to Open Source this (the process is like pulling teeth, long story), but I can still show it off.

(6) Neuro genetic Algorithms - Theory and Applications


An interesting branch of AI/ML with some elegant applications. Again live demo of a couple of interesting algorithms and talk about design/performance trade-offs.

(7) Denotational, Operational and Axiomatic Semantics - Designing programming languages with mathematics

This is of interest to people building their own languages. Most language implementations are adhoc "hacks". They don't have to be.

If you plan to attend, let me know which of these topics strike your fancy. And if you are a reader of this blog, find me and say Hello.

See you at DevCamp!

The New American Militarism

Excerpt from the preface of Andrew Bacevich's "The New American Militarism: How Americans are Seduced by War"

The final point concerns my understanding of history. Before moving into a career focused on teaching and writing about contemporary U.S. foreign policy, I was trained as a diplomatic historian. My graduate school mentors were scholars of great stature and enormous gifts, admirable in every way. They were also splendid teachers, and I left graduate school very much under their influence. My own abbreviated foray into serious historical scholarship bears the earmarks of their approach, ascribing to Great Men—generals, presidents, and cabinet secretaries—the status of historical prime movers.

I have now come to see that view as mistaken. What seemed plausible enough when studying presidents named Wilson or Roosevelt breaks down completely when a Bush or Clinton occupies the Oval Office. Not only do present-day tendencies to elevate the president to the status of a demigod whose every move is recorded, every word parsed, and every decision scrutinized for hidden meaning fly in the face of republican precepts. They also betray a fundamental misunderstanding of how the world works.

What is most striking about the most powerful man in the world is not the power that he wields. It is how constrained he and his lieutenants are by forces that lie beyond their grasp and perhaps their understanding. Rather than bending history to their will, presidents and those around them are much more likely to dance to history’s tune. Only the illusions churned out by public relations apparatchiks and perpetuated by celebrity-worshipping journalists prevent us from seeing that those inhabiting the inner sanctum of the West Wing are agents more than independent actors. Although as human beings they may be interesting, very few can claim more than marginal historical significance. So while the account that follows discusses various personalities—not only politicians but also soldiers, intellectuals, and religious leaders—it uses them as vehicles to highlight the larger processes that are afoot.

Appreciating the limits of human agency becomes particularly relevant when considering remedial action. If a problem is bigger than a particular president or single administration—as I believe the problem of American militarism to be—then simply getting rid of that president will not make that problem go away. To pretend otherwise serves no purpose.

..................

The bellicose character of U.S. policy after 9/11, culminating with the American-led invasion of Iraq in March 2003, has, in fact, evoked charges of militarism from across the political spectrum. Prominent among the accounts advancing that charge are books such as The Sorrows of Empire: Militarism, Secrecy, and the End of the Republic, by Chalmers Johnson; Hegemony or Survival: America’s Quest for Global Dominance, by Noam Chomsky; Masters of War: Militarism and Blowback in the Era of American Empire, edited by Carl Boggs; Rogue Nation: American Unilateralism and the Failure of Good Intentions, by Clyde Prestowitz; and Incoherent Empire, by Michael Mann, with its concluding chapter called “The New Militarism.”

Each of these books appeared in 2003 or 2004. Each was not only written in the aftermath of 9/11 but responded specifically to the policies of the Bush administration, above all to its determined efforts to promote and justify a war to overthrow Saddam Hussein.

As the titles alone suggest and the contents amply demonstrate, they are for the most part angry books. They indict more than explain, and whatever explanations they offer tend to be ad hominem. The authors of these books unite in heaping abuse on the head of George W. Bush, said to combine in a single individual intractable provincialism, religious zealotry, and the reckless temperament of a gunslinger. Or if not Bush himself, they finger his lieutenants, the cabal of warmongers, led by Vice President Dick Cheney and senior Defense Department officials, who whispered persuasively in the president’s ear and used him to do their bidding. Thus, according to Chalmers Johnson, ever since the Persian Gulf War of 1990–1991, Cheney and other key figures from that war had “wanted to go back and finish what they started.” Having lobbied unsuccessfully throughout the Clinton era “for aggression against Iraq and the remaking of the Middle East,” they had returned to power on Bush’s coattails. After they had “bided their time for nine months,” they had seized upon the crisis of 9/11 “to put their theories and plans into action,” pressing Bush to make Saddam Hussein number one on his hit list.6 By implication, militarism becomes something of a conspiracy foisted on a malleable president and an unsuspecting people by a handful of wild-eyed ideologues.


By further implication, the remedy for American militarism is self-evident: “Throw the new militarists out of office,” as Michael Mann urges, and a more balanced attitude toward military power will presumably reassert itself.

As a contribution to the ongoing debate about U.S. policy, The New
American Militarism rejects such notions as simplistic. It refuses to lay the
responsibility for American militarism at the feet of a particular president
or a particular set of advisers and argues that no particular presidential election holds the promise of radically changing it. Charging George W. Bush with responsibility for the militaristic tendencies of present-day U.S. foreign policy makes as much sense as holding Herbert Hoover culpable for the Great Depression: whatever its psychic satisfactions, It is an exercise in scapegoating that lets too many others off the hook and allows society at large to abdicate responsibility for what has come to pass.
The point is not to deprive George W. Bush or his advisers of whatever
credit or blame they may deserve for conjuring up the several large-scale
campaigns and myriad lesser military actions comprising their war on terror. They have certainly taken up the mantle of this militarism with a verve not seen in years. Rather it is to suggest that well before September 11, 2001, and before the younger Bush’s ascent to the presidency a militaristic predisposition was already in place both in official circles and among
Americans more generally. In this regard, 9/11 deserves to be seen as an event that gave added impetus to already existing tendencies rather than as a turning point. For his part, President Bush himself ought to be seen as a player reciting his lines rather than as a playwright drafting an entirely new script.


In short, the argument offered here asserts that present-day American
militarism has deep roots in the American past. It represents a bipartisan
project. As a result, it is unlikely to disappear anytime soon, a point
obscured by the myopia and personal animus tainting most accounts of
how we have arrived at this point.




Great Book. Worth Reading.

Acquistion Aftertaste

Mini-msft asks

How big was the original iPhone team? How big was the KIN team? Why did one result in a lineage of amazingly successful devices in the marketplace, and the other become a textbook extended definition for "dud" ?

He goes on to quote an ex Danger employee

"And finally, one Danger-employee's point of view of why they became demotivated:

To the person who talked about the unprofessional behavior of the Palo Alto Kin (former Danger team), I need to respond because I was one of them.

You are correct, the remaining Danger team was not professional nor did we show off the amazing stuff we had that made Danger such a great place. But the reason for that was our collective disbelief that we were working in such a screwed up place. Yes, we took long lunches and we sat in conference rooms and went on coffee breaks and the conversations always went something like this..."Can you believe that want us to do this?" Or "Did you hear that IM was cut, YouTube was cut? The App store was cut?" "Can you believe how mismanaged this place is?" "Why is this place to dysfunctional??"

Please understand that we went from being a high functioning, extremely passionate and driven organization to a dysfunctional organization where decisions were made by politics rather than logic.

Consider this, in less than 10 years with 1/10 of the budget Microsoft had for PMX, we created a fully multitasking operating system, a powerful service to support it, 12 different device models, and obsessed and supportive fans of our product. While I will grant that we did not shake up the entire wireless world (ala iPhone) we made a really good product and were rewarded by the incredible support of our userbase and our own feelings of accomplishment. If we had had more time and resources, we would of come out with newer versions, supporting touch screens and revamping our UI. But we ran out of time and were acquired and look at the results. A phone that was a complete and total failure. We all knew (Microsoft employees included) that is was a lackluster device, lacked the features the market wanted and was buggy with performance problems on top of it all.

When we were first acquired, we were not taking long lunches and coffee breaks. We were committed to help this Pink project out and show our stuff. But when our best ideas were knocked down over and over and it began to dawn on us that we were not going to have any real affect on the product, we gave up. We began counting down to the 2 year point so we could get our retention bonuses and get out.

I am sorry you had to witness that amazing group behave so poorly. Trust me, they were (and still are) the best group of people ever assembled to fight the cellular battle. But when the leaders are all incompetent, we just wanted out.


I guess we need another ThinkWeek paper on how to successfully acquire companies, too. Between this and aQuantive, we only excel at taking the financial boon of Windows and Office and giving it over to leadership that totally blows it down the drain like an odds-challenged drunk in Vegas. And the shareholders continue to suffer in silence. And the drunks are looking for their next cash infusion."

Hilarious. You couldn't invent this stuff. But after my last stint at MegaCorp where my group blew through millions of dollars and delivered zilch (I got out early!), I am not surprised. Dumb dumber management structures have the irresistible property of stifling any innovation or effectiveness.

Something I am watching is HP's acquisition of Palm. I know a couple of good people at HP but by and large the company is bloated and dysfunctional. It will be interesting to see what they end up doing with the Palm assets.


The comments are hilarious on Mini's post.

"If Roz and/or Andy doesn't go, what does that say about our supposed value of "accountability?" I for one am tired of accountability meaning "we move them over here and give them a smaller project and hope they resign."

Heh heh! Something like this just happened to someone at my ex employer. I've come to the conclusion that there is only one MegaCorp worldwide and the idea of separate companies is probably an illusion fostered to give the loser employee types the illusion of changing jobs in the hopes of life getting better ;-)

Why learn Compiler Implementation?

Yesterday a friend called up and asked what I was doing and, among other things, I said " I am building a compiler for a language with features X, Y and Z". He replied "But why do that?".

Well I like building compilers, interpreters etc but there are good reasons why "mainstream" programmers should learn this stuff.

As Hal Abelson of MIT (co author of SICP ) said

"If you don't understand compilers, you can still write programs - you can even be a competent programmer- but you can't be a master"

Steve Yegge has an interesting, if more verbose, post at http://steve-yegge.blogspot.com/2007/06/rich-programmer-food.html

One minor side effect of knowing this stuff is that you are immune to language fads. The local fanboi crowd is jumping off the somewhat creaky Ruby bandwagon and onto the gleaming Clojure one. Watch out for a lot of half baked blather on the wonders of Lisp by people with not much of a clue. But it will be amusing and help pass the time.

Work from Home job for Kernel Hackers

Someone I know from Hacker News sent me this email (lightly edited)


"I am likely to hire one high-class Linux kernel engineer in November
time. I would be interested to have information on any possible
candidates that may fit the work.

I am looking for people who are:

- Enthusiastic about linux kernel work
- Have working knowledge of CPUs, cache internals, SMP, concurrency,
memory management. ARM/Embedded knowledge would be a plus.
- Ability to bring out a solution on his own, even architect a design
without my intervention.
- Familiar with open source work flow, i.e. using git, knows how to
create a clean patch that is well tested and send it by email.
- Good communication skills, i.e. ability to write full English
sentences with correct wording and no typos - you would be surprised to
see how people even lack this.
- Past work experience on the linux kernel would be helpful.


The work environment involves telecommuting from home with occasional
yearly meetups. Most communication is done by email. We are all about
core kernel development"

When he originally talked to me about this I had someone in mind but he decided to go to grad school and will be leaving India in August for the USA. If I had any kernel dev experience I would have taken it up myself - looks to be an awesome job.


So if anyone has the qualifications listed above and wants a cool job working from home, contact me with details of what you've done and I'll connect you to the hirer.

NB:I've got some emails wrt this post. Just to clarify, contact me with details of what you've done wrt kernel hacking/systems work. Links to submitted patches would be nice, as would any links to core systems code of any kind. This is *not* a GSOC kind of mentored position and needs people with prior experience.

This post is now obsolete. Do NOT repeat NOT keep sending me CVs. They go straight into the garbage folder.