The answer to "Will you mentor me?" is

No.

Thanks for understanding.

Ok that was the nutshell version. If that answers your question, that's great.

The more detailed answer is "No, I won't mentor you,but in this blog entry I will tell you what to do instead, to get where you want to go". And I can reply with the url to this post the next time someone requests mentoring.

I once wrote a comment on Hacker News about what *I* learned about ending up with awesome mentors. Here it is, slightly edited so it reads a little better.

(The OP asked) Recently I have tried approaching a few good developers through their blogs about various matters including advice on how to go about some projects I'm undertaking but I am surprised at the unfriendly responses I have received. Maybe I have been going about it the wrong way but it got me thinking; Shouldn't the guys whose work we look up to be keen on what some of us young aspiring developers have to contribute to the community? I mean sure, we don't have the experience or skills some of these guys have(yet) but we still have some ideas that are viable with the right technical skills to back them. If any of them want to reach out and help nurture some potential talent, it may very well benefit all them in the end, whether financially or in terms of new ideas and experiences.


I commented thus


I have some experience in this, so let me try to explain a couple of things that I learned in the "school of hard knocks".

Once upon a time I was in a situation where I thought I could contribute to something one of the best programmers in the world was working on so I sent an email (I got the address from his webpage) and said something to the effect of " you say on this webpage you need this code and I have been working on something similair in my spare time and I could write the rest for you over the next few months because I am interested in what you are doing" and I got a 2 line reply which said (paraphrased) " A lot of people write to me saying they'll do this , but I've never seen any code yet so I am a little skeptical. Don't take it personally. Thanks. bye.".

So in the next email (sent a minute after I received his reply) I sent him a zipped file of code with an explanation that "this is what I've done so far which is about 70% of what you want" and he immediately replied saying "Whoa you are serious. That is refreshing .. ' and opened up completely, giving me a lot of useful feedback and very specific advice. He is a (very valued) mentor to this day.

Another time, I was reading a paper from a (very famous) professor at Stanford, and I thought I could fill in some gaps in that paper so I wrote a "You know your paper on X could be expanded to give results Y and Z. I could use the resulting code in my present project. Would you be interested in seeing the expanded results or code" email and I got a very dismissive one line email along he lines of " That is an old paper and incomplete in certain respects, Thanks".

So a few days later, I sent along a detailed algorithm that expanded his idea, with a formal proof of correctness and a code implementation and he suddenly switched to a more expansive mode, sending friendly emails with long and detailed corrections and ideas for me to explore.

Now I am not in the league of the above two gentlemen, but perhaps because I work in AI and Robotics in India,which isn't too common, I receive frequent emails to the effect of "please mentor me", often from students. I receive too many of these emails to answer any in any detail, but if I ever get an email with "I am interested in AI/ Robotics. This is what I've done so far. Here is the code. I am stuck at point X. I tried A, B, C nothing worked. What you wrote at [url] suggests you may be the right person to ask. can you help?" I would pay much more attention than to a "please mentor me" email.

In other words, when you asks for a busy person's time for "mentorship" or "advice" or whatever, show (a) you are serious and have gone as far as you can by yourself (b) have taken concrete steps to address whatever your needs are and (optionally. but especially with code related efforts)(c) how helping you could benefit them/their project.

Good developers are very busy and have so much stuff happening in their lives and more work than they could ever hope to complete that they really don't have any time to answer vague emails from some one they've never heard of before.

As an (exaggerated) analogy, think of writing an email to a famous director or movie star or rock star, saying "I have these cool ideas about directing/acting/ music. Can you mentor me/give me advice?"

I am replacing the words "app" and "technical" in your sentence below with "film" and "film making".

"if I have an idea for a film that I want to develop, but my film making skills limit me, it would be nice to have people to bounce the idea off and have it implemented. "(so .. please mentor me/give me advice/make this film for me).

Do you think a top grade director (say Spielberg) would respond to this?

The fact that you at least got a 2 line response shows that the developers you wrote to are much nicer than you may think. They care enough not to completely dismiss your email, though they receive dozens of similar emails a week.

As someone else advised you on this thread, just roll up your sleeves and get to work. If your work is good enough, you'll get all the "mentoring" you'll need. "Mentoring" from the best people in your field is a very rare and precious resource and like anything else in life that is precious, should be earned.

My 2 cents. Fwiw. YMMV.


That says most of what I want to say.

Some minor points now, addressing some points raised in the latest emails.

If you claim to be "very passionate about X" but have never done anything concrete in X I find it difficult to take you seriously. People who are really passionate about anything don't wait for "leaders" or "mentors" before doing *concrete* work in the area of their passion, however limited. Specifically wrt to programming/machine learning etc in the days of the internet and with sites like Amazon or the MIT OCW you have no limits except those you impose on yourself.

I hate to sound all zen master-ey but in my experience, it is doing the work that teaches you what you need to do next. Walking the path reveals more of the map. All the mentoring a truly devoted student needs is an occasional nudge here or an occasional brief warning there. Working with uncertainty is part of the learning. Waiting for mentorship/leadership/"community"[1]/ whatever to start working is a flaw that guarantees you will never achieve anything worthwhile.

Ok pseudo-zen-master-mode off. More prosaic version - "shut up and code". Or make a movie on your webcam, Or write that novel. Whatever. Your *work* will, in time, bring you all the mentoring and community or whatever else you need.

As always My 2 cents. Fwiw. YMMV. Have a nice day.

[1] For some reason Bangalore is crawling with people who first want to form a community and then start learning/working/whatever. These efforts almost invariably peter out uselessly. First do the work. Then if you feel like "communing" talk to others who are also working hard.Please read this , sent to me by my friend Prakash Swaminathan.

My Schedule for the rest of the year

- Starting tomorrow hack 12 hours a day as part of my current project. ( C & Haskell, Machine Learning, if anyone is interested). Will be traveling to places without Internet connectivity. So expect to be mostly offline.

- Oct end / Beginning of Nov: Back in Bangalore . Back online. Yay!

- Nov end: complete paperwork/documentation/training blah blah, Project handover.

- Nov end. This (phase of this) project done. Whew.

- December - somewhat free. I hope to release some Open Source code before EOY. Fairly old Scala code (so needs to be updated to Scala 2.8, add some comments and so on) but should be useful to others. Paperwork for Open Source release should come through before then.

Jan 1, 2010. New Year. No definite plans but lots of nice opportunities. Problems of plenty. Touch Wood. (Update: "No definite plans" is no longer true. A couple of VERY interesting opportunities in the air. Life is good.)

The Secret of Professional Happiness

I was talking to Prakash Swaminathan the other day and he said something that I thought encapsulated the essence of having a great professional life.

(a) Work with people you admire, (b) on interesting projects and (c) work from home as much as possible.

I could imagine dropping (c) if the other two criteria were met (though it does make a lot of sense in today's networked world) but whenever I've compromised on (a) or (b) life has sucked, without exception.

So children,learn from your elders. Always work with great people on great projects and avoid the corporate politics bullshit and you'll be happy professionally.

Of course this assumes you are skilled enough (or are willing to work to get there) that awesome people want you on awesome projects but that is a different post altogether.

Who (and what) I would like to see at DevCamp

Comments, requests and suggestions on my last post are pouring in (re: my last post). Thanks everyone. One of the folks who sent me email asked "Who would *you* like to see speaking at DevCamp, assuming they are in India and willing to deliver a talk, and on what?"

Hmm. Interesting question. I haven't really thought about this very deeply but here is a quick response(very busy day, no time to edit, link to home pages etc, sorry)

In no particular order,

(1) Debashish Ghosh on deep Scala programming. This guy is really good.

(2) Baishampayan Ghose on the technical aspects of paisa.com

(3) Bhasker Kode on Erlang at Hover.in

(4) Peter Thomas - guru on things Wicket-ey, speaking of things wicket-ey. (Due disclosure , old friend of mine)

(5) Narayan Raman on *the evolution* of Sahi (and on running a company based around an open source tool he wrote. How cool is that?)

(6) Anyone from c42 or ActiveSphere on the challenges of setting an n-man (n < 7) consultancy and competing with the big boys (due disclosure both companies built by ex tw -ers. I know a few of them)

(7) Anyone (technical) from FlipKart. They seem to be doing good things (I am a satisfied customer) and I am interested in how they tackle the huge challenges in building (for e.g) reccomendation systems.

(8) Anyone at all in India doing serious work in Haskell (Scala would do).

(9) Anyone building/working in a *technically* challenging startup (Notion Ink, say) on their *technical* challenges.

(10) ThoughtWorkers hacking on stuff, on what they are hacking on. TW ers in general have all kinds of side projects going. The two Viveks (Prahalad and Singh) would be a good start.

Speaking at DevCamp 2010

I'll be speaking at DevCamp 2010. As in 2008, I have a "menu" of topics that people can vote on and will select the topic at the very last minute. Since I don't use slides(in general) this isn't very hard to pull off. More on this below.

Dev Camp is interesting because in India, there aren't any "developer to developer" conferences. Most are either company sponsored events (e.g Sun/Oracle/Adobe Tech days) or are overrun by "evangelists" hired by MegaCorps to sell their crapware to developers. DevCamp attempts to head these people off by stating "DevCamp is an annual BarCamp style unconference for hackers, by hackers and of hackers that began in Bangalore in 2008 with code and hacking as its core themes" Some of these "evangelists" are shameless enough to crash the conf anyway, but the Law of Two Feet often takes care of that.

So what do you talk about at DevCamp? (everything that follows is *my* opinion. I have nothing to do with the organizing of DevCamp)

If you are any kind of hacker, you have a pet project running on the side. You are learning or doing something that might be of interest to other developers. So in the last devcamp I attended (in 2008) someone was trying to replace JMeter with an equivalent Erlang tool and he gave a very interesting talk on the advantages and challenges of this approach.

Bring your laptop and show us what you are working on. *Don't* make one of those slide heavy "Introduction to Blah" type talks that are prevalent at most Indian conferences (last year's PyConf India was a good example of this iirc. Hopefully this year is better). Your audience consists of professional developers who are quite comfortable with looking up stuff on the Internet.

As the Dev Camp page puts it, "Assume a high level of exposure and knowledge on the part of your audience and tailor your sessions to suit. Avoid 'Hello World' and how-to sessions which can be trivially found on the net. First hand war stories, in-depth analysis of topics and live demos are best. ". Again some folks so try to sneak in "Introduction to Blah" where X is the latest "hot" topic (Clojure or Android would fit the bill these days for e.g), but again "The Law of Two Feet" (mostly) takes care of them.

If you want to talk about Clojure don't do "An Introduction to Clojure". In the days of YouTube, Rich Hickey can do that much better than you could. Talk about "How I built a Text Processing/WebcCrawler library in Clojure" or "My startup runs on Clojure" (and show us the code). Tell us what *you* know that few others do ("in-depth analysis") and/or show us interesting code you wrote ("live demos"). If someone were to do a talk on (for e.g) how the Clojure *compiler* works and the tradeoffs in its design, that would be interesting to me. If you are recycling "Clojure has macros, woot!" I don't care.

The other interesting aspect about DevCamps is how lightweight it is. There is none of the stuffiness associated with the usual company conferences. It is an *un* conference, like Barcamp, but without the legion of SEO marketing people, "bloggers", non-tech "founders" trawling for naive developers who'll work for free on their latest "killer idea"s etc who swarm Barcamp. BarCamp (imo) attracts fringe lunatics. DevCamp attracts (or should attract when it works well) competent developers.

So, these are the things I could talk about at DevCamp. Since I work on Machine Learning and Compilers, the topics reflect that experience. I could talk about how to build a Leasing System in Java but I doubt I'd have anything interesting to say ;-).

Send me email if you have a preference (or leave a comment here). I'll talk about whatever has the highest number of votes on Sep 4. "Customer Development" for sessions woot? Email > comments here > twitter but any and all forms of media are acceptable.

The topics from highest to lowest number of votes registered at the time of writing are

(1)An In Depth Look at the Category Theory Bits in Haskell (expanded version of the old Monad tutorial)

At DevCamp 2008, I presented a talk on "Understanding Monads" where the idea was that someone who knew nothing about Monads should come to the talk and walk out knowing how they work and when to use them. Instead of giving vague analogies("monads are space stations/containers/elephants.." you build monads from the ground up using first class functions. The talk included, in its first iteration, the List, Maybe and State monads. Later versions (over the years I have given the talk a few times) broke down the Category Theory behind monads and how it helps in structuring programs.

The latest version encompasses all the hairy Category Theory related bits and pieces(Applicatives, Monoids, Functors , Monad Transfomers...) which impede programmers trying to learn Haskell/Scala/ML etc. I don't assume any theory/math background from the audience and introduce required formalisms. The good news is that this is a very polished and popular topic (and is trending highest in the number of "votes") . The bad news is that I am bored of this talk (but will still use it if it scores the highest number of votes).


(2) Building a Type Inferencer in 45 minutes


Static Type Systems, especially those more powerful than the Java/C# variety are a mystery to most programmers. This can be seen for example in how developers with a Java background write "Java in Scala" than idiomatic Scala. The best way (and the Hacker's way) to understand how a Type Inferencer works is to build one. This session builds a Hindley Milner type checker with a couple of extensions.

(3) WarStory: How I escaped Enterprise SW and became a Machine Learning Dev

Self explanatory ;-)


(4) Proof Technique for Programmers - A Developer's gateway to Mathematics (and Machine Learning)

This comes out of something I observed in the Bangalore Dev community. A lot of people read "Programming Collective Intelligence" (a terrible book - read my HN "review" here - I am "plinkplonk". See also comments by brent) and fancy themselves "Machine Learning" people ("we aren't experts but we know the basics". Ummm . No, you don't :-P. )

The sad truth is, you can't do any serious machine learning (or Computer Vision, or Robotics, or NLP or Algorithm heavy) development without high levels of mathematics. "Pop" AI books like PCI are terrible in teaching you anything useful.

To quote Peter Norvig from his review of Chris Bishop's Neural network book (emphasis mine)

"To the reviewer who said "I was looking forward to a detailed insight into neural networks in this book. Instead, almost every page is plastered up with sigma notation", that's like saying about a book on music theory "Instead, almost every page is plastered with black-and-white ovals (some with sticks on the edge)." Or to the reviewer who complains this book is limited to the mathematical side of neural nets, that's like complaining about a cookbook on beef being limited to the carnivore side. If you want a non-technical overview, you can get that elsewhere (e.g. Michael Arbib's Handbook of Brain Theory and Neural Networks or Andy Clark's Connectionism in Context or Fausett's Fundamentals of Neural Networks), but if you want understanding of the techniques, you have to understand the math. Otherwise, there's no beef. "

The "if you want understanding of the techniques, you have to understand the math" bit is true for all areas of ML, not just Neural networks. The biggest stumbling block (there are many ;-)) for most developers attempting to grok the underlying mathematics is the proof based learning method most higher level Math/Machine Learning books assume.

E.g here is the *first* exercise of the *second* chapter of "Elements of Statistical Learning", a which (unlike PCI) book you *should* read if you plan to do Machine Learning-ey things

"Suppose each of K-classes has an associated target tk , which is a
vector of all zeros, except a one in the kth position. Show that classifying to
the largest element of y amounts to choosing the closest target, mink ||tk − y ||, if the elements of y sum to one."


This "Given X, Prove Y" structure is how almost all books in the field teach things. Sure you should code up the algorithms, but doing such problems is how you get *insight* into the field. And algorithms have their own problems (pun intended). Open Cormen et al's "Introduction to Algorithms" and you'll find questions like (randomly opening the third edition)

Problem 20.1 (e) Prove that, under the assumption of simple uniform hashing, your RS-vEB-TREE-INSERT (Note vEB tree == van Emde Boas tree) and RS-vEB-TREE-SUCCESSOR run in O(lg lg u) expected time.

Thus it turns out that for getting into many areas of interest, a knowledge of how to prove things is critical. You will make very slow or zero progress without that understanding. That is the bad news. The good news is, proofs are (relatively) easy for programmers to understand when presented the right way (acquiring skill takes a while). I wasted many years learning this stuff in inefficient ways. Don't make the same mistake.

Zero math background required. Just bring some paper to write on.


(5) Trika - A Hierarchical Reinforcement Learning framework in Scala

A demo and discussion on an RL framework I built. I haven't yet cleared the paperwork to Open Source this (the process is like pulling teeth, long story), but I can still show it off.

(6) Neuro genetic Algorithms - Theory and Applications


An interesting branch of AI/ML with some elegant applications. Again live demo of a couple of interesting algorithms and talk about design/performance trade-offs.

(7) Denotational, Operational and Axiomatic Semantics - Designing programming languages with mathematics

This is of interest to people building their own languages. Most language implementations are adhoc "hacks". They don't have to be.

If you plan to attend, let me know which of these topics strike your fancy. And if you are a reader of this blog, find me and say Hello.

See you at DevCamp!

The New American Militarism

Excerpt from the preface of Andrew Bacevich's "The New American Militarism: How Americans are Seduced by War"

The final point concerns my understanding of history. Before moving into a career focused on teaching and writing about contemporary U.S. foreign policy, I was trained as a diplomatic historian. My graduate school mentors were scholars of great stature and enormous gifts, admirable in every way. They were also splendid teachers, and I left graduate school very much under their influence. My own abbreviated foray into serious historical scholarship bears the earmarks of their approach, ascribing to Great Men—generals, presidents, and cabinet secretaries—the status of historical prime movers.

I have now come to see that view as mistaken. What seemed plausible enough when studying presidents named Wilson or Roosevelt breaks down completely when a Bush or Clinton occupies the Oval Office. Not only do present-day tendencies to elevate the president to the status of a demigod whose every move is recorded, every word parsed, and every decision scrutinized for hidden meaning fly in the face of republican precepts. They also betray a fundamental misunderstanding of how the world works.

What is most striking about the most powerful man in the world is not the power that he wields. It is how constrained he and his lieutenants are by forces that lie beyond their grasp and perhaps their understanding. Rather than bending history to their will, presidents and those around them are much more likely to dance to history’s tune. Only the illusions churned out by public relations apparatchiks and perpetuated by celebrity-worshipping journalists prevent us from seeing that those inhabiting the inner sanctum of the West Wing are agents more than independent actors. Although as human beings they may be interesting, very few can claim more than marginal historical significance. So while the account that follows discusses various personalities—not only politicians but also soldiers, intellectuals, and religious leaders—it uses them as vehicles to highlight the larger processes that are afoot.

Appreciating the limits of human agency becomes particularly relevant when considering remedial action. If a problem is bigger than a particular president or single administration—as I believe the problem of American militarism to be—then simply getting rid of that president will not make that problem go away. To pretend otherwise serves no purpose.

..................

The bellicose character of U.S. policy after 9/11, culminating with the American-led invasion of Iraq in March 2003, has, in fact, evoked charges of militarism from across the political spectrum. Prominent among the accounts advancing that charge are books such as The Sorrows of Empire: Militarism, Secrecy, and the End of the Republic, by Chalmers Johnson; Hegemony or Survival: America’s Quest for Global Dominance, by Noam Chomsky; Masters of War: Militarism and Blowback in the Era of American Empire, edited by Carl Boggs; Rogue Nation: American Unilateralism and the Failure of Good Intentions, by Clyde Prestowitz; and Incoherent Empire, by Michael Mann, with its concluding chapter called “The New Militarism.”

Each of these books appeared in 2003 or 2004. Each was not only written in the aftermath of 9/11 but responded specifically to the policies of the Bush administration, above all to its determined efforts to promote and justify a war to overthrow Saddam Hussein.

As the titles alone suggest and the contents amply demonstrate, they are for the most part angry books. They indict more than explain, and whatever explanations they offer tend to be ad hominem. The authors of these books unite in heaping abuse on the head of George W. Bush, said to combine in a single individual intractable provincialism, religious zealotry, and the reckless temperament of a gunslinger. Or if not Bush himself, they finger his lieutenants, the cabal of warmongers, led by Vice President Dick Cheney and senior Defense Department officials, who whispered persuasively in the president’s ear and used him to do their bidding. Thus, according to Chalmers Johnson, ever since the Persian Gulf War of 1990–1991, Cheney and other key figures from that war had “wanted to go back and finish what they started.” Having lobbied unsuccessfully throughout the Clinton era “for aggression against Iraq and the remaking of the Middle East,” they had returned to power on Bush’s coattails. After they had “bided their time for nine months,” they had seized upon the crisis of 9/11 “to put their theories and plans into action,” pressing Bush to make Saddam Hussein number one on his hit list.6 By implication, militarism becomes something of a conspiracy foisted on a malleable president and an unsuspecting people by a handful of wild-eyed ideologues.


By further implication, the remedy for American militarism is self-evident: “Throw the new militarists out of office,” as Michael Mann urges, and a more balanced attitude toward military power will presumably reassert itself.

As a contribution to the ongoing debate about U.S. policy, The New
American Militarism rejects such notions as simplistic. It refuses to lay the
responsibility for American militarism at the feet of a particular president
or a particular set of advisers and argues that no particular presidential election holds the promise of radically changing it. Charging George W. Bush with responsibility for the militaristic tendencies of present-day U.S. foreign policy makes as much sense as holding Herbert Hoover culpable for the Great Depression: whatever its psychic satisfactions, It is an exercise in scapegoating that lets too many others off the hook and allows society at large to abdicate responsibility for what has come to pass.
The point is not to deprive George W. Bush or his advisers of whatever
credit or blame they may deserve for conjuring up the several large-scale
campaigns and myriad lesser military actions comprising their war on terror. They have certainly taken up the mantle of this militarism with a verve not seen in years. Rather it is to suggest that well before September 11, 2001, and before the younger Bush’s ascent to the presidency a militaristic predisposition was already in place both in official circles and among
Americans more generally. In this regard, 9/11 deserves to be seen as an event that gave added impetus to already existing tendencies rather than as a turning point. For his part, President Bush himself ought to be seen as a player reciting his lines rather than as a playwright drafting an entirely new script.


In short, the argument offered here asserts that present-day American
militarism has deep roots in the American past. It represents a bipartisan
project. As a result, it is unlikely to disappear anytime soon, a point
obscured by the myopia and personal animus tainting most accounts of
how we have arrived at this point.




Great Book. Worth Reading.

Acquistion Aftertaste

Mini-msft asks

How big was the original iPhone team? How big was the KIN team? Why did one result in a lineage of amazingly successful devices in the marketplace, and the other become a textbook extended definition for "dud" ?

He goes on to quote an ex Danger employee

"And finally, one Danger-employee's point of view of why they became demotivated:

To the person who talked about the unprofessional behavior of the Palo Alto Kin (former Danger team), I need to respond because I was one of them.

You are correct, the remaining Danger team was not professional nor did we show off the amazing stuff we had that made Danger such a great place. But the reason for that was our collective disbelief that we were working in such a screwed up place. Yes, we took long lunches and we sat in conference rooms and went on coffee breaks and the conversations always went something like this..."Can you believe that want us to do this?" Or "Did you hear that IM was cut, YouTube was cut? The App store was cut?" "Can you believe how mismanaged this place is?" "Why is this place to dysfunctional??"

Please understand that we went from being a high functioning, extremely passionate and driven organization to a dysfunctional organization where decisions were made by politics rather than logic.

Consider this, in less than 10 years with 1/10 of the budget Microsoft had for PMX, we created a fully multitasking operating system, a powerful service to support it, 12 different device models, and obsessed and supportive fans of our product. While I will grant that we did not shake up the entire wireless world (ala iPhone) we made a really good product and were rewarded by the incredible support of our userbase and our own feelings of accomplishment. If we had had more time and resources, we would of come out with newer versions, supporting touch screens and revamping our UI. But we ran out of time and were acquired and look at the results. A phone that was a complete and total failure. We all knew (Microsoft employees included) that is was a lackluster device, lacked the features the market wanted and was buggy with performance problems on top of it all.

When we were first acquired, we were not taking long lunches and coffee breaks. We were committed to help this Pink project out and show our stuff. But when our best ideas were knocked down over and over and it began to dawn on us that we were not going to have any real affect on the product, we gave up. We began counting down to the 2 year point so we could get our retention bonuses and get out.

I am sorry you had to witness that amazing group behave so poorly. Trust me, they were (and still are) the best group of people ever assembled to fight the cellular battle. But when the leaders are all incompetent, we just wanted out.


I guess we need another ThinkWeek paper on how to successfully acquire companies, too. Between this and aQuantive, we only excel at taking the financial boon of Windows and Office and giving it over to leadership that totally blows it down the drain like an odds-challenged drunk in Vegas. And the shareholders continue to suffer in silence. And the drunks are looking for their next cash infusion."

Hilarious. You couldn't invent this stuff. But after my last stint at MegaCorp where my group blew through millions of dollars and delivered zilch (I got out early!), I am not surprised. Dumb dumber management structures have the irresistible property of stifling any innovation or effectiveness.

Something I am watching is HP's acquisition of Palm. I know a couple of good people at HP but by and large the company is bloated and dysfunctional. It will be interesting to see what they end up doing with the Palm assets.


The comments are hilarious on Mini's post.

"If Roz and/or Andy doesn't go, what does that say about our supposed value of "accountability?" I for one am tired of accountability meaning "we move them over here and give them a smaller project and hope they resign."

Heh heh! Something like this just happened to someone at my ex employer. I've come to the conclusion that there is only one MegaCorp worldwide and the idea of separate companies is probably an illusion fostered to give the loser employee types the illusion of changing jobs in the hopes of life getting better ;-)

Why learn Compiler Implementation?

Yesterday a friend called up and asked what I was doing and, among other things, I said " I am building a compiler for a language with features X, Y and Z". He replied "But why do that?".

Well I like building compilers, interpreters etc but there are good reasons why "mainstream" programmers should learn this stuff.

As Hal Abelson of MIT (co author of SICP ) said

"If you don't understand compilers, you can still write programs - you can even be a competent programmer- but you can't be a master"

Steve Yegge has an interesting, if more verbose, post at http://steve-yegge.blogspot.com/2007/06/rich-programmer-food.html

One minor side effect of knowing this stuff is that you are immune to language fads. The local fanboi crowd is jumping off the somewhat creaky Ruby bandwagon and onto the gleaming Clojure one. Watch out for a lot of half baked blather on the wonders of Lisp by people with not much of a clue. But it will be amusing and help pass the time.

Work from Home job for Kernel Hackers

Someone I know from Hacker News sent me this email (lightly edited)


"I am likely to hire one high-class Linux kernel engineer in November
time. I would be interested to have information on any possible
candidates that may fit the work.

I am looking for people who are:

- Enthusiastic about linux kernel work
- Have working knowledge of CPUs, cache internals, SMP, concurrency,
memory management. ARM/Embedded knowledge would be a plus.
- Ability to bring out a solution on his own, even architect a design
without my intervention.
- Familiar with open source work flow, i.e. using git, knows how to
create a clean patch that is well tested and send it by email.
- Good communication skills, i.e. ability to write full English
sentences with correct wording and no typos - you would be surprised to
see how people even lack this.
- Past work experience on the linux kernel would be helpful.


The work environment involves telecommuting from home with occasional
yearly meetups. Most communication is done by email. We are all about
core kernel development"

When he originally talked to me about this I had someone in mind but he decided to go to grad school and will be leaving India in August for the USA. If I had any kernel dev experience I would have taken it up myself - looks to be an awesome job.


So if anyone has the qualifications listed above and wants a cool job working from home, contact me with details of what you've done and I'll connect you to the hirer.

NB:I've got some emails wrt this post. Just to clarify, contact me with details of what you've done wrt kernel hacking/systems work. Links to submitted patches would be nice, as would any links to core systems code of any kind. This is *not* a GSOC kind of mentored position and needs people with prior experience.

This post is now obsolete. Do NOT repeat NOT keep sending me CVs. They go straight into the garbage folder. 

Thieving Tharoor Gets His Due

(Warning:- Indian Politics)

My judgement about Shashi Tharoor is vindicated. The philandering, corrupt, "new hope" has been kicked out of the Cabinet for his clumsy attempt to reward his girlfriend with a 15 million dollar stake in an IPL team. His fanboi legion is out there on the internetz banging away with such gems like " What if he is corrupt? Others are even more corrupt" and "start a new party and you will be Prime Minister".


Please Mr T, do try.

His NRI following, who choose not to live in India, but want to have a say in its politics, must be heart broken. I mean, I can understand the appeal- Spend your productive years in the West and when the time comes to retire persuade a party to give you a parliament seat and after a 3 week campaign in a constituency you've never seen before and not even speaking the language of your constituents, become Member of Parliament and then immediately a minister with zero experience as a politician or an MP, thus setting you free to "contribute" to India, enriching your girlfriend by millions of dollars - what's not to like? It is the ultimate NRI fantasy.

Like all fantasies projected onto reality it does work once in a while, and invariably ends disastrously.


As MJ Akbar said in his blog entry

"Tharoor is writhing between a mistake and a misfortune. His mistake was to gatecrash a party without an invitation. He thought he could buy entry with Dubai and Gujarat money and spin out collateral political benefits by name-association with Kochi. He leapt to take the political credit when Kochi won the franchise. He is alleged to have taken financial rewards more surreptitiously. His friend Sunanda Pushkar's feeble claim that she is not a proxy is silly. You do not get sweat equity in perpetuity, which means free and forever, with a starting value of Rs 70 crore, for being an unknown executive of a Dubai company. There hasn't been a case of "cheque-payment culpability" of this order since the transactions that ended the chief ministership of A R Antulay in 1981. Nearly 30 years ago, Congress inexplicably tried to defend the indefensible before dumping Antulay so hard it virtually broke the warhorse's back. Mystery repeats itself.

Tharoor's misfortune was to encounter an adversary who could out-Twitter him at high noon in the gunfight at IPL corral. Tharoor and Lalit Modi have more in common than sharp suits, sharp wits and a dogged commitment to the television cameras. Having achieved so much through effective use of the media, they were convinced their favourite weapon remained the best option. They went to war through the media. A veteran like Sharad Pawar would have told them, had they but asked that children in glasshouse nurseries shouldn't throw stones.

Modi has one advantage over Tharoor; he is in the private sector. His accountability is to fiscal laws. Tharoor affects the image of the Congress at a time when the party cannot afford a greasy controversy. Tharoor is the first Congress minister in the Sonia Gandhi-Manmohan Singh government to be publicly pilloried for alleged corruption."


I wouldn't put it past His Sliminess to wriggle his way back into some position of power. But for now, Good Riddance.

Anyway he promised to make Trivandrum a "world class city" if he got elected. As Member of Parliament he can still work towards that. I supect he won't get a chance next time ;-). It would help if he knew some people in his constituency and their problems and/or spoke the language. Oh well now he has the time for all that. We'll wait and watch.


Mood: Delighted.

Some advice for Mr Tharoor: You have about 3 friends in the Congress Party hierarchy. Fortunately for you one of them is the Prime Minister. Get Dr Singh to have a pliable CBI officer do an "investigation" and then declare you innocent. Hey Presto get your ministry back and then we can figure out how to make 70 crores back(with interest). It would help if you could keep your mouth shut and not tweet your usual inanities while all this is going on. Good Luck!

Meanwhile (non political) temperatures are soaring in Delhi. 52 degrees centigrade the other day. Any more and people will burst into flame.

PS: I don't have access to the internet these days ( I had to make a special effort for this post - the taste of "I told you so" is too sweet ) so if any of the ex-minister's fanboys (yeah you "ppl" or "tweeple" or whatever you morons call yourself these days) want to leave nasty anonymous comments like the last time I wrote about Saint Tharoor, you'll have to wait till I get back online (the middle of May, more or less).

What to do when "they" hate Indian developers

Some fellow posted on proggit

"Currently the top rated link on proggit is

"What did I do wrong? (or, how are you supposed to hire a programmer?)"


And the first rated comment is

What did I do wrong?
In the end, we went with the Indian company
Is this some kind of a clever troll?



As an Indian (and a partner at an Indian software consulting company), am I not supposed to be offended by this? Would your answer had been same had the company been an Israeli one? I see similar highly voted comments on proggit all the time, and its very infuriating, as obv. I generally like hanging out here."


and titled it "Dear Proggit: why the hatred for India?"

My answer

"(I am an Indian programmer living in India) You are just being too sensitive. Except for some xenophobic rednecks, no one hates India. There are plenty of unskilled Indian "developers" especially in the enterprise sw outsourcing companies who can't code to save their lives and it is only natural that people who have been burned once are leery about outsourcing work to Indian companies.

The way to fix this is to do good work, more specifically write great code. The Japanese did exactly this in manufacturing, turning around their reputation form a producer of cheap low quality gee gwas to master sof mnufacturing. Whining is useless.

Just do good work. Then do better. Rinse. repeat. The reputation and "hate" will take care of themselves.
"


I am so tired of Indian developers being ultra "sensitive" and doing everything but write good code.


An even shorter version of my answer is "Shut up and Code."

full thread

No getting away from me

An email (lightly edited) I got from a friend working on Financial Software in London

"Here I am minding my own business, doing my job testing/developing Trading strategies designed by one of the Quants + Traders and what do I hear -

someone in the corner is talking about something very
mathematical - it all sounds gobbledegook to me and then I hear the word "Re-inforcement Learning" (I know you work on something like that)

I walk over, say hi (to some French mathematicians from Ecole Poly) and casually inquire what they are talking about, comes the reply - Modern Statistical Learning for predicting market behavior and tuning algorithmic trade strategies,

On the Screen - Java code - author @Ravi Mohan

There is no running away from you is there :-)
"

No there isn't!

;-)

A Haskell Journey

Over the last couple of years, I've been using Haskell (oddly enough) as a scripting/shell language, somewhat akin to bash, to tie together various bits and pieces of code written in other languages. (See http://www.cse.unsw.edu.au/~dons/data/Basics.html for some shell like utilities).

One of my goals for this year is to really master Haskell so I can use it as a primary language. As I dig in, I find there are two levels of Haskell.

Level 1 consists of algebraic data types, pure (and lazy) functions, typeclasses and modules. Someone fluent in another language can (relatively) easily wrap their heads around this portion of the language and start programming. There are plenty of tutorials and books (including "Real World Haskell") that teach this style of Haskell usage.

But, sooner or later one comes across code that is written in "level 2" Haskell - making heavy use of Monads, Monoids, Arrows and all the other Category Theory goodness. Monads get an abnormal mindshare among would-be Haskell developers but there is more to Haskell than Monads. No one has really written a comprehensive guide to this bit of Haskell. The nearest we have is the TypeClassopedia (warning, PDF) but that is more a collection of links for further reading than a detailed exposition. The Wikibook is uneven. RWH does a hop-skip-and jump over these bits - a correct decision given the focus and size of the book.

What we really need is an "Advanced Haskell" book which assumes a knowledge of Level 1 Haskell and then lays out the CT bits in an orderly fashion. Monads for example, are best understood from a Category Theory perspective than through some tortured analogy to Space Stations or Elephants or whatever. Some exposition of Type Theory would help too - a knowledge of kinds, for example is very useful to decode some of the advanced bits ( TAPL's last chapters have a good exposition iirc).

In the absence of an "Advanced Haskell" book, the best option is to read the various papers, trawl the mailing lists and so on for answers to specific questions. I've learned some Category Theory before (and am fairly comfortable with Type Theory) so I haven't found this to be particularly hard but I can see how it could be (very) hard for someone without this background. Mastery of this level would be when I can code *fluently* with Comonads and such. I am not there yet.

That said, Haskell is the most elegant language I've ever used and I plan to write a lot of code in it. I plan to open source some Haskell code over the next couple of months, so we'll see how much I've really understood!

Learning about Machine Learning

Bradford Cross has posted an awesome blog post (edit: removed link, since Bradford took down the post) titled "Learning about Statistical Learning". If you plan to work in ML, read the post, buy some of the books and work through them.

Could save you years of work if you are systematic from the beginning (I wasn't), especially if you are self taught (I am).


I work on different domains (Robotics/Computer Vision/Simulation) from Bradford and so have a different list of books. Please read Bradford's lists first. This is a supplement to his awesome post rather than a replacement.

I assume you are a good developer and you have a solid grip on algorithm analysis etc.(though, that said, see reccomendations for Discrete Math books below)

The first step..

Learn proof techniques *first*. You'll make no serious progress till you do. The best book is

Velleman's "How to Prove It" - reccomended by Bradford but I am repeating it here because this is critical.

Mathematics

In my experience you need to be somewhat comfortable with 6 branches of Mathematics before you can tackle ML. Imo, best to take a year and get these right before venturing into ML proper. (I know, it sounds awfully boring. I wasted a lot of time trying to shorten this step. In this case, the long way is the real shortcut)

(1) Calculus - best "lite" book - Calculus by Strang (free download) ,

best "heavy" books - Calculus by Spivak, Principles of Mathematical Analysis a.k.a "Baby Rudin"


(2) Some book on Discrete Math (don't know what to recommend here - I don't like Rosen's book) + a good book on say Introduction to Algorithms by Cormen et al will do [*]

(3) Linear Algebra (First work through Strang's book, then Axler's)

(4) Probability (Bertsekas is a good book for those with no prior exp) and

(5) Statistics (I would recommend Devore and Peck for the total beginner but it is a damn expensive book. So hit a library or get a bootlegged copy to see if it suits you before buying a copy, see brad's list for advanced stuff.)

(6) Information Theory (MacKay's book is freely available online)

Basic AI.

Brad suggests Mitchell's book.

I think AIMA (3d Edition) is much better. ( I am biased. I wrote and maintained the Java code for a long while -- children, don't do this. Java is an terrible language to develop AI algorithms in. If you need the JVM use Scala or Clojure -- and I think it covers a lot more than Mitchell does. Take a look at both. Pick one).


Machine Learning.

NB: you need all the linear algebra, calculus etc worked through before you hit this point

In order,

"Pattern Recognition and Machine Learning" by Christopher Bishop,


*then* "Elements of Statistical Learning" (free download).

Neural Networks:

In order,

Neural Network Design Hagan Demuth and Beale,

Neural Networks, A Comprehensive Foundation (2nd edition) - By Haykin (there is a newer edition out but I don't know anything about that, this is the one I used)

and Neural Networks for Pattern Recognition ( Bishop).

At this point you are in good shape to read any papers in NN. My reccomendations - anything by Yann LeCun and Geoffrey Hinton. Both do amazing research.

Reinforcement Learning (again this is just stuff *I* happened to specialize in for various projects, so feel free to ignore)

Reinforcement Learning - An Introduction by Barto and Sutton (follow up with "Recent Advances In reinforcement Learning" (PDF) which is an old paper but a GREAT introduction to *Hierarchical* Reinforcement learning


Neuro Dynamic Programming by Bertsekas


Computer Vision

Introductory Techniques for 3-D Computer Vision, by Emanuele Trucco and Alessandro Verri.

An Invitation to 3-D Vision by Y. Ma, S. Soatto, J. Kosecka, S.S. Sastry. (warning TOUGH!!)

Robotics.

I know only about the software/algorithms side of Robotics and that too only Probabilistic Robotics. I don't know anything about hardware, electronics or Physics.

Probabilistic Graphical Models: Principles and Techniques (Adaptive Computation and Machine Learning) (strictly speaking not a robotics book, but a lot of the theory in this book is behind the algorithms in the next book

Probabilistic Robotics (Intelligent Robotics and Autonomous Agents) by Thrun, Burgard and Fox (trivia Thrun also wrote the Robotics chapter in AIMA - did I tell you AIMA rocks as a first introduction to AI?)

And that's all folks. Happy hacking!


[*] working though Cormen et al is a humungous task and can easily consume a year or more of work. Something like Sally Goldman's new Algorithm book maybe more suited to programmers.


PS: I have been getting a lot of email asking *how* one should learn X or Y. I have no idea really. The above is a list of books that worked for me and is provided only in the spirit of "these are good books that worked for me I don't know if they'll work for you."

As to how I learned, I just read books and papers, try to understand, (a lot of banging head against wall at this point) and try to solve problems and code stuff. Beyond that I have no advice on how to learn effectively etc. I am entirely self taught and have no idea how to teach this stuff. You probably need to talk to a good prof.