An attempt at setting the information balance right

The problem is not that there is too much information, but that there is too little of the right kind

In his brilliant book, A Short History of Nearly Everything, Bill Bryson shows through many examples how history often credits the wrong person. They show how being in the right place at the right time or publishing your ideas in the right publication so that the right people notice them is some times more important than having the idea.

For instance, today the great astronomer Edward Hubble is credited to have discovered that we live in an ever-expanding universe. However, it was an astronomer with the cheerily intergalactic name Vesto Slipher who should have got the credit.

PSM_V31_D740_Carl_Wilhelm_Scheele
Carl Scheele (Credit: Wikimedia)

Or take the example of Carl Scheele, a Swedish chemist, who discovered eight new elements—oxygen, nitrogen, chlorine, fluorine, manganese, barium, molybdenum and tungsten—and received credit for none of them. His work was either overlooked or made it late to publication after someone else had made the same discovery independently. The credit instead went to chemists of the English-speaking world.

And, if you’re still not convinced, try Josiah Willard Gibbs, who Bryson calls the “most brilliant person that most people have never heard of”. Between 1875 and 1878 he produced a series of papers on the thermodynamic principles of nearly everything but published them in the Transactions of the Connecticut Academy of Arts and Science, a journal that “managed to be obscure even in Connecticut”. Although Gibbs was recognised later in life, most of the work he did remained hidden for too long at great cost to the scientific enterprise.

Information games

The reason I am telling you all this is not just because it is interesting, but also because there is a lesson we can learn from these examples. All three scientists who got scooped lived in the age when information was scarce and the fastest it travelled was at the speed of a moving vehicle. 

We live in the age of information excess and the fastest it travels is the fastest it will ever travel (ie at the speed of light). However, we are still stuck with one problem that those gentlemen of the 19th century faced and perhaps it has become worse—who receives what information matters even more today. With the internet throwing up interesting things on our screens every day, are we getting the information we really need?

I’ll explain the problem with two examples. The first comes from how we learn history. This came to my attention during my first months in Oxford. Ask any non-Indian what they thought about British colonisation of India and you got a view that was quite different to what I was taught in school in India.

Partisan views

Most people acknowledged that there was imperial excess. They bemoaned the human cost of the partition between India and Pakistan, for instance. But they, particularly British friends, also praised how the British gave Indians railways, law and order, the English language, and, some even suggested, the Indian identity. Most importantly, they saw Indian independence as British leaving India. This was a time, one said, that Britain was relinquishing control of other colonies too following the troubles that World War II had caused for Britain domestically.

Sardar Patel. Credit: Govt of India
Sardar Patel. Credit: Govt of India

This wasn’t what I was taught in school. Indian textbooks glorify the independence struggle, it seemed. I was surrounded by some of the smartest people I had ever met, and instead of questioning them I had to go back and read history. What I realised soon though was that neither my British friends nor I had quite the balanced view that one ought to have of this important period in world history.

I wasn’t expecting my British friends to know about Mangal Pandey or Bhagat Singh, but I thought they would be aware of the Quit India Movement and Jallianwallah Bagh massacre. That they would recognise the importance of the role of freedom fighters, such as Sardar Patel, in accelerating India’s independence. Equally, I suspect my British friends thought I should have more appreciation for the things the British left in India.

The point here is that where the information came from changed how people viewed the world. In this case the topic was a well-studied period of history, and so I was able to educate myself enough to get a balanced view. But what about more recent events?

History’s value

In The Sceptical Patriot, Sidin Vadukut analysed history textbooks used by Indian kids today. He found that none of them devote any space to post-Independence period. Beyond a little bit on the Indian constitution, there is nothing about the wars with Pakistan and China or about the Naxal movement, which is considered the greatest threat to India’s internal security.

Why should Indian kids learn about this stuff? Vadukut’s story might give you an answer:

Sometime in 2002 or 2003, a group of Japanse Hibakusha, or atomic bomb survivors, visited Chennai. The city was a stop on what I think was a global tour to promote peace and condemn nuclear weapons. They decided to visit a primary school and tell the students why the idea of nuclear weapons was a bad one.

I read about the school visit in one of the local newspapers. I don’t recall which one, and no amount of searching online has thrown up the original news report. But the broad details of what happened are seared into my mind.

After the presentation, the Hibakusha asked the children:  should countries go to war? No, they all said in chorus. Should countries use nuclear weapons? No. Should India use nuclear weapons? Never. What if the enemy is Pakistan? Oh, Pakistan is a special case, the kids said, we should totally nuke them.

Every time I retell this story at a public forum, there is an explosion of laughter followed by an awkward silence.

Absence of proper facts

This brings me to the second example, which is closer to me—science journalism. The media landscape of the west is changing. The old media houses are losing audience and revenue at such a pace that it feels like a crisis. The first to lose their jobs in such cases tend to be specialist journalists such as those covering science, environment and health. Sometimes the sections live on, but they shrink in size and depth. The reporting gets done by general reporters instead.

There have been some positive changes with new media organisations trying to fill the gap. And despite all that, even with shrinking newsrooms, western media’s science coverage remains so much better than Indian media’s. A survey I did a few years ago of Indian newspapers gives the same results today—proper science reporting doesn’t exist.

This is a problem. Most of the time if you come across a science story in Indian newspapers, it happens to be one from an international wire service, such as AP, Reuters or The Guardian. Despite the international nature of science, Indian readers are fed the writing of western journalists. The stories become less relevant and thus less interesting. While the hard-science stories are still somewhat valuable, those about the environment, health or even technology aren’t.

This matters because policy depends on the quality of information that decision-makers get. One more example from Vadukut’s book makes that case absolutely clear. In March 2008, Daggubati Purandeswari, the then minister of state for human resource development, when talking about India’s education system, parroted “facts” about Indians abroad, which had been forwarded in a hoax chain email.

“Sir, as rightly pointed out by the honourable member, our students have been placed very well globally. For example 12% scientists in the United States are Indians. We have 38% if the doctors in the US who are again Indians. 36% of NASA scientists are again Indians. So, the students are doing very well, and they are reaching places which again reflects on the quality of education that is being provided to our children in the country.”

All of these “facts” can be easily verified as being false, but the honourable MP did not think she needed to do that. Her words were then reported in Times of India the next day as “facts”. And that is how facts get made up.

Science suffers

That aside, had there been good science journalists writing about the achievements of scientists in India, perhaps Daggubati would not have relied on information from dodgy sources. India’s premier institutes, such as the Indian Institutes of Technology, never figure in the world university rankings. From the information available to the minister today, she won’t be able to figure out whether the absence of IITs in such rankings says something about the poor quality of research or the lack of communication of that research.

There is also another side to this story. In my time at Oxford, whenever there was a paper published in an Indian or Chinese journal, I was explicitly advised not to give it too much value. The suggestion was that research in these journals was not reliable, which in other words means that the researchers were making up data.

This kind of opinion may have been formed by experience. But mostly this opinion was the result of western media reporting negative science stories emerging from China and India, which make academics wary of trusting such research. This, I believe, must lead to missing out on genuinely good research being produced in these countries.

The solution, of course, is one that will require change from the big editorial houses. In my conversations with Indian journalists, I’ve been told many times that there is thirst for good science content but no publication is ready to provide it.

A different solution that a friend and I are pursuing is to collate good science stories from around the web that have their focus on the other side of the world. While this doesn’t solve the need for more science reporting, at least it provides a central place to find good content related to India. Some of it exists, but sadly it tends not to be at one place but spread across different publications. (You can sign up for our newsletter here.)

Those gentlemen in the 19th century suffered personal loss because key information didn’t reach the right audience. Today’s tragedy is the same but the reason is different—key information is not reaching the right audience because either there is too much information and very poor filters or there aren’t enough people to collect and present the information that is so desperately needed.

Lead image: teresaling. Thanks to Deeksha Sharma and Vasudevan Mukunth for reading drafts of this post.

Soliciting negative feedback is hard, but you must do it often if you care about progressing

Last week, I tried an experiment in self-promotion. I made a birthday wish that I shared among my Facebook friends, wishing that they would read more of my writing. The experiment went well. That post became one of the most read pieces since I moved my blog to this website last year. I got lots of friends to subscribe, and many told me how they had already been enjoying my work.

But I wanted feedback on whether this experiment was really worth it. After all, I didn’t want it to sound like a sales pitch. This was a genuine request, and I wanted to know if it came across that way.

I’m lucky to have a group of really smart people whom I can ask for critical feedback. The group approved of my experiment, and the prompt led to a valuable discussion on feedback, which started, as many things do, with Elon Musk.

Feedback loop

Most of the time we walk around thinking that we are doing the right thing. That is important, of course, because if we were not confident in our abilities then we would not be able to function. But from time to time we must solicit feedback to help us spot faults and find better ways of doing things.

This might seem like common sense, but Elon Musk, one of the most successful entrepreneurs alive, says that most people don’t seek feedback that matters. He says we must not just seek feedback, but we must specifically seek negative feedback. (As a side, the operative word here should be critical, which means negative and analytically founded.)

When asking for feedback if you don’t ask for negative feedback, chances are you will never get it because people usually withhold such feedback for fear of hurting our feelings. This human folly to be soft on others leads to ineffectiveness. Even the times when negative feedback has to be given, it is usually sugar-coated, which often does not lead to the action that is needed.

Forget niceties

Truth is a hard apple to throw and a hard apple to bite.” These are slightly modified words of the American author Donald Barthelme. One way of allowing such hard apples to reach you, at least on an individual level, is to set up a system for soliciting feedback anonymously. With such an option, those giving feedback can forget niceties and really get to the point. It is also easy to do. For instance, here is a simple Google form where you can leave anonymous feedback for me.

However, before you jump to setting up your own form, you have to remember that negative feedback can (and will) hurt. You need to be sure that you are ready to hear nasty stuff. Smarter people than I have thought about this and they’ve developed rules that might help.

If you are thinking of soliciting anonymous feedback, try to abide by Crocker’s Rules (in full):

Declaring yourself to be operating by “Crocker’s Rules” means that other people are allowed to optimise their messages for information, not for being nice to you.

It means that you have accepted full responsibility for the operation of your own mind—if you’re offended, it’s your fault. Anyone is allowed to call you a moron and claim to be doing you a favour.

While Crocker’s Rules are simple, they are not easy to follow. In launching my own anonymous form, I’m taking a risk. But I do believe that the payoff will be worth it.

Mass change

On an organisation level, most places already have regular appraisals in place. However, these tend to be too formal for their own good. This can hurt an organisation, especially one that is growing rapidly or one where roles change quite often.

For this to work, on such a level, there will need to be behavioural change, which is hard. People will need to be encouraged to give feedback and a system will need to be in place to help them manage this feedback. Organisations can’t force people to follow Crocker’s Rules. But the human resource department can do something to help, if they want such a culture to flourish.

An experiment that has worked at some leading tech firms is that of radical transparency. Except for 100% personal emails, every email is shared with everyone else in the organisation. So someone new to a project can go read all the emails, all the way back if they want, and problems are uncovered more quickly. It’s hard to pretend everything’s going well with the customer when the email thread shows it’s not. (Of course email volume will be high, but email filters and selective reading can go a long way.)

One way or another, you must do your best to solicit negative feedback and do it often. If you care about progressing quickly, that is.

***

Thanks to Alex Flint, Christo Fogelberg and Xiao Cai for ideas and feedback. Image: gforsythe

***


Crocker’s Rules in full

Declaring yourself to be operating by “Crocker’s Rules” means that other people are allowed to optimise their messages for information, not for being nice to you.

Crocker’s Rules means that you have accepted full responsibility for the operation of your own mind—if you’re offended, it’s your fault. Anyone is allowed to call you a moron and claim to be doing you a favour. (Which, in point of fact, they would be. One of the big problems with this culture is that everyone’s afraid to tell you you’re wrong, or they think they have to dance around it.)

Two people using Crocker’s Rules should be able to communicate all relevant information in the minimum amount of time, without paraphrasing or social formatting. Obviously, don’t declare yourself to be operating by Crocker’s Rules unless you have that kind of mental discipline.

These rules don’t mean you can insult people; it means that other people don’t have to worry about whether they are insulting you. Crocker’s Rules are a discipline, not a privilege. Taking advantage of Crocker’s Rules does not imply reciprocity. How could it? Crocker’s Rules are something you do for yourself, to maximise information received—not something you grit your teeth over and do as a favour. The rules are named after Lee Daniel Crocker.

Bill Gates on the world and things he cares about

Jeff Goodall of Rolling Stone magazine does a great job of interviewing one of the greatest men of our times. Gates falls in the five greatest thinkers I follow. As Goodall summarises, he is one of the most optimistic persons alive who thinks “the world is a giant operating system that can be debugged”. That is why a long interview with him is much welcome. Here I have snippets and some crystallised gems from it:

On comparison with Zuckerberg:

I start with architecture, and Mark Zuckerberg starts with products, and Steve Jobs started with aesthetics.

On Edward Snowden:

I think he broke the law, so I certainly wouldn’t characterize him as a hero.

On taxation:

In general, on taxation-type things, you’d think of me as a Democrat. That is, when tax rates are below, say, 50 percent, I believe there often is room for additional taxation.

On helping the poor:

Should the state be playing a greater role in helping people at the lowest end of the income scale? Poverty today looks very different than poverty in the past. The system’s ability to distinguish between somebody who has family that could take care of them versus someone who’s really out on their own is not very good, either. It’s a totally gameable system – not everybody games it, but lots of people do. Why aren’t the technocrats taking the poverty programs, looking at them as a whole, and then redesigning them?

On government’s ability to tackle big problems:

You have to have a certain realism that government is a pretty­ blunt instrument and without the constant attention of highly qualified people with the right metrics, it will fall into not doing things very well. If I could wave a wand and fix one thing, it’d be political deadlock, the education system or health care costs.

On money in politics:

I’m not sure you’d want money to be completely out of politics. We’ve got a system with a lot of checks and balances. When you get into a period of crisis where the overwhelming majority agrees on something, government can work amazingly well, like during World War II.

On innovation:

Innovation can actually be your enemy in health care if you are not careful. If you accelerate certain things but aren’t careful about whether you want to make those innovations available to everyone, then you’re intensifying the cost in such a way that you’ll overwhelm all the resources.

On prioritisation:

I want to focus on things where I think my experience working with innovation gives me an opportunity to do something unique. The majority of the foundation’s money goes to a finite number of things that focus on health inequity – why a person from a poor country is so much worse off than somebody from a country that’s well-off. It’s mostly infectious diseases.

On “no poor countries by 2050”:

Assuming there’s no war or anything, we ought to be able to take even the coastal African countries and get them up to a reasonable situation over the next 20 years. You get more leverage because the number of countries that need aid is going down, and countries like China and India will still have problems, but they’re self-sufficient. And over the next 20 years, you get better tools, new vaccines, a better understanding of diseases and, hopefully, cheaper ways of making energy.

On investment:

(Not a quote) Philanthropy has a better hit rate than venture capital. And yet, because of successes like Google, people give it more importance.

On climate change:

It’s a big challenge, but I’m not sure I would put it above everything. I think it’s a real test of the boundary of science and politics – and an acid test of people’s time horizons. I happen to think we should explore geo-engineering.­

On alternative power:

Intermittent energy sources like wind and solar . . . you can crank those up, depending on the quality of the grid and the nature of your demand. You can scale that up 20 percent, 30 percent and, in some cases, even 40 percent. But when it comes to climate change, that’s not interesting. You’re talking about needing factors of, like, 90 percent.

With so many problems, people feel pessimistic:

Really? That’s too bad. I think that’s overly focusing on the negatives. I think it’s a pretty bright picture, myself. But that doesn’t mean I think, because we’ve always gotten through problems in the past, “just chill out, relax, someone else will worry about it.” I don’t see it that way.

What is your biggest fear?

I understand how every healthy child, every new road, puts a country on a better path, but instability and war will arise from time to time, and I’m not an expert on how you get out of those things. I wish there was an invention or advance to fix that. There’ll be some really bad things that’ll happen in the next 50 or 100 years, but hopefully none of them on the scale of, say, a million people that you didn’t expect to die from a pandemic, or nuclear or bioterrorism.

On immigration:

US immigration laws are bad – really, really bad. I’d say treatment of immigrants is one of the greatest injustices done in our government’s name.

Do you believe in God?

I agree with people like Richard Dawkins that mankind felt the need for creation myths. Before we really began to understand disease and the weather and things like that, we sought false explanations for them. Now science has filled in some of the realm – not all – that religion used to fill. I think it makes sense to believe in God, but exactly what decision in your life you make differently because of it, I don’t know.