Tag: big data

Curious Bends — commoner panthers, space diplomacy, big data sells big cars and more

Akshat RathiLeave a comment

1. Why the GM debate in India won’t abate

It is a sign of its inadequacy that the debate on genetically modified crops in India is still on, with no end in sight. Although public consensus is largely polarised, the government has done its bit to postpone resolution. For one, decisions on GM crops are made as if they were “technical answers to technical questions”. For another, no formal arena of debate exists that also addresses social anxieties. (8 min read)

2. Black panthers are commoner in India than thought

Camera traps installed by the Wildlife Conservation Society of India have shown that about one in ten of all leopard images belong to black leopards (that is, black panthers). These melanistic big cats have been spotted in wildlife reserves in Kerala and Karnataka, and seem commoner in the wetter forests of the Western Ghats. In fact, written records of sightings in these parts date from 1879, and could aid conservation efforts in a country that lost its cheetahs in 1960. (2 min read)

3. One foot on Earth and another in the heavens

For smaller and middle income nations, strengthening institutional and technical capacity on the ground might be a better option than to launch satellites because more than vanity, the choice makes them better positioned to gather useful data. And if such a nation is in South Asia, then India’s planned SAARC satellite could make that choice easier, providing a finer balance between “orbital dreams and ground realities”. (5 min read)

+ The author, Nalaka Gunawardene, is a journalist and science writer from Colombo, Sri Lanka.

4. Do big carmakers know their way around big data?

When sales slumped, Mahindra & Mahindra, an Indian car-maker, used data gleaned from the social media to strip its former best-selling XUV500 model of some features and sell it cheaper. The company declined to give further details. This isn’t unique—big car-makers around the world are turning to big data to widen margins. But do they know how best to use the data or is it just that putting the squeeze on this lemon is a fad? (6 min read)

5. A geothermal bounty in the Himalayas

As the developing world edges toward an energy sufficiency crisis, scientists, environmental conservationists and governments get closer to a Mexican standoff. This is no better highlighted than with the gigawatts of geothermal energy locked up in the Himalayas. A 20-MW plant could “save three million litres of diesel”, $2 million and 28,000 tons of carbon dioxide in northern India per year. Why isn’t it being used? (2 min read)

Chart of the week

20140726_irc374-e1406691005265

“Both [female genital mutilation and child marriage] stem from deeply rooted social norms which can only be changed by educating parents about the harm they cause. Making foreign aid conditional on results gives governments an extra incentive not just to pass laws, but to enforce them. Police and women’s activists in some countries have set up phone hotlines and safe houses for victims or girls at risk. Most important is to make sure that girls go to school and finish their studies.” The Economist has more.

If you learnt something new from Curious Bends, why not spread the word? Forward this email to your friends and ask them to subscribe. Have a good day!

When free will, causality and privacy are all at stake

Akshat RathiLeave a comment

Review of Viktor Mayer-Schonberger and Kenneth Cukier’s Big Data

We live in a world where flu outbreaks are predicted faster and more accurately by analysing Google search results rather than by doctors or clinicians, where traffic jams are better judged by crunching data from cellphone signals rather than from direct reports from people on the ground, and where your shopping habits might reveal that you’re pregnant before anyone else in your family knows.

This is the power of big data. It is defined not by the sheer volume of information, but by what that large volume enables us to do that similar smaller volume of data wouldn’t. For example, Google’s flu trends will hardly work if the amount of queries made per second were not in the thousands. Another aspect of big data is that using it means shedding our obsession for causality and embracing correlations. The important thing is to know what rather than the why.

Take the example of how Google’s page ranking system works. The computer algorithm at the heart of Google search that is analysing data from all across the web is not trying to understand what the websites say or mean, so much as it is trying to correlate what people want when they type something in the search query. More queries followed by more clicks on relevant sites will better the algorithm that ranks pages, helping it to make better predictions which links will work best. Now add to it information like a person’s search history, location, time of the day, etc. and Google is able to give near perfect search results.

The book also rightly argues that despite data’s ubiquitous use today, the revolution has only just begun. A lot of the information in the world still remains locked or wasted. Consider the example of electrocardiography (ECG). When a patient undergoes ECG, hundreds of data points are collected every second but most of it gets thrown away. Instead the capability of easy storage (thus never needing to throw away any data) can be used to make better predictions of the patient’s health in the future. Datafication, which is recording everything possible, can unlock information around us. Things which might seem uninteresting could, in combination with other data, reveal insights that we could not have guessed before

Big data’s use does not paint a uniformly rosy picture. Governments are desperately trying to control more and more data from their citizens’ lives under the guise of security concerns. But, just like private companies do, the data can easily be employed by governments for uses that citizens would not approve of, if they were asked to give consent. The story of Minority Report could come true. In it the government develops a system that is used to predict the future occurrence of crime and make arrests in time to stop it. These sort of uses are still science fiction, but not for long. They risk taking away from humanity its most dear capacity—to act on “free will”.

It is this dual-edged sword of big data that makes Messrs Cukier and Mayer-Schonberger’s book timely and important. Written beautifully and convincingly, it makes for a great read. Where I don’t agree with the book is that big data “will transforming how we live, work and think”. I think it already has.

Geek philanthropy

Akshat RathiLeave a comment

An innovative charity rallies geeks for a good cause

Businesses avidly mine data to improve their efficiency. Non-profit groups have plenty of information, too. But they can rarely afford to hire number-crunchers. Now a bunch of philanthropic geeks at DataKind, a New York-based charity, are helping other do-gooders work more productively and quantify their achievements for donors, who like to see that their money is well spent.

A typical DataKind two-day “hackathon” last month in London attracted 50 people who worked in three teams. One pored over the records of Place2Be, which offers counselling to troubled schoolchildren. Crunching the data showed that boys tend to respond better than girls, though girls who lived with only their fathers showed the biggest improvements of all. The charity did not know that.

The expertise is far beyond what is available to a typical charity. The small-talk among the volunteers was of dizzyingly complex statistical and artificial-intelligence techniques. Volunteers included an analyst at Teradata, a data-analytics firm. Around 20 employees attended from Aimia, a firm that mines data from consumer-loyalty programs.

In a previous hackathon in San Francisco, DataKind volunteers analysed the data from Mobilising Health, a non-profit group that connects rural patients in India with doctors in cities that are usually many hours away. Volunteers record symptoms and relay them by cellphones. The doctors then may prescribe drugs or recommend a hospital visit. The charity wanted to use the many months’ worth of accumulated text messages to evaluate the medics’ performance. Thanks to DataKind the charity was able to rejig the system to take more account of urgency and to direct requests to the most responsive doctors.

Thomas Levine, a data scientist at ScraperWiki, a provider of data-processing services, says he has attended DataKind events out of altruism but also for education. Would anyone care to measure that benefit?

First published in The Economist. Article written with Kenn Cukier. Also available in audio here.

Image from here.