Data brain… How do you draw the best insights from your data? Photo: iStock/lawrence.dutrieux
It was the bold title of a conference this month at the Massachusetts Institute of Technology, and of a widely read article in The Harvard Business Review last October: “Big Data: The Management Revolution.”
Andrew McAfee, principal research scientist at the MIT Center for Digital Business, led off the conference by saying that Big Data would be “the next big chapter of our business history.” Next on stage was Erik Brynjolfsson, a professor and director of the MIT center and a co-author of the article with McAfee. Big Data, Brynjolfsson said, will “replace ideas, paradigms, organisations and ways of thinking about the world.”
These drumroll claims rest on the premise that data like web-browsing trails, sensor signals, GPS tracking, and social network messages will open the door to measuring and monitoring people and machines as never before. And by setting clever computer algorithms loose on the data troves, you can predict behavior of all kinds: shopping, dating and voting, for example.
The results, according to technologists and business executives, will be a smarter world, with more efficient companies, better-served consumers and superior decisions guided by data and analysis.
I’ve written about what is now being called Big Data a fair bit over the years, and I think it’s a powerful tool and an unstoppable trend. But at year-end, I thought, might be a time for reflection, questions and qualms about this technology.
Quest for insights
The quest to draw useful insights from business measurements is nothing new. Big Data is a descendant of Frederick Winslow Taylor’s “scientific management” of more than a century ago. Taylor’s instrument of measurement was the stopwatch, timing and monitoring a worker’s every movement. Taylor and his acolytes used these time-and-motion studies to redesign work for maximum efficiency. The excesses of this approach would become satirical grist for Charlie Chaplin’s Modern Times. The enthusiasm for quantitative methods has waxed and waned ever since.
Big Data proponents point to the internet for examples of triumphant data businesses, notably Google. But many of the Big Data techniques of math modeling, predictive algorithms and artificial intelligence software were first widely applied on Wall Street.
At the MIT conference, a panel was asked to cite examples of big failures in Big Data. No one could really think of any. Soon after, though, Roberto Rigobon could barely contain himself as he took to the stage. Rigobon, a professor at MIT’s Sloan School of Management, said the financial crisis certainly humbled the data hounds. “Hedge funds failed all over the world,” he said.
The problem with math
The problem is that a math model, like a metaphor, is a simplification. This type of modeling came out of the sciences, where the behavior of particles in a fluid, for example, is predictable according to the laws of physics.
In so many Big Data applications, a math model attaches a crisp number to human behavior, interests and preferences. The peril of that approach, as in finance, was the subject of a recent book by Emanuel Derman, a former quant at Goldman Sachs and now a professor at Columbia University. Its title is “Models. Behaving. Badly.”
Claudia Perlich, chief scientist at Media6Degrees, an online ad-targeting start-up in New York, puts the problem this way: “You can fool yourself with data like you can’t with anything else. I fear a Big Data bubble.”
The bubble that concerns Perlich is not so much a surge of investment, with new companies forming and then failing in large numbers. That’s capitalism, she says. She is worried about a rush of people calling themselves “data scientists,” doing poor work and giving the field a bad name.
Indeed, Big Data does seem to be facing a workforce bottleneck.
“We can’t grow the skills fast enough,” says Perlich, who formerly worked for IBM Watson Labs and is an adjunct professor at the Stern School of Business at New York University.
A report last year by the McKinsey Global Institute, the research arm of the consulting firm, projected that the United States needed 140,000 to 190,000 more workers with “deep analytical” expertise and 1.5 million more data-literate managers, whether retrained or hired.
Step one: defining the problem
Thomas H. Davenport, a visiting professor at the Harvard Business School, is writing a book called Keeping Up With the Quants to help managers cope with the Big Data challenge. A major part of managing Big Data projects, he says, is asking the right questions: How do you define the problem? What data do you need? Where does it come from? What are the assumptions behind the model that the data is fed into? How is the model different from reality?
Society might be well served if the model makers pondered the ethical dimensions of their work as well as studying the math, according to Rachel Schutt, a senior statistician at Google Research.
“Models do not just predict, but they can make things happen,” says Schutt, who taught a data science course this year at Columbia. “That’s not discussed generally in our field.”
Models can create what data scientists call a behavioural loop. A person feeds in data, which is collected by an algorithm that then presents the user with choices, thus steering behavior.
Consider Facebook. You put personal data on your Facebook page, and Facebook’s software tracks your clicks and your searches on the site. Then, algorithms sift through that data to present you with “friend” suggestions.
Understandably, the increasing use of software that microscopically tracks and monitors online behavior has raised privacy worries. Will Big Data usher in a digital surveillance state, mainly serving corporate interests?
Personally, my bigger concern is that the algorithms that are shaping my digital world are too simple-minded, rather than too smart. That was a theme of a book by Eli Pariser, titled The Filter Bubble: What the Internet Is Hiding From You.
It’s encouraging that thoughtful data scientists like Perlich and Schutt recognise the limits and shortcomings of the Big Data technology that they are building. Listening to the data is important, they say, but so is experience and intuition. After all, what is intuition at its best but large amounts of data of all kinds filtered through a human brain rather than a math model?
At the MIT conference, Schutt was asked what makes a good data scientist. Obviously, she replied, the requirements include computer science and math skills, but you also want someone who has a deep, wide-ranging curiosity, is innovative and is guided by experience as well as data.
“I don’t worship the machine,” she said.
I recently spoke at a UKTI (UK Trade & Investment) conference during a recent whirlwind visit to SE Asia by UK’s Prime Minister David Cameron. The PM and his 86 strong entourage graced 4 Asian countries in 5 days, pushing for stronger bilateral trade ties, and reinvigorating the relationship between the UK and its Asian partners.
Not only is such an initiative much needed in a time of economic difficulty, but more so a necessity in times where trade partnerships with the East become increasingly more influential, and inward investment into the UK most welcomed (to say the least). I must say however, that despite the difficulties at home in the UK, the UKTI had embarked on a very bold and consistent campaign themed the ‘GREAT Campaign’, basically blowing the trumpet of how GREAT BRITAIN, was once a dominant economic force. The campaign ran across an entire gamut of integrated Print, Digital, and Outdoor, which illustrate the many facets of commercial attractiveness of conducting business with the UK.
(The writer Bob Chua speaking about trade opportunities between ASEAN and the UK)
The challenge and opportunity which this particular campaign promoted, was the fact that it is usually extremely difficult to develop a key message that promotes a country based on its numerous traits, be it heritage, commerce, tourism, or trade. The GREAT Campaign not only nailed this in my opinion, but also stuck to a common theme that will hopefully put the GREAT, back into GREAT BRITAIN.
KUALA LUMPUR, Mar 15 – Malaysian companies learnt why the UK is still an attractive businesses destination, especially when looking at going global at a breakfast forum organised by the UK Trade & Investment (UKTI) and Pulse Group PLC at the Ritz-Carlton, Kuala Lumpur recently.
Pulse is a leading digital research agency founded by enterprising Malaysian entrepreneur Bob Chua, who decided to set up a global office in the UK in 2008. In 33 months since setting up overseas, Pulse floated on London’s PLUS market, a stock exchange providing cash trading and listing, derivatives and technology services.
Chua shared the secret of his success with the attendees, many of whom were looking at opportunities to invest in various sectors in the UK, including infrastructure, education, oil and gas, creative services, information technology and property.
“The UK is one of the largest market research buying regions, so we have had business there from day one. The UK is known to have good corporate governance and for operating a solid financial system, so having a base there lent us an element of credibility,” said Chua, a recipient of the Ernst & Young Emerging Entrepreneur of the year Award in 2008.
Chua added that the initial 90 per cent of Pulse’s business came from international companies, and thanked UKTI for their support and introduction to UKTI’s network of contacts.
With professional advisers across 96 international markets, UKTI helps UK-based companies succeed in the global economy and assists overseas companies to bring high quality investments in to the UK.
Tony Collingridge OBE, Director of Trade & Investment at the British High Commission said, “The UK has maintained its position as the most popular location for inward investment in Europe. The UK economy is open, diverse, well-regulated and competitive. It is quick and easy to start a business and take advantage of the high levels of skills, innovation and value that has helped so many overseas companies grow their global business through the UK.”
According to Ernst & Young’s 2011 European Attractiveness Survey, the UK has maintained its leadership in FDI projects and FDI jobs in Europe with investors leveraging on its strength in services and increasingly its industry, investing in business services (14% of the projects received), machinery and equipment (11%), computers (7%) and software (7%).
About Bob Chua
Bob Chua is Founder and CEO of Pulse Group PLC, one of the industry’s fastest growing Digital Research Agency’s, having grown from start-up to IPO (on London’s PLUS market) in a mere 33 months. Prior to Pulse, Bob lived and worked in London, Sydney, Hong Kong, and spent a significant amount of time in the US. His global outlook and management style has seen him achieve significant roles within some of the largest Fortune 500 organisations, as well as consulting some of the leading companies throughout the world.
Bob is a successful Malaysian Entrepreneur with a vast global network, and experience in starting-up, fund-raising, nurturing hyper-growth, M&A’s, and bringing companies public. He is also a winner of the prestigious ‘Ernst & Young Emerging Entrepreneur of the Year Award 2008’.
Bob also advises the Malaysian Government on policies and matters relating to technology, economics, and innovation by appointment of the Malaysian Ministry of Science, Technology and Innovation (MOSTI). Bob is a Graduate from Griffith Business School, Australia.
For media enquiries, please contact Sheikh Zain, Head of Marketing, Pulse Group at 603-2167 6666 or email firstname.lastname@example.org
At the British High Commission, please contact Vivienne Pal, Media Officer, at 03-2170 2200 ext 263 or email Vivienne.Pal@fco.gov.uk