Facebook's Other Big Disruption

Quentin Hardy, The New York Times, January 17, 2013

Facebook just made a potentially game-changing announcement. It got less fanfare than Tuesday’s announcement that it is going into the social search business, but this other announcement may have bigger long-term implications for the technology industry.

Put simply, some of the world’s biggest computing systems just got a little cheaper, and a lot easier to configure. As a consequence, the companies that supply the hardware to these systems may have to scramble to remain as profitable. The reason is a Facebook-led open source project.

In 2011 Facebook began the Open Compute Project, an effort among technology companies to use open-source computer hardware. Tech companies similarly shared intellectual property with Linux software, which lowered costs and spurred innovation. Facebook’s project has attracted many significant participants, including Goldman Sachs, Arista Networks, Rackspace, Hewlett-Packard and Dell.

At a user summit on Wednesday Intel, another key member of the Open Compute Project, announced it would release to the group a silicon-based optical system that enables the data and computing elements in a rack of computer servers to communicate at 100 gigabits a second. That is significantly faster than conventional wire-based methods, and uses about half the power.

More important, it means that elements of memory and processing that now must be fixed closely together can be separated within a rack, and used as needed for different kinds of tasks. There is a lot of waste in data centers today simply because, when there is an upgrade in servers, lots of other associated data-processing hardware has to be changed, too.

There were other announcements, like a computer motherboard called Grouphug that allows different manufacturers’ chips to be interchanged without altering other parts of the machine. Before, they were custom made. Put together, such innovations potentially lower the cost and complexity of running big and small data centers to an extent that works for a lot of companies.

“Who wouldn’t want a cheaper, more efficient server?” said Frank Frankovsky, vice president of hardware design at Facebook, and the chairman of Open Compute. “The problem we’re solving is much larger than Facebook’s own challenges. There is a massive amount of data in the world that people expect to have processed quickly.”

To be sure, it’s in Facebook’s interest to attack expensive hardware. The company makes money from a service that requires hundreds of thousands of computer servers distributed in big centers around the world. Google and Amazon.com, which are not members of the project, maintain proprietary systems which they apparently felt gave them a competitive edge.

For Facebook, the difference seems to be more in the software. To the extent hardware costs drop, that’s great for them. Mr. Frankovsky argued that, while “this puts challenges on the incumbents” in hardware, “it also helps them. They have a finite number of engineering resources, and this way they hear from a community about whether there is an interest for a product.” Intel may hope to benefit from its open-source release, since it could see an overall rise in demand for its chips with the move toward cheaper computing.

The real test is whether Facebook can increase the number of potential buyers for Open Compute equipment. “The question is, can they extend this beyond a few Web businesses like Facebook and Rackspace, or a few financial exercises at Goldman, and bring this to industries like oil or aerospace?” said Matt Eastwood, an analyst with IDC, a technology research firm. “That will take it from 20 or 30 companies to hundreds of companies.”

The issue isn’t so much a technical one, he argues, as it is one of getting corporate information technology professionals interested in radical design changes. Mr. Frankovsky is aware of the problem. Recently he and his colleagues led a seminar in Texas for BP, Shell and other oil giants on how they could use Open Compute hardware in their data centers.

This will not change things dramatically this year, and possibly even next, but over the long haul it could remake a lot of businesses. Linux, remember, was around for several years as a minor player, but eventually undid Sun Microsystems and others.

The Dunbar Number, From the Guru of Social Networks

Drake Bennett, Bloomberg Business Week, January 2013


A little more than 10 years ago, the evolutionary psychologist Robin Dunbar began a study of the Christmas-card-sending habits of the English. This was in the days before online social networks made friends and "likes" as countable as miles on an odometer, and Dunbar wanted a proxy for meaningful social connection. He was curious to see not only how many people a person knew, but also how many people he or she cared about. The best way to find those connections, he decided, was to follow holiday cards. After all, sending them is an investment: You either have to know the address or get it; you have to buy the card or have it made from exactly the right collage of adorable family photos; you have to write something, buy a stamp, and put the envelope in the mail. These are not huge costs, but most people won't incur them for just anybody.

Working with the anthropologist Russell Hill, Dunbar pieced together the average English household's network of yuletide cheer. The researchers were able to report, for example, that about a quarter of cards went to relatives, nearly two-thirds to friends, and 8 percent to colleagues. The primary finding of the study, however, was a single number: the total population of the households each set of cards went out to. That number was 153.5, or roughly 150.

This was exactly the number that Dunbar expected. Over the past two decades, he and other like-minded researchers have discovered groupings of 150 nearly everywhere they looked. Anthropologists studying the world's remaining hunter-gatherer societies have found that clans tend to have 150 members. Throughout Western military history, the size of the company—the smallest autonomous military unit—has hovered around 150. The self-governing communes of the Hutterites, an Anabaptist sect similar to the Amish and the Mennonites, always split when they grow larger than 150. So do the offices of W.L. Gore & Associates, the materials firm famous for innovative products such as Gore-Tex and for its radically nonhierarchical management structure. When a branch exceeds 150 employees, the company breaks it in two and builds a new office.

For Dunbar, there's a simple explanation for this: In the same way that human beings can't breathe underwater or run the 100-meter dash in 2.5 seconds or see microwaves with the naked eye, most cannot maintain many more than 150 meaningful relationships. Cognitively, we're just not built for it. As with any human trait, there are outliers in either direction—shut-ins on the one hand, Bill Clinton on the other. But in general, once a group grows larger than 150, its members begin to lose their sense of connection. We live on an increasingly urban, crowded planet, but we have Stone Age social capabilities. "The figure of 150 seems to represent the maximum number of individuals with whom we can have a genuinely social relationship, the kind of relationship that goes with knowing who they are and how they relate to us," Dunbar has written. "Putting it another way, it's the number of people you would not feel embarrassed about joining uninvited for a drink if you happened to bump into them in a bar."

While Dunbar has long been an influential scholar, today he is enjoying newfound popularity with a particular crowd: the Silicon Valley programmers who build online social networks. At Facebook (FB) and at startups such as Asana and Path, Dunbar's ideas are regularly invoked in the attempt to replicate and enhance the social dynamics of the face-to-face world. Software engineers and designers are basing their thinking on what has come to be called Dunbar's Number. Path, a mobile photo-sharing and messaging service founded in 2010, is built explicitly on the theory—it limits its users to 150 friends.

"What Dunbar's research represents is that no matter how the march of technology goes on, fundamentally we're all human, and being human has limits," says Dave Morin, one of Path's co-founders. To developers such as Morin, Dunbar's insistence that the human capacity for connection has boundaries is a challenge to the ethos of Facebook, where one can stockpile friends by the thousands. Dunbar's work has helped to crystallize a debate among social media architects over whether even the most cleverly designed technologies can expand the dimensions of a person's social world. As he puts it, "The question is, 'Does digital technology in general allow you to retain the old friends as well as the new ones and therefore increase the size of your social circle?' The answer seems to be a resounding no, at least for the moment."

At 65, Dunbar is thickening slightly, with a scholar's slouch, although he tends to take stairs two at a time. A professor at the University of Oxford, he lunches regularly in the senior common room of Magdalen College, where he's a fellow. The cozy space, with oil portraits of long-dead scholars in robes and wigs, looks out on a baize-like lawn. In November, over thin, gray lamb chops, he told a bit of his story. He grew up in Tanzania, where his father was an electrical engineer, and as a teenager he'd dive and sail off the coast and drive into the bush to shoot elephants. When he was at graduate school in the early 1970s, his original research interest was not human friendship but the social life of the gelada, a monkey found only in the Ethiopian highlands and closely related to the baboon.

Dunbar has a quick, ironic smile and speaks sleepily, in long, fluent dilations. What attracted him to the gelada, he says, were "the peculiarities of their social system, which is based around small family groups which come together into large herds. It's kind of vaguely similar to what you see in modern hunter-gatherers. It's called a fission-fusion social system, and it only occurs in two monkeys out of all the 300-odd primates—aside from humans."

It was the monkeys' grooming habits that really interested him. For geladas, as for many other primates, grooming is only partly about cleanliness. It's also a form of bonding. Gelada life is rife with intrigue—there are cabals and coups and uneasy alliances—and the monkeys cement friendships by picking through each other's fur for parasites and kneading the skin beneath. In an early paper, Dunbar showed that the amount of time geladas spend grooming is not a function of body size, which would suggest a solely hygienic purpose, as bigger bodies take longer to pick over. Instead, it's a function of group size. The bigger the troop, the more time its members spend trying to curry favor with each other through massage. Dunbar began to wonder what other characteristics might correlate with group size.

In 1992, Dunbar published his answer: brain size. Scientists have long been intrigued by the question of why primates have such big brains. It's nice to be smart, of course, but big brains demand an enormous amount of energy and require years to grow to full size, and the larger skulls that protect them make childbirth much more dangerous. Plenty of species have thrived on this planet without much of a brain at all.

Dunbar's argument, laid out in the Journal of Human Evolution, was that big brains evolved to solve the problem of social life. Living in large groups confers significant advantages, chief among them better protection against predators. But living together is also difficult. Members compete for food and access to mates. They have to guard against bullies and cheats—and pick their own spots to bully or cheat. "For very social species, and this applies particularly to primates, the group is an adaptation to solve particular ecological problems," Dunbar explains. "But the group itself triggers a whole series of problems at the individual level. It's essentially the social contract problem: People tread on your toes; they steal your food just as you've unearthed it."

As group size grows, a dizzying amount of data must be processed. A group of five has a total of 10 bilateral relationships between its members; a group of 20 has 190; a group of 50 has 1,225. Such a social life requires a big neocortex, the layers of neurons on the surface of the brain, where conscious thought takes place. In his 1992 paper, Dunbar plotted the size of the neocortex of each type of primate against the size of the group it lived in: The bigger the neocortex, the larger the group a primate could handle. At the same time, even the smartest primate—us—doesn't have the processing power to live in an infinitely large group. To come up with a predicted human group size, Dunbar plugged our neocortex ratio into his graph and got 147.8.

Dunbar was not the first to suggest that social dynamics explained the evolution of higher intelligence, but the simple arithmetic of his argument—bigger brains equal bigger groups—gave it resonance, and he's now seen as the father of what's known as the social brain hypothesis. "It's been very influential," says Simon Reader, an evolutionary biologist at McGill University. "It has been the dominant hypothesis."

The Dunbar Number has made its namesake an intellectual celebrity. Much of his recent writing has been for popular audiences. For a while he contributed regularly to the New Scientist magazine and the Scotsman newspaper. He has spoken at TED and written books for lay readers; the most recent of them, The Science of Love, was published in the U.S. in November. Although he's an engaging writer, his more recent books give the impression of having been written quickly. In The Science of Love there's an amusing page-long description of the erotic effects of the steroid androstadienone. That description also appears, almost word for word, in his previous book. Asked about this, he says, "You tend to slip into these sorts of standard formulations, I think. I don't think there's anything that's directly cut and pasted."

In person Dunbar retains a certain remove, not exactly aloof and not exactly shy. Sitting and speaking in his cinder-block-walled office at Oxford's department of experimental psychology, he twists metronomically in his swivel chair, leaning back and running his eyes over the spines of the books on his bookshelves. He gives the impression of someone not actively looking to increase his number of bilateral relationships. Asked whether as a scholar of social behavior he thinks of himself as a particularly social person, he says, "I guess I'm sort of about average. I'm certainly not hypersocial, that's for sure." Over the course of one afternoon, he is interrupted twice by phone calls. The first is a major book festival asking him to be a guest speaker. The second is BBC News asking him to come on that evening. He says no to both, the first one with a trace of annoyance—he'd already declined by e-mail, he explains later.

His professional network spans an array of disciplines. He's collaborating on projects with linguists, computer scientists, physicists, classicists, economists, archeologists, anthropologists, and literary scholars. All the projects are related to the social brain hypothesis. One study looks at laughter, its physiological effects, and the role it might play in cementing social bonds. Another considers, in a similar way, dancing. His collaborators universally praise him. "For me, Robin is the sort of person you can't help liking within about five minutes of meeting him," says Felix Reed-Tsochas, a theoretical physicist at Oxford who's collaborated with him. "He's full of really, really interesting ideas and insights, which just kind of gives you a buzz."

In the fall of 2010, Dunbar got a phone call from Morin, who had been the executive in charge of Facebook's app platform and co-invented Facebook's Connect feature. Earlier that year he'd left the company to help found Path. He had discovered Dunbar's work years earlier as a freshman economics major at the University of Colorado.

Morin, now 32, grew up in Helena, Mont., a town of 28,000 people, and he talks about small-town life in the key of John Mellencamp. "America was built on the backs of these small communities," he says, sitting in a conference room at Path's offices in a downtown San Francisco skyscraper with a view of the Bay. Although Morin has spent his adult life in cities, he's used online networks to create communities with the closeness of his hometown.

Path, he says, provides a way for anybody to be able to do that. The service allows people to post photos from their smartphones. Users can message each other and comment on and search through the material others have posted. One of its more intimate features allows someone to tell everyone in his network when he's going to sleep and when he's woken up. But that network cannot be larger than 150 people. Path, in essence, is for clans.

"People feel like they can put things on Path they can't put anywhere else," Morin says. "Fundamentally, once you go beyond this number of people you can keep in your head, you begin to filter yourself, you change what you share and how much, you put on your public face." The service recently passed 5 million users, and Morin says keeping its network size small has rewarded the company with a remarkably engaged user base.
Morin and Dunbar's first conversation lasted a couple of hours. Among other things, they talked about Dunbar's research on how long the average friendship can survive in the absence of face-to-face contact (6 to 12 months), and about how, according to Dunbar, a woman can have two best friends (including her romantic partner), but a man only one. Since then the two have spoken every few months. The search algorithm Path uses to find a user's closest friends is based on Dunbar's work. Morin says the service is launching several features this year that grow out of the psychologist's ideas, although he declines to describe them.

Morin likes to point out that it's misleading to talk about a single Dunbar Number. Dunbar actually describes a scale of numbers, delimiting ever-widening circles of connection. The innermost is a group of three to five, our very closest friends. Then there is a circle of 12 to 15, those whose death would be devastating to us. (This is also, Dunbar points out, the size of a jury.) Then comes 50, "the typical overnight camp size among traditional hunter-gatherers like the Australian Aboriginals or the San Bushmen of southern Africa," Dunbar writes in his book How Many Friends Does One Person Need? Beyond 150 there are further rings: Fifteen hundred, for example, is the average tribe size in hunter-gatherer societies, the number of people who speak the same language or dialect. These numbers, which Dunbar has teased out of surveys and ethnographies, grow by a factor of roughly three. Why, he isn't sure.

The venture capitalist Jerry Murdock is one of Path's investors; his firm, Insight Venture Partners, also invested in Twitter and Tumblr. Murdock, who has a numerological streak, sees Dunbar's Number as a sort of social Fibonacci sequence, a simple mathematical relationship revealing a deeper truth about the workings of the universe. He believes the two sets of numbers may be related. "What Dunbar's theory does, like all good theories, is it explains constraints, constraints in nature," he says. "And it's the constraints that make great architecture. It's the constraints that make great companies."

Just as simplicity has popularized Dunbar's ideas, it has opened him up to the charge of reductionism. "We want to apply this single monolithic idea that reduces all the complexity of the world to just one dimension and just one number," says Duncan Watts, a network theorist and research scientist at Microsoft (MSFT). As he sees it, Dunbar's model of friendship, as a series of circles of intimacy, is a massive oversimplification: In real life, people don't have better friends and worse friends, they have different sorts of friends they go tor different things. "If you're saying there's only 150 people who matter, my response is, 'Matter to what?' " he says. "Depending on what you're trying to do, the people who matter may be your co-workers, they may be your old high school friends, they may be your current social circle, they may be your family. The challenge for social networking sites is to solve that problem."

Others, anthropologists and brain scientists in particular, challenge the evolutionary story Dunbar tells, arguing that it discounts other factors that might have driven the development of the big human brain—the pressure to figure out more efficient ways to forage, or the need to surmount the defense mechanisms of the plants and animals our ancestors wanted to eat. "Ecological pressures like avoiding predators, finding food and shelter, choosing habitats—all these kinds of decisions. I think they played a role" in brain growth, says Reader, the biologist.

Researchers who've used different methods to measure the size of a person's social circle have come up with numbers that don't match Dunbar's. One set of studies by the anthropologist Russell Bernard and the network scientist Peter Killworth found a mean social network size of 291. Another paper, published this month in the Journal of the American Statistical Association, came up with 611.

Among social network architects, there are those who see the Dunbar Number less as a wall and more as a hurdle. When Morin was at Facebook, he used to discuss behavioral science with Dustin Moskovitz, one of its co-founders. In 2008, Moskovitz, along with the programmer Justin Rosenstein, left Facebook to found Asana, a company that offers task-management software meant to improve how work teams collaborate. Whereas Path fits itself to the contours of the social limits Dunbar describes, Asana seeks to explode them.
To Moskovitz and Rosenstein, a tool such as Asana—or Facebook, for that matter—is like a telescope. It's a technology that extends the range of our abilities. "It gives us more capacity for keeping track of these relationships, for annotating them, knowing what people are doing, developing an understanding of their strengths and weaknesses, without necessarily having a bunch of one-on-one conversations," says Moskovitz. Rosenstein adds: "Certainly that's one of our semisecret sub-missions: to increase Dunbar's Number."

At Facebook itself, Dunbar still comes up often. "We do talk about it. In a lot of contexts it's a compelling framing of some of the data that we have about people's relationships," says Cameron Marlow, a sociologist and the head of the company's data science team.

Dunbar is familiar with the critiques of his work, and he has responses to them. He agrees with Watts, for example, that people have different social networks for different purposes, but that doesn't mean there isn't some basic emotional bond we reserve for some people, independent of their utility to us: "Someone like your boss, or the person you borrow $50 from to pay the drug dealer, these people are meaningful in your life, but they're not meaningful to you as relationships." He also continues to find his number popping up all around him. A paper published in 2011 found that on Twitter the average number of other people a user regularly interacts with falls between 100 and 200. And though the limit on how many Facebook friends one can have is a generous 5,000, the average user has 190—more than 150, but within what Dunbar sees as the margin of error.

Dunbar himself has zero Facebook friends. He occasionally peers over his wife's shoulder when she logs on at home, but he isn't on the social network. He has a LinkedIn (LNKD) account, he says, "by mistake." He opened a Path account but never uses it.

Dunbar does not rule out the possibility that human beings might be able to reset the cognitive limits on our social lives—we've done it before. The reason we're able to function in so much larger groupings than our primate cousins, Dunbar argues, is because, tens of thousands of years ago, we taught ourselves to talk. Whereas baboons bond by taking turns picking each others' nits, we have rhetoric and gossip and half-time speeches, not to mention singing and storytelling and jokes, to bring and hold us together. Language, he says, is how humans used their big brains to get to 150. And until something as revolutionary as that comes along, 150 is where he thinks we'll stay.

Why The Simmering Social Revolution In The Workplace Will Boil Over In 2013

Brett Caine, Forbes, December 24

Brett Caine is senior vice president and general manager for the Citrix Online Services division.

So much has been said and written about being social – we tweet, we pin, we like. We have become a society that communicates and shares just about everything we do, with one notable exception – work. Work is the place where social firewalls go up when they really should come down. After all, our teams are about teamwork. Social is the perfect tool to get our teams to work more collaboratively.

And as it catches on, productivity is improving – people can work and play from anywhere and (finally) debunking the notion that workers need to be in an office to produce. The number of work-at-home employees is increasing dramatically and not just day-extenders. For the first time we are seeing companies implement work-at-home policies and practices that make it possible to work from home as a full member of the team. Everyone wants flexibility, more and more ask for it and the millennials will demand it. What does this changing workforce (and workplace) mean for leaders and managers in the workplace?

In full openness, I work for a company whose business, in part, is to help other businesses collaborate socially. We also use the tools ourselves. In embracing these trends, our company has delivered extremely high employee engagement scores over the past few years – increasing each year despite the challenges and complexities of an ever increasing distributed and global workforce.

With these trends as the backdrop, I predict that in 2013, we’ll start seeing some distinct changes in the workplace – changes that mark a leap forward in the “social” work revolution.

Here are five areas where the changes will take place – areas that all managers and employees need to be ready to address.

·       Email evolves into obscurity
Email is not the right tool for office communications. It’s been default as the best way to do it for too long. It’s broken and it (we) can’t keep up in a fast paced, continuous adaption of teams in the workforce.

Email was never intended to be used as a collaboration tool. It slows us down, kills productivity and leaves critical information in silos rather than being shared. This argument was made profoundly by a recent study, which found that the average worker spends nearly a third of his or her work week managing email and nearly 20 percent of his or her time looking for internal information and/or tracking down colleagues who can help with specific tasks. Leaders of major companies are shutting down internal email – simply to improve communication and get more work done.

In 2013, we will see a shift in how emails are used in the workplace. Increasingly, emails will evolve into notifications that will signal to readers that something is happening somewhere else – e.g., a brainstorming call at 2 p.m. The content and reference materials will be in the cloud and continuously updated, synchronized and easily available from anywhere.

·       Social collaboration becomes invisible
With more and more businesses turning to the cloud for effective collaboration solutions, the social enterprise isn’t the latest idea for companies. Microsoft, Salesforce and IBM proved that with major acquisitions of social networking companies for enterprises. When multiple, large industry players enter a space, it’s the beginning of a major trend and unlikely to fade away. In my view, the trend will become so common, social becomes the frame for collaboration in the future.

If you really want to social enable your company, figure out how to actually get to “work” with social collaboration instead of just talking about it. This will be the tipping point for users and companies to get the benefit in 2013 and beyond.

·       Needs for collaboration increase as we get better at preparing for unlikely events
Unfortunately, natural and man-made disasters are likely to increase in the years ahead. Companies therefore are updating their crisis management plans. As part of that preparation, it makes perfect sense for companies to invest in social collaboration tools that allow them to maintain business under any circumstances. Increasingly, companies and employee commutes and routines are getting disrupted by storms, transit strikes, outages and fires to name a few. In 2013, get ahead of this and invest in both tools and policies to enable work from anywhere business models. Certainly, social collaboration tools, remote access and support for employees regardless of location are smart steps to take to prepare your business to stay open even when the most unlikely events occur.

·       Your personal cloud will have a single access point
As businesses move to the cloud, this will also be the year where you will start seeing one point of access to your data regardless of where it’s stored. You will no longer have to login and check five, six or even seven different storage repositories for the information you need for work. One app will connect you with all your social channels – personal, professional, you name it. That ease-of-use is going to drive adoption in droves.

·       Casual Fridays will become Work-from-Home Fridays
The tablet generation has already redefined the workplace. This on-the-go generation is not defined by four walls and a desktop computer. They are mobile. Mobile is the new normal. It’s not the exception. We will see more small- and medium sized companies take this generation into consideration and consider virtual offices and workshifting to incentivize their employees.

With rising gas prices and the high costs of commercial real estate, it makes financial sense for SMBs in particular to consider alternatives to traditional corporate headquarters and hubs. Office-sharing in some regions has become quite popular, particularly among start-ups. In return, we will have a more dynamic workforce and work environment instead of the same old stagnant cube nation.

The future is indeed bright for going social in the workplace. It’s great to see work catch up to everyday life. It’s something we can all like.

Social Media a Healthcare Data Gold Mine

April Dembosky, The Financial Times, December 11, 2012

Bill Schmarzo envisions a future where holiday photos posted on Facebook become a gauge of a person’s weight loss or gain over time.

The chief technology officer for EMC’s consultant services acknowledges that privacy advocates are unlikely to allow his fantasy to become reality, but the technology that can measure minute body changes in photographs and feed it into someone’s electronic health record already exists.

 “The scary thing is if that data could be used to deny care and insurability,” Mr Schmarzo says.
Healthcare companies are loathe to tread into such sensitive territory, but they are keenly aware of the gold mine of health data stored in people’s social media accounts.

“Studies have shown that people are more willing to share more private medical information in social media than they’re willing to share with their medical providers,” says Martin Kohn, chief medical scientist at IBM.

Pharmaceutical companies already analyse social media sites to track reports of side-effects of their drugs. That data can help correct formulas more quickly than waiting for the results of years-long clinical trials; it can also be used to set prices and test marketing slogans.

Public health officials are also interested in social media data as a source of information on disease outbreaks. Several start-ups are working on algorithms that study Facebook, Twitter, and blog posts to track early signs of infectious disease outbreaks, as people generally complain to their friends long before public health agencies can collect doctors’ reports and issue official warnings.

One outbreak of the Norovirus stomach bug at a student journalism conference in Canada was live-tweeted earlier this year, with posts such as “Motion that nobody else on this bus puke” and “36 hours without leaving my hotel room” signalling its coming and going before any traditional surveillance system had noted it.

The trouble with social media data are that they are fragmented and incomplete, warns James Kaufman, manager of public health research at IBM’s Silicon Valley lab, and there are limits to the types of insights that can be drawn from such patient-reported data.

“Colds, flus, sure people report that,” he says, “but no one’s going to report Aids or haemorrhoids.”

Social Media Report 2012 (Nielsen)

Social Media Use Exploded in 2012, Led by Pinterest

Stephanie Mlot, PC Magazine, December 3, 2012

It's hard for some to remember life before the Internet, let alone social media, and now it appears that these sites are coming of age.

Consumers spend more time on social networks than any other site — about 20 percent via a PC, and 30 percent using a mobile device, Nielsen's Social Media Report revealed.

Still not impressed? The report also tips a 37 percent increase in the total time spent on social media in the U.S., reaching 121 billion minutes in July, compared to 88 billion in summer 2011.

The ultimate social network, Facebook, remains the most-visited in the U.S., and earned the title of most popular Web brand in the U.S. this year. It reached 152.2 million PC visitors, 78.4 million app users, and 74.3 million mobile Web surfers. That dwarfs social sites in all categories (see graphic below), beating No. 2 Blogger by more than 93.7 million PC users. In the mobile app race, Facebook, Twitter, Foursquare, Google+, and Pinterest carry the top five spots.

Top networks like Facebook and Twitter have profound staying power in a world where social media options are expanding every day, including breakout star Pinterest, which boasted what Nielsen reported is the largest year-over-year increase — 1,698 percent. For more, check out PCMag's Pinterest Board.
Overall Internet connections are on the rise, as well, drawing more than 80 percent increases in mobile Web and app usage in the last year; PCs dropped 4 percent. Meanwhile, tablets, handheld music players like the iPod touch, game consoles, Internet-enabled TVs, and e-readers are slowly boosting connectivity.

With so many mobile options, it appears nearly a third of people ages 18 to 24 write Facebook comments, send tweets, and perhaps even blog from the comfort of their bathroom. Those ages 25 to 34 are more likely to use social networking in the office.

For all of the celebrity accounts and political usage accrued by social media, it seems that people still focus on connecting with friends and family members. More than 60 percent of people turn on Facebook to keep up with someone they know in real life, while 9 percent initiate LinkedIn contact because of a person's physical attractiveness, Nielsen reported.

Report at http://blog.nielsen.com/nielsenwire/social/2012/

Lady Gaga: The Value is in the Details

Ravi Mattu, The Financial Times, November 30, 2012

When Lady Gaga, the singer and social media star – with over 31m Twitter followers, more than anyone else, and over 51m Facebook likes – finished watching a screening of The Social Network, she called her manager Troy Carter. "She said she'd like to build a social network for her fans" – she calls them her little monsters – "and build a community where they could congregate and have conversations," he says. "So I called some of my friends in the Valley."

One of those friends was Joe Lonsdale, co-founder of the Palo Alto-based data management company Palintir. "He said, 'Send me all the data you have.' So, we sent him everything and he said it was the worst data he had ever seen in his life." The problem wasn't the amount of data – they had lots of it, from Ticketmaster, Ladygaga.com and merchandise sales – but the quality. Existing social media platforms weren't much better. "When you deal with Facebook, the information you get is geographical – what city people are logging in from, what time of day – but you don't get the behavioural information to help you build a better experience."

So, with the help of Mr Lonsdale and backing from Google Ventures among others, Mr Carter created Backplane, a new social media platform on which sits Littlemonsters.com. The site is designed to cater to the "hardcore 1m" Lady Gaga fans because their behaviour is more valuable than trying to decipher what happens to a mass audience.

"Our bet is on the future of micronetworks," he says. "Facebook wasn't wired to build a relationship between fans and artists. It's more about communicating with family and friends and old girlfriends or your classmates; 51m likes doesn't mean we're going to sell 51m albums or concert tickets."

It is not just about deeper insight. It is also about getting rid of the middleman. It is a "misconception when people talk about a direct relationship between artists and their fans or brands and consumers through social media. The reality is that these platforms own the relationship. So as much as you can talk directly to a customer or a fan, you still have this intermediary . . . that controls the data. And at any given time, if they turn it off or they change an algorithm, like Facebook did with its newsfeed algorithm last year, it changes the way you're able to communicate with that fan or customer."

Listening to the sharp-suited 40-year-old at the FT Innovate conference calmly skewer the shortcomings of a technology company he claims is a friend – he is at pains to say that his efforts are complementary rather than competitive, and spoke to Facebook and others before launching – it is hard not to see him as a disrupter taking on the social media elite.

It is one reason he is here. His use of social media, particularly with Lady Gaga, has made him a much sought-after voice among the corporates drowning in the sea of big data.

Given his increasingly high profile, it is sometimes difficult to imagine the entrepreneur's childhood in inner-city Philadelphia. His single-parent mother worked at a hospital for 30 years, cleaning surgical instruments, while raising him and his brothers. She often worked long shifts, starting at 5:30 in the morning, so "the streets were raising us at the same time".

For an African-American boy growing up in that world, there were not a lot of options. "You got a couple of choices: drug dealers were the role models – you didn't have doctors and hedge fund managers that looked like you," he says. Or music. "At that time, hip hop culture was exploding . . .  and coming from the family I came from, drugs was not an option."

While that may sound like a scene from The Wire, Mr Carter says being an outsider has been key to his success. "Being born in the adolescent years of hip hop helped us learn about flux. And when you're in an industry that is constantly growing, changing, maturing . . . you get a chance to try different things out and a chance to fail."

In high school, he got to know fellow Philadelphians DJ Jazzy Jeff and the Fresh Prince – the actor Will Smith – and became an assistant carrying the hip hop duo's records from gig to gig, before setting out on his own as a music promoter. It was during this time that he met rapper and producer Sean Combs, now known as P Diddy, who gave him a job at his Bad Boy Records.

Mr Carter says this is where he learnt about the record business and P Diddy's example taught him "that you can be a young black entrepreneur with no college degree or any sort of experience and people will give you a shot in this business".

He later set up a boutique talent management company, which he sold to the Sanctuary Group, then part of Universal Music. He quickly discovered that being in a big company was not for him. "Instead of me being able to be creative with the artists, I was sitting in finance meetings a couple of times a week. It killed my spirit as an entrepreneur."

But he also understood the value of getting the organisational culture right. When he launched Atom Factory, hiring other outsiders was essential. "My COO didn't come from the music industry, my vice-president of creative was actually a schoolteacher," he says. "It was important we had people who came from an outside perspective, who didn't come from selling CDs."

As well as Lady Gaga, the group represents John Legend and Bollywood star Priyanka Chopra as she launches a music career outside India.

Alongside Atom Factory sit AF Square, an angel investment fund with stakes in a number of mostly tech start-ups, including news app Summly, the taxi-hailing app Uber and music streaming site Spotify, and A/Idea, an ideas lab. Mr Carter has announced plans to launch a drink called Pop Water.

He declines to disclose profit figures but says the group has grown 60-70 per cent year on year for the past four years and he is sole shareholder.

Mr Carter is still best known as the manager of Lady Gaga, partly because of how he used technology to circumvent mainstream radio when she struggled to get her music played on it.

Once again, via Littlemonsters.com, she is a beta test with a view to understanding how Backplane could be employed by other companies to build communities. He is working with a shoe brand on tapping into "sneaker culture", for example.

"Right now, we're planting the seeds of an oak tree. What we are planting today, we may not see the full benefit for five to 10 years," he says, pointing to the fact that many of the core tenets of the music business, such as digital rights arrangements, could change.

Still, the data being collected is already informing commercial decisions. For example, until Littlemonsters.com went live six months ago, Lady Gaga had never toured or been promoted in South America, a region that is not a big music market in terms of album sales and downloads. But "once we launched the website, we were able to get a lot of info about fans and specifically the numbers of them in South America", leading to the decision to add a number of dates there to the singer's Born This Way Ball tour.
But however excited one gets about data, Mr Carter offers a word of advice for anyone thinking about using it to tinker with the creative side of the business: don't. "I stay away from the arts . . . writing songs, being creative – those are downloads from god. You can't do data analytics on art."

WSJ - CEO Council on Big Data

Big Data: Opportunities and Risks Co-Chairs
Tim Armstrong, Chairman and CEO, AOL Inc.
Dominic Barton, Global Managing Director, McKinsey & Co.
R. Marcelo Claure, Chairman, President and CEO, Brightstar Corp.
Subject Expert
David J. Rothkopf, President and CEO, Garten Rothkopf

Big Data: The Top Four Recommendations
1. Big Data Is Opportunity
Industry already recognizes that the advent of Big Data is a potential engine of significant economic growth. Policy makers must address issues such as security and privacy but not restrict this opportunity. Consumers, government and companies all have a role to play in defining policy. Countries need to recognize that the treatment of data is ultimately an issue of national competitiveness.

2. Create Rules of the Road
The government should define which activities concerning data are legal or illegal. For activities that are legal, consumers should then define what is public or private in their own cases through opting in or out in terms of what they choose to reveal about themselves.

3. Enact Cybersecurity Standards
The private sector, government, the military and consumers should jointly develop detailed standards and incidence-reporting practices for cybersecurity, building on existing industry best practices.

4. Create a Global Data Treaty
The U.S. should play a leadership role in coordinating with international bodies to pass a treaty to establish clear, universal standards on data privacy and ownership.

It's a phenomenon commonly referred to as Big Data, and it has generated widespread debate over a host of issues. What should companies do with such data? How can it be used profitably? How should it be treated? Should there be laws governing its use? Internationally recognized safeguards for consumer privacy? And how do U.S. companies stay competitive as more countries learn how to process and interpret such data?

The Wall Street Journal's John Bussey moderated a task-force discussion about just such issues. Here are edited excerpts of their presentations to the CEO Council:

Opportunity First
JOHN BUSSEY: Our group ended up with essentially two principles and two action items. Dominic, if you could take our first, please?

DOMINIC BARTON: Our group felt very strongly that this is a huge opportunity, and we shouldn't focus on how to protect or regulate Big Data before we recognize how important it is. Companies that use Big Data effectively get about a 6% productivity improvement versus others.

We're at the early stages. It's in every sector. And we all felt it can be the next wave of productivity growth if we use this data effectively.

Depending on how you protect or use data, it can actually lead to a country's competitive advantage. We've got small city-states like Singapore and Abu Dhabi that are allowing, for example, consumer medical information to be publicly available to get innovation.
So let's not lose sight of how important it can be for productivity improvement as we think about the protection.

MR. BUSSEY: Marcelo, our second principle.
MARCELO CLAURE: First of all, we had a fascinating group. We had a tremendous amount of interaction. And we quickly realized that all of us live in a hyper-connected world. By that I mean we're passing a huge amount of data every day through a connected car, a connected cellphone, a connected tablet, a connected home.

Big Data: The Top Four Recommendations
1. Big Data Is Opportunity
Industry already recognizes that the advent of Big Data is a potential engine of significant economic growth. Policy makers must address issues such as security and privacy but not restrict this opportunity. Consumers, government and companies all have a role to play in defining policy. Countries need to recognize that the treatment of data is ultimately an issue of national competitiveness.

2. Create Rules of the Road
The government should define which activities concerning data are legal or illegal. For activities that are legal, consumers should then define what is public or private in their own cases through opting in or out in terms of what they choose to reveal about themselves.

3. Enact Cybersecurity Standards
The private sector, government, the military and consumers should jointly develop detailed standards and incidence-reporting practices for cybersecurity, building on existing industry best practices.

4. Create a Global Data Treaty
The U.S. should play a leadership role in coordinating with international bodies to pass a treaty to establish clear, universal standards on data privacy and ownership.

Companies around the world increasingly collect and process vast amounts of customer data, particularly data tracking consumer behavior on the Web.

 We see it as a tremendous opportunity, but what is the government role?

Some members of our group argued that government shouldn't be allowed to regulate or do anything related to Big Data. But the rest of the group agreed that we've got to define the basics of what the government is going to do.

And where we came out at the end was, government should have the role of defining what is legal and what is illegal in the use of Big Data. And that's where the government should stop. We think it should be left up to the consumer to define what he or she wants to share and doesn't want to share, what is public and what is private.

Limiting government control to what's legal and illegal will allow the consumer to make more choices. As a consumer, you should be able to determine how much of your Facebook profile you want to share, or whether you want to share information about where you shopped last.

Security and Treaty
MR. BUSSEY: And Tim has our two action items.

TIM ARMSTRONG: The first one is probably more important than it seems on the surface, especially for private companies. I think we started this as kind of a government conversation, but mostly it's private companies that actually own the infrastructure that makes the company or the country work.

So cybersecurity is something that really needs to be dealt with. And specifically, having standards around this is important in two different directions.

One is standards of what happens when something gets attacked. How do you report it, and what is the infrastructure to help companies with that?

The second piece which is important and which came up in our discussion is when your data lives outside the country, or when you're going to do things with data in other countries. There are groups like CFIUS [the U.S. government's Committee on Foreign Investment in the U.S., which reviews foreign investment deemed to have an effect on national security] which are inside the government, which, if you haven't dealt with them, you will, over these type of data assets. Cybersecurity is really important.

The last action item is coming up with a global data treaty. The U.S. is probably the largest economy involved in Big Data right now, and we think it's important for the U.S. to take a leading role in defining what some of the Big Data policies and standards should be in such a treaty. We would hope that we, the U.S., could define a basic treaty to start off, and then other countries could either adopt or augment the treaty over time.

Obama's Approach to Big Data: Do As I Say, Not As I Do

Politicians' Policy Decisions May Stymie Tools That Got Them Elected

Kate Kaye, Ad Age, November 16, 2012


One of the keys to success for President Barack Obama's reelection bid was its masterful use of data. But lost in the hype is this: The administration supports a browser-based do not track system that, if pervasive, would throw a wrench into the data-collection tactics that empowered the campaign.

Even today BarackObama.com features data-tracking cookies from several online ad and analytics firms.
The Mitt Romney and Obama campaigns spent hundreds of thousands of dollars in 2012 on data and related services to enhance their own voter contact information, inform their online and offline messaging and target ads. At the same time, Congress is inspecting the practices of firms that buy, sell and filter consumer data for corporate marketers.

"The Obama administration and the GOP should confront head-on the privacy issues raised by [their] far-reaching use of digital profiling and targeting data," argued privacy advocate Jeffrey Chester, founder of the Center for Digital Democracy. "It would be unfortunate for the administration's work to advance Do Not Track and other key safeguards if they failed to tackle the use of powerful data targeting technologies by political campaigns."

Industry and privacy wonks actually agree
It's a rare occurrence, but both Mr. Chester and the ad industry are in agreement on one thing: They both appreciate the attention the Obama data machine is getting. Privacy groups want to raise awareness of data collection and usage in the hopes of generating public support for curbing what they see as an increasingly infiltrative violation of personal privacy by marketers and the mushrooming data industry.

"Protecting the privacy of consumers and citizens should require policymakers from both sides to confront the civil liberties implications of what has been unleashed," added Mr. Chester, noting that the 2012 campaigns should divulge what data they collected, how they targeted ads and what will happen to the information now that the election is over.

Industry players, especially their Capitol Hill lobbyists, aim to convince legislators that the very data practices some of them criticize are helping them and their colleagues win races.

"Big data isn't going to help Todd Aken," said Mike Zaneis, general counsel of the Interactive Advertising Bureau, referring to the disgraced Congressman from Missouri who lost his Senate campaign after claiming women can ward off pregnancy resulting from "legitimate rape." Continued Mr. Zaneis, "But the Obama campaign used a lot of online data and a tremendous amount of offline data to go precinct-by-precinct to get-out-the-vote."

Third-party tags
More than a week after the election, BarackObama.com houses an array of third-party tags that track users for ad targeting and campaign and site analytics. Yesterday, around fifteen ad company tags were surfaced by Evidon's Ghostery software, including tags from BlueKai, which calls itself a "big data activation solution," and Appnexus, which among other things allows advertisers to use a variety of user behavioral data to target ads to those users on Facebook.

Both the Obama and Romney campaigns used social-media-widget and data provider ShareThis to target fundraising ads and identify issues and trends swing state voters were interested in, according to ShareThis CEO Kurt Abrahamson. The company tracks when people visit web pages and share them on Twitter, Facebook, LinkedIn or other popular social sites and allows advertisers to target ads using that anonymized information.

Clashing goals of campaigning and governing
Data tracking tools and techniques that have helped legislators on both sides of the aisle build supporter lists, generate donations and get out the vote could be stymied by a do-not-track browser standard or restrictive privacy legislation.

In February, the Federal Trade Commission and the ad industry announced they'd work together with browser companies to develop a DNT standard. At the same time, the U.S. Commerce Department introduced a consumer privacy bill of rights that guided companies to provide individual control over data collection, better data security measures, and transparency of data use, and also called for "a reasonable amount of data collection by companies." Secretary of Commerce John Bryson said at the time the department would work with Congress to implement the privacy bill of righs -- which some deem to be supportive of industry's self-regulatory approach -- through legislation.

The Digital Advertising Alliance, a large coalition of ad industry trade groups, has conducted an "ongoing dialogue with the FTC as recently as yesterday to figure out how to implement the [DNT] standard," said Stu Ingis, counsel to the DAA, on Wednesday. The DAA oversees the industry's Ad Choices program, which allows people to opt-out from online ad targeting through display ads that include the group's small triangular symbol. It's not entirely clear whether the FTC is confident that the DAA's self-regulatory program is enough to protect consumer privacy.

As reported by Politico earlier this month, FTC Chairman Jon Leibowitz said, "If by the end of the year or early next year, we haven't seen a real Do Not Track option for consumers, I suspect the commission will go back and think about whether we want to endorse legislation." Mr. Leibowitz is expected by beltway insiders to step down at the end of the year, and some believe his goal to finalize a DNT standard before he leaves is pressurizing the situation.

A free pass for political data?
Enter the Bipartisan Congressional Privacy Caucus. The group recently received responses to inquiries into several data firms that manage and analyze, and in some cases buy and sell, online and offline consumer data. Nine firms -- Acxiom, Epsilon, Equifax, Experian, Harte-Hanks, Intelius, Fair Isaac, Merkle, and Meredith Corp. -- submitted lengthy and often vague answers to a series of questions about their data businesses and practices.

"Many questions about how these data brokers operate have been left unanswered, particularly how they analyze personal information to categorize and rate consumers," said lawmakers in a joint statement regarding the companies' responses.

Absent from the list of data firms questioned were similar companies that deal mainly in voter file and political information that is often enhanced with consumer demographic, shopping and other data. For instance, NGP Van, the Democratic data powerhouse favored by the Obama team was not part of the inquiry. The Obama campaign and DNC spent hundreds of thousands of dollars with NGP Van this election cycle alone. The firm matches its voter data with data from TargetSmart, which offers "the richest set of consumer and interest data, allowing the most sophisticated targeting," according to the NGP Van site.

Other political data firms left out of the inquiry include Catalist, another Democratic data firm; Campaign Grid, which offers Republican data and online ad targeting; and Aristotle, a well-established non-partisan political data company. People involved with the congressional inquiry deny that political data firms were left off the list for any strategic reason.

In a press release about the data broker responses, the Privacy Caucus stated it "will push for whatever steps are necessary to make sure Americans know how this industry operates and are granted control over their own information."

Rep. Ed Markey, a Democrat from Massachusetts and Caucus co-chair, has sponsored a Do Not Track Kids Act and a mobile privacy bill.

Observers don't expect a privacy bill to be passed anytime soon; if that does happen, it may not apply to political campaigns or groups anyway. For instance, political messages are exempt from CAN-SPAM laws, and political organizations are not restricted by the Do Not Call Registry.

"Often when data laws are being proposed and put forward, the politicians exempt themselves," said Don Hinman, senior VP for data strategy at Epsilon, which gets some of its data from political advertisers but mainly is a purveyor of consumer information.

Mr. Ingis considers it exemption for political messages to be a first amendment issue. "It would be very hard for such a limitation on political messages to be restricted. . . . and I think that would have been true in the context of Do Not Call if they would have gone there," he said.

How 'Social Intelligence' Can Guide Decisions

By offering decision makers rich real-time data, social media is giving some companies fresh strategic insight.

Martin Harrysson, Estelle Metayer, and Hugo Sarrazin, McKinsey Quarterly, November 2012

In many companies, marketers have been first movers in social media, tapping into it for insights on how consumers think and behave. As social technologies mature and organizations become convinced of their power, we believe they will take on a broader role: informing competitive strategy. In particular, social media should help companies overcome some limits of old-school intelligence gathering, which typically involves collecting information from a range of public and propriety sources, distilling insights using time-tested analytic methods, and creating reports for internal company “clients” often “siloed” by function or business unit.

Today, many people who have expert knowledge and shape perceptions about markets are freely exchanging data and viewpoints through social platforms. By identifying and engaging these players, employing potent Web-focused analytics to draw strategic meaning from social-media data, and channeling this information to people within the organization who need and want it, companies can develop a “social intelligence” that is forward looking, global in scope, and capable of playing out in real time.

This isn’t to suggest that “social” will entirely displace current methods of intelligence gathering. But it should emerge as a strong complement. As it does, social-intelligence literacy will become a critical asset for C-level executives and board members seeking the best possible basis for their decisions.

In this article, we explore four distinct ways social technologies can augment the intelligence-gathering approaches of companies. As Exhibit 1 makes clear, social media has little effect on some aspects of the intelligence cycle—in particular, the need to identify priorities for exploration and decision making over the next 6 to 12 months, as well as the use of assembled information to make unbiased decisions. But social technologies can play a surprisingly central role in how information is sourced, collected, analyzed, and distributed.

Pew: Mobile is the Needle; Social is the Thread

Kathryn Zickuhr, Pew Research Center’s Internet & American Life Project, October 18, 2012

"Examining more than a decade of data on the social impact of technology in America, Pew Internet Research Analyst Kathryn Zickuhr discussed the patterns and trends shaping the new messaging realities of the digital age at the WSU Elliott School of Communications’ annual Comm Week conference."

Data from Health Care Reviews Could Power "Yelp for Health Care" Startups

Data-driven decision engines will need patient experience to complete the feedback loop.

Alex Howard, O'Reilly Radar, October 17, 2012

Given where my work and health has taken me this year, I’ve been thinking much more about the relationship of the Internet and health data to accountability and patient-driven health care.

When I was looking for a place in Maine to go for care this summer, I went online to look at my options. I consulted hospital data from the government at HospitalCompare.HHS.gov and patient feedback data on Yelp, and then made a decision based upon proximity and those ratings. If I had been closer to where I live in Washington D.C., I would also have consulted friends, peers or neighbors for their recommendations of local medical establishments.

My brush with needing to find health care when I was far from home reminded me of the prism that collective intelligence can now provide for the treatment choices we make, if we have access to the Internet.

Patients today are sharing more of their health data and experiences online voluntarily, which in turn means that the Internet is shaping health care. There’s a growing phenomenon of “e-patients” and caregivers going online to find communities and information about illness and disability.

Aided by search engines and social media, newly empowered patients are discussing health conditions with others suffering from disease and sickness — and they’re taking that peer-to-peer health care knowledge into their doctors’ offices with them, frequently on mobile devices. E-patients are sharing their health data of their own volition because they have a serious health condition, want to get healthy, and are willing.

From the perspective of practicing physicians and hospitals, the trend of patients contributing to and consulting on online forums adds the potential for errors, fraud, or misunderstanding. And yet, I don’t think there’s any going back from a networked future of peer-to-peer health care, anymore than we can turn back the dial on networked politics or disaster response.

What’s needed in all three of these areas is better data that informs better data-driven decisions. Some of that data will come from industry, some from government, and some from citizens.

This fall, the Obama administration proposed a system for patients to report medical mistakes. The system would create a new “consumer reporting system for patient safety” that would enable patients to tell the federal government about unsafe practices or errors. This kind of review data, if validated by government, could be baked into the next generation of consumer “choice engines,” adding another layer for people, like me, searching for care online.

There are precedents for the collection and publishing of consumer data, including the Consumer Product Safety Commission’s public complaint database at SaferProducts.gov and the Consumer Financial Protection Bureau’s complaint database. Each met with initial resistance by industry but have successfully gone online without massive abuse or misuse, at least to date.

It will be interesting to see how medical associations, hospitals and doctors react. Given that such data could amount to government collecting data relevant to thousands of “Yelps for health care,” there’s both potential and reason for caution. Health care is a bit different than product safety or consumer finance, particularly with respect to how a patient experiences or understands his or her treatment or outcomes for a given injury or illness. For those that support or oppose this approach, there is an opportunity for public comment on proposed data collection at the Federal Register.

The power of performance data
Combining patients review data with government-collected performance data could be quite powerful in helping to drive better decisions and adding more transparency to health care.
In the United Kingdom, officials are keen to find the right balance between open data, transparency and prosperity.

“David Cameron, the Prime Minister, has made open data a top priority because of the evidence that this public asset can transform outcomes and effectiveness, as well as accountability,” said Tim Kelsey, in an interview this year. He used to head up the United Kingdom’s transparency and open data efforts and now works at its National Health Service.

“There is a good evidence base to support this,” said Kelsey. “Probably the most famous example is how, in cardiac surgery, surgeons on both sides of the Atlantic have reduced the number of patient deaths through comparative analysis of their outcomes.”

More data collected by patients, advocates, governments and industry could help to shed light on the performance of more physicians and clinics engaged in other expensive and lifesaving surgeries and associated outcomes.

Should that be extrapolated across the medical industry, it’s a safe bet that some medical practices or physicians will use whatever tools or legislative influence they have to fight or discredit websites, services or data that puts them in a poor light. This might parallel the reception that BrightScope’s profiles of financial advisors have received in industry.

When I talked recently with Dr. Atul Gawande about health data and care givers, he said more transparency in these areas is crucial:

“As long as we are not willing to open up data to let people see what the results are, we will never actually learn. The experience of what happens in fields where the data is open is that it’s the practitioners themselves that use it.”

In that context, health data will be the backbone of the disruption in health care ahead. Part of that change will necessarily have to come from health care entrepreneurs and watchdogs connecting code to research. In the future, a move to open science and perhaps establish a health data commons could accelerate that change.

The ability of caregivers and patients alike to make better data-driven decisions is limited by access to data. To make a difference, that data will also need to be meaningful to both the patient and the clinician, said Dr. Gawande. He continued:

“[Health data] needs to be able to connect the abstract world of data to the physical world of what really happens, which means it has to be timely data. A six-month turnaround on data is not great. Part of what has made Wal-Mart powerful, for example, is they took retail operations from checking their inventory once a month to checking it once a week and then once a day and then in real-time, knowing exactly what’s on the shelves and what’s not. That equivalent is what we’ll have to arrive at if we’re to make our systems work. Timeliness, I think, is one of the under-recognized but fundamentally powerful aspects because we sometimes over prioritize the comprehensiveness of data and then it’s a year old, which doesn’t make it all that useful. Having data that tells you something that happened this week, that’s transformative.”

Health data, in other words, will need to be open, interoperable, timely, higher quality, baked into the services that people use, and put at the fingertips of caregivers, as US CTO Todd Park explains in the video below:

There is more that needs to be done than simply putting “how to live better” information online or into an app. To borrow a phrase from Robert Kirkpatrick, for data to change health care, we’ll need to apply the wisdom of the crowds, the power of algorithms and the intuition of experts to find meaning in health data and help patients and caregivers alike make better decisions.

That isn’t to say that health data, once published, can’t be removed or filtered. Witness the furor over the removal of a malpractice database from the Internet last year, along with its restoration.

But as more data about doctors, services, drugs, hospitals and insurance companies goes online, the ability of those institutions to control public perception of the institutions will shift, just as it has with government and media. Given flaws in devices or poor outcomes, patients deserve such access, accountability and insight.

Enabling better health-data-driven decisions to happen across the world will be far from easy. It is, however, a future worth building toward.