Algorithms Get a Human Hand in Steering Web

Steve Lohr, The New York Times, March 10, 2013

Trading stocks, targeting ads, steering political campaigns, arranging dates, besting people on "Jeopardy" and even choosing bra sizes: computer algorithms are doing all this work and more.

But increasingly, behind the curtain there is a decidedly retro helper — a human being.

Although algorithms are growing ever more powerful, fast and precise, the computers themselves are literal-minded, and context and nuance often elude them. Capable as these machines are, they are not always up to deciphering the ambiguity of human language and the mystery of reasoning. Yet these days they are being asked to be more humanlike in what they figure out.

"For all their brilliance, computers can be thick as a brick," said Tom M. Mitchell, a computer scientist at Carnegie Mellon University.

And so, while programming experts still write the step-by-step instructions of computer code, additional people are needed to make more subtle contributions as the work the computers do has become more involved. People evaluate, edit or correct an algorithm's work. Or they assemble online databases of knowledge and check and verify them — creating, essentially, a crib sheet the computer can call on for a quick answer. Humans can interpret and tweak information in ways that are understandable to both computers and other humans.

Question-answering technologies like Apple's Siri and I.B.M.'s Watson rely particularly on the emerging machine-man collaboration. Algorithms alone are not enough.

Twitter uses a far-flung army of contract workers, whom it calls judges, to interpret the meaning and context of search terms that suddenly spike in frequency on the service.

For example, when Mitt Romney talked of cutting government money for public broadcasting in a presidential debate last fall and mentioned Big Bird, messages with that phrase surged. Human judges recognized instantly that "Big Bird," in that context and at that moment, was mainly a political comment, not a reference to "Sesame Street," and that politics-related messages should pop up when someone searched for "Big Bird." People can understand such references more accurately and quickly than software can, and their judgments are fed immediately into Twitter's search algorithm.

"Humans are core to this system," two Twitter engineers wrote in a blog post in January.

Even at Google, where algorithms and engineers reign supreme in the company's business and culture, the human contribution to search results is increasing. Google uses human helpers in two ways. Several months ago, it began presenting summaries of information on the right side of a search page when a user typed in the name of a well-known person or place, like "Barack Obama" or "New York City." These summaries draw from databases of knowledge like Wikipedia, the C.I.A. World Factbook and Freebase, whose parent company, Metaweb, Google acquired in 2010. These databases are edited by humans.

When Google's algorithm detects a search term for which this distilled information is available, the search engine is trained to go fetch it rather than merely present links to Web pages.
 "There has been a shift in our thinking," said Scott Huffman, an engineering director in charge of search quality at Google. "A part of our resources are now more human curated."

Other human helpers, known as evaluators or raters, help Google develop tweaks to its search algorithm, a powerhouse of automation, fielding 100 billion queries a month. "Our engineers evolve the algorithm, and humans help us see if a suggested change is really an improvement," Mr. Huffman said.

Katherine Young, 23, is a Google rater — a contract worker and a college student in Macon, Ga. She is shown an ambiguous search query like "what does king hold," presented with two sets of Google search results and asked to rate their relevance, accuracy and quality. The current search result for that imprecise phrase starts with links to Web pages saying that kings typically hold ceremonial scepters, a reasonable inference.

Her judgments, Ms. Young said, are "not completely black and white; some of it is subjective." She added, "You try to put yourself in the shoes of the person who typed in the query."

I.B.M.'s Watson, the powerful question-answering computer that defeated "Jeopardy" champions two years ago, is in training these days to help doctors make diagnoses. But it, too, is turning to humans for help.

To prepare for its role in assisting doctors, Watson is being fed medical texts, scientific papers and digital patient records stripped of personal identifying information. Instead of answering questions, however, Watson is asking them of clinicians at the Cleveland Clinic and medical school students. They are giving answers and correcting the computer's mistakes, using a "Teach Watson" feature.

Watson, for example, might come across this question in a medical text: "What neurological condition contraindicates the use of bupropion?" The software may have bupropion, an antidepressant, in its database, but stumble on "contraindicates." A human helper will confirm that the word means "do not use," and Watson returns to its data trove to reason that the neurological condition is seizure disorder.
 "We're using medical experts to help Watson learn, make it smarter going forward," said Eric Brown, a scientist on I.B.M.'s Watson team.

Ben Taylor, 25, is a product manager at FindTheBest, a fast-growing start-up in Santa Barbara, Calif. The company calls itself a "comparison engine" for finding and comparing more than 100 topics and products, from universities to nursing homes, smartphones to dog breeds. Its Web site went up in 2010, and the company now has 60 full-time employees.

Mr. Taylor helps design and edit the site's education pages. He is not an engineer, but an English major who has become a self-taught expert in the arcane data found in Education Department studies and elsewhere. His research methods include talking to and e-mailing educators. He is an information sleuth.

On FindTheBest, more than 8,500 colleges can be searched quickly according to geography, programs and tuition costs, among other criteria. Go to the page for a university, and a wealth of information appears in summaries, charts and graphics — down to the gender and race breakdowns of the student body and faculty.

Mr. Taylor and his team write the summaries and design the initial charts and graphs. From hundreds of data points on college costs, for example, they select the ones most relevant to college students and their parents. But much of their information is prepared in templates and tagged with code a computer can read. So the process has become more automated, with Mr. Taylor and others essentially giving "go fetch" commands that the computer algorithm obeys.

The algorithms are getting better. But they cannot do it alone.

The Google Glass feature no one is talking about

Mark Hurst, Creative Good, February 28, 2013

Google Glass might change your life, but not in the way you think. There's something else Google Glass makes possible that no one – no one – has talked about yet, and so today I'm writing this blog post to describe it.

To read the raving accounts of tech journalists who Google commissioned for demos, you'd think Glass was something between a jetpack and a magic wand: something so cool, so sleek, so irresistible that it must inevitably replace that fading, pitifully out-of-date device called the smartphone.

Sergey Brin himself said as much yesterday, observing that it is "emasculating" to use a smartphone, "rubbing this featureless piece of glass." His solution to that piece of glass, of course, is called Glass. And his solution to that emasculation is – well, as VentureBeat put it, "Sergey Brin calls smartphones 'emasculating' – but dorky Google Glass [is] A-OK."

Like every other shiny innovation these days, Google Glass will live or die solely on the experience it creates for people. The immediate, most visible problem in the Glass experience is how dorky the user looks while wearing it. No one wants to be the only person in the bar dressed like a cyborg from a 1992 virtual-reality movie. It's embarrassing. Early adopters will abandon Google Glass if they don't sense the social approval they seek while wearing it.

Google seems to have calculated this already and recently announced a partnership with Warby Parker, known for its designer glasses favored by the all-important younger demographic. (My own proposal, posted the day before, jokingly suggested that Google look into monocles.)

Except for the awkward physical design, the experience of using Google Glass has won high praise from reviewers. Seeing your bitstreams floating in the air in front of you, it would seem, is an ecstatic experience. Weather! Directions! Social network requests! Email overload! All floating in front of you, never out of your sight! For people who delight in a deluge of digital distractions, this is much more exciting than a smartphone, which forces you back to the boring offline world, every so often, when you put the phone away. Glass promises never to do that. In fact, in a feat of considerable chutzpah, Google is attempting to pitch Glass as an antidote to distraction, since users don't have to look down at a phone. Right, because now the distractions are all conveniently placed directly into your eyeball! (For a more accurate exploration of Glass-enabled distraction, see this darkly comic parody video. Even edgier is this parody – warning, some spicy language.)

As if all that wasn't enough, Google Glass comes with yet another, even more important feature: lifebits, the ability to record video of the people, places, and events around you, at all times. Veteran readers will remember that I predicted this six years ago in my book Bit Literacy. From Chapter 13:

The life bitstream will raise new and important issues. Should it be socially acceptable, for example, to record a private conversation with a friend? How will anyone be sure they're not being recorded, in public or private? … Corporations, police, even friends with 'life recorders' will capture the actions and utterances of everyone in sight, whether they like it or not.

Today, finally, that future has arrived: a major company offering the ability to record your life, store it, and share it – all with a simple voice command.

And this is where our story takes a turn, toward a ramification that dwarfs every other issue raised so far on Google Glass. Yes, the glasses look dorky – Google will fix that. And sure, Glass forces users to be permanently plugged-in to Google's digital world – that's hardly a concern for the company or, for that matter, most users out there. No. The real issue raised by Google Glass, which will either cause the project to fail or create certain outcomes you may not want (which I'll describe), has to do with the lifebits. Once again, it's an issue of experience.

The Google Glass feature that (almost) no one is talking about is the experience – not of the user, but of everyone other than the user. A tweet by David Yee introduces it well:

There is a kid wearing Google Glasses at this restaurant which, until just now, used to be my favorite spot.

The key experiential question of Google Glass isn't what it's like to wear them, it's what it's like to be around someone else who's wearing them. I'll give an easy example. Your one-on-one conversation with someone wearing Google Glass is likely to be annoying, because you'll suspect that you don't have their undivided attention. And you can't comfortably ask them to take the glasses off (especially when, inevitably, the device is integrated into prescription lenses). Finally – here's where the problems really start – you don't know if they're taking a video of you.

Now pretend you don't know a single person who wears Google Glass… and take a walk outside. Anywhere you go in public – any store, any sidewalk, any bus or subway – you're liable to be recorded: audio and video. Fifty people on the bus might be Glassless, but if a single person wearing Glass gets on, you – and all 49 other passengers – could be recorded. Not just for a temporary throwaway video buffer, like a security camera, but recorded, stored permanently, and shared to the world.

Now, I know the response: "I'm recorded by security cameras all day, it doesn't bother me, what's the difference?" Hear me out – I'm not done. What makes Glass so unique is that it's a Google project. And Google has the capacity to combine Glass with other technologies it owns.

First, take the video feeds from every Google Glass headset, worn by users worldwide. Regardless of whether video is only recorded temporarily, as in the first version of Glass, or always-on, as is certainly possible in future versions, the video all streams into Google's own cloud of servers. Now add in facial recognition and the identity database that Google is building within Google Plus (with an emphasis on people's accurate, real-world names): Google's servers can process video files, at their leisure, to attempt identification on every person appearing in every video. And if Google Plus doesn't sound like much, note that Mark Zuckerberg has already pledged that Facebook will develop apps for Glass.

Finally, consider the speech-to-text software that Google already employs, both in its servers and on the Glass devices themselves. Any audio in a video could, technically speaking, be converted to text, tagged to the individual who spoke it, and made fully searchable within Google's search index.

Now our stage is set: not for what will happen, necessarily, but what I just want to point out could technically happen, by combining tools already available within Google.

Let's return to the bus ride. It's not a stretch to imagine that you could immediately be identified by that Google Glass user who gets on the bus and turns the camera toward you. Anything you say within earshot could be recorded, associated with the text, and tagged to your online identity. And stored in Google's search index. Permanently.

I'm still not done.

The really interesting aspect is that all of the indexing, tagging, and storage could happen without the Google Glass user even requesting it. Any video taken by any Google Glass, anywhere, is likely to be stored on Google servers, where any post-processing (facial recognition, speech-to-text, etc.) could happen at the later request of Google, or any other corporate or governmental body, at any point in the future.
Remember when people were kind of creeped out by that car Google drove around to take pictures of your house? Most people got over it, because they got a nice StreetView feature in Google Maps as a result.
Google Glass is like one camera car for each of the thousands, possibly millions, of people who will wear the device – every single day, everywhere they go – on sidewalks, into restaurants, up elevators, around your office, into your home. From now on, starting today, anywhere you go within range of a Google Glass device, everything you do could be recorded and uploaded to Google's cloud, and stored there for the rest of your life. You won't know if you're being recorded or not; and even if you do, you'll have no way to stop it.

And that, my friends, is the experience that Google Glass creates. That is the experience we should be thinking about. The most important Google Glass experience is not the user experience – it's the experience of everyone else. The experience of being a citizen, in public, is about to change.
Just think: if a million Google Glasses go out into the world and start storing audio and video of the world around them, the scope of Google search suddenly gets much, much bigger, and that search index will include you. Let me paint a picture. Ten years from now, someone, some company, or some organization, takes an interest in you, wants to know if you've ever said anything they consider offensive, or threatening, or just includes a mention of a certain word or phrase they find interesting. A single search query within Google's cloud – whether initiated by a publicly available search, or a federal subpoena, or anything in between – will instantly bring up documentation of every word you've ever spoken within earshot of a Google Glass device.

This is the discussion we should have about Google Glass. The tech community, by all rights, should be leading this discussion. Yet most techies today are still chattering about whether they'll look cool wearing the device.

Oh, and as for that physical design problem. If Google Glass does well enough in its initial launch to survive to subsequent versions, forget Warby Parker. The next company Google will call is Bausch & Lomb. Why wear bulky glasses when the entire device fits into a contact lens? And that, of course, would be the ultimate expression of the Google Glass idea: a digital world that is even more difficult to turn off, once it's implanted directly into the user's body. At that point you'll not even know who might be recording you. There will be no opting out.

Smartphones May Enable Smart Cities

Bernie Meyerson, The Financial Times, March 4, 2013

Connected and predictive cities will transform city living, says IBM Fellow Bernie Meyerson

People who live in cities are starting to get a lot more information to help plan their daily lives. Smarter City systems are telling residents about traffic congestion, the time the next bus will arrive, the potential for flooding on a block-by-block basis, and a host of other useful updates.

The same wealth of data enables city service leaders to proactively send crews out to mitigate future damage, such as going out to replace water lines before small leaks become catastrophic sinkholes.

This is happening because cities are harnessing the streams of information from sensors to manage municipal systems better. Cities are complex systems of systems that have always generated vast amounts of information. Information flowing in from many small subsystems, such as local flood catch-basins, is integrated with other sources to provide guidance on the overall operation of a city's storm sewer system. This integrated data, from such systems of systems, is then given to city administrators in useful formats that can transform city living for the better.

The need for cities to maximise the efficient use of infrastructure and help residents be more productive without spending vast sums on new facilities is driving this process. The more efficient use of data and predictive analytics promises to help accomplish that goal.

In some cases, cities have moved well beyond just monitoring events in real time. They have begun to accurately predict future events likely to cause damage, and take action to avoid it. Predicting and altering the future may sound like science fiction, but it is here today and represents the future for urban management.
This feat begins with gathering as much data as possible on the issues that need to be addressed. Using the "internet of things", sensors embedded in a wide array of systems serving the public (traffic signals, buses, trains, parking spaces, water systems) report the status of the system they are monitoring via the Internet.
The number of devices such as these exceeded the number of people on the internet in 2008, and will reach 50bn in 2020, according to an article in the 8th November issue of the Journal of Sensor and Actuator Networks. Cities now gather the critical data required to build the models and predictive abilities to understand and alter the future for the better.

All these devices can constantly send information to central command posts where digital streams of data can be analysed and used to alert emergency responders or divert traffic in case of trouble.

Sensors that instantly detect a broken down subway car on a busy track can give repair crews valuable extra minutes of work time to correct matters. However, a true game-changer is the use of sensors to show that a bearing on a train car is running warmer than usual, allowing analytics software to alert maintenance workers and eliminate an impending problem long before a dangerous failure occurs.

IBM, has been using predictive analytics to help cities function better for almost a decade. For example, we have helped city police in Richmond, Virginia deploy their forces to predicted trouble spots and avert crimes. We have helped schools in Mobile, Alabama identify potential dropouts and target them for additional remediation.

In Rio de Janeiro, the city has built a sophisticated central command center in preparation for the World Cup in 2014, and Rio's mayor is counting on advanced analytics to monitor the deluge of data coming in from the region and spot problems long before any one individual could.

Rio currently employs sophisticated weather prediction systems capable of forecasting weather for areas as small as one square kilometer to alert city officials to the potential for heavy rainfall to cause highly localised mud slides in a specific neighborhood located on a steep mountain side. As the predictions are very precise, public safety officials can concentrate their safety teams at key locations, alerting residents to the potential danger and moving to evacuate them if deemed appropriate.

With super-cities of tens of millions of people now emerging, at a scale and complexity never before encountered, this ability to focus resources using deep understanding of the "big picture" becomes ever more critical. In 2010, for the first time in history, more people on the planet lived in cities than lived in rural areas. More importantly, in the next 20 years, some 2bn more people will move into cities, marking one of the most significant migrations in human history.

In some countries, India and China as examples, governments are building new cities to accommodate this transformation. However, on a worldwide basis, most migrants will move to existing cities where they may have personal relationships, prospects of a job, or some other incentive.

In those existing cities, the coming population growth will challenge even the most skilled city managers, and the legacy infrastructure of many such cities is not equipped with the ability to gather the data needed to guide city operations. Despite the seeming lack of data gathering infrastructure, another option exists given the shift from desktop computing to mobile devices and tablets.

Mobile technology has advanced dramatically in the past decade to the point where a lightweight smartphone may gather all manner of data, such as its location, velocity, local temperature and background noise levels. Furthermore, in the near future, additional capabilities are easily envisioned, where sensors could be embedded to detect both particulate and chemical pollution, local lighting conditions, vibration levels, and more.
This will enable a new era of participatory engagement by a city's population, where the very data required to optimise city operations is in part willingly provided by its citizens via smartphone apps.

It also raises complex societal issues that must be addressed as we face this new era, but similarly provides an unprecedented opportunity for the citizenry of a community to take an active role in the betterment of their community, not only in the reporting of critical environmental and transportation data, but in taking an active role in mitigating the very challenges their own data identifies.

It will be a cooperative and informed effort, with information technology enabling societal transformation, to address the growing challenges facing our cities in the coming decades.

Bernie Meyerson is an IBM fellow and vice president for innovation

Less Innovation, More Inequality

Edmund S. Phelps, The New York Times, February 24, 2013

ONE source of the outsize inequalities in America is the dynamism that made economic activity so rewarding. An economy open to new concepts and novel ventures is bound to generate unequal gains. To tax all of those gains would close off the prospects for success that many entrepreneurs need if they are to undertake ambitious ventures — a big mistake. But it would also be a mistake to misunderstand the relation of inequality and innovation. It is less innovation — not more — that has widened inequality in the United States in recent decades.

America's peak years of indigenous innovation ran from the 1820s to the 1960s. There were a few financial panics and two depressions, to be sure. But in this period, a frenzy of creative activity, economic competition and rapid growth in national income provided widening economic inclusion, rising wages for all and engaging careers for most. Innovations gave workers better tools to work with and better products to make, thus lifting their wages. Then this innovation began to retreat, most of it to an area of land along the West Coast. In the early 1970s the rate of indigenous innovation (as measured by its estimated contribution to the rate of growth in labor productivity) dropped by about half — to around 1 percent since then, from about 2 percent before then.

The economist Robert J. Gordon has noted this slowdown in innovation, which he lays to the end of big breakthroughs. My view is that innovation has declined in the everyday processes that businesses tinker with incrementally as they try to become more productive over time. This decline of innovation across many fields — with notable exceptions like Silicon Valley, biotechnology and clean energy — has set back much of the earlier gains in productivity in American history.

It forced a broad devaluation of business assets, including employees. Wage restraint and reduced hiring followed, especially in the heavy manufacture of capital goods. As a result, wages of workers on the low rungs of the ladder slowed more than the wages of those in the middle, and the rising gap leveled off only in the 1990s. Unemployment rates tend to rise and fall in roughly equal proportion at all rungs of the ladder, and that happened between 1973 and 1985. (Over that time, the rate for white men went from 4.3 to 6.2 percent, for black men from 9.4 to 15.1 percent.) But the rise in unemployment was of greater consequence to those on the bottom rung, since their economic precariousness had been higher to start with. This is the heart of the inequality story.

The gap between the less advantaged and the more advantaged widened as the gap between high-school graduates and dropouts and Americans with college and graduate degrees rose. Many of the more advantaged could opt to retire on their resources, which fewer of the less advantaged could do. No wonder that between the mid-1970s and the mid-1990s, the labor force participation rate of white men — who were already relatively advantaged — drifted down while, in contrast, the participation rates of black and Latino men did not. (The rise of female participation rates, white and nonwhite, is another story.)

The inequality gaining attention recently is different in scale but not kind. In my 1997 book "Rewarding Work: How to Restore Participation and Self-Support to Free Enterprise," I wrote that wages had declined in the middle of the income distribution — "approximately at the border between the working class and the middle class" — relative to the affluent.

In short, there has been a widening between the middle and the top in both employment and wages. This "decompression," as some economists call it, began swelling with the return, around 2004, of the meager innovation and slow growth that had plagued the economy since the 1970s until the brief respite offered by the Internet boom that began in the mid-1990s.

The question that confronts policy makers is what steps to take.

There has been a drumbeat for investment in infrastructure. Advocates in business and government assert that such projects would create more work while they last and leave productivity higher in the end — though maybe not high enough to earn the revenue to cover the cost. The impact on jobs is clear over the short run. Economists have been sifting data for evidence that cities and states have won measurable gains in productivity from their capital projects. Whatever the answer, the debate has missed the point that the government will have to keep on finding new projects as old ones are completed. Such an endless series of projects will run into the law of diminishing returns. And even if returns hold up, bricks and mortar are not a solution to the decline in dynamism that — largely if not wholly — lies behind the slowdown in innovation.
Two ideas about how to revive the nation's dynamism are much in discussion.

Some observers attribute the bulk of the American economy's innovation — new products and new methods — to commercial applications of recent scientific advances. This pool of scientific advances, the argument goes, has pretty much run dry, with little replenishment in recent years, and so American innovation has run out of fuel. These scientists then say that stepping up government funding for scientific research could refill the pool, creating new possibilities for innovation into the indefinite future.

But this thesis is based on a mistaken premise. There is no evidence that innovating in America is or has been tethered to scientific advances. Some historians find that innovations largely ran ahead of scientific advances in the 19th century. The myriad new products of recent decades were mostly created by new commercial ideas and tinkering, not by new scientific advances.

A second approach, advocated by some economists and policy makers, is the explicit adoption of an industrial policy. It is argued that the government can spark innovation in the private sector by providing finance for development and marketing of new products or methods in companies or industries that offer promise, at least in the government's view, of boosting innovation. President Obama has implicitly endorsed such an approach. In his State of the Union address last month, he vowed to make America "a magnet for new jobs and manufacturing," favorably cited a "manufacturing innovation institute" in Youngstown, Ohio, that engages in 3-D printing, and announced the start of three more such manufacturing hubs, which will work with the Defense and Energy Departments "to turn regions left behind by globalization into global centers of high-tech jobs."

But this thesis that the government can adequately make the decisions once made by a well-functioning private sector raises serious doubts. Granted, enterprises in the private sector are prone to making mistakes when deciding to develop new products — since feasibility, cost and market reception are all unknown. The difficulty with a national industrial policy is that it places those decisions in the hands of government officials who are remote from the local expertise and insights that companies draw on for dynamic innovation. It is hard enough for venture capitalists and early-stage investors to make the right choices. It is unimaginable that the government can do it well. Besides, there is a moral hazard. Operating an industrial policy runs the risk that government officials — perhaps unconsciously — will do what is best for their political prospects rather than what they might agree was best if they were not directly involved.

What, then, can be done to address the slowdown in innovation and the attendant rise in inequality? There is no question that effective initiatives can be taken to address particular inequalities. Subsidies for employers to hire low-wage workers is one initiative that could be taken to address a particularly serious inequality. But there is no way to restore the sense of equality that prevailed as late as the 1960s without remedying the ills that caused inequalities to widen: the narrowing of high innovation to a handful of industries and the consequent slowing of economic growth to a snail's pace.

I am convinced that a return to the productivity growth and broad economic inclusion of the past will require nothing less than a revival of the high dynamism that underpinned that performance.

The needed revival will require a reform of the financial sector and of the business sector. In the financial sector it is necessary to put an end to the short-term thinking that unduly focuses on hitting quarterly earnings targets instead of aiming for long-range profitability and growth. Financial institutions' addiction to liquidity has made lending to business less attractive, while an addiction to diversified investments has left very few financial institutions willing to make money the old-fashioned way — by lending, or investing in projects for new products and methods.

In the business sector, it is necessary to put an end to infighting in established companies and the shortsightedness of chief executives who know they have only a few years in which to haul in some big bonuses. Better corporate oversight by boards and by government regulators is also essential.

Little of this will happen, however, and any government reforms will be undermined without a wider embrace of the old ethos of imagination, exploration, experiment and discovery. It is that ethos that laid the foundation for the broad-based prosperity of the American middle class in the postwar years, and without its revival, no amount of government intervention can fully mitigate the widening inequality that the slowdown in innovation has helped create.

Edmund S. Phelps, a Nobel laureate in economics, is director of the Center on Capitalism and Society at Columbia University and author of the forthcoming book "Mass Flourishing: How Grassroots Innovation Created Jobs, Challenge and Change."

______________________________________________________________
Stefaan G. Verhulst
Chief Research and Development Officer
The Governance Lab


Wagner Graduate School of Public Service
New York University
The Puck Building
295 Lafayette Street
New York, NY 10012
Tel: (212) 998 7530
Cell: (646) 573 1361

Bill gates on Metrics and Big Data

From the fight against polio to fixing education, what's missing is often good measurement and a commitment to follow the data. We can do better. We have the tools at hand.

Bill Gates, The Wall Street Journal, January 25, 2013

By custom, many Ethiopian parents won't name a child for weeks, in case the baby dies. Sebsebila Nassir, pictured above with a health worker, named her newborn daughter Amira—'princess' in Arabic—on her immunization card the day she was born.

We can learn a lot about improving the 21st-century world from an icon of the industrial era: the steam engine.

Harnessing steam power required many innovations, as William Rosen chronicles in the book "The Most Powerful Idea in the World." Among the most important were a new way to measure the energy output of engines and a micrometer dubbed the "Lord Chancellor" that could gauge tiny distances.

Such measuring tools, Mr. Rosen writes, allowed inventors to see if their incremental design changes led to the improvements—such as higher power and less coal consumption—needed to build better engines. There's a larger lesson here: Without feedback from precise measurement, Mr. Rosen writes, invention is "doomed to be rare and erratic." With it, invention becomes "commonplace."

In the past year, I have been struck by how important measurement is to improving the human condition. You can achieve incredible progress if you set a clear goal and find a measure that will drive progress toward that goal—in a feedback loop similar to the one Mr. Rosen describes.

This may seem basic, but it is amazing how often it is not done and how hard it is to get right. Historically, foreign aid has been measured in terms of the total amount of money invested—and during the Cold War, by whether a country stayed on our side—but not by how well it performed in actually helping people. Closer to home, despite innovation in measuring teacher performance world-wide, more than 90% of educators in the U.S. still get zero feedback on how to improve.

An innovation—whether it's a new vaccine or an improved seed—can't have an impact unless it reaches the people who will benefit from it. We need innovations in measurement to find new, effective ways to deliver those tools and services to the clinics, family farms and classrooms that need them.

I've found many examples of how measurement is making a difference over the past year—from a school in Colorado to a health post in rural Ethiopia. Our foundation is supporting these efforts. But we and others need to do more. As budgets tighten for governments and foundations world-wide, we all need to take the lesson of the steam engine to heart and adapt it to solving the world's biggest problems.

One of the greatest successes in terms of using measurement to drive global change has been an agreement signed in 2000 by the United Nations. The Millennium Development Goals, supported by 189 nations, set 2015 as a deadline for making specific percentage improvements across a set of crucial areas—such as health, education and basic income. Many people assumed the pact would be filed away and forgotten like so many U.N. and government pronouncements. The decades before had brought many well-meaning declarations to combat problems from nutrition to human rights, but most lacked a road map for measuring progress. However, the Millennium goals were backed by a broad consensus, were clear and concrete, and brought focus to the highest priorities.

When Ethiopia signed on to the Millennium goals in 2000, the country put hard numbers to its ambition to bring primary health care to all of its citizens. The concrete goal of reducing child mortality by two-thirds created a clear target by which to measure success or failure. Ethiopia's commitment attracted a surge of donor money toward improving the country's primary health-care services.

With help from the Indian state of Kerala, which had built a successful network of community health-care posts, Ethiopia launched its own program in 2004 and today has more than 15,000 health posts staffed by 34,000 workers. (This is one of the greatest benefits of measurement—the ability it gives government leaders to make comparisons across countries and then learn from the best.)

Last March, I visited the Germana Gale Health Post in the Dalocha region of Ethiopia, where I saw charts of immunizations, malaria cases and other data plastered to its walls. This information goes into a system—part paper-based and part computerized—that helps government officials see where things are working and to take action in places where they aren't. In recent years, data from the field have helped the government respond more quickly to outbreaks of malaria and measles. Perhaps even more important, the government previously didn't have any official record of a child's birth or death in rural Ethiopia. It now tracks those metrics closely.

The health workers provide most services at the posts, though they also visit the homes of pregnant women and sick people. They ensure that each home has access to a bed net to protect the family from malaria, a pit toilet, first-aid training and other basic health and safety practices. All these interventions are quite simple, yet they've dramatically improved the lives of people in this country.

Consider the story of one young mother in Dalocha. Sebsebila Nassir was born in 1990, when about 20% of all children in Ethiopia did not survive to see their fifth birthdays. Two of Sebsebila's six siblings died as infants. But when a health post opened its doors in Dalocha, life started to change. Last year when Sebsebila became pregnant, she received regular checkups. On Nov. 28, Sebsebila traveled to a health center where a midwife was at her bedside during her seven-hour labor. Shortly after her daughter was born, a health worker gave the baby vaccines against polio and tuberculosis.

According to Ethiopian custom, parents wait to name a baby because children often die in the first weeks of life. When Sebsebila's first daughter was born three years ago, she followed tradition and waited a month to bestow a name. This time, with more confidence in her new baby's chances of survival, Sebsebila put "Amira"—"princess" in Arabic—in the blank at the top of her daughter's immunization card on the day she was born. Sebsebila isn't alone: Many parents in Ethiopia now have the confidence to do the same.

Ethiopia has lowered child mortality more than 60% since 1990, putting the country on track to achieve the Millennium goal of lowering child mortality two-thirds by 2015, compared with 1990. Though the world won't quite meet the goal, we've still made great progress: The number of children under 5 years old who die world-wide fell to 6.9 million in 2011, down from 12 million in 1990 (despite a growing global population).
Another story of success driven by better measurement is polio. Starting in 1988, global health organizations (along with many countries) established a goal of eradicating polio, which focused political will and opened purse strings to pay for large-scale immunization campaigns. By 2000, the virus had nearly been wiped out; there are now fewer than 1,000 cases world-wide.

But getting rid of the very last cases is the hardest part. In order to stop the spread of infections, health workers have to vaccinate nearly all children under the age of 5 multiple times a year in polio-affected countries. There are now just three countries that have not eliminated polio: Nigeria, Pakistan and Afghanistan. I visited northern Nigeria four years ago to try to understand why eradication is so difficult there. I saw that routine public health services were failing: Fewer than half the kids were getting vaccines regularly. One huge problem was that many small settlements in the region were missing from vaccinators' hand-drawn maps and lists documenting the locations of villages and numbers of children.

To fix this, the polio workers walked through all high-risk areas in the northern part of the country, which enabled them to add 3,000 previously overlooked communities to the immunization campaigns. The program is also using high-resolution satellite images to create even more detailed maps. As a result, managers can now allocate vaccinators efficiently.

What's more, the program is piloting the use of phones equipped with a GPS application for the vaccinators. Tracks are downloaded from the phone at the end of the day so managers can see the route the vaccinators followed and compare it to the route they were assigned. This helps ensure that areas that were missed can be revisited.

I believe these kinds of measurement systems will help us to finish the job of polio eradication within the next six years. And those systems can be used to help expand routine vaccination and other health activities, which means the legacy of polio eradication will live beyond the disease itself.

Another place where measurement is starting to lead to vast improvements is in education.

In October, Melinda and I sat among two dozen 12th-graders at Eagle Valley High School near Vail, Colo. Mary Ann Stavney, a language-arts teacher, was leading a lesson on how to write narrative nonfiction pieces. She engaged her students, walking among them and eliciting great participation. We could see why Mary Ann is a master teacher, a distinction given to the school's best teachers and an important component of a teacher-evaluation system in Eagle County.

Ms. Stavney's work as a master teacher is informed by a three-year project our foundation funded to better understand how to build an evaluation and feedback system for educators. Drawing input from 3,000 classroom teachers, the project highlighted several measures that schools should use to assess teacher performance, including test data, student surveys and assessments by trained evaluators. Over the course of a school year, each of Eagle County's 470 teachers is evaluated three times and is observed in class at least nine times by master teachers, their principal and peers called mentor teachers.

The Eagle County evaluations are used to give a teacher not only a score but also specific feedback on areas to improve and ways to build on their strengths. In addition to one-on-one coaching, mentors and masters lead weekly group meetings in which teachers collaborate to spread their skills. Teachers are eligible for annual salary increases and bonuses based on the classroom observations and student achievement.
The program faces challenges from tightening budgets, but Eagle County so far has been able to keep its evaluation and support system intact—likely one reason why student test scores have improved in Eagle County over the past five years.

I think the most critical change we can make in U.S. K–12 education, with America lagging countries in Asia and Northern Europe when it comes to turning out top students, is to create teacher-feedback systems that are properly funded, high quality and trusted by teachers.

And there are plenty of other areas where our ability to measure can improve people's lives in powerful ways—areas where we are falling short, unnecessarily.

In poor countries, we still need better ways to measure the effectiveness of the many government workers providing health services. They are the crucial link bringing tools such as vaccines and education to the people who need them most. How well trained are they? Are they showing up to work? How can measurement enable them to perform their jobs better?

In the U.S., we should be measuring the value being added by colleges. Currently, college rankings are focused on inputs—the scores and quality of students entering college—and on judgments and prejudices about a school's "reputation." Students would be better served by measures of which colleges were best preparing their graduates for the job market. They then could know where they would get the most for their tuition money.

In agriculture, creating a global productivity target would help countries focus on a key but neglected area: the efficiency and output of hundreds of millions of small farmers who live in poverty. It would go a long way toward reducing poverty if we had public scorecards showing how developing-country governments, donors and others are helping those farmers.

And if I could wave a wand, I'd love to have a way to measure how exposure to risks like disease, infection, malnutrition and problem pregnancies impact children's potential—their ability to learn and contribute to society. Measuring that could help us quantify the broader impact of those risks and help us tackle them.
The lives of the poorest have improved more rapidly in the past 15 years than ever before. And I am optimistic that we will do even better in the next 15 years. The process I have described—setting clear goals, choosing an approach, measuring results, and then using those measurements to continually refine our approach—helps us to deliver tools and services to everybody who will benefit, be they students in the U.S. or mothers in Africa. Following the path of the steam engine long ago, thanks to measurement, progress isn't "doomed to be rare and erratic." We can, in fact, make it commonplace.

Important Computing Development!

Scientists have stored audio and text on fragments of DNA and then retrieved them with near-perfect fidelity a technique that eventually may provide a way to handle the overwhelming data of the digital age.They later were able to retrieve them with 99.99% accuracy. 

See:
Naik, Gautam. "Storing Digital Data in DNA." The Wall Street Journal, January 24, 2013.


Cookson, Clive. "Scientists Look to DNA for Data Storage." The Wall Street Journal, January 23, 2013. 

Goldman, Nick, et al. "Towards Practical, High-Capacity, Low-Maintenance Information Storage in Synthesized DNA." Nature, January 23, 2013.  

Vivek Kundra: Release Data, Even If It's Imperfect

Joel Schectman, The Wall Street Journal, January 15, 2013

Vivek Kundra, former CIO of the federal government, says organizations will be more innovative if their data is rapidly released and shared, even if it’s imperfect.

Real-time data sharing promotes competitiveness and accountability, Kundra told CIO Journal Editor Michael Hickins at the Wall Street Journal CIO Network conference in San Diego. Soon after joining the federal government in 2009, Kundra created a Web-based tool that allows the public to track the progress of federal IT projects. “The default is that people will go after things that may not be 100% accurate,” Kundra said. “My view is it’s much, much better for the government to put out data that’s not 100% accurate, then to hold it in a secretive and opaque way.”

Kundra, now an emerging markets chief at Salesforce.com , said the release of government data is helping the private sector create a new wave of innovative apps, like applications that will help patients choose better hospitals. Those apps are built atop anonymized Medicare information. Kundra says he had a simple litmus test for assessing the risk of releasing data, which may include imperfections. “Assessing the risk, I asked a very simple question: were we using this data to make public policy decisions or investments. And inevitably, the answer would be yes,” Kundra said. “Then the question was, we’re using it, but why can’t let the American people see it? My view was let’s put it out there because we have to be able to trust that there’s somebody much smarter than us…it becomes a feedback loop that makes the data quality improve, and it makes the processes improve internally.”

Approaching Illness as a Team

The New York Times, December 25, 2012

The Cleveland Clinic, long considered a premier medical system, is gaining new renown for innovation in improving the quality of care while holding down costs.

In its most fundamental reform, the clinic in the past five years has created 18 “institutes” that use multidisciplinary teams to treat diseases or problems involving a particular organ system, say the heart or the brain, instead of having patients bounce from one specialist to another on their own.

The Neurological Institute, for example, provides both inpatient or outpatient care for those 
with strokes and brain tumors, as well as those with epilepsy, multiple sclerosis, depression and sleep disorders, among other conditions.

On a recent visit, we observed one such team, consisting of a neurosurgeon, a neurologist, a neuroradiologist, a neurologist with advanced training in intensive care, a physical and rehabilitation doctor, a medical resident, a physical therapist and a nurse. As they made rounds from patient to patient, they had a portable computer that displayed electronic medical records so that the whole team could see how the patient was doing and plan the course of care for the day.

This team approach can improve the quality of care because all the experts are involved in deciding the best treatment option, which can save time and money. The neurological team, by consensus, has been better able to determine which acute stroke patientsneed a risky and expensive treatment that involves threading a catheter through an artery in the leg up into the brain to destroy a clot. It cut the use of that treatment in half, reducing costs and deaths and improving outcomes.

The Cleveland Clinic has strong leverage to drive such reforms because its staff physicians are salaried and are granted only one-year contracts and subjected to annual performance reviews. Those reviews apply measures of quality, like patient improvement, patient satisfaction and cost reductions. It raises the pay of those who get high marks, reduces the pay of poor performers and even terminates some doctors who fall short. This approach could become more widespread as more hospitals and doctors move toward the salary-based model.

Data analysis to evaluate how well treatments work is also a big part of the medical practice. For instance, the clinic analyzed outcomes for heart surgery patients and found that those who had received blood transfusions during surgery had higher complication rates afterward and a lower long-term survival rate. As a result, it has adopted strict guidelines that limit the use of transfusions.

Such judgments about a treatment’s effectiveness are made by doctors, not by financial administrators, so they tend to be accepted. One analysis found that suturing could be done as well with a $5 silk stitch as with a $400 staple, leading to a big drop in the use of the staples. At the same time, the clinic has also carried out simpler reforms, like improving sterile conditions, which has reduced catheter-related bloodstream infections by more than 40 percent and urinary tract infections by 50 percent. All this has happened in a remarkably short time. Patients seem to like the treatment they get. A federal government survey of patient opinion last fall found that 80 percent of the patients gave the Cleveland Clinic a high rating over all and 84 percent would recommend it to others, well above the state and national averages in the 69 percent to 71 percent range.

Still, many patients are clearly unhappy. A series this year about confusing medical bills and unexpectedly high charges by The Plain Dealer of Cleveland elicited hundreds of patients’ complaints mostly directed against the clinic, because it had reclassified off-campus physician practices and health centers as hospital outpatient facilities and tacked on a “facility fee” for services previously billed at lower doctor’s office rates. The clinic says the added fees are justified because it provides better quality controls and health information technologies in its outpatient units than that available in a typical doctor’s office.

Medicare’s spending per patient at the clinic for an episode of illness that requires hospitalization is below the national median, suggesting that the clinic’s cost-cutting efforts are working. The University HealthSystem Consortium, an alliance of the nation’s leading nonprofit academic medical centers and teaching hospitals, gave the clinic one of its “rising star” awards in September for significant improvements over the previous year in quality, patient safety and clinical effectiveness, an indication that its quality efforts are taking hold.
The Cleveland Clinic’s progress in restructuring itself, said Michael Porter, a Harvard professor who analyzes health care delivery and organizational change, is “light speed” compared with other institutions. The clinic is “a model of where we need to go,” he said, “Not perfect, not done, but far along.”

Longterm perspective: A Timeline of Information History

A Timeline of Information History

This timeline presents significant events and developments in the innovation and management of information and documents from cave paintings (ca 30,000 BC) to the present. To keep recent electronic developments from dominating the listing, only the most significant digital innovations are included.

The Five Forces Shaping the 21st Century

Vivek Ranadivé, Forbes, November 27, 2012

The 21st century didn’t start in the year 2000. It started in 2010, the same way the 20th century began in 1908 with the advent of the automobile. It became the century of highways and freeways, the century of the auto—the American century. Similarly, if you look at what happened a couple of years ago, there were all kinds of crossover points that happened around the same time: more cell phones than landlines, more laptops than desktops, more debit cards than credit cards, more farmed fish than wild fish, more girls in college than boys.


I am dedicated to the belief that if you get the right information to the right place at the right time and in the right context, you can make the world a better place. This is something I call the two-second advantage. In order to achieve that, you need to understand five forces shaping the 21st century.

The first is the massive explosion of data. Look at all the data that was created from the beginning of mankind until a couple of years ago. Since then, ten times as much data has been created. Think about it: 10 times as much data just in the last two years then in all of history. The amount of video content that will go up on YouTube today will be far more than all that Hollywood has created since its inception.

The second force is the rise of mobility. It took 100 years for there to be a billion landlines, 10 years for there to be a billion cell phones and just one year for there to be a billion smart cell phones. We live in a time where everyone on the planet will have one of these smart cell phones.

The third is the emergence of platforms—social, cloud and so on. It used to be that if you wanted to reach an audience of millions or tens of millions, you had to be a large corporation. Today, platforms like YouTube, the iPhone app store and Facebook, allow individuals to reach massive global audiences. Just recently a Korean rapper put his song up on YouTube and it became a global phenomenon within days. That’s the third factor to consider… how do you leverage the platforms that are out there?

The fourth is the rise of Asia. A few hundred years ago, India and China were about two-thirds of the world’s economy. Most economists predict that at some point in the next century, we will revert to that same state. That is because anything that can be done in India and China will be done in India and China. Any 21st-century strategy must take that into account.

The fifth and final force is that Math is trumping Science. If the 20th century was the century of Science, I believe the 21st century will be the century of Math. When I say “Math trumping Science,” I mean that you no longer have to know the why of something, you have to know the what. You simply have to know that if A and B happen, then C will happen; you have to find the pattern. For years, AIDS researchers tried to find the secret of how the AIDS virus mutated and they were not able to. About a year ago, they converted it into a Math problem and put it into a game called Foldit. Within a week, gamers had found the answer—something scientists had not been able to find for years.

Anyone can build their business by understanding and harnessing these five forces and utilizing the two-second advantage. Think outside the box, be passionate, innovative and creative, and you, too, can make the world a better place.

Andrew McAfee : Let the Crowd Fix Your Product's Bugs

Andrew McAfee, Harvard Business Review Blog, November 6, 2012

I'm starting to come to the conclusion that of all the myths businesses and their leaders tell themselves, one of the most harmful is that they know where the expertise is. The more I learn about the results from crowdsourcing and open innovation efforts, the more I believe that the smart strategy is to expose your problems and challenges to as many people as possible and let them show you what they can do. Here's my most recent example of the power of this approach.

The online startup Kaggle assembles a diverse group of people from around the world to work on tough problems submitted by organizations. The company runs data science competitions, where the goal is to arrive at a better prediction than the submitting organization's starting 'baseline' prediction. Results from these contests are striking in a couple ways. For one thing, improvements over the baseline are usually substantial. In one case, Allstate submitted a dataset of vehicle characteristics and asked the Kaggle community to predict which of them would have later personal liability claims filed against them. The contest lasted approximately three months, and drew in more than 100 contestants. The winning prediction was more than 270% better than the insurance company's baseline.

Another interesting fact is that the majority of Kaggle contests are won by people who are marginal to the domain of the challenge — who, for example, made the best prediction about hospital readmission rates despite having no experience in health care — and so would not have been consulted as part of any traditional search for solutions. In many cases, these demonstrably capable and successful data scientists acquired their expertise in new and decidedly digital ways.

Between February and September of 2012 Kaggle hosted two competitions sponsored by the Hewlett Foundation about computer grading of student essays. Improvements in this area are important because essays are better at capturing student learning than multiple choice questions, but much more expensive to grade when human raters are used. So automatic grading of written answers would both improve the quality of testing and lower its cost. Kaggle and Hewlett worked with many education experts to set up the competitions, and as they were preparing to launch some of these people were worried.

The first contest was to consist of two rounds. Eleven established educational testing companies would compete against each other in the first, with members of Kaggle's community of data scientists invited to join in, individually or in teams, in the second. The experts were worried that the Kaggle crowd would simply not be competitive. After all, each of the testing companies had been working on automatic grading for some time, and had devoted substantial resources to the problem. Their hundreds of man years of accumulated experience and expertise seemed like an insurmountable advantage over a bunch of novices.

They needn't have worried. Many of the 'novices' drawn to the challenge outperformed all of the testing companies in the essay competition, and came closer to the consensus score of the human graders than did any of the humans themselves. The surprises continued when Kaggle investigated who the top performers were. In both competitions, none of the top three finishers had any previous significant experience with either essay grading or natural language processing. And in the second competition, none of the top three finishers had any formal training in artificial intelligence beyond a free online course offered by Stanford AI faculty and open to anyone in the world who wanted to take it. And people all over the world did, and learned a lot from it. The top three individual finishers were from, respectively, America, Slovenia, and Singapore.

Businesses certainly know where a lot of the relevant expertise is in any situation, but results like those from Kaggle show me that they certainly don't know where all of it is. As the open source software advocate Eric Raymond famously observed, with enough eyeballs all bugs are shallow. So why not expose your tough problems to as many eyeballs as possible?

Building A Culture Around Big Data

Deanna Glick, AOL Government, October 16, 2012

A report released today by the Partnership for Public Service aims to educate federal managers on how agencies can do just that. The report, From Data to Decisions II: Building an Analytics Culture, examines how to best use data – not anecdotes – to base decisions.

Building on an original report released last November that examined how several federal agencies use data, the new report identifies strategies for how to develop and grow an analytics culture within agencies and incorporate it into how federal workers perform the mission. It profiles seven agencies using analytics to achieve better results and the strategies used in a budget-cutting climate.

Both reports were joint efforts between the partnership and the IBM Center for The Business of Government.

"By sharing compelling stories of how agencies are developing, growing and sustaining their analytics and performance-management approaches, we hope to shed light on key steps and processes that are transferable to other agencies," the report states.

To complete the report, the organizations studies how agencies are using analytics; how they got started; what conditions helped to grow their approaches; what challenges arose and why; and what success looks like.

"We found many parallels in approach across agencies and programs," according to the report. "Driven by budget realities and the push for more data-driven actions, agency managers were examining their programs in a disciplined, comprehensive way to determine how they conduct their business."

The report features details of analytics efforts at agencies within the departments of Homeland Security, Health and Human Services, Interior, Defense and Treasury.

A common successful first step in creating a culture around analytics, researchers found, was agencies tying specific activities directly to what they are intended to achieve and linking them to goals. Focusing on these details help agencies employ a data-driven approach to managing programs, identify critical information to gauge progress and results, and ensure that only those activities that are key or essential to meeting desired results are performed.

To improve airport security, for example, a federal security director with the Transportation Security Administration worked with a team to break down the job of a transportation security officer at checkpoint and baggage areas. After analyzing and brainstorming around specific tasks related to the job, his team identified more than 1,300 knowledge areas, values and skills for a transportation security officer. Based on this analysis, they identified vulnerabilities in security screening and uncovered weaknesses in training, procedures or technology. They then pinpointed what could be improved through training and better application of procedures or policy and where technology could support improved performance.

"By instituting these types of systematic processes, agencies start building analytic cultures so they can look critically at what they do and thoroughly understand how their activities can lead to better results," the report states. "The reward for their meticulous appraisal is the enhanced ability to serve the American public cost-effectively and efficiently."
For more news and insights on innovations at work in government, please sign up for the AOL Gov newsletter. For the quickest updates, like us on Facebook.

IBM's Watson Is Learning Its Way To Saving Lives

A few years ago, IBM’s new computer was a game-playing curiosity. Now Watson is poised to change the way human beings make decisions about medicine, finance, and work.

Jon Gertner, Fast Company, October 15, 2012.

The woman was gravely ill. Her name was Ms. Yamato. Thirty-seven years old, born in Osaka, Japan, she had never smoked, and yet there it was anyway: a spot on her lung.
A doctor had already performed a bronchoscopy and had made the diagnosis of cancer. Then he referred the patient to Mark Kris, an oncologist at Memorial Sloan-Kettering Cancer Center in New York. Seated alongside me in his office on the Upper East Side of Manhattan, Kris is showing me Ms. Yamato's electronic medical record on an iPad. "I'm preparing for the first visit," he explains, swiping the screen to show what that entails. He's interested in running at least two tests on the patient. The first is an MRI, to find out if the cancer has spread to her brain. The second involves a deeper diagnostic regimen. Lung cancer tumors are not all the same; there are thousands of variations. So a test that examines the mutations within a tumor will be crucial, he says. It so happens that cancer patients born in East Asia who have never smoked often have a particular mutation that responds well to a medication by the name of Erlotinib. That may be the case here. One can hope.

Over the past year, IBM executives have come to believe that Watson represents the first machine of the third computer age.

The woman is not real. She happens to be a character within an app that IBM has created for Watson, its new computer. Watson's special talent, its reason for being, is a singular ability to grasp the intricacies of human language and answer exceedingly difficult questions. You may have heard about Watson already. Back in 2007, a group of computer engineers at IBM's research labs in upstate New York began building the machine--named for IBM's founder, Thomas J. Watson--with the goal of creating a question-and-answer technology that would be more authoritative and powerful than anything on the planet. The initial objective of the Watson group was simple: to win in the game show Jeopardy!, something Watson famously achieved in February 2011. Yet the group had a far more important goal: to turn Watson into a business, hopefully one of some scale. So starting in late 2009, a business development team at IBM began holding meetings outside the company in an effort to understand the ultimate worth of this new technology. No doubt it could be a business one day. But what kind of business?

"The first thing that hit us about Watson," recalls John Kelly, IBM's chief of research, "was that this thing could be applied almost anywhere." Early on, IBM executives decided to focus on a field in which Watson could have a notable social impact while also proving its ability to master a complex body of knowledge. The team chose medicine. They believed Watson could help doctors make diagnoses and, even more important, select treatments. Specifically, they thought Watson could be the perfect tool to chart the complex decision trees that cancer specialists like Kris negotiate every day as they weigh treatment options that might involve radiation, surgery, and any of countless chemotherapy drugs. Watson can ingest more data in a day than any human could in a lifetime. It can read all of the world's medical journals in less time than it takes a physician to drink a cup of coffee. All at once, it can peruse patient histories; keep an eye on the latest drug trials; stay apprised of the potency of new therapies; and hew closely to state-of-the-art guidelines that help doctors choose the best treatments. Watson never goes on vacation. And it never forgets a fact. On the contrary, it keeps learning.

This fall, after six months of teaching their treatment guidelines to Watson, the doctors at Sloan-Kettering will begin testing the IBM machine on real patients. The Ms. Yamato app shows how it will work. After Kris inputs the results of her medical tests, Watson begins deliberating. "It's going through its algorithms," Kris says as we stare at the iPad. "It's seeing where the data sends it today." On the screen, a colorful globe spins. In a few seconds, Watson offers three possible courses of chemotherapy, charted as bars with varying levels of confidence--one choice above 90% and two above 80%. "Watson doesn't give you the answer," Kris says. "It gives you a range of answers." Then it's up to Kris to make the call. He regards the options on the screen and wonders how they might change if Ms. Yamato happened to develop a common symptom: hemoptysis, or coughing up blood.

"Let's try that," he says. He inputs the information and shows me the result approvingly. Watson has dropped one drug from the top chemo regimen. That's just what Kris would have done.

To make sense of all this--that is, to gauge both the value of Watson to a hospital like Sloan-Kettering and its potential to change forever the worlds of medicine and business--you could follow two different paths. You might consider Watson's evolutionary promise. Watson can almost certainly generate huge administrative benefits. Already, one large health insurer--Indiana-based Wellpoint--has begun using a Watson computer in its Virginia data center to speed along the authorization for medical procedures. Usually, authorizations are evaluated by a team of trained nurses and can sometimes take weeks to come through. Watsonizing the process would speed it up--a boon for a doctor like Kris, who now must wait while assistants exchange faxes with insurers before he can get clearance for any expensive tests.

Kris shows me what happens when Watson's treatment plan calls for an MRI. A button pops up on his screen to ask for preauthorization. "I just click that," he says, and it's done instantly.

I ask him what if Watson's request is denied.

Kris seems amused by the question. Watson has already consulted the latest medical literature, and it's been trained by the best cancer doctors in the world. "Who is the authority that is going to trump that?" he asks. Insurers balk at paying for unnecessary procedures; Watson's expert opinion essentially guarantees the necessity.

But the more intriguing path is the second one--a consideration of Watson's potential to do something revolutionary. This is the trail that captivates Kris. Eventually, he thinks, Watson could provide any doctor anywhere with the world's best second opinion. A physician in a community hospital in the Midwest, or at a remote medical center in China, could have instant access to everything that the medical field's best oncologists--people like Kris and his colleagues at Sloan-Kettering--have taught Watson. What is more, Watson will be able to excavate facts beyond the ken of Sloan-Kettering's current lineup of specialists. As Kris says, "We could ask Watson: What is the best treatment for this rare condition based on all of Sloan-Kettering's records?" It could then go through several years of cancer cases looking for the most successful outcomes. In time, it could even look at hospital records from around the world. As Manoj Saxena, the IBM executive now in charge of commercializing Watson, tells me: "It's like being able to take a knowledge worker--cancer specialist, nurse, bond trader, portfolio manager, whatever--and equip that person with the best knowledge, and have it available at their fingertips." As Watson evolves, Saxena believes, these knowledge banks will significantly alter how, and how well, humans make decisions.

Within a few years, for instance, Watson may be reaching well beyond oncology to assist patients suffering from any chronic disease and help general practitioners make diagnoses in their offices. Ultimately, Saxena believes, Watson could play an essential role in the diagnosis and treatment of mental health; in the financial services industry, where Citibank is testing it now; and in education. It could become the world's smartest dietitian.

How Watson Works
IBM's Watson computer begins trials in the health care industry this fall. The initial goal is to help oncologists make better decisions for cancer treatment; eventually, the computer will also aid in the diagnosis and treatment of other chronic diseases.

1. For well over a year, the Watson computers have been "trained" in science and medicine. Technicians feed Watson medical textbooks and journals, patient histories, and treatment guidelines.

2. At Memorial Sloan-Kettering Cancer Center in New York, doctors have begun using a Watson appon a tablet to access the computer through the cloud. The doctor logs in to Watson and begins to input data and ask questions.

3. When the oncologist queries Watson about a course of treatment for a lung or breast cancer patient, the computer--with its ability to understand natural language--notes keywords in the query, such as the particular type of cancer and the genomic variant of the tumor.

4. Watson then springs into action, using its massively parallel processors to review millions of pages of text in seconds. It explores the patient's medical history, medications, and other existing conditions. It then combines this information with recent data from the patient's medical tests and may comb through studies of patient groups at Sloan-Kettering who have had similar types of cancer. It also reviews doctors' and nurses' notes, recent medical research, journal articles, and treatment guidelines.

5. Watson then generates hypotheses for treatment. On the tablet app, these appear as separate options with varying levels of confidence. For instance, Watson might score one treatment option--a combination of chemotherapy drugs--with a 95% confidence level, suggesting it would be the most sensible path. It might also highlight options with lower scores as alternative treatment courses. The doctor then weighs the options and makes the call.

Saxena now commands a team of about 200 people who are working to adapt Watson's skills for various IBM clients. He and I are discussing his progress over lunch one day near IBM's upstate New York headquarters when he leans back and tells me that after creating two successful tech startups, both of which he sold (the second to IBM), his current job is far and away the most meaningful endeavor of his life. Those startups, he confides, were exciting, important. "But this," he says of the Watson rollout, "this is stuff that is going to change the course of history."

Over the past year, IBM executives have come to believe that Watson represents the first machine of the third computer age, a category now referred to within the company as cognitive computing. As Kelly describes it, the first generation of computers were tabulating machines that added up figures. "The second generation," he says, "were the programmable systems--the mainframe, the first IBM 360, PCs, all the computers we have today." Now, Kelly believes, we've arrived at the cognitive moment--a moment of true artificial intelligence. These computers, such as Watson, can recognize important content within language, both written and spoken. They do not ask us to communicate with them in their coded language; they speak ours. And perhaps most important, they can learn, so they improve without constant human instruction.

Siri, on the iPhone, might be considered an elementary example. Watson is industrial strength. "Computers do numerical calculations, they move data around, and they've been doing that forever," David Ferrucci, the IBM researcher who commanded the team that built the first Watson computer, tells me one day at IBM's research labs. "When I think about Watson, it's interpreting the information in human terms. It's saying: What does this mean to me? And that's a big deal." Also significant is how Watson renders an answer. Unlike its responses in Jeopardy!, in the real world it will perform as it did for Kris at Sloan-Kettering--by giving not a single solution but a range of probable solutions, each backed up by Watson's evidence and ranked by its level of confidence. In the lingo of computer science, that makes the machine probabilistic rather than deterministic. One might say this trait gives Watson a humanizing glow of humility and diminishes concerns that it marks a stride toward a computer-led dystopia. Watson, in IBM's marketing schema, is here to help with our questions, rather than solve them. In the case of medicine, it--for Watson is not really a he--is here to support doctors, not replace them.

The Watson of today is not precisely the same machine that won in Jeopardy! IBM has fine-tuned its software and algorithms for medical applications (or, in the case of Citibank, financial services applications). Watson has shrunk, too, from a row of about a dozen server racks that would have filled a small bedroom to an assemblage about the size of a double-door refrigerator. But for all the concentrated power, it doesn't look like anything special. Its sleek black servers are standard IBM Power 750s. You could wander around Watson and regard its blinking lights, as I did on a quiet midsummer afternoon at IBM's research labs, and not think something unusual is happening inside it. But there is. The way Watson solves problems--or, rather, the way it looks for answers, simultaneously sending out thousands of inquiries in all directions and then scoring the evidence it collects--is different from how other computers work. One person at IBM likens Watson's process to (1) gathering hundreds or thousands of possible solutions from a vast data bank, (2) pouring them into a giant funnel, (3) stirring with a dash of algorithms, and (4) letting only the best drip out of the bottom.

At the moment, a half-dozen Watsons are scattered around the country. Some are on the premises of IBM clients, as with the insurer Wellpoint, while others are cloud based, which is how hospitals such as Sloan-Kettering will access Watson. "Effectively, there's no limit to how many Watsons there can be," Bernie Meyerson, IBM's VP of innovation, tells me. Watson is a creation of software, not hardware. "That's the beauty of it," he says.

Watson is different from big servers and mainframes in other ways, too. The best computers of today have the extraordinary processing power needed to create, say, complex supply chains for building a new automobile or planning a satellite launch. These machines are good at manipulating the vast amounts of clearly defined data--numbers and facts--known as structured information. But most of the world's information is more ambiguous and less precise and lies beyond their reckoning. "We now have this proliferation of what we call Big Data," Saxena, Watson's business manager, tells me, referring to the flood of information created by our computers, our electronic sensors, and ourselves. "Ninety percent of the world's information was created in the last two years," he says. "But 80% of that 90% is unstructured or semistructured information, like doctor's notes or product reviews on Amazon." This near infinitude also includes tweets, blogs, emails--all the noise and scribble of modern life. So any company that aspired to manage the data of all the world's businesses would today be able to analyze only a small part of it. Watson, though, is a genius at reading unstructured information. And it's precisely this facility that explains why IBM sees such a rich business opportunity here.

It likewise explains why medicine is a logical first choice. While some health information is indeed structured--think of blood-pressure readings or cholesterol counts--the vast majority is unstructured. This cache includes textbooks, medical journals, patient records, and nurse and doctor evaluations. In fact, medicine embodies so much unstructured information that its proliferation has, by the account of many medical professionals, far outstripped the ability of doctors to keep up. Neither better training nor continuing education could ever wholly remedy this problem. When I meet with Herbert Chase, a professor of clinical medicine at Columbia University who consulted with IBM during the early stages of the Watson project, he says it is "not humanly possible" for a busy doctor to keep abreast of the current literature.

One result of information overload is a high rate of misdiagnosis and consequently incorrect treatment. By some estimates, Saxena tells me, 20% of initial diagnoses of cancer are eventually altered. "Imagine the implications of cancer care if there is a one in five chance that for the next six months whatever therapy they're giving you is wrong," he says.
Deciding on a course of treatment is even tougher than making a diagnosis. "It's still possible for a doctor to know the ways that people get sick," says Chase, who is also a kidney specialist. "But what is unmanageable, and what has been for decades, is knowing what the best option is today." Some applications now available to doctors are meant to alleviate this problem; one popular web-based tool is named Isabel. But Watson, in Chase's view, reaches a different level of sophistication. "I'll give you an example of a test we thought up for Watson," he tells me one day in his Manhattan office. "A patient was pregnant, had Lyme disease, and was also allergic to penicillin. And Watson came up with a drug. The first thing I thought was, Watson made a mistake. That drug can't be given to someone allergic to penicillin." But Chase was wrong, not Watson. "My knowledge was about five years old," he says. "And in the past couple of years, all the muckety-mucks had reviewed all the studies and had concluded yes, you can give that drug to someone who's allergic to penicillin."
To Chase, this proves a point: If you're a patient, you don't want to believe your doctor doesn't know everything. But he or she doesn't, and can't. At its best, the dispensation of treatment is inefficient today. "At its worst," Chase says, "it's subpar, incorrect, wrong therapy," and doesn't reach the standard of care to which his profession aspires. "As you can imagine," he adds, "this is not something we like talking about."

Last year, IBM turned 100 years old, which sets it apart from West Coast counterparts like Amazon, Apple, Google, HP, and Microsoft--all younger and ostensibly the tech world's leading innovators. To delve into IBM's recent research, though, is to wonder if our perception of technological leadership sometimes suffers from the distortions of branding and familiarity. We use iPhones and search engines and laser printers every day. But IBM's technologies are lodged deeper within the infrastructure of daily life; you're tapping into them whenever you send an email, for instance, or log on to a website. IBM has been granted more patents than any other company in the world for 19 years in a row. Yet since getting out of the laptop business in 2004, it has not produced a single product that it sells directly to the consumer.

If you're a patient, you don't want to believe your doctor doesn't know everything. But he or she doesn't, and can't.

To understand how Watson figures into the company's culture of ideas, or to see how it represents the kind of large-scale innovation that arguably lies beyond the capabilities of any startup, it helps to understand what the company actually does these days. IBM has operations in 172 countries and an organizational chart that resembles a vast Soviet bureaucracy. It employs about 433,000 men and women. Though IBM still sells hardware--big mainframe computers, silicon chips, and supercomputers--mainly it makes money selling software and consulting services to businesses and governments. The company's strategy has been validated of late by its performance: IBM's stock price has been on an upward trek for the past five years, and its winning streak has attracted the likes of Warren Buffett, who last year decided the company merited an investment of $10.7 billion. Meanwhile, as one of the few global titans to invest staggering sums on R&D ($6 billion to $7 billion a year), IBM maintains one of the world's last great industrial laboratories. At its main research center in Yorktown Heights, New York, a jet-age dream of glass curtain walls and rusticated stone designed by the Finnish-American architect Eero Saarinen, IBM employs the bulk of what is likely the world's largest mathematics department, with 300 members. If you're looking for a new PC design, you're out of luck here. But if you're shopping around for a new or better algorithm, IBM can build you one.

Not everyone is impressed by the direction of IBM's management. A relentless focus on earnings and cost cutting has led to a significant offshoring of domestic jobs, and a vocal corps of disillusioned or laid-off IBMers regularly take to the web to lament that the company's best days are behind it. IBM has also had its share of technological stumbles, apparently bungling several high-profile government contracts in recent years (in Texas and Indiana, for example) that left the company embroiled in disagreements with unhappy clients. And though these flare-ups may be uncommon, the company otherwise rarely quickens the pulse, with a long-standing reputation for being slow, steady, reliable, and maybe a little dull. IBM doesn't have big growth spikes or ballyhooed product launches; rather, it has plodding, long-term client contracts built around its ability to help optimize, say, a company's global IT services or a public utility's electrical grid. The corporation moves along like a supertanker. "IBM's annual revenue base is huge--$100 billion," says Toni Sacconaghi, a technology analyst for Sanford C. Bernstein. "So to move the needle is tough. It's hard to find big new products."

The managers and engineers keep looking anyway. One way IBM tries to infuse the troops with a sense of mission is through its periodic attempts to create for itself a Grand Challenge, such as the construction of Deep Blue, a chess-playing computer, or, more recently, Watson. The Grand Challenges are focused and expensive efforts--IBM will not verify Watson's cost, but estimates put the sum between $100 million and $1 billion--to push the company beyond the competition.

Watson's origins can arguably be traced back some years to a more modest annual initiative IBM calls the Global Technology Outlook, or GTO. Anyone at IBM can contribute to the outlook, and most of the results are eventually made public. The GTO tries to identify future business opportunities by putting a spotlight on various technology trends. A while ago, the IBM outlook pointed to analytics as a potentially huge field. Not long after, then-CEO (and current chairman) Sam Palmisano green-lighted IBM's acquisition of about $16 billion in smaller companies that had computer technologies to do this kind of work--essentially, to comb through vast stores of data, both structured and unstructured, and help extract nuggets from the global corporate babel.

Like Big Data or cloud computing, analytics is one of those contemporary catchphrases that everyone talks about but no one pauses to define. Bernie Meyerson, IBM's VP of innovation, argues that the great promise of analytics is not just to spot trends or glean information for boosting sales but to use computers and software to change the future. "Analytics is the capability to see what no human can," he says. Recently, at a public event, Meyerson was asked if IBM missed out by not building a tablet to compete with the iPad. He responded that as part of its Smarter Cities Initiative, IBM had just spent several years gathering all of the data on car transportation in Singapore; it then fed the data into a model it had built to predict the time and location of traffic jams. "We know from history what happens in Singapore if you slow the lights down in one direction by three seconds, and how to tweak the model so the jam never happens," he told his questioner. "And so there will be a traffic jam that never occurs because we can predict what happens 20 minutes from now, because we can take enough Big Data and crunch it, and do analytics on it. So we're predicting the future, and changing it. And you're asking me if I'm worried about a tablet?"

Watson, too, fits into Meyerson's conception of analytics, though it aims to change not the future of a traffic jam but of illness and investing. And by all indications, that tantalizing promise is not lost on the business community. "I have my shoulder against the door," Saxena tells me. He means he is turning clients away--something I heard from several other sources, too--until IBM executives feel confident Watson has proved its credibility at places like Wellpoint and Sloan-Kettering. Saxena seems certain that Watson will be a multibillion-dollar business, though he will only go so far as to say that by 2015, IBM will have annual revenues of about $16 billion from its analytics portfolio, of which Watson will be a part. When I put the question of Watson's potential to John Kelly, IBM's chief of research, he says: "It's like asking, at the very beginning, How big will the PC industry be?"

Kelly notes that the business model for Watson is still to be determined. He isn't sure whether selling Watson as a computer or marketing it as a service will make the most sense. But he feels he has time to decide. None of IBM's competitors, more than a year after the Jeopardy! victory, has announced a Q&A technology like Watson. "I think we have a huge lead," Kelly tells me. "When people realize this is not a one-off game machine but a new era of computing, then you'll see other companies tripling down to catch up."

I asked a number of people, both within IBM and outside of it, whether other organizations could have built this machine first. The consensus was probably not. The reasons did not precisely connect to IBM's technological capabilities--Google and Microsoft have plenty of computer prodigies in their ranks too. Rather, it was the combination of assets at IBM that made the difference. The company had its vast corporate lab, huge sums it was ready to invest, a profound expertise in hardware as well as software, and a collaborative culture that brought in lots of help from academia. And crucially, it had its business clients. In this respect, being a company that doesn't cater to consumers has advantages. Watson is only as bright as its teachers. Without the staff at Sloan-Kettering, where doctors like Mark Kris teach it oncology, Watson would not be nearly so smart. In fact, it might be kinda dumb. Or it might get all sorts of things wrong, like Siri does, except you'll be looking not for a pizza parlor but for a tumor.

From the start, the team that originally built Watson under David Ferrucci has worked out of a big room on the second floor of IBM's Hawthorne Labs in Westchester County, New York. Hawthorne is a large glass cube of a building situated about 30 miles north of New York City. Inside the Watson work space are five fake wood-grained tables, each home to a group of computer engineers who sit around and alternately immerse themselves in their screens or break to discuss coding with a neighbor. The mood here is sober. The staffers bring water bottles, not junk food. These aren't the unlined faces you'll see at a startup. Indeed, Ferrucci, who sits off to the side, is a suburban dad who looks like he'd be just as comfortable standing in front of a grill with a basting brush as he is overseeing his team. The walls here are covered with huge whiteboards crammed with the hieroglyphics of computer science. Overhead lights cast the room in gloomy fluorescence. The place has the neglected feel of a finished basement in a 1970s-era subdivision.

In early fall, the Watson team, now about 45 strong, began moving its work to a gleaming new space in IBM's main Yorktown Heights research laboratory--a promotion that reflects their importance as they support Saxena's much larger business development group while simultaneously working on the next iteration of Watson, known as Watson 2.0. One of the team's goals is to make Watson adaptable enough so that it doesn't require several dozen people spending a year to get it ready for every new application, such as medicine or financial services. But a more immediate project is to help Watson through the U.S. Medical Licensing Examination, the complex test all med-school graduates must take before practicing. If it passes, says Ferrucci, "that doesn't mean I can have a computer be a doctor." But IBM would gain what he calls "a crisp metric" that proves Watson has a real proficiency in medicine. The credential would no doubt help Watson's standing with health insurers, doctors, and patients, too. Passing the licensing exam is a difficult task--far harder than winning at Jeopardy!--but in early September, Ferrucci seemed pleased by the results. The computer is doing "interestingly well," he said. He sounded confident that Dr. Watson will ace the test by year's end.

Harder to intuit is how soon afterward Watson will infiltrate society. When I ask Jaime Carbonell, a computer science professor at Carnegie Mellon, he says he has no doubt the impact of Watson will be significant. "But I don't think there will be one moment of, 'Now we have it and yesterday we didn't,'" Carbonell remarks. "It will take time to permeate. Like cell phones, which were big, clumsy things you could barely carry at first." Was there a year, or month, or day, he asks, when cell phones began to change the world? "I can't think of when that was," he says. "But now we can't do without them."

Such is the course of technology: Electronic tools initially available only to the elite grow ever faster, smaller, cheaper. Kelly tells me he believes that eventually Watson will shrink to the size of a handheld device. Randy Katz, a computer science professor at UC Berkeley, sees a more approachable Watson, too. "Can the person in the street ask Watson a question now? No, he can't," says Katz. "But in five or 10 years, will there be systems like that--like Siri, but much better? I think the answer is yes."

In many of my conversations at IBM, the talk often drifts to applications of Watson. All sorts of intriguing scenarios are presented to me--for instance, that Watson will soon analyze not just words but images, such as MRIs and EKGs. Or it will diagnose a spider bite on a child's arm in a crop field in Africa, transmitted via smartphone by his worried father to a U.S. hospital. One afternoon, Saxena suggests this one: When you think you're coming down with the flu, Watson will be able to discern, before you even arrive at the doctor's office, that it might be a ragweed allergy, based on your medical record (you've had the same symptoms twice before at this time of year); your symptoms (gleaned from the insurance claim and diagnostic information in journals); and recent news (it just read an article in the Austin-American Statesman on a ragweed outbreak near your hometown).

It all sounds amazing. It's also speculative. Watson has not yet saved a life or a dollar of medical costs, or added anything, really, to IBM's bottom line. It has not yet faced its resistors--doctors who may find the technology objectionable and slow its adoption. It has not yet, as Saxena believes it will, changed the course of history. It has only won a television game show.

Still, Saxena predicts the computer will begin to scale up dramatically late next year. "By then," he says, "we will have built the technology, demonstrated it, built the tooling and methods around it. We will have the recipe book, and then we'll just push it out." But he will only have reached the end of Watson's beginning.

A version of this article appears in the November 2012 issue of Fast Company.