Big Data Challenge to Transform Health Care Delivery

Health Leaders Launch Big Data Challenge to Transform Health Care Delivery

National Contest Calls on Health Care Organizations to Identify Roadblocks to Implementing New Care Models

April 16, 2013

http://bipartisanpolicy.org/news/press-releases/2013/04/health-leaders-launch-big-data-challenge-transform-health-care-delivery

Media Contact:

Ashley Berrang
aberrang@bipartisanpolicy.org
202-637-1456

Pete Simpkinson
simpkinp@advisory.com
202-266-6645

Washington, D.C. – Today, the Bipartisan Policy Center (BPC), Heritage Provider Network (HPN), and The Advisory Board Company launched the Care Transformation Prize Series, a national contest to address the most daunting data problems U.S. health care organizations face as they implement new delivery system and payment reforms.

The goal of this big data challenge is to help health care organizations more effectively use data to drive improvements in health care cost and quality. The series was announced at a BPC-hosted event today that featured a forward-thinking discussion on the strategies that providers, health plans, and states are using to harness data to help Americans lower their health care costs and improve care.

The Care Transformation Prize Series is the latest competition sponsored by Dr. Richard Merkin, President and CEO of HPN, in an ongoing effort to spur innovations that improve quality and reduce inefficiencies as organizations work to implement new delivery system and payment reforms. This prize series expands upon lessons learned from past HPN data competitions to ensure that we are identifying the questions that will best support transformations in care.

“Today, we call on the entire health care community – particularly those serving on the front lines in providing care – to submit the most difficult data challenges facing their organizations as they implement new models of care,” said Dr. Merkin.

“Over the next two years, this contest will inspire innovative solutions for health care organizations as they learn how to leverage large data, particularly clinical data sets, for reform efforts.”

A Prize Board made up of prominent leaders from many sectors of health care will determine which challenges will be addressed by competing teams of leading data scientists.

Heritage Provider Network and The Advisory Board Company will offer at least three quarterly prizes of $100,000 to the teams that develop the best solutions to the selected challenges. The winning algorithms will then be made available to health care organizations and the public.

Every day, health care providers are being asked to improve care while lowering costs. Yet few organizations have the knowledge, expertise and resources to create and utilize data-driven models of care. “To be successful, providers need to access and analyze large electronic data sets, which are now becoming more available through investments in health IT,” said BPC Health Innovation Initiative Director Janet Marchibroda. “This contest is an innovative way to create ground-breaking solutions to some of the toughest problems facing health care organizations today.”

Faced with continuing concerns about rising health care costs and uneven quality, the federal government, states, and private sector are rapidly adopting new models of delivery and payment reforms that promise to improve the quality and cost-effectiveness of health care.

“The Advisory Board Company is privileged to be providing the data analytics platform with which our unparalleled network of member organizations is delivering better care, more efficiently,” said Richard Schwartz, EVP at The Advisory Board Company. “We’re excited to tap into the broader community of data and care experts to help accelerate these efforts.”

For more information about The Care Transformation Prize Series or to submit a question, please visit http://www.caretransformationprize.com.

 

 

http://www.technologyreview.com/news/513016/how-wireless-carriers-are-monetizing-your-movements/

 

BUSINESS NEWS

·         8 COMMENTS

How Wireless Carriers Are Monetizing Your Movements

Data that shows where people live, work, and play is being sold to businesses and city planners, as mobile operators seek new sources of revenue.

·         By Jessica Leber on April 12, 2013

WHY IT MATTERS

Better information about how people move around the physical world could lead to more efficient and greener cities and help small businesses.

Remapping Cities: A San Francisco startup’s software shows real-world analytics about who goes where.

Wireless operators have access to an unprecedented volume of information about users’ real-world activities, but for years these massive data troves were put to little use other than for internal planning and marketing.

This data is under lock and key no more. Under pressure to seek new revenue streams (see “AT&T Looks to Outside Developers for Innovation”), a growing number of mobile carriers are now carefully mining, packaging, and repurposing their subscriber data to create powerful statistics about how people are moving about in the real world.

More comprehensive than the data collected by any app, this is the kind of information that, experts believe, could help cities plan smarter road networks, businesses reach more potential customers, and health officials track diseases. But even if shared with the utmost of care to protect anonymity, it could also present new privacy risks for customers.

Verizon Wireless, the largest U.S. carrier with more than 98 million retail customers, shows how such a program could come together. In late 2011, the company changed its privacy policy so that it could share anonymous and aggregated subscriber data with outside parties. That made possible the launch of its Precision Market Insights division last October.

The program, still in its early days, is creating a natural extension of what already happens online, with websites tracking clicks and getting a detailed breakdown of where visitors come from and what they are interested in.

Similarly, Verizon is working to sell demographics about the people who, for example, attend an event, how they got there or the kinds of apps they use once they arrive. In a recent case study, says program spokeswoman Debra Lewis, Verizon showed that fans from Baltimore outnumbered fans from San Francisco by three to one inside the Super Bowl stadium. That information might have been expensive or difficult to obtain in other ways, such as through surveys, because not all the people in the stadium purchased their own tickets and had credit card information on file, nor had they all downloaded the Super Bowl’s app.

Other telecommunications companies are exploring similar ideas. In Europe, for example, Telefonica launched a similar program last October, and the head of this new business unit gave the keynote address at new industry conference on “big data monetization in telecoms” in January.

“It doesn’t look to me like it’s a big part of their [telcos’] business yet, though at the same time it could be,” says Vincent Blondel, an applied mathematician who is now working on a research challenge from the operator Orange to analyze two billion anonymous records of communications between five million customers in Africa.

The concerns about making such data available, Blondel says, are not that individual data points will leak out or contain compromising information but that they might be cross-referenced with other data sources to reveal unintended details about individuals or specific groups (see “How Access to Location Data Could Trample Your Privacy”).

Already, some startups are building businesses by aggregating this kind of data in useful ways, beyond what individual companies may offer. For example, AirSage, an Atlanta, Georgia, a company founded in 2000, has spent much of the last decade negotiating what it says are exclusive rights to put its hardware inside the firewalls of two of the top three U.S. wireless carriers and collect, anonymize, encrypt, and analyze cellular tower signaling data in real time. Since AirSage solidified the second of these major partnerships about a year ago (it won’t specify which specific carriers it works with), it has been processing 15 billion locations a day and can account for movement of about a third of the U.S. population in some places to within less than 100 meters, says marketing vice president Andrea Moe.

As users’ mobile devices ping cellular towers in different locations, AirSage’s algorithms look for patterns in that location data—mostly to help transportation planners and traffic reports, so far. For example, the software might infer that the owners of devices that spend time in a business park from nine to five are likely at work, so a highway engineer might be able to estimate how much traffic on the local freeway exit is due to commuters.

Other companies are starting to add additional layers of information beyond cellular network data. One customer of AirSage is a relatively small San Francisco startup,Streetlight Data which recently raised $3 million in financing backed partly by the venture capital arm of Deutsche Telekom.

Streetlight buys both cellular network and GPS navigation data that can be mined for useful market research. (The cellular data covers a larger number of people, but the GPS data, collected by mapping software providers, can improve accuracy.) Today, many companies already build massive demographic and behavioral databases on top of U.S. Census information about households to help retailers choose where to build new stores and plan marketing budgets. But Streetlight’s software, with interactive, color-coded maps of neighborhoods and roads, offers more practical information. It can be tied to the demographics of people who work nearby, commute through on a particular highway, or are just there for a visit, rather than just supplying information about who lives in the area.

“If you’re a retailer and you are on someone’s commute path, they may pass by you 600 times a year,” says founder and CEO Laura Schewel, a transportation researcher who is also finishing her PhD at the University of California, Berkeley. “If they never come in, that’s a big missed opportunity.”

Streetlight’s work shows why such data has the potential to improve city planning. One of the company’s first customers is the Oakland Business Development Corporation, which is trying to attract businesses to a city with a reputation for crime and poverty, says Schewel. Streetlight’s data shows the patterns by which droves of wealthier people commute through the city from outlying suburbs on their way to San Francisco, and it also shows that young people visit Oakland for the growing nightlife scene. The group’s goal is to convince national chains that don’t know Oakland well and might look only at household demographics that it makes sense to open up a shop there, and then help them chose ideal locations.

Of course, all the companies involved are aware that people may have privacy concerns, and they wrestle with the challenge of conveying what “anonymous” and “aggregated” data means to people who are increasingly aware that they are carrying around a tracker in their pocket. (Verizon Wireless also does allow its customers to opt out of the program).

The companies appear to be taking many precautions to protect privacy. In Streetlight’s case, the data is obtained in batches and algorithms are used to tease out patterns that help it match locations to other sources of data, such as DMV records or U.S. Census information. The company says it can’t track any individual device on a map or target ads to any one specific person with the data it buys. Schewel also emphasizes that the data is at least a day, if not a month, old and its software doesn’t report statistics for groups of less than 15 devices.

A more important question, probably, is how people will balance their privacy concerns with the benefits of making their location data, or other mobile data about them, more available. Research and experience suggest that in practice most people don’t mind, or don’t care as much as they think they do about privacy (see “The Value of Privacy”).

There will be many uses for such data. Some will involve the race to become the platform that provides the best mobile ads and deals. Verizon, for example, now has a pilot effortwithin its Precision Market Insights program that allows customers to opt in to (rather than opt out of) giving Verizon permission to share more information about who and where they are, including Web browsing and app usage data. In return, they might receive deals and offers on their phone from marketers who want to reach them.

The data could also save budget-strapped governments some money. In California, the state is undertaking an unusual pilot experiment by licensing GPS data from navigation providers rather than repairing its crumbling network of sensors and detectors in the state’s roadways. Alexandre Bayen, a researcher at the University of California, Berkeley, who is helping put together the request for bidders, believes markets that give governments access to mobility data will eventually become common.

Schewel, who came to the idea for Streetlight as she was trying to give green transportation advocates better data about who uses the roads, also views her company as a long-term way to create more efficient cities by helping businesses locate closer to their customers and reducing shopping trips.

“A green transportation startup, where the core value proposition of the company is saving transportation fuel, is not a scalable idea,” she says. “But a marketing analytic startup that spits out mile reduction as its side effect? I think that is the most powerful way to be an agent for change.

The personal computer is in crisis

Computer Sales in Free Fall

Quarterly Shipments Drop 14% as Windows 8 Fails to Stem Advance of iPads

By IAN SHERR And SHIRA OVIDE

http://online.wsj.com/article/SB10001424127887324695104578414973888155516.html?mod=WSJ_hp_mostpop_read

An employee at an Electric Avenue store in Miami shows a Dell computer to potential buyers.

The personal computer is in crisis, and getting little help from Microsoft Corp.'s Windows 8 software once seen as a possible savior.

What's next for the personal-computer industry, following startling numbers showing a sharp decline in new sales? Spencer Ante joins Markets Hub.

Research firm IDC issued an alarming report Wednesday for PC makers such as Dell Inc. and Hewlett-Packard Co., saying world-wide shipments of laptops and desktops fell 14% in the first quarter from a year earlier. That is the sharpest drop since IDC began tracking this data in 1994 and marks the fourth straight quarter of declines.

Gartner Inc., a rival research firm, estimated global shipments sank 11.2%, which it called the worst drop since the first quarter of 2001. Gartner blamed the rise of tablets and smartphones, which are sapping demand for personal computers.

Shipments of personal computers have plummeted in spite of Microsoft’s introduction of Windows 8 – which was designed to make PCs appear more like tablets. As improvements in PC technology become more incremental, consumers are increasingly gravitating toward mobile devices.

Microsoft, whose software is on a majority of the world's PCs, last fall introduced Windows 8, a completely overhauled operating system with touch-screen capabilities.

But there is little sign that buyers are responding. In a surprisingly harsh assessment, IDC said Windows 8 hasn't only failed to spur more PC demand but has actually exacerbated the slowdown—confusing consumers with features that don't excel in a tablet mode and compromise the traditional PC experience.

Bloomberg

Shoppers at a Best Buy store in Orem, Utah, check out laptops running Microsoft's Windows 8.

More

• Heard: Microsoft Can't Keep Up in a Mobile World
• Microsoft Plans 7-Inch Tablet

"The reaction to Windows 8 is real," said Jay Chou, an IDC analyst, about the negative sentiment.

"The PC market is evolving and highly dynamic," a Microsoft spokeswoman said. "Along with our partners we continue to bring even more innovation to market across tablets and PCs."

Mr. Chou said not only has Windows 8 failed to attract consumers, but businesses are keeping their distance as well. Chief information officers at several companies echoed his opinion Wednesday.

Ricoh Americas Corp., which replaces about a third of its 17,000 PCs every three years and upgrades to the most current operating system available, said this year it is sticking with Windows 7, released in 2009. Tracey Rothenberger, the company's chief operating officer, said the benefits of switching to the new software aren't worth the effort of training employees to use it.

"I don't think there's anything wrong with [Windows 8]," said Mr. Rothenberger. "But I think there's minimal value in the incremental changes that are there."

Microsoft executives have said Windows 8 would take many months to catch on with consumers and especially with businesses that generate the majority of the company's profits.

Mikako Kitagawa, an analyst at Gartner, wasn't so quick to blame Microsoft for the industry's woes, arguing instead that secular trends such as customer interest in touch-based mobile devices beyond PCs are playing a larger role.

"Windows 8 is the right direction," Ms. Kitagawa said. "It will be a while before Windows 8 is adopted."

H-P, the world's largest PC maker, showed the steepest drop in global shipments in the quarter with a 24% decline, according to IDC. Dell, ranked fourth in world-wide shipments, posted a 11% drop as it makes headlines for grim financial troubles and efforts to take the company private.

Even Apple Inc., whose products ignited consumer interest in mobile devices, may not be immune to the troubles. U.S. shipments pegged it as the third largest PC maker, though its results fell 7.5%, IDC said. By contrast, Gartner said Apple's U.S. computer shipments rose 7.4% from the previous year.

The first quarter's declines come after a particularly troubling run up to last year's holiday quarter, during which industry researchers began sounding alarm bells that the PC market would post its first annual contraction in more than a decade. Meanwhile, tablets like Apple's iPad flew off the shelves.

Many computer-industry executives hoped Windows 8 would spark a revival in their PC and tablet sales. PC makers have mounted a series of responses to the iPad, shipping an array of computers with new looks including portables that convert from tablet mode to a clamshell design with a built-in keyboard.

Microsoft executives have said to make Windows 8 computing devices more appealing, they are urging a wider variety of touch-screen computers and tablets at a broader range of prices than were available over the holiday season.

But consumers don't appear to be responding and now analysts are expecting PC sales to suffer at least through the first half of 2013.

Carrie Soukup, a 32-year old tech support contractor living in Austin, Texas, is among the skeptics about the new software.

As she has been researching a new laptop she plans to purchase this year, she said a PC running Windows 7 was more appealing than the newer designs. "I'm not entirely thrilled with Windows 8," she said. "There's a lot of show I don't see the functionality in."

Other consumers, like Gene Elmer Munson, have held off buying PCs all together as they increasingly rely on mobile gadgets.

Mr. Munson, a 40-year-old human resources-and-finance worker, has a desktop, a laptop and a smartphone and 7-inch tablet powered by Google Inc.'s Android software.

He bought the 7-inch Google Nexus tablet in December, and now finds himself using his laptop less and his tablet more, especially for personal work, including the novels he writes in his spare time.

"If I didn't have a smartphone and a tablet I would have replaced it by now," said Mr. Munson of his H-P laptop. "I don't see myself buying a laptop in the foreseeable future."

Roughly 350 million personal computers are sold each year, but combined sales of smartphones and tablets are dwarfing those PC figures.

People are expected to buy about 919 million smartphones this year, and IDC expects nearly 200 million tablets will be sold this year.

In another blow to Microsoft particularly, sales of tablets powered Windows 8 also haven't been strong. Windows 8 accounted for only 1% of global tablet shipments in 2012, with its share expected to rise to 7.4% by 2017, IDC says.

Stephen Baker, an analyst at NPD Group, said the future for Windows-based machines is looking increasingly bleak. "In a PC form factor you won't see Windows 8 with touch have any impact," he said.

Meanwhile, few businesses are buying many more PCs, and when they are, they aren't buying them with Windows 8—at least not yet.

Some corporate chief information officers say they don't see a compelling reason to upgrade to Windows 8.

Barry Libenson, chief information officer of Land O'Lakes Inc., manages a fleet of around 7,000 PCs, and says the number "is increasing annually, but not by a significant amount."

His company, on a three-year refresh cycle, runs the PCs on Windows 7 and doesn't have plans to migrate to Windows 8 until either the performance of Windows 7 deteriorates, or if a licensing deal with Microsoft were to make cost a nonissue.

Windows 8 "will not be a driver for increased adoption" of PCs, he said.

IDC said companies have also begun extending the life of their computers. The firm noted many companies that used to buy new PCs every three years before the recession have pushed out their purchases to every four or five years these days.

The market's troubles have been exaggerated by the dramatic corporate changes at H-P and Dell, which IDC said magnified negative trends. Rival Lenovo Group Ltd., by comparison, has been able to maintain its shipments worldwide, and actually increased its results in the U.S.

IDC said the market's steeper-than-expected contraction may spur the firm to revise its worldwide shipment expectations even lower than the gloomy outlook of a 1.3% contraction for the year. IDC's data doesn't include shipments of iPads or any computers whose screens can be detached from their keyboards. Gartner said some hybrid laptops are included, but they still represent very small numbers.

—Clint Boulton
and Michael Hickins contributed to this article.

Write to Ian Sherr at ian.sherr@dowjones.com

 

Big data could mean big savings in health care – but here’s what has to happen first

By Ki Mae Heussner

Apr. 5, 2013 - 3:29 PM PDT

Summary:

A new report from McKinsey estimates that big data could save the health care industry up to $450 billion, but it has to overcome a few obstacles first.

Properly exploiting big data in health care could mean up to $450 billion in savings health care organizations and consumers according to a recent report from consulting firm McKinsey. But don’t get too excited yet – that data-optimized future isn’t just going to fall in our laps.

An abundance of newly available information — from research and development data aggregated by pharmaceutical companies to digitized patient records to recently-released health information from the federal government and other public sources — combined with new technology has the potential to transform health. But, according to the report, to really “jujitsu” that data (as the country’s CTO Todd Park likes to say), the industry may need to shift its thinking and scale some obstacles first.

“Stakeholders will only benefit from big data if they take a more holistic, patient-centered approach to value, one that focuses equally on health-care spending and treatment outcomes,” McKinsey’s report said.

To do that, the analysis laid out a few guidelines, including:

• encouraging patients to take more active steps to improve their health
• developing an integrated approach to care in which all caregivers have access to the same information
• selecting health care providers based on skill sets, not job titles (for example, having nurses perform tasks traditionally doctor-required duties)
• looking for ways to improve value while improving care quality (for example through systems that tie provider reimbursement to patient outcomes)
• identifying new and innovative approaches to treatment and care delivery

With those approaches, McKinsey estimates that the use of big data could save health care stakeholders $300 to $450 billion in health care costs, or 12 to 17 percent of the $2.6 trillion baseline in U.S. health-related spending.

Already, big companies and emerging startups are leading the way in the smart use of data. At GigaOM’s recent Structure: Data conference, Aetna’s head of innovation Michael Palmer talked about how the company is using data to prevent diabetes and heart attacks. Startup Asthmapolis (which this week raised $5 million) is using GPS data collected via sensors attached to inhalers to help individuals, physicians and public health officials uncover asthma-related patterns.

 

Institute for Data Sciences and Engineering

The Potential and the Risks of Data Science

By STEVE LOHR

http://bits.blogs.nytimes.com/2013/04/07/the-potential-and-the-risks-of-data-science/

 

Columbia University held a daylong symposium on Friday as a kind of brainy coming-out party for its new Institute for Data Sciences and Engineering. The institute is a collection of interdisciplinary centers including ones for cybersecurity, financial analytics, health analytics, new media and smart cities. It points to the direction universities will have to take if the bundle of technologies called Big Data — new data and artificial intelligence tools — really are to transform industries, as its champions predict.

The symposium, “From Big Data To Big Ideas,” was mainly a celebration of the promise of the technology in fields from health care to transportation, with presentations from Columbia professors and computer scientists from companies like Google, Facebook, Microsoft and Bloomberg.

The privacy and surveillance perils of Big Data came up only in passing. But during a question-and-answer portion of one panel, Ben Fried, Google’s chief information officer, expressed a misgiving. “My concern is that the technology is way ahead of society,” Mr. Fried said. There is danger, he suggested, if only a technical elite understand Big Data and its implications, with the risk of a runaway technology or a public rejection.

I spoke with Mr. Fried briefly afterward. “I think it is a mistake if conversations about this technology leave out the humanities,” he said. Broader social concerns, he explained, should be a guide and will affect the spread and use of Big Data technology.

Mr. Fried works for a company that has at times tested the limits of Big Data technology, notably the privacy threat posed by overaggressive data collection. But he makes a good point, and he’s not the only one making it.

Alex Pentland, a computational social scientist at the MIT Media Lab, is leading a group at MIT and elsewhere in exploring the implications of what he calls “a data-driven society.”

At Columbia, Mark Hansen, a professor of journalism and director of the institute’s New Media Center, has his own plan for bringing the humanities into Big Data. He teaches students from Columbia’s Graduate School of Journalism how to do some data programming. The goal, Mr. Hansen explains, is not to make them professional programmers, but mainly to give journalists — whom he calls “society’s explainers of last resort” — a firmer understanding of computer technology. Software algorithms, he said, are not impartial. They are written by people, and can embody human values and biases.

Mr. Hansen’s small-scale educational program recalls the major initiative at Dartmouth College in the 1960s, when mainframe computers, the transforming technology of the day, came into widespread use in business, government and science. It was there that two professors, John Kemeny and Thomas Kurtz, developed Basic, a simplified programming language, initially for Dartmouth students. They were influenced by the concerns raised by C.P. Snow, the English scientist and novelist, in his 1959 lecture, “The Two Cultures,” which analyzed the difference between scientific and literary intellectuals, and pointed to the danger of the schism.

The Columbia data sciences institute is just getting under way. New centers could be added. One of the panelists on Friday suggested Columbia might want to add something like the Berkman Center for Internet and Society at Harvard University, which focuses on the impact of technology on society.

The Columbia institute is a science-led undertaking, and its director is a computer scientist, Kathleen R. McKeown. Incidentally, Ms. McKeown, who was the first female professor to receive tenure in the engineering school, holds a bachelor’s degree in comparative literature.

 

McKinsey: Planning for big data

Planning for big data
A special package on mckinsey.com

Companies can exploit big data and advanced analytics to improve their competitiveness, productivity, and decisions, yet actually realizing that promise is hard. The first requirement is a plan: not only a strategy but also a road map for investing in technology and data and for building commitment, reinventing processes, and changing the organizational culture.

“Big data: What’s your plan?” unifies insights from many real-world analytics projects to show you how to get started. After you read it, check out three related videos. In the first, McKinsey director Tim McGuire explains big data’s triple challenge: deciding which data to use, deploying analytics, and using its insights to transform operations. In a second video, McKinsey expert Matt Ariker focuses on the human element: developing skills; integrating new capabilities, people, and roles; and making the necessary changes in mind-sets and behavior. Finally, McKinsey expert Matthias Roggendorf presents the business case for such a profound data transformation.

Two additional articles—the first set in a series on how analytics has been changing key industries—apply these insights specifically to health care. “The big-data revolution in US health care: Accelerating value and innovation” explains why analytics could transform the sector. “How big data can revolutionize pharmaceutical R&D” shows a way forward for companies struggling with declining success rates and stagnant pipelines. Both include video interviews with McKinsey directors, who discuss the challenges that organizations face.

 

We need a data democracy, not a data dictatorship

By Derrick Harris

photo: Shutterstock / Tiago Jorge da Silva Estima

 

http://gigaom.com/2013/04/07/we-need-a-data-democracy-not-a-benevolent-data-dictatorship/

 

Summary:

A data democracy built to last needs tools that empower everyone to work with data rather than relying on apps and data scientists. Tableau helped ignite the data revolution, and its IPO could help it keep going.

The democratization of data is a real phenomenon, but building a sustainable data democracy means truly giving power to the people. The alternative is just a shift of power from traditional data analysts within IT departments to a new generation of data scientists and app developers. And this seems a lot more like a dictatorship than a democracy — a benevolent dictatorship, but a dictatorship nonetheless.

These individuals and companies aren’t entirely bad, of course, and they’re actually necessary. Apps that help predict what we want to read, where we’ll want to go next or what songs we’ll like are certainly cool and even beneficial in their ability to automate and optimize certain aspects of our lives and jobs. In the corporate world, there will always be data experts who are smarter and trained in advanced techniques and who should be called upon to answer the toughest questions or tackle the thorniest problems.

Last week, for example, Salesforce.com introduced a new feature of its Chatter intra-company social network that categorizes a variety of data sources so employees can easily find the people, documents and other information relevant to topics they’re interested in. As with similarly devised services — LinkedIn’s People You May Know, the gravitational search movement, or any type of service using an interest graph — the new feature’s beauty and utility lie in its abstraction of the underlying semantic algorithms and data processing.

The problem, however, comes when we’re forced to rely on these people, features and applications to decide how data can affect our lives or jobs, or what questions we can answer using the troves of data now available to us. In a true data democracy, citizens must be empowered to make use of their own data as they see fit and they must only have to rely apps and experts by choice or when the task really requires an expert hand. At any rate, citizens must be informed enough to have a meaningful voice in bigger decisions about data.

The democratic revolution is underway

The good news is that there’s a whole new breed of startups trying to empower the data citizenry, whatever their role. Companies such as 0xdata, Precog and BigML are trying to make data science more accessible to everyday business users. There are next-generation business intelligence startups such as SiSense, Platfora and ClearStory rethinking how business analytics are done in an area of HTML5 and big data. And then there are companies such as Statwing, Infogram and Datahero (which will be in beta mode soon, by the way) trying to bring data analysis to the unwashed non-data-savvy masses.

Combined with a growing number of publicly available data sets and data marketplaces, and more ways of collecting every possible kind of data —  personal fitness, web analytics, energy consumption, you name it — these self-service tools can provide an invaluable service. In January, I highlighted how a number of them can work by using my own dietary and activity data, as well as publicly available gun-ownership data and even web-page text. But as I explained then, they’re still not always easy for laypeople to use, much less perfect.

Statwing spells out statistics for laypeople.

Can Tableau be data’s George Washington?

This is why I’m so excited about Tableau’s forthcoming IPO. There are few companies that helped spur the democratization of data over the past few years more than Tableau. It has become the face of the next-generation business intelligence software thanks to its ease of use and focus on appealing visualization, and its free public software has found avid users even among relative data novices like myself. Tableau’s success and vision no doubt inspired a number of the companies I’ve already referenced.

Assuming it begins its publicly traded life flush with capital, Tableau will not just be financially sound — it will also be in a position to help the burgeoning data democracy evolve into something that can last. More money means being able to develop more features that Tableau can use to bolster sales (and further empower business users with data analysis), which should mean the company can afford to also continually improve its free service and perhaps put premium versions in the hands of more types of more non-corporate professionals for free.

Tableau is already easy (I made this) — but not easy enough.

The bottom-up approach has already proven very effective in the worlds of cloud computing, software as a service and open-source software, and I have to assume it’s a win-win situation in analytics, too. Today’s free users will be tomorrow’s paying users once they get skilled enough to want to move onto bigger data sets and better features. But the base products have to be easy enough and useful enough to get started with, or companies will only have a lot of registrations and downloads but very few avid users.

And if Tableau steps ups its game around data democratization, I have to assume it will up the ante for the company’s fellow large analytics vendors and even startups. A race to empower the lower classes on the data ladder would certainly be in stark contrast to the historical strategy of building ever-bigger, ever-more-advanced products targeting only the already-powerful data elite. That’s the kind of revolution I think we all can get behind.

Feature image courtesy of Shutterstock user Tiago Jorge da Silva Estima.

 

JAMA Viewpoint: The Inevitable Application of Big Data to Health Care

Viewpoint | April 03, 2013

The Inevitable Application of Big Data to Health Care

Travis B. Murdoch, MD, MSc; Allan S. Detsky, MD, PhD

JAMA. 2013;309(13):1351-1352. doi:10.1001/jama.2013.393.

Text Size: A A A

Article

References

The amount of data being digitally collected and stored is vast and expanding rapidly. As a result, the science of data management and analysis is also advancing to enable organizations to convert this vast resource into information and knowledge that helps them achieve their objectives. Computer scientists have invented the term big data to describe this evolving technology. Big data has been successfully used in astronomy (eg, the Sloan Digital Sky Survey of telescopic information), retail sales (eg, Walmart's expansive number of transactions), search engines (eg, Google's customization of individual searches based on previous web data), and politics (eg, a campaign's focus of political advertisements on people most likely to support their candidate based on web searches).

 

FULL article at http://jama.jamanetwork.com/article.aspx?articleid=1674245

 

EOWilson in WSJ: Great Scientist ≠ Good at Math

Great Scientist ≠ Good at Math

E.O. Wilson shares a secret: Discoveries emerge from ideas, not number-crunching

By E.O. WILSON

http://online.wsj.com/article/SB10001424127887323611604578398943650327184.html

For many young people who aspire to be scientists, the great bugbear is mathematics. Without advanced math, how can you do serious work in the sciences? Well, I have a professional secret to share: Many of the most successful scientists in the world today are mathematically no more than semiliterate.

Vahram Muradyan

Many of the most successful scientists in the world today are mathematically no more than semiliterate.

During my decades of teaching biology at Harvard, I watched sadly as bright undergraduates turned away from the possibility of a scientific career, fearing that, without strong math skills, they would fail. This mistaken assumption has deprived science of an immeasurable amount of sorely needed talent. It has created a hemorrhage of brain power we need to stanch.

I speak as an authority on this subject because I myself am an extreme case. Having spent my precollege years in relatively poor Southern schools, I didn't take algebra until my freshman year at the University of Alabama. I finally got around to calculus as a 32-year-old tenured professor at Harvard, where I sat uncomfortably in classes with undergraduate students only a bit more than half my age. A couple of them were students in a course on evolutionary biology I was teaching. I swallowed my pride and learned calculus.

I was never more than a C student while catching up, but I was reassured by the discovery that superior mathematical ability is similar to fluency in foreign languages. I might have become fluent with more effort and sessions talking with the natives, but being swept up with field and laboratory research, I advanced only by a small amount.

Fortunately, exceptional mathematical fluency is required in only a few disciplines, such as particle physics, astrophysics and information theory. Far more important throughout the rest of science is the ability to form concepts, during which the researcher conjures images and processes by intuition.

Everyone sometimes daydreams like a scientist. Ramped up and disciplined, fantasies are the fountainhead of all creative thinking. Newton dreamed, Darwin dreamed, you dream. The images evoked are at first vague. They may shift in form and fade in and out. They grow a bit firmer when sketched as diagrams on pads of paper, and they take on life as real examples are sought and found.

Pioneers in science only rarely make discoveries by extracting ideas from pure mathematics. Most of the stereotypical photographs of scientists studying rows of equations on a blackboard are instructors explaining discoveries already made. Real progress comes in the field writing notes, at the office amid a litter of doodled paper, in the hallway struggling to explain something to a friend, or eating lunch alone. Eureka moments require hard work. And focus.

Ideas in science emerge most readily when some part of the world is studied for its own sake. They follow from thorough, well-organized knowledge of all that is known or can be imagined of real entities and processes within that fragment of existence. When something new is encountered, the follow-up steps usually require mathematical and statistical methods to move the analysis forward. If that step proves too technically difficult for the person who made the discovery, a mathematician or statistician can be added as a collaborator.

In the late 1970s, I sat down with the mathematical theorist George Oster to work out the principles of caste and the division of labor in the social insects. I supplied the details of what had been discovered in nature and the lab, and he used theorems and hypotheses from his tool kit to capture these phenomena. Without such information, Mr. Oster might have developed a general theory, but he would not have had any way to deduce which of the possible permutations actually exist on earth.

Over the years, I have co-written many papers with mathematicians and statisticians, so I can offer the following principle with confidence. Call it Wilson's Principle No. 1: It is far easier for scientists to acquire needed collaboration from mathematicians and statisticians than it is for mathematicians and statisticians to find scientists able to make use of their equations.

This imbalance is especially the case in biology, where factors in a real-life phenomenon are often misunderstood or never noticed in the first place. The annals of theoretical biology are clogged with mathematical models that either can be safely ignored or, when tested, fail. Possibly no more than 10% have any lasting value. Only those linked solidly to knowledge of real living systems have much chance of being used.

If your level of mathematical competence is low, plan to raise it, but meanwhile, know that you can do outstanding scientific work with what you have. Think twice, though, about specializing in fields that require a close alternation of experiment and quantitative analysis. These include most of physics and chemistry, as well as a few specialties in molecular biology.

Newton invented calculus in order to give substance to his imagination. Darwin had little or no mathematical ability, but with the masses of information he had accumulated, he was able to conceive a process to which mathematics was later applied.

For aspiring scientists, a key first step is to find a subject that interests them deeply and focus on it. In doing so, they should keep in mind Wilson's Principle No. 2: For every scientist, there exists a discipline for which his or her level of mathematical competence is enough to achieve excellence.

—Dr. Wilson is a professor emeritus at Harvard University. His many books include "On Human Nature" and "The Social Conquest of Earth." This piece is adapted from his new book "Letters to a Young Scientist" (Liveright).

A version of this article appeared April 6, 2013, on page C2 in the U.S. edition of The Wall Street Journal, with the headline: Great Scientist ≠ Good at Math.

 

Big Data and the Big Opportunity to Reform Education

Posted April 4, 2013

http://smartdatacollective.com/andreesa/116531/big-data-and-big-opportunity-reform-education?utm_source=feedburner&utm_medium=feed&utm_campaign=Smart+Data+Collective+%28all+posts%29

 

30

In big data terms, every human being is a rich resource. Students, from enrollment to grades to the clubs they join, generate as much data as any of us. Institutions of both higher and elementary learning are already working towards using this information to better the education they provide, the way they handle different student demographics and even the way they research fields like literature.

Behavior, Matchmaking and New Knowledge

From test scores to online behavior, every single student generates a unique data trail. By running and analyzing student data, institutions can see if they’re spending the right amount of money in the right places, treating their online students the way they should be, and a host of other insights.

The WICHE Cooperative for Educational Technologies has built a database of 640,000 students from six colleges, enabling analytical access to 3 million course-level records, according to Inside Higher Ed. Researchers are able to run demographics, performance and student retention, among other data, in order to analyze what works and doesn’t work in each institution.

This higher-ed data mining project has already disproven one widely held belief about enrollment. For students at high risk of dropping out, it is better to start an online education with a limited number of courses, rather than taking a full load as was commonly believed.

In addition, one of the colleges involved, Rio Salado College, is starting to build a foundation for what could eventually become a sort of matchmaking service for students and colleges. The college is measuring each of its 40,000+ online students’ online interactions, the number of times they view textbooks, their rate of response to instructor feedback and overall performance. Combined with demographic information, this data will provide the college with conclusions about which programs are best for which types of students.

Big Data for Elementary Schools

On the elementary school level, a project named InBloom is aiming to create middleware that places the information systems of ten school districts across the country onto the cloud. This will create a pool of information that includes, among other things, student test scores and attendance data, according to the Hechinger Report.

The company is aiming to provide an API that enables app developers to construct new education apps, such as learning games and performance tracking dashboards, to aid school employees and teaching. The idea is to give schools more insight into how their initiatives are performing, where their money is working best and which types of programs are working for which students. Personalized learning is another goal of the program.

Optimizing Academic Research on the Cloud

From personal genomics to better medical care, big data has already had a powerful effect on science. Even astronomy can now be done from a computer screen rather than a telescope. By running and analyzing images taken from space, astronomers have been able to collaborate in order to make new discoveries. Some crowdsourced research projects let mainstream Internet users to view the same images astronomers are looking at. This recently led to a Dutch schoolteacher discovering an important new kind of gas cloud, the Chronicle of Higher Education reports.

The Chronicle also points out that big data is being used to find patterns in music. One project is logging 23,000 hours of music and displaying patterns in a student-accessible cloud-based application. Students are not only studying the rhythmic trends in the music of a single composer, but the historical trends across many composers for an entire time period.

The same kind of historical analysis is becoming available to the humanities, in the fields of language, history and literature. Stanford now has something called the Literature Lab. There, the text of tens of thousands of books—scanned by Google for the Google Books project—can be analyzed to draw new insights on writing trends, the Chronicle reports.

Literary data mining is producing a new trend of “distant reading,” where students analyze the data about books—available in the form of maps, graphs, charts and the like—in order to analyze them. Researchers can draw new conclusions about, for example, common author sentiment in Edwardian England.

The Chronicle also points out that this literary project is based on Google Books, the result of the search giant’s effort to put all printed books on the cloud. This has created the largest digital library in human history, and also spurred an arms race as to which academic institution will own rights to it. So far, Stanford, home of the Literature Lab, is one of the front runners.

The Future of Education

In the not-too-distant future, big data will become so pervasive in education that we’ll find it hard to imagine life without it. Many students will work on tablets, for example, that will generate assignments at a difficulty level that matches their abilities. Online students will be able to enter their criteria for finding a university and view a list of customized results. Even the study of humanities may have a data mining component. The brave new world of education is here.

(image: big data education / shutterstock)

 

New Yorker: Steamrolled by Big Data

Steamrolled by Big Data

Posted by Gary Marcus

http://www.newyorker.com/online/blogs/elements/2013/04/steamrolled-by-big-data.html?printable=true&currentPage=all

 

Five years ago, few people had heard the phrase “Big Data.” Now, it’s hard to go an hour without seeing it. In the past several months, the industry has been mentioned in dozens of New York Times stories, in every section from metro to business. (Wired has even already declared it passé: “STOP HYPING BIG DATA AND START PAYING ATTENTION TO ‘LONG DATA’.”) At least one corporation, the business-analytics firm SAS, has a Vice-President of Big Data. Meanwhile, nobody seems quite sure exactly what the phrase means, beyond a general impression of the storage and analysis of unfathomable amounts of information, but we are assured, over and over, that it’s going to be big. Last summer, Jon Kleinberg, a computer scientist at Cornell, said in the Times that “The term itself is vague, but it is getting at something that is real… Big Data is a tagline for a process that has the potential to transform everything.”

Most of what’s written about Big Data is enthusiastic, like Kenneth Cukier and Viktor Mayer-Schonberger’s gushing ode “Big Data: A Revolution That Will Transform How We Live, Work, and Think,” which is currently selling briskly on Amazon, or the recent Times article on Mayor Bloomberg’s geek squad, and how “Big Data’s moment, especially in the management of cities, has powerfully and irreversibly arrived.” But despite the sense of excitement and promise surrounding the industry, Big Data isn’t nearly the boundless miracle that many people seem to think it is.

* * *

The reason scarcely anybody used to talk about Big Data is that, until very recently, it didn’t exist—most data had been, by current standards, small potatoes. Now, Big Data is mainly measured in terabytes (trillions) and petabytes (quadrillions); within a decade, even those numbers may seem quaint.

As companies like Google have shown, more data often means newer and better solutions to old problems. Last year, I wrote about how Google significantly improved spell-checkers by using massive databases of users’ self-corrections to do work that previously required hand-crafted algorithms focussed on the intricacies of English spelling and the psychology of typing. Google’s new trick wouldn’t work if you only had a few user searches to draw on, but if you have trillions of searches from many millions of users, across a hundred and forty-six languages, it’s pure genius—and a technique that can be rapidly applied to different languages with relatively little manual labor. And it’s just one of hundreds or perhaps thousands of innovations driven by the sheer mass of data that we’re capable of storing, wrangling, and manipulating. Cukier and Mayer-Schonberger’s book, for example, explains how the artificial-intelligence researcher Oren Etzioni created Farecast (eventually sold to Microsoft, and now part of Bing Travel), which scraped data from the Web to make good guesses about whether airline fare would rise or fall. Coupled with some advances in statistical techniques, Big Data is now de rigueur—to the point, almost, of being a kind of new religion, nicely parodied in Dilbert last summer: “In the past, our company did many evil things,” Dilbert’s pointy-haired boss says, “but if we store Big Data in our servers we will be saved.”

* * *

Companies like the PalmPilot co-founder Jeff Hawkins’s Numenta offer the promise of something even more transformational: universal, one-size-fits-all, real-time predictive analytics. According to Numenta’s Web site, their software, Grok, “finds complex patterns in data streams and generates actionable predictions in real time…. Feed Grok data, and it returns predictions that generate action. Grok learns and adapts automatically.” Numenta boasts that “As the age of the digital nervous system dawns, Grok represents the type of technology that will convert massive data flows into value.” The company doesn’t lack for press, either; the Times (“JEFF HAWKINS DEVELOPS A BRAINY BIG DATA COMPANY”), Technology Review, Forbes, and Bloomberg News have all covered Numenta with gusto.

According to Quentin Hardy of the Times, “Jeff Hawkins has been a pioneer of mobile devices, a distinguished lecturer in neuroscience, and a published author of a revolutionary theory of how the brain works. If he’s right about Big Data, a lot of people are going to wish he’d never gone into that field.” Why? “From initially observing the data flow, [Grok] begins making guesses about what will happen next. The more data, the more accurate the predictions become.” (It also promises to obviate the need for massive hard drives: by analyzing incoming data so quickly, there will be no need to store the old information.)

Read a few paragraphs further down, though, and the article reveals that—for all of Numenta’s billing as a one-size-fits-all automatic solver—Grok is “still in limited release, with just a few customers in the fields of energy, media, and video processing.” Numenta champions data but so far gives little concrete support of its own claims. (A company spokesperson noted that Grok “is in private beta and pilots with select customers in a variety of vertical markets, including electric energy, I.T. management, online advertising, and finance.”)

If fifty years of research in artificial intelligence has taught us anything, it’s that every problem is different, that there are no universally applicable solutions. An algorithm that is good at chess isn’t going to be much help parsing sentences, and one that parses sentences isn’t going to be much help playing chess. A faster computer will be better than a slower computer at both, but solving problems will often (though not always) require a fair amount of what some researchers call “domain knowledge”—specific information about particular problems, often gathered painstakingly by experts. So-called machine learning can sometimes help (spell-checking is a case where it can help a lot, and ditto for speech recognition), but nobody has ever, for example, built a world-class chess program by taking a generally smart machine, endowing it with enormous data, and letting it learn for itself. If Grok really did what its Web site promises, a complex problem like chess would be grist for its mill; there are tons of chess games online, often live, and the rules of chess could be programmed into Grok in an hour. But I’ll eat my hat, and send Hawkins a personal apology, if he can get Grok to mine that stream of chess games well enough to beat Magnus Carlsen or Garry Kasparov without relying on a whole lot of expert knowledge of the game, even using whatever high-end hardware Numenta presumably has access to. (Nevertheless, a Numenta spokesperson noted that “Grok is ideally suited for working with fast data streams.”)

* * *

Of course, Numenta is not the only company working with Big Data. Almost every expert I spoke to for this story mentioned interesting work being done elsewhere in the field. I.B.M., for example, used Big Data (along with many other techniques) to great effect in its “Jeopardy”-winning Watson, and products such as Siri and Google search depend heavily on it, without quite making the lavish promises of human-free automaticity that Numenta implies.

Some problems do genuinely lend themselves to Big Data solutions. The industry has made a huge difference in speech recognition, for example, and is also essential in many of the things that Google and Amazon do; the Higgs Boson wouldn’t have been discovered without it. Big Data can be especially helpful in systems that are consistent over time, with straightforward and well-characterized properties, little unpredictable variation, and relatively little underlying complexity.

But not every problem fits those criteria; unpredictability, complexity, and abrupt shifts over time can lead even the largest data astray. Big Data is a powerful tool for inferring correlations, not a magic wand for inferring causality. The field has thus far apparently yielded only modestly improved weather prediction, and had little, if any, impact on challenges such as getting computers to program themselves. “Jeopardy” is a feasible application because most of the required knowledge derives from titles on Wikipedia pages; it’s largely an exercise in data retrieval, to which Big Data is well-suited. Chess, in contrast, is an exercise in novelty that demands an enormous amount of precision. Every position is different and has its own best move, often a function of a great number of interdependent pieces with complex relationships that are highly sensitive to exact details. In Google Translate, gist often suffices; in chess, nobody can win at a grandmaster level by picking moves that are only roughly correct.

In fact, one could see the entire field of artificial intelligence as an inadvertent referendum on Big Data, because nowadays virtually every problem that has ever been addressed in A.I.—from machine vision to natural language understanding—has also been attacked from a data perspective. Yet most of those problems are unsolved, Big Data or no. Even with the world’s largest databases, for example, the challenge of machine vision remains largely open. Last summer, the “cat detector” at Google (which I’ve mentioned before) was trained on ten million images, using a thousand machines, for three days, and if the program managed to “learn” what a cat looks like, its overall visual performance was poor. Cats that were bigger than usual, smaller than usual, or slightly out of frame caused significant decrements in performance. It’s also a good bet that the system would do a lot poorer in more complex scenes with many objects. That’s not because Big Data isn’t useful; in vision, as in so many things, Big Data is only a small part of a solution.

As one skeptical industry insider, Anthony Nyström, of the Web software company Intridea, put it to me, selling Big Data is a great gig for charlatans, because they never have to admit to being wrong. “If their system fails to provide predictive insight, it’s not their models, it’s an issue with your data.” You didn’t have enough data, there was too much noise, you measured the wrong things. The list of excuses can be long.

* * *

In reality, most computational models of most things have, historically speaking, been wrong—or at least incomplete, effective in some circumstances, not all. Even Google, which likely has the biggest data of anyone, still uses humans to hand-curate some of it, because unanalyzed gobs of information are no guarantee of anything, and giant servers still can’t serve as fully trustworthy replacements for human judgment.

For perspective, it might help to consider the challenge of inferring the structure of protein from its underlying DNA sequence, a problem with an enormous number of applications in medicine, and throughout biology. Hundreds if not thousands of researchers have worked on the problem for fifty years, and for the last decade have had large databases to help; yet, in the words of a review published a few months ago in Science, “no single group [of researchers] yet consistently produces accurate models,” especially with more complex DNA sequences that don’t closely resemble genes that are already well understood. The more complex a problem is, and the more particular instances differ from those that came before, the less likely Big Data is to be a sure thing.

In the years to come, scientists and engineers will develop a clearer picture of the circumstances in which Big Data can and can’t make a big difference; for now, hype needs to be tempered with caution and a sensitivity to when humans should and should not remain in the loop. As Alexei Efros, one of the leaders in applying Big Data to machine vision, put it, Big Data is “a fickle, coy mistress,” inviting, yet not without risk.

Illustration by Joost Swarte

 

book review (big data/healt): A Prescription for Frustration

April 1, 2013

A Prescription for Frustration

By ABIGAIL ZUGER, M.D.

Time spent in hospitals brings out the inner Chekhov in some doctors, the inner Che in others. Then there are the occasional hybrids, the storytellers who secretly plot revolution and the revolutionaries who wind up telling fairy tales.

One might argue that Dr. Leana Wen and Dr. Joshua Kosowsky belong to the latter group. At least, their impressive “When Doctors Don’t Listen” is a manifesto motivated by very active imaginations, not that this necessarily diminishes the book’s importance.

The authors, both emergency room physicians at Brigham and Women’s Hospital in Boston, do a fine job of sorting through most of the serious problems in American medicine today, including the costs, overtesting, overprescribing, overlitigation and general depersonalization. All are caused at least in part, they argue, by the increasing use of algorithms in medical care.

Algorithms are flow charts, created by groups ranging from individual hospitals to large professional organizations, dictating what tests doctors should order and what medications they should prescribe in hundreds of different situations. Deployed throughout medicine, the algorithms are perhaps used most frequently in emergency rooms, where any single word a patient utters may set off a long cascade of programmed activity.

Thus, people who say “fever” and “cough” are likely to find themselves being evaluated and treated according to a “pneumonia” algorithm; a mention of chest pain will land them on a “heart attack” algorithm. And such is the power of these decision trees that even if you are a circus employee and Jumbo the elephant has just stepped on your chest, the chances are that once you say “chest pain” you will wind up on that heart attack pathway, just in case.

There are good reasons behind this kind of rote decision making. The great river of suffering and complaint that churns through E.R. doors demands some kind of quickly imposed external order. Algorithms help standardize medical care and ensure that no life-threatening conditions are accidentally overlooked.

But doctors who use too many algorithms practice what is commonly disparaged as “cookbook” medicine — you combine ingredients like blood tests and CT scans, and out pops a diagnosis. (Or, more often, the absence of one: “Well, at least we know it wasn’t a heart attack!”) Even dummies can do it.

And just as world-class chefs eschew standard recipes, so medical experts chafe at formulaic care, pointing out that hard-earned medical instincts should not be ignored, that people usually want to know what they have rather than what they don’t have, and that trainees who rely on algorithms will never learn to think.

Instead of letting key words dominate practice, Dr. Wen and Dr. Kosowsky argue, doctors should listen more carefully and use a little more common sense. “We advocate for a type of personalized medicine that involves taking a personalized history, performing a personalized physical exam and approaching each patient as an individual rather than as a list of variables,” they write. In other words, remember the elephant even if he’s not in the room.

The book lists many instances in which cookbook medicine fails or backfires. There is the college student suffering through an expensive evaluation for the brain hemorrhage she does not have; the world-class hangover she had is almost gone by the time she finally ends the madness by running out the E.R. door. The chatty woman with a million complaints isn’t so lucky: she almost dies from an infected gallbladder while her doctors are busy worrying about her heart.

The book’s insights and cautionary tales should appeal to medical and lay readers alike: they combine into a superb analysis of how doctors listen and think, and offer detailed suggestions for how they could do both better.

But when the authors embark on an earnest campaign for patients to grab the reins and steer their own wayward doctors gently but firmly onto the right path — there, I would argue, is where the fantasy begins.

“Participate in your physical exam,” they urge their readers. “Make the differential diagnosis together.” A healthy person might take on that difficult assignment, especially the advice about learning how to tell an effective story (start at the beginning, don’t leave anything out, avoid technical terms).

But most acutely ill people just want an experienced person to take over, to do what has to be done and do it fast. That doctors who fall ill are often mismanaged just like other patients testifies to an illness’s power to limit effective participation in one’s own care.

Still, the democratic examination room is a delightfully seductive fantasy. Even writers of medical fiction don’t venture that far. Like Dr. Louise Aronson in “A History of the Present Illness,” they generally stay so firmly grounded in grim reality that their fiction reads like memoir by another name.

Dr. Aronson, a geriatrician in San Francisco, joins the ranks of those immortalizing the small, realistic details of modern medical care. She offers up the requisite coming-of-age tales of studenthood, residency and disillusioned midcareer, but is at her best in the world of the much too old — the rooms where dyed hair grows out platinum white and small bodies lie in bed “the way a letter lies in its envelope.”

Dr. Aronson writes lovely, nuanced description; if her dialogue is occasionally a little stiff and some of her plots stretch the bounds of plausibility, it is still worth staying with her: no revolution in sight, just keen, calm observation, punctuated by sighs.