Katie Fehrenbacher, Gigaom, Jan. 29, 2012, 9:30pm
One of the most obvious trends from the big smart grid conference DistribuTECH last week was how much analytics and big data tools will be used to try to remake energy in 2012, from curbing energy consumption, to reducing energy loss, to adding in more clean power to the grid. Here’s 10 ways that analytics and big data will start to shape the production and consumption of energy in the world:
1). Weather data: Having a finger on the pulse of constantly changing weather data on a micro and macro level can help utilities, building owners and consumers optimize their energy consumption habits and promote energy efficiency. Startup EnergyHub recently partnered with sensor network player Earth Networks to use weather data to make a more efficient form of demand response (utilities controlling power consumption). Other startups like EcoFactor, Opower and Tendril also use weather data as part of their energy behavioral analytics.
IBM has long sold a weather prediction service called Deep Thunder to municipalities, organizations and utilities, which use it to do things like tailor their services, change routes, or generate more or less power. I think weather data could some day provide a platform for some very important next generation services and applications for energy efficiency, much in the way that location data is used as a platform for a variety of services.
2). Cell phone data: Cell phones in our pockets are essentially palm-sized sensors and computers sending a constant stream of information to the cloud where companies could one day use that data to create energy efficiency and better energy products. And yes, a lot of that data is private information, but after that data is anonymized it can be used for the greater good of the community — particularly via the billions of cell phones in developing countries. A startup called Jana does research projects around cell phone data in developing countries, and looks to work with NGOs on programs to create better infrastructure, energy infrastructure and resources.
3). Connected thermostat data: One of the biggest trends from DistribuTECH this year was the overwhelming amount of smart thermostats that are now being sold and marketed. Companies can incorporate that thermostat data into data bases that can be used to promote energy efficiency. EcoFactor’s service remembers every time a home owner overrides the automated smart thermostat system and changes the personalized service to accommodate that manual override. Using 100,000 connected thermostats (which produce 5 billion data points each month) EnergyHub found some interesting statistics like folks in cold climates have a lower average heating temperature set point than households in warmer states.
4). Hadoop & energy databases: The open source data base tool Hadoop is well known — and oft used — in the computing worlds. But in the energy and utility worlds it’s quite rare. However, as the amount of energy data has started to rapidly grow from the smart grid, some companies are embracing Hadoop as a key way to manage energy info. Opower tells me it’s using Hadoop (and the company commercializing Hadoop, Cloudera) as an important way to manage its massive energy data streams. Likewise PJM has turned to Hadoop as a way to organize the energy data coming off of a synchophaser sensor project.
5). Clean power data: One of the main goals for the smart grid is to enable the addition of more variable clean power, which is far more unreliable than fossil fuels (the sun doesn’t shine and the wind doesn’t blow 24/7). Analytics crunching the data from a utilities’ energy supply and demand can help make clean power a little less variable, by being able to more accurately predict the environmental conditions, as well as more accurately assess demand from energy users.
6). Electric car data: Electric cars will by their nature be connected cars, using information technology to manage the vehicle charge and location. Utilities will be closely tracking the charging habits of electric car owners in order to make sure that the grid isn’t overloaded in some early adopter neighborhoods.
7). Power line sensors: One of the areas of low hanging fruit for the power grid is the simple task of helping utilities find blackouts more easily and be able to monitor and manage grid outages. That’s partly where sensor systems called synchophasers come in, which can in real time monitor the health of power lines, collecting multiple data streams per second. Expect all major networks to have synchophaser systems installed over the coming years.
8). Real estate data: Startups like First Fuel Software can use big data to make super accurate assessments about buildings and ways to reduce the energy consumption of buildings — without any extra hardware or monitoring software being installed at the building. Things like weather around the building, demographics of the people in the building, and the building’s historical energy consumption can be used to create an accurate projection. The best way to make a building more energy efficient is by getting as much data about the building;s energy use as possible.
9). Variable pricing: Some day when electricity is sold throughout the world at different prices dependent on supply and demand, massive data bases will be needed. This type of variable pricing is offered in some places in the world, but if it ever becomes ubiquitous it will help curb consumption, by offering high prices when energy is being over used.
10). Using behavioral analytics to curb energy consumption: Getting into the brains of energy users is the job of startups like Opower and Tendril (after it acquired Gr0unded Power.) Essentially these companies have collected data on consumers and demographics and they are using it to try to guess the best way to influence the consumer to do things like upgrade their home appliances and lights to more efficient ones.
Monday, January 30, 2012
WSJ Opinion: The Coming Tech-led Boom
Three breakthroughs are poised to transform this century as much as telephony and electricity did the last.
Mark P. Mills & Julio M. Ottino, The Wall Street Journal, January 30, 2012
In January 1912, the United States emerged from a two-year recession. Nineteen more followed—along with a century of phenomenal economic growth. Americans in real terms are 700% wealthier today.
In hindsight it seems obvious that emerging technologies circa 1912—electrification, telephony, the dawn of the automobile age, the invention of stainless steel and the radio amplifier—would foster such growth. Yet even knowledgeable contemporary observers failed to grasp their transformational power.
In January 2012, we sit again on the cusp of three grand technological transformations with the potential to rival that of the past century. All find their epicenters in America: big data, smart manufacturing and the wireless revolution.
Information technology has entered a big-data era. Processing power and data storage are virtually free. A hand-held device, the iPhone, has computing power that shames the 1970s-era IBM mainframe. The Internet is evolving into the "cloud"—a network of thousands of data centers any one of which makes a 1990 supercomputer look antediluvian. From social media to medical revolutions anchored in metadata analyses, wherein astronomical feats of data crunching enable heretofore unimaginable services and businesses, we are on the cusp of unimaginable new markets.
The second transformation? Smart manufacturing. This is the first structural shift since Henry Ford launched the economic power of "mass production." While we see evidence already in automation and information systems applied to supply-chain management, we are just entering an era where the very fabrication of physical things is revolutionized by emerging materials science. Engineers will soon design and build from the molecular level, optimizing features and even creating new materials, radically improving quality and reducing waste.
Devices and products are already appearing based on computationally engineered materials that literally did not exist a few years ago: novel metal alloys, graphene instead of silicon transistors (graphene and carbon enable a radically new class of electronic and structural materials), and meta-materials that possess properties not possible in nature; e.g., rendering an object invisible—speculation about which received understandable recent publicity.
This era of new materials will be economically explosive when combined with 3-D printing, also known as direct-digital manufacturing—literally "printing" parts and devices using computational power, lasers and basic powdered metals and plastics.
Already emerging are printed parts for high-value applications like patient-specific implants for hip joints or teeth, or lighter and stronger aircraft parts. Then one day, the Holy Grail: "desktop" printing of entire final products from wheels to even washing machines.
The era of near-perfect computational design and production will unleash as big a change in how we make things as the agricultural revolution did in how we grew things. And it will be defined by high talent not cheap labor.
Finally, there is the unfolding communications revolution where soon most humans on the planet will be connected wirelessly. Never before have a billion people—soon billions more—been able to communicate, socialize and trade in real time.
The implications of the radical collapse in the cost of wireless connectivity are as big as those following the dawn of telegraphy/telephony. Coupled with the cloud, the wireless world provides cheap connectivity, information and processing power to nearly everyone, everywhere. This introduces both rapid change—e.g., the Arab Spring—and great opportunity. Again, both the launch and epicenter of this technology reside in America.
Few deny that technology fuels economic growth as well as both social and lifestyle progress, the latter largely seen in health and environmental metrics. But consider three features that most define America, and that are essential for unleashing the promises of technological change: our youthful demographics, dynamic culture and diverse educational system.
First, demographics. By 2020, America will be younger than both China and the euro zone, if the latter still exists. Youth brings more than a base of workers and taxpayers; it brings the ineluctable energy that propels everything. Amplified and leavened by the experience of their elders, youth and economic scale (the U.S. is still the world's largest economy) are not to be underestimated, especially in the context of the other two great forces: our culture and educational system.
The American culture is particularly suited to times of tumult and challenge. Culture cannot be changed or copied overnight; it is a feature of a people that has, to use a physics term, high inertia. Ours is distinguished by incontrovertibly powerful features, namely open-mindedness, risk-taking, hard work, playfulness, and, critical for nascent new ideas, a healthy dose of anti-establishment thinking. Where else could an Apple or a Steve Jobs have emerged?
Then there's our educational system, often criticized as inadequate to global challenges. But American higher education eludes simple statistical measures since its most salient features are flexibility and diversity of educational philosophies, curricula and the professoriate. There is a dizzying range of approaches in American universities and colleges. Good. One size definitely does not fit all for students or the future.
We should also remember that more than half of the world's top 100 universities remain in America, a fact underscored by soaring foreign enrollments. Yes, other nations have fine universities, and many more will emerge over time. But again the epicenter remains here.
What should our politicians do to help usher in this new era of entrepreneurial growth? Liquid financial markets, sensible tax and immigration policy, and balanced regulations will allow the next boom to flourish. But the essential fuel is innovation. The promise resides in the tectonic technological shifts under way.
America's success isn't preordained. But the technological innovations circa 2012 are profound. They will engender sweeping changes to our society and our economy. All the forces are in place. It's just a matter of when.
Mr. Mills, a physicist and founder of the Digital Power Group, writes the Forbes Energy Intelligence column. Mr. Ottino is dean of the McCormick School of Engineering and Applied Sciences at Northwestern University.
Mark P. Mills & Julio M. Ottino, The Wall Street Journal, January 30, 2012
In January 1912, the United States emerged from a two-year recession. Nineteen more followed—along with a century of phenomenal economic growth. Americans in real terms are 700% wealthier today.
In hindsight it seems obvious that emerging technologies circa 1912—electrification, telephony, the dawn of the automobile age, the invention of stainless steel and the radio amplifier—would foster such growth. Yet even knowledgeable contemporary observers failed to grasp their transformational power.
In January 2012, we sit again on the cusp of three grand technological transformations with the potential to rival that of the past century. All find their epicenters in America: big data, smart manufacturing and the wireless revolution.
Information technology has entered a big-data era. Processing power and data storage are virtually free. A hand-held device, the iPhone, has computing power that shames the 1970s-era IBM mainframe. The Internet is evolving into the "cloud"—a network of thousands of data centers any one of which makes a 1990 supercomputer look antediluvian. From social media to medical revolutions anchored in metadata analyses, wherein astronomical feats of data crunching enable heretofore unimaginable services and businesses, we are on the cusp of unimaginable new markets.
The second transformation? Smart manufacturing. This is the first structural shift since Henry Ford launched the economic power of "mass production." While we see evidence already in automation and information systems applied to supply-chain management, we are just entering an era where the very fabrication of physical things is revolutionized by emerging materials science. Engineers will soon design and build from the molecular level, optimizing features and even creating new materials, radically improving quality and reducing waste.
Devices and products are already appearing based on computationally engineered materials that literally did not exist a few years ago: novel metal alloys, graphene instead of silicon transistors (graphene and carbon enable a radically new class of electronic and structural materials), and meta-materials that possess properties not possible in nature; e.g., rendering an object invisible—speculation about which received understandable recent publicity.
This era of new materials will be economically explosive when combined with 3-D printing, also known as direct-digital manufacturing—literally "printing" parts and devices using computational power, lasers and basic powdered metals and plastics.
Already emerging are printed parts for high-value applications like patient-specific implants for hip joints or teeth, or lighter and stronger aircraft parts. Then one day, the Holy Grail: "desktop" printing of entire final products from wheels to even washing machines.
The era of near-perfect computational design and production will unleash as big a change in how we make things as the agricultural revolution did in how we grew things. And it will be defined by high talent not cheap labor.
Finally, there is the unfolding communications revolution where soon most humans on the planet will be connected wirelessly. Never before have a billion people—soon billions more—been able to communicate, socialize and trade in real time.
The implications of the radical collapse in the cost of wireless connectivity are as big as those following the dawn of telegraphy/telephony. Coupled with the cloud, the wireless world provides cheap connectivity, information and processing power to nearly everyone, everywhere. This introduces both rapid change—e.g., the Arab Spring—and great opportunity. Again, both the launch and epicenter of this technology reside in America.
Few deny that technology fuels economic growth as well as both social and lifestyle progress, the latter largely seen in health and environmental metrics. But consider three features that most define America, and that are essential for unleashing the promises of technological change: our youthful demographics, dynamic culture and diverse educational system.
First, demographics. By 2020, America will be younger than both China and the euro zone, if the latter still exists. Youth brings more than a base of workers and taxpayers; it brings the ineluctable energy that propels everything. Amplified and leavened by the experience of their elders, youth and economic scale (the U.S. is still the world's largest economy) are not to be underestimated, especially in the context of the other two great forces: our culture and educational system.
The American culture is particularly suited to times of tumult and challenge. Culture cannot be changed or copied overnight; it is a feature of a people that has, to use a physics term, high inertia. Ours is distinguished by incontrovertibly powerful features, namely open-mindedness, risk-taking, hard work, playfulness, and, critical for nascent new ideas, a healthy dose of anti-establishment thinking. Where else could an Apple or a Steve Jobs have emerged?
Then there's our educational system, often criticized as inadequate to global challenges. But American higher education eludes simple statistical measures since its most salient features are flexibility and diversity of educational philosophies, curricula and the professoriate. There is a dizzying range of approaches in American universities and colleges. Good. One size definitely does not fit all for students or the future.
We should also remember that more than half of the world's top 100 universities remain in America, a fact underscored by soaring foreign enrollments. Yes, other nations have fine universities, and many more will emerge over time. But again the epicenter remains here.
What should our politicians do to help usher in this new era of entrepreneurial growth? Liquid financial markets, sensible tax and immigration policy, and balanced regulations will allow the next boom to flourish. But the essential fuel is innovation. The promise resides in the tectonic technological shifts under way.
America's success isn't preordained. But the technological innovations circa 2012 are profound. They will engender sweeping changes to our society and our economy. All the forces are in place. It's just a matter of when.
Mr. Mills, a physicist and founder of the Digital Power Group, writes the Forbes Energy Intelligence column. Mr. Ottino is dean of the McCormick School of Engineering and Applied Sciences at Northwestern University.
Thursday, January 26, 2012
Big Data at Davos
At Davos, Discussions of a Global Data Deluge
Nick Bilton, The New York Times, January 25, 2012
DAVOS, Switzerland — Each year, a new theme reverberates through technology conferences around the world. A few years ago it was predictions of the coming wave of social media, location-based services and mobile, long before they became mainstream.
Today’s topic de jour: data. Lots of data.
What makes the data discussion different than in previous years is that it is being discussed in high-profile nontechnology meetings too. This is clearly evident at this year’s annual World Economic Forum.
Meetings here this week include: “From data to decisions: How are new approaches to data intelligence transforming decision-making?” “Data deluge and citizen science.”
“Incidents from digital crime to massive incidents of data theft are increasing significantly, with major political, social and economic implications.” “How is big data being used to uncover individual and collective human dynamics?”
The discussions are not confined to technology attendees either. Chancellors, bankers and educators meeting at the conference are being asked to discuss what the forum calls a growing data deluge and how to manage it.
A 2012 report released by the World Economic Forum, titled, “Big Data, Big Impact: New Possibilities for International Development,” outlines some of the possibilities data can bring around the globe to business and education. It also warns of its potential privacy implications.
The report says data is a new economic asset class, which touches all aspects of society, regardless of income or location.
“Big data represents one of these seismic economic shifts that happens every 10 years,” said Zach Bogue, co-founder of a stealth data investment fund called Data Collective, who was attending Davos. “In some sense, this data has always existed, but until now the bandwidth, storage capability and compute power haven’t existed to harness it.”
Yet as there is talk of data, the discussion of privacy is not far behind.
Earlier this week in Munich, Viviane Reding, the European justice commissioner, repeatedly talked about data in respect to privacy. Ms. Reding said there were 27 laws that apply to data in Europe, most of which date back more than a decade and don’t properly protect consumers today.
Ms. Reding outlined new regulations that were presented in Brussels on Wednesday and were designed to implement one sweeping data protection regulation that would apply to all of Europe.
The new regulations are part of the discussion at Davos as these new rules would drastically affect the way companies operate and collect data. For example, one component of this legislation will require companies to communicate to users why they are collecting this data and how long it is being stored on company servers.
Although the proposed data law being presented by Ms. Reding will apply to companies in Europe, it will clearly affect technology companies around the globe too, including Facebook, Twitter and Google in the United States.
At the World Economic Forum, the discussion is also focusing on one of the toughest regulation challenges in regards to data collection: How to manage a balancing act between governments overseeing data collection and its actions stifling innovation. More importantly, regulators hope to figure out a global solution to data collection.
As the World Economic Forum reports says: ”Concerted action is needed by governments, development organizations and companies to ensure that this data helps the individuals and communities who create it.”
Nick Bilton, The New York Times, January 25, 2012
DAVOS, Switzerland — Each year, a new theme reverberates through technology conferences around the world. A few years ago it was predictions of the coming wave of social media, location-based services and mobile, long before they became mainstream.
Today’s topic de jour: data. Lots of data.
What makes the data discussion different than in previous years is that it is being discussed in high-profile nontechnology meetings too. This is clearly evident at this year’s annual World Economic Forum.
Meetings here this week include: “From data to decisions: How are new approaches to data intelligence transforming decision-making?” “Data deluge and citizen science.”
“Incidents from digital crime to massive incidents of data theft are increasing significantly, with major political, social and economic implications.” “How is big data being used to uncover individual and collective human dynamics?”
The discussions are not confined to technology attendees either. Chancellors, bankers and educators meeting at the conference are being asked to discuss what the forum calls a growing data deluge and how to manage it.
A 2012 report released by the World Economic Forum, titled, “Big Data, Big Impact: New Possibilities for International Development,” outlines some of the possibilities data can bring around the globe to business and education. It also warns of its potential privacy implications.
The report says data is a new economic asset class, which touches all aspects of society, regardless of income or location.
“Big data represents one of these seismic economic shifts that happens every 10 years,” said Zach Bogue, co-founder of a stealth data investment fund called Data Collective, who was attending Davos. “In some sense, this data has always existed, but until now the bandwidth, storage capability and compute power haven’t existed to harness it.”
Yet as there is talk of data, the discussion of privacy is not far behind.
Earlier this week in Munich, Viviane Reding, the European justice commissioner, repeatedly talked about data in respect to privacy. Ms. Reding said there were 27 laws that apply to data in Europe, most of which date back more than a decade and don’t properly protect consumers today.
Ms. Reding outlined new regulations that were presented in Brussels on Wednesday and were designed to implement one sweeping data protection regulation that would apply to all of Europe.
The new regulations are part of the discussion at Davos as these new rules would drastically affect the way companies operate and collect data. For example, one component of this legislation will require companies to communicate to users why they are collecting this data and how long it is being stored on company servers.
Although the proposed data law being presented by Ms. Reding will apply to companies in Europe, it will clearly affect technology companies around the globe too, including Facebook, Twitter and Google in the United States.
At the World Economic Forum, the discussion is also focusing on one of the toughest regulation challenges in regards to data collection: How to manage a balancing act between governments overseeing data collection and its actions stifling innovation. More importantly, regulators hope to figure out a global solution to data collection.
As the World Economic Forum reports says: ”Concerted action is needed by governments, development organizations and companies to ensure that this data helps the individuals and communities who create it.”
Why 3-D Printing Will Go the Way of Virtual Reality
Extruding, printing, and sintering are not the same as manufacturing.
Christopher Mims, Technology Review, January 25, 2012
There is a species of magical thinking practiced by geeks whose experience is computers and electronics—realms of infinite possibility that are purposely constrained from the messiness of the physical world—that is typical of Singularitarianism, mid-90s missives about the promise of virtual reality, and now, 3-D printing.
As 3-D printers come within reach of the hobbyist—$1,100 for MakerBot's Thing-O-Matic—and The Pirate Bay declares "physibles" the next frontier of piracy, I'm seeing usually level-headed thinkers like Clive Thompson and Tim Maly declare that the end of shipping is here and we should all start boning up on Cory Doctorow's science fiction fantasies of a world in which any object can be rapidly synthesized with a little bit of energy and raw materials.
This isn't just premature, it's absurd. 3-D printing, like VR before it, is one of those technologies that suggest a trend of long and steep adoption driven by rapid advances on the systems we have now. And granted, some of what's going on at present is pretty cool—whether it's in rapid prototyping, solid-fuel rockets, bio-assembly or just giant plastic showpieces.
But the notion that 3-D printing will on any reasonable time scale become a "mature" technology that can reproduce all the goods on which we rely is to engage in a complete denial of the complexities of modern manufacturing, and, more to the point, the challenges of working with matter.
Let's start with the mechanism. Most 3-D printers lay down thin layers of extruded plastic. That's great for creating cheap plastic toys with a limited spatial resolution. But printing your Mii or customizing an iPhone case isn't the same thing as firing ceramics in a kiln or smelting metal or mixing lime with sand at high temperatures to produce glass—unless you'd like everything that's currently made from those substances to be replaced with plastic, and there are countless environmental, health, and durability reasons you don't.
Advocates of 3-D printing also neglect entirely the fact that so much of what we use continues to be made out of natural substances, and for good reason. By any number of measures, wood is pound-for-pound stronger than steel, and the move toward natural products for packaging suggests that the strength and affordability of paper, bamboo and even mushrooms mean that in the future there will be more and not less of all of these.
The desire for 3-D printing to take over from traditional manufacturing needs to be recognized for what it is: an ideology. Getting all of our goods from a box in the corner of our home has attractive implications, from mass customization to "the end of consumerism." With stakes like those, who wouldn't want to be a true believer?
Hype is inevitably followed by some level of backlash, or at least disinterest, and it would be a shame for 3-D printing to head into a too-deep trough of the Gartner hype cycle. There will be plenty of interesting applications for 3-D printing, but I'll bet the ones that will have the biggest impact will be within traditional factories, where rapid prototyping is already having a huge impact.
Khosla: Do We Need Doctors or Algorithms?
Vinod Khosla, TechCrunch, January, 2012
Editor’s note: This is Part II of a guest series written by legendary Silicon Valley investor Vinod Khosla, the founder of Khosla Ventures. In Part I, he laid the groundwork by describing how artificial intelligence is a combination of human and computer capabilities. In Part III, he will talk about how technology will sweep through education.
Later, I got to cogitating about what I had said and why, and how embarrassingly wrong that might be. But the more I think about it the more I feel my gut reaction was probably right. The beginnings of “Doctor Algorithm” or Dr. A for short, most likely (and that does not mean “certainly” or “maybe”) will be much criticized. We’ll see all sorts of press wisdom decrying “they don’t work” or “look at all the silly things they come up with.” But Dr A. will get better and better and will go from providing “bionic assistance” to second opinions to assisting doctors to providing first opinions and as referral computers (with complete and accurate synopses and all possible hypotheses of the hardest cases) to the best 20% of the human breed doctors. And who knows what will happen beyond that?
Assessing Current Healthcare
Let’s start with healthcare (or sickcare, as many knowledgeable people call it). Think about what happens when you visit a doctor. You have to physically go to the hospital or some office, where you wait (with no real predictability for how long), and then the nurse probably takes you in and checks your vitals. Only after all this does the doctor show up and, after some friendly banter, asks you to describe your own symptoms.
The doctor assesses them and hunts around (probably in your throat or lungs) for clues as to their source, provides the diagnosis, writes a prescription, and sends you off.
The entire encounter should take no more than 15 minutes and usually takes probably less than that. Sometimes a test or two may be ordered, if you can afford it. And, as we all know, most of the time, it turns out to be some routine diagnosis with a standard treatment . . . something a computer algorithm could do if the treatment involved no harm, or at least do as well as the median doctor (I am not talking about the top 20% of doctors here—80% of doctors are below the “top 20%” but that is hard for people to intuit!).
So what’s wrong with this situation? This is by no means an exhaustive list, but it sets up a nice springboard:
Physically having to go to your doctor’s office makes sense for the most part, except that a lot of the basic tests are either visual (tongue and throat check) or auditory (listening to the breath and vibrations in the abdomen). Time plus cost will often discourage people from taking that first step to visit a doctor. Most of the time a Dr. A could at least advise you when it is worth visiting based on your normal body functions, your current indications, and your locality’s current infections and other symptom trends.
- A lot of the vitals being tested for (e.g. blood pressure, pulse) can now be routinely done at home or even with the help of an iPhone and an explosion of additional possibilities will emerge in the next decade.
- You are the one telling the doctor your symptoms.
- The doctor has to inquire (probably every time) into any possible history of each symptom, test results, and illnesses, except when he does not have time for you in that village in India.
- The prescriptions are still done on paper, requiring you to, again, physically go to a pharmacy and pick up what you need there. So compliance is an issue.
Going down the list, we find a pretty negative assessment. The vital signs could all be determined with the help of mobile devices, the operation of which do not require years of training and a certification. You will be able to do this by yourself—Philips already is using the iPhone camera to try to measure vital indicators, others will be even more innovative and as an insurance company it would be cost-effective to give them to every insured person for free. Skin Scan is measuring your risk of skin cancer from a photograph of a skin lesion. Telemedicine is accelerating and a Qualcomm company is measuring heart rates using an iPhone. Cell phones that display your vital signs and take ultrasound images of your heart or abdomen are in the offing as well as genetic scans of malignant cells that match your cancer to the most effective treatment. Ear infection and skin rash pictures and more will all be mobile phone based, often supplemented by the kind of (fractal) analysis that Skin Scan does, and more than what the doctors naked eye could usually see.
The history of symptoms, illnesses, and test results could be accessed, processed, and assessed by a computer to see any correlation or trends with the patient’s past. You are the one providing the doctor with the symptoms anyway after all!
Any follow-up hunts for clues could again be done with mobile devices. The prescriptions—along with the medical records—could relocate to electronic and digital methods, saving paper, reducing bureaucracy, and easing the healing process. If 90% of the time the doctor knows exactly the right kind of diagnosis from these very few and superficial inputs (we haven’t even considered genetics yet!), does it really require 10+ years of intense education for every diagnostician?
The fault is not entirely with the doctors, though. Most of us don’t know what set of symptoms warrant the full-scale attention of medical personnel, so we either go all the time or we do not go at all (save for emergencies). We also cannot realistically expect any (even our family) doctor to remember every single symptom and test result over the years, definitely not in a government hospital in China. Similarly, we cannot expect our doctor to be able to remember everything from medical school twenty years ago or memorize the whole Physicians Desk Reference (PDR) and to know everything from the latest research, and so on and so forth. This is why, every time I visit the doctor, I like to get a second opinion. I do my Internet research and feel much better.
Identifying Emerging Trends In Healthcare
But I always wonder why I cannot input my specific test numbers and have a system offer me a “second opinion” on the diagnosis since it has all the data that the doctor has and can use all my current and historical data effectively. In fact, it is not hard to imagine it having more data than the doctor has since my full patient record would be at the tip of its digital brain, unlike the average doctor who probably doesn’t remember my blood glucose levels or my ferritin from two years ago. He does not remember all the complex correlations from med school in which ferritin matters—there are three thousand or more metabolic pathways, I was once told, in the human body and they impact each other in very complex ways. These tasks are perfect for a computer to model as “systems biology” researchers are trying to do.
Add to it my baseline numbers from when I was not sick, which most doctors don’t have and if they did 80% of physicians would be too lazy to use or not know how to use. Applied Proteomics can extract tens of gigabytes of proteomics—what my genes are actually doing instead of what they can do—baseline data from one drop of blood.
Oh, by the way I have my 23andMe data to add my genetic propensities (howsoever imprecise today, but improving rapidly with time and more data). The doctor uses a lot of imprecise judgments too as most good doctors will readily admit. My very good doctor did not check that I have relative insensitivity, genetically, to Metformin, a diabetes drug. It is easy to input the PDR (the Physicians Desk Reference), the massively thick, small-font book that all physicians are supposed to know backwards and forwards. They often don’t remember everything they read, in med school but it is a piece of cake for computers. The book on your typical doctor’s desk is probably not current on the leading-edge science either. Confirmed science and emerging science are different things and each has a role. Doctors mostly use confirmed science, the average doctor not understanding and pros and cons of each or the expected value of a treatment (benefit and harm). And our 18th century tradition of “first do no harm” dictates that if a treatment hurts ten patients a year but saves a thousand lives we reject it.
With enough examples, today’s techniques for language translation (or newer techniques) can translate from human lingo for symptoms (“I feel itchy” or “buzzy” or “reddish bubbly rash with pimples” or “less energy in the morning” or “sort of a stretch in my tendon” and the myriad of imprecise ways symptoms are described and results interpreted — these are highly amenable to big data analysis) into medical lingo matching the PDR. With easy input of real medical results into a computer and long-standing historical data per patient and per population, which a human cannot possibly handle, and patient and population genetics, I suspect getting a second opinion of my diagnosis from Dr. A is a reasonable expectation, and it should certainly be better than a middling physician’s (especially in less developed countries like India, where there is a dire shortage of trained physicians).
I may still need a surgeon (though robotic surgeons like those from Intuitive Surgical are on the way too) or other specialists for some tasks for a little while and the software may move from “second opinion” (in three years? Or seven?) to “bionic software” for the physicians (in five or ten years, with enough patient data?). Bionic software, again, defined here as software which augments and amplifies human understanding.
But I doubt very much if within 10-15 years (given continued investment and innovation and keeping the AMA from quashing such efforts politically) I won’t be able to ask Siri’s great great grandchild (Version 9.0?) for an opinion far more accurate than the one I get today from the average physician. Instead of asking Siri 9.0, “I feel like sushi” or “where can I dispose a body” (try it…it’s fairly accurate!) and with your iPhone X or Android Y with all the power of IBM’s current Watson computer in the mobile phone and an even more powerful “Nvidia times 10-100” server which will cost far less than med school with terabytes or petabytes of data on hundreds of millions (billions?) of patients, including their complete genomics and proteomics (each sample costing about the same as a typical blood test).
IBM’s Watson computer, I understand, is now being applied to medical diagnosis after handling imprecise and vague tasks like winning at Jeopardy, which experts a few years ago would have said could not be done. “Computers cannot match the judgment of humans on these kinds of tasks!” And with enough data, medical diagnosis or 90% of it is an easier task than Jeopardy.
Already Kaiser Permanent already has 10 million real-time medical records with details of 30,000,000 e-visits last year with caregivers and computer modeling of key diseases per individual that data scientists would love to get their hand on. Already, according to IDC 14% of the US population is using their phones for medical help and 200 million health and fitness related mobile applications have been downloaded according to pyramid research. Fun stuff, though early. They are probably two generations away from systems that are actually useful.
A more elaborate vision, one that is not very useful today because of lack of enough data and enough science, is defined in Experimental Man and websites like Quantified Self. Though they feel like toys today, they are much further along than the mobile phone was pre-iPhone in January of 2007. And data, the key ingredient to useful analysis, and diagnosis, is starting to explode exponentially—be it genetic data, proteomic data or physical data about my steps, my exercises, my stress levels or my normal heart and respiration rates.
My UP wristband or something like it (disclosure: I am an investor in Jawbone)) will know all my sleep patterns when I am healthy and how many steps I take each day and may have more data on my mobility if I ever get depressed than any psychiatrist ever will know what to do with. Within a few years, my band will know my heart rate at all times, my respiration rate, my galvanic skin resistance (one parameter among multiple ones used to measure my stress level), my metabolic rate (should cost about $10 to add to the band by measuring my CO2 in my breath and may detect changes in my body chemistry too like when I get a certain type of cancer and traces of it show up in my breath).
All my “health data” as well as my “sick data” and my “activity data” will be accessible to Dr. A (and location when I was stressed or breathing hard or getting the allergic reaction and what chemicals were nearby or in the air—did toluene exposure cause me to break out in a rash from that new carpet or trigger a systemic reaction from my body?). I doubt I will be prescribed an arthritis medicine without Dr. A knowing my genetics and the genetics of my autoimmune disease. Or a cancer medicine without the genetics of my cancer when the genetic sequence (once per life) costs far less than a single dose of medicine. In fact all my infectious disease treatments may be based on analysis of my full genome and my history of exposure to viruses, bacteria and toxic chemicals.
Constant everyday health data from non-medical devices will swamp the “sickness tests” used in most medical diagnosis and be supplemented by detailed genetic, proteomic and sick data with bionic software and machine learning systems. Siri might even remind me one day that my heart rate while sleeping has gone up abnormally over the last year, so I should go run some heart sickness cardiograms or imaging tests.
Obviously, Siri’s children and its server friends will be able to keep up with the latest research and decide on optimal strategies based on patient preference (“I prefer to live longer even if it means all the fancy treatments” or “I want to live a normal life and die. I prefer to spend more of my time with my children than at the hospital” or “I like taking risky treatments”). They will take into account known research, early pioneering approaches, very complex interrelationships and much more.
My best guess is that today a physician’s bias makes all these personal decisions for patients in a majority of the cases without the patient (or sometimes even the physician) realizing what “preferences “ are being incorporated into their recommendations. The situation gets worse the less educated or economically less well-off the patient is, such as in developing countries, in my estimation.
Envisioning Future Healthcare
Eventually, we won’t need the average doctor and will have much better and cheaper care for 90-99% of our medical needs. We will still need to leverage the top 10 or 20% of doctors (at least for the next two decades) to help that bionic software get better at diagnosis. So a world mostly without doctors (at least average ones) is not only not reasonable, but also more likely than not. There will be exceptions, and plenty of stories around these exceptions, but what I am talking about will most likely be the rule and doctors may be the exception rather than the other way around.
However fictionalized, we will be aiming to produce doctors like Gregory House who solve biomedical puzzles beyond our best input ability. And India, China and other countries may not have to worry about the investment in massive healthcare or massive inequalities in the type of physicians they might have access to. And hopefully our bionic software (or independent software someday) will be free of the influence of heavily marketed but only minimally effective drugs or treatment regimes or branding campaigns against generics or lower-cost and equally effective, more affordable drugs and treatments. Dr. A will be able to do a cost optimization too both at the patient level and at the policy level (but we may choose, at least for a decade or two, to reject its recommendations—we will still be free to be stupid or political).
What is important to realize is how medical education and the medical profession will change toward the better as a result of these trends. The vision I am proposing here, though, is one in which those decades of learning and experience are used where they actually matter. We consider doctors some of the most learned people in our society. We should aim to use their time and knowledge in the most efficient manner possible.
And everybody should have access to the skills of the very best ones instead of only having access to the average doctor. And the not so “Dr. House’ doctors will help us with better patient skills, bedside manners, empathy, advice and caring, and they will have more time for that too. If computers can drive cars and deal with all the knowledge in jeopardy, surely their next to next to next…generation can do diagnosis, treatment and teaching in these far less uncertain domains and with a lot more data. Further the equalizing impact of both electronic doctors and teaching environments has hugely positive social implications. Besides, who wants to be treated by an “average” doctor? And who does not want to be an empowered patient?
The best way to predict this future is not to extrapolate the past and what has or has not worked, but to invent the future we want, the one we believe possible!
Image credit: Shutterstock/koya979
P Please consider the environment before printing this e-mail.
Wednesday, January 25, 2012
The State of the Union and the Power of Technological Change
Robert J. Shapiro, Sonecon, January 25th, 2012
President Obama made inequality a major theme of his State of the Union address last night, an unsurprising choice as he prepares to face Mitt Romney.
Everyone now knows that just last year Mr. Romney paid a smaller share of his $21 million income in taxes than the average American paid on a $50,000 salary. But if inequality was the President’s theme, his main subject was jobs. For Obama, faster job growth depends on more government. We need Washington, for example, to retrain workers, reduce college costs, and provide special supports for manufacturers. For Romney, the answer for job creation is, what else, less government: Washington needs only to cut regulation and reduce taxes, especially for the wealthy people and corporations who, in the Romney worldview, create the jobs. But not so fast — there are other options as well. A new report from the NDN think tank suggests that certain kinds of new technologies can spur job creation more effectively than most government programs or tax cuts. The new study, conducted by Kevin Hassett of the American Enterprise Institute and myself, found that the rapid spread of new 3G wireless devices from 2007 to 2011 led directly to the creation of nearly 1.6 million new jobs. And those job gains occurred even as the overall economy was shedding 5.3 million other jobs.
Our analysis tracked shifts by consumers and businesses from 2G wireless phones to 3G smart phones and tablets, quarter by quarter and state by state, from July 2007 to December 2011. We then analyzed the links between the shift to the more powerful 3G devices and changes in employment, quarter to quarter and state by state. We did the math and found that every 10 percentage point increase in the use of those devices generated more than 231,000 new jobs within a year.
It makes clear and compelling economic sense. As a growing share of Internet use shifts to wireless devices, the people and businesses that use them become more efficient and productive. Those gains, in turn, create new value which ultimately leads to more job creation. The spread of 3G wireless devices also created a platform for new services — for example, in mobile e-commerce, mobile social networking, and location-based services. The growth of those new services also led to more job creation.
And the best news for jobs is that another technological shift is occurring right now, from 3G to 4G wireless devices. 4G wireless networks and the Internet infrastructure that supports them have the potential to drive significant new efficiencies and innovations across the economy. Jobs already are being created in 4G-dependent areas such as cloud-based services and mobile health applications. According to industry analysts, 4G wireless networks in the near future could be used to create a Smart Electricity Grid and a national public safety system.
This analysis, then, can provide a new direction for job creation efforts: Adopt spectrum and other policies that will promote the broad and rapid deployment of 4G
Still, there are also kernels of economic truth in the Romney and Obama positions.
Romney is not wrong, for example, when he says that lower taxes are usually better for the economy than higher taxes. But there’s no evidence that lower taxes on wealthy people or corporations would produce many jobs. And in a period of trillion-dollar budget deficits, calls for tax cuts seem at best irrelevant, and at worst politically cynical and misleading.
The President is on firmer ground. Greater access to higher education and retraining should increase productivity and growth, at least over the long haul. Since the direct benefits from those efforts would presumably go to people from modest or middle-income households, Obama’s approach also could help address inequality. And since the President seems prepared to raise the revenues to finance his proposals, they could be more than political window dressing.
For all of these good points, these approaches are not the answer to slow job creation. For that, President Obama and Mr. Romney have to directly address the forces that actually create and destroy private-sector jobs. One such force is technology, and our new analysis shows that the 3G and 4G wireless technologies can create many more jobs than they may destroy, and do so quickly. Another approach could focus on reducing the additional costs that businesses bear directly when they create new jobs.
That could mean cuts on the employer side of the payroll tax or new measures to slow increases in the health care costs that businesses bear for their employees. At a minimum, any of these approaches would produce more economic benefits for more people than all of the tax cuts promoted by Obama’s opponents.
SEE ALSO REPORT on
The Employment Effects of Advances in Internet and Wireless Technology at: http://www.sonecon.com/docs/studies/Wireless_Technology_and_Jobs-Shapiro_Hassett-January_2012.pdf
President Obama made inequality a major theme of his State of the Union address last night, an unsurprising choice as he prepares to face Mitt Romney.
Everyone now knows that just last year Mr. Romney paid a smaller share of his $21 million income in taxes than the average American paid on a $50,000 salary. But if inequality was the President’s theme, his main subject was jobs. For Obama, faster job growth depends on more government. We need Washington, for example, to retrain workers, reduce college costs, and provide special supports for manufacturers. For Romney, the answer for job creation is, what else, less government: Washington needs only to cut regulation and reduce taxes, especially for the wealthy people and corporations who, in the Romney worldview, create the jobs. But not so fast — there are other options as well. A new report from the NDN think tank suggests that certain kinds of new technologies can spur job creation more effectively than most government programs or tax cuts. The new study, conducted by Kevin Hassett of the American Enterprise Institute and myself, found that the rapid spread of new 3G wireless devices from 2007 to 2011 led directly to the creation of nearly 1.6 million new jobs. And those job gains occurred even as the overall economy was shedding 5.3 million other jobs.
Our analysis tracked shifts by consumers and businesses from 2G wireless phones to 3G smart phones and tablets, quarter by quarter and state by state, from July 2007 to December 2011. We then analyzed the links between the shift to the more powerful 3G devices and changes in employment, quarter to quarter and state by state. We did the math and found that every 10 percentage point increase in the use of those devices generated more than 231,000 new jobs within a year.
It makes clear and compelling economic sense. As a growing share of Internet use shifts to wireless devices, the people and businesses that use them become more efficient and productive. Those gains, in turn, create new value which ultimately leads to more job creation. The spread of 3G wireless devices also created a platform for new services — for example, in mobile e-commerce, mobile social networking, and location-based services. The growth of those new services also led to more job creation.
And the best news for jobs is that another technological shift is occurring right now, from 3G to 4G wireless devices. 4G wireless networks and the Internet infrastructure that supports them have the potential to drive significant new efficiencies and innovations across the economy. Jobs already are being created in 4G-dependent areas such as cloud-based services and mobile health applications. According to industry analysts, 4G wireless networks in the near future could be used to create a Smart Electricity Grid and a national public safety system.
This analysis, then, can provide a new direction for job creation efforts: Adopt spectrum and other policies that will promote the broad and rapid deployment of 4G
Still, there are also kernels of economic truth in the Romney and Obama positions.
Romney is not wrong, for example, when he says that lower taxes are usually better for the economy than higher taxes. But there’s no evidence that lower taxes on wealthy people or corporations would produce many jobs. And in a period of trillion-dollar budget deficits, calls for tax cuts seem at best irrelevant, and at worst politically cynical and misleading.
The President is on firmer ground. Greater access to higher education and retraining should increase productivity and growth, at least over the long haul. Since the direct benefits from those efforts would presumably go to people from modest or middle-income households, Obama’s approach also could help address inequality. And since the President seems prepared to raise the revenues to finance his proposals, they could be more than political window dressing.
For all of these good points, these approaches are not the answer to slow job creation. For that, President Obama and Mr. Romney have to directly address the forces that actually create and destroy private-sector jobs. One such force is technology, and our new analysis shows that the 3G and 4G wireless technologies can create many more jobs than they may destroy, and do so quickly. Another approach could focus on reducing the additional costs that businesses bear directly when they create new jobs.
That could mean cuts on the employer side of the payroll tax or new measures to slow increases in the health care costs that businesses bear for their employees. At a minimum, any of these approaches would produce more economic benefits for more people than all of the tax cuts promoted by Obama’s opponents.
SEE ALSO REPORT on
The Employment Effects of Advances in Internet and Wireless Technology at: http://www.sonecon.com/docs/studies/Wireless_Technology_and_Jobs-Shapiro_Hassett-January_2012.pdf
Vivek Kundra: Digital Fuel of the 21st Century: Innovation through Open Data and the Network Effect
Digital Fuel of the 21st Century: Innovation through Open Data and the Network Effect
Vivek Kundra
Shorenstein Center Fellow,
Fall 2011
Executive Vice President of Emerging Markets, Salesforce.com; Former U.S. Chief Information Officer
Read the full paper (PDF).
Excerpt:
A Shift in Power
In the information economy, data is power and we face a choice between democratizing it and holding on to it for an asymmetrical advantage. For example, before a 13-year race to crack the code of life was complete,1 a team of international scientists gathered in Bermuda to discuss the strategy for managing the Human Genome Project data. They made a set of critical decisions to make the human genome data freely available in the public domain. These decisions came to be known as the "Bermuda Principles."2 This decision gave rise to an ecosystem of scientists and companies that have advanced everything from personalized medicine to creating economic activity that improves the human condition.
In today's age of information, data is supremely important. We generate and store more data today than any other time in history. In fact, data is predicted to continue along its exponential growth curve to 1.8 zettabytes in 2011. To get a sense of this exponential growth, a zettabyte is a trillion gigabytes; that's 1,000,000,000,000,000,000,000.3 If this data isn't sliced, diced and cubed to separate signal from noise, it can be useless. But, when made available to the public and combined with the network effect—defined by Reed's Law,4 which asserts that the utility of large networks, particularly social networks, can scale exponentially with the size of the network—society has the potential to drive massive social, political and economic change.
In today's world, open data leveraged by networks is the fuel that powers important decisions at each level of society—from government, to business, to community, to households—but it is also a product of our every activity at every level of our existence. Channeling the power of this open data and the network effect can help:
1. Fight government corruption, improve accountability and enhance government services
2. Change the default setting of government to open, transparent and participatory
3. Create new models of journalism to separate signal from noise to provide meaningful insights
4. Launch multi-billion dollar businesses based on public sector data
Technology enables the disruption of institutions that was structurally not even possible before. The adoption of social, mobile and cloud technologies is lowering the co-efficient of friction and giving rise to the network effect that has already changed the power dynamics between large institutions and the people.
Facebook has more than 800 million active users5 and Twitter has over 100 million active users.6 These are still early days in the social space when you consider that there are approximately 7 billion people7 in the world and 5 billion people8 with mobile devices that enable them to connect in ways that were impossible just years ago. The ability of 5 billion people to instrument the world and share their experiences in a low-cost manner has forever shifted power away from the hands of the few to the network.
1. U.S. Department of Energy, "The Human Genome Project," http://www.ornl.gov/sci/techresources/Human_Genome/project/about.shtml.
2. Michael Nielson, Reinventing Discovery: The New Era of Networked Science (Princeton University Press, 2011), 7.
3. Juan Enriquez, "The Glory of Big Data." Popular Science, October 31, 2011, http://www.popsci.com/technology/article/2011-10/glory-big-data.
4. David P. Reed, "The Law of the Pack," Harvard Business Review, (2001): 1.
5. Facebook, "Facebook Statistics," http://www.facebook.com/press/info.php?statistics.
6. Twitter, "One hundred million voices," http://blog.twitter.com/2011/09/one-hundred-million-voices.html.
7. Census Bureau, "U.S. & World Population Clocks," http://www.census.gov/main/www/popclock.html.
8. BBC News, "Over 5 billion mobile phone connections worldwide," http://www.bbc.co.uk/news/10569081.
Vivek Kundra
Shorenstein Center Fellow,
Fall 2011
Executive Vice President of Emerging Markets, Salesforce.com; Former U.S. Chief Information Officer
Read the full paper (PDF).
Excerpt:
A Shift in Power
In the information economy, data is power and we face a choice between democratizing it and holding on to it for an asymmetrical advantage. For example, before a 13-year race to crack the code of life was complete,1 a team of international scientists gathered in Bermuda to discuss the strategy for managing the Human Genome Project data. They made a set of critical decisions to make the human genome data freely available in the public domain. These decisions came to be known as the "Bermuda Principles."2 This decision gave rise to an ecosystem of scientists and companies that have advanced everything from personalized medicine to creating economic activity that improves the human condition.
In today's age of information, data is supremely important. We generate and store more data today than any other time in history. In fact, data is predicted to continue along its exponential growth curve to 1.8 zettabytes in 2011. To get a sense of this exponential growth, a zettabyte is a trillion gigabytes; that's 1,000,000,000,000,000,000,000.3 If this data isn't sliced, diced and cubed to separate signal from noise, it can be useless. But, when made available to the public and combined with the network effect—defined by Reed's Law,4 which asserts that the utility of large networks, particularly social networks, can scale exponentially with the size of the network—society has the potential to drive massive social, political and economic change.
In today's world, open data leveraged by networks is the fuel that powers important decisions at each level of society—from government, to business, to community, to households—but it is also a product of our every activity at every level of our existence. Channeling the power of this open data and the network effect can help:
1. Fight government corruption, improve accountability and enhance government services
2. Change the default setting of government to open, transparent and participatory
3. Create new models of journalism to separate signal from noise to provide meaningful insights
4. Launch multi-billion dollar businesses based on public sector data
Technology enables the disruption of institutions that was structurally not even possible before. The adoption of social, mobile and cloud technologies is lowering the co-efficient of friction and giving rise to the network effect that has already changed the power dynamics between large institutions and the people.
Facebook has more than 800 million active users5 and Twitter has over 100 million active users.6 These are still early days in the social space when you consider that there are approximately 7 billion people7 in the world and 5 billion people8 with mobile devices that enable them to connect in ways that were impossible just years ago. The ability of 5 billion people to instrument the world and share their experiences in a low-cost manner has forever shifted power away from the hands of the few to the network.
1. U.S. Department of Energy, "The Human Genome Project," http://www.ornl.gov/sci/techresources/Human_Genome/project/about.shtml.
2. Michael Nielson, Reinventing Discovery: The New Era of Networked Science (Princeton University Press, 2011), 7.
3. Juan Enriquez, "The Glory of Big Data." Popular Science, October 31, 2011, http://www.popsci.com/technology/article/2011-10/glory-big-data.
4. David P. Reed, "The Law of the Pack," Harvard Business Review, (2001): 1.
5. Facebook, "Facebook Statistics," http://www.facebook.com/press/info.php?statistics.
6. Twitter, "One hundred million voices," http://blog.twitter.com/2011/09/one-hundred-million-voices.html.
7. Census Bureau, "U.S. & World Population Clocks," http://www.census.gov/main/www/popclock.html.
8. BBC News, "Over 5 billion mobile phone connections worldwide," http://www.bbc.co.uk/news/10569081.
FT on Big Data
Decisions, decisions … will ‘Big Data’ have ‘Big’ impact?
Corey Yulinsky, The Financial Times, February 24, 2012
A phone book with a billion pages would reach about 80 miles into the sky. ‘Big Data’ is about utilising computer databases containing that scale of information - and more. This new management buzzword is clearly here to stay.
Such the attention is well-deserved. The potential advantage to companies able to take trillions of bytes of information, mine relevant data, convert it into insights, and make it useful for customers and employees is enormous. Information abundance and technology advances have taken us from a long era of information scarcity and dropped us into the deep end of information overload. The real question facing companies is how to ensure that the big potential of Big Data is actually realised.
The answer is to recognise that the most powerful benefits of Big Data come from changing the core elements of an organisation’s operating model as much if not more so than its technology. The first step is to understand “what’s so big about Big Data?”
Data becomes Big Data with the confluence of four factors
1. Information ubiquity:The breadth, volume and timeliness of available data, structured and unstructured;
2. Speed: The ability to store, process, and retrieve massive quantities in dramatically reduced timeframes, often near real-time;
3. Machine intelligence: Processing power driving algorithms that access vast data sources and drive decisions – amplified by feedback loops enabling continuous learning and improvement;
4.Economics: The cost of all of this data collection, storage, processing and delivery has dropped radically.
Big Data only has an impact when and organisation can capitalise on these attributes to change the decisions it makes or the way decisions are made. Don’t forget that the predecessor of Big Data was often called “decision support.” Otherwise, it is simply an interesting but low-value diversion of resources. Organisations that consciously and explicitly adapt decision-making processes to the opportunities afforded by Big Data analytics will reap the benefits. Others will experience the frustration and bottlenecks we saw with CRM (customer relationship management) and similar data-driven initiatives.
An approach to revolutionising decision-making starts with recognising four broad categories of decision-making and how they involve Big Data:
1. Episodic decisions: Decisions made infrequently, sometimes on a regular cycle and sometimes in response to triggering events. Often strategic in nature (which businesses/markets to enter/exit, which customer sets to pursue) these decisions are highly discretionary in nature, and require synthesis of multiple information sources. Increasingly, simulation models (combining descriptive and predictive approaches) are used to drive scenario analyses making tradeoffs more clear.
2.Fixed cycle decisions: All businesses have an ongoing cadence for key processes (demand forecasting, sales calling, marketing campaigns, pricing, inventory replenishment). Historically made as discretionary decisions backed by “rear view mirror” performance data, they are now often supported by predictive analytics leveraging Big Data flows.
3.Embedded algorithmic decisions: These “machine-made” continuous decisions driven by optimisation algorithms built-in to consumer (or frontline) facing interfaces are most often the face of Big Data decision making. They include dynamic online offers, real-time credit and pricing decisions, automated underwriting and call routing. These decisions driving “personalised” outcomes are the type most often changed by the data explosion.
4.“Controller” decisions: Evaluation, iteration and refinement of decision-making is usually focused on the effectiveness and ROI of algorithmic decisions. Machine learning ensures that the algorithms constantly get better, but there also has to be a rigorous layer of analysis to make sure these algorithms stay aligned with the company’s objectives and targets.
When viewed through this decision lens, it becomes clear that there are three major ways to create winning impact with Big Data.
1.The Battle of Business Rules – Who will create the most compelling and effective algorithms? Companies who have the capability to understand which data and which analytic insights will drive desired outcomes will create the foundation for cumulative learning and increasing advantage. Machines still need to be “taught” foundational business rules and the smartest teachers will win.
2.Decision Migration—Big Data’s analytics and processing aspects create the opportunity to transform episodic or fixed cycle decisions into algorithmic decisions, with potentially lucrative impacts for disrupters. In effect, this happened in segments of the advertising business where a large portion of traditional negotiated price (episodic/fixed cycle) ads were replaced by online ads driven by real-time auction pricing (algorithmic continuous). Another example can be found in the stock market with the advent of high frequency trading. Identifying which decisions can be migrated and how to do so will be another competitive arena.
3.Information Transformation – More rarely, we expect to see instances where new ways to play emerge in existing industries driven by companies that reset industry economics through by applying new analysis driven by Big Data capabilities. Featuring capabilities and cultures that are very different from their competitors, their strategic decisions are made differently and manifest themselves across the organisation. Two early examples are Harrah’s (now Caesars) in the gaming industry and Capital One in credit cards. In both cases, a CEO saw the transformative power of intensely data-driven decision processes to catalyse their businesses. The nature of this kind of transformation is complex relative to the other ways to win, but the pay-offs can be terabyte-big.
For Big Data to go from buzzword to providing value to your bottom line, the quickest route is to identify, manage and evolve the way your company uses data to make decisions and then focus on using Big Data to transform the key decisions that drive your business model.
Corey Yulinsky, a New York-based Partner in Booz $ Company’s Financial Services practice
Corey Yulinsky, The Financial Times, February 24, 2012
A phone book with a billion pages would reach about 80 miles into the sky. ‘Big Data’ is about utilising computer databases containing that scale of information - and more. This new management buzzword is clearly here to stay.
Such the attention is well-deserved. The potential advantage to companies able to take trillions of bytes of information, mine relevant data, convert it into insights, and make it useful for customers and employees is enormous. Information abundance and technology advances have taken us from a long era of information scarcity and dropped us into the deep end of information overload. The real question facing companies is how to ensure that the big potential of Big Data is actually realised.
The answer is to recognise that the most powerful benefits of Big Data come from changing the core elements of an organisation’s operating model as much if not more so than its technology. The first step is to understand “what’s so big about Big Data?”
Data becomes Big Data with the confluence of four factors
1. Information ubiquity:The breadth, volume and timeliness of available data, structured and unstructured;
2. Speed: The ability to store, process, and retrieve massive quantities in dramatically reduced timeframes, often near real-time;
3. Machine intelligence: Processing power driving algorithms that access vast data sources and drive decisions – amplified by feedback loops enabling continuous learning and improvement;
4.Economics: The cost of all of this data collection, storage, processing and delivery has dropped radically.
Big Data only has an impact when and organisation can capitalise on these attributes to change the decisions it makes or the way decisions are made. Don’t forget that the predecessor of Big Data was often called “decision support.” Otherwise, it is simply an interesting but low-value diversion of resources. Organisations that consciously and explicitly adapt decision-making processes to the opportunities afforded by Big Data analytics will reap the benefits. Others will experience the frustration and bottlenecks we saw with CRM (customer relationship management) and similar data-driven initiatives.
An approach to revolutionising decision-making starts with recognising four broad categories of decision-making and how they involve Big Data:
1. Episodic decisions: Decisions made infrequently, sometimes on a regular cycle and sometimes in response to triggering events. Often strategic in nature (which businesses/markets to enter/exit, which customer sets to pursue) these decisions are highly discretionary in nature, and require synthesis of multiple information sources. Increasingly, simulation models (combining descriptive and predictive approaches) are used to drive scenario analyses making tradeoffs more clear.
2.Fixed cycle decisions: All businesses have an ongoing cadence for key processes (demand forecasting, sales calling, marketing campaigns, pricing, inventory replenishment). Historically made as discretionary decisions backed by “rear view mirror” performance data, they are now often supported by predictive analytics leveraging Big Data flows.
3.Embedded algorithmic decisions: These “machine-made” continuous decisions driven by optimisation algorithms built-in to consumer (or frontline) facing interfaces are most often the face of Big Data decision making. They include dynamic online offers, real-time credit and pricing decisions, automated underwriting and call routing. These decisions driving “personalised” outcomes are the type most often changed by the data explosion.
4.“Controller” decisions: Evaluation, iteration and refinement of decision-making is usually focused on the effectiveness and ROI of algorithmic decisions. Machine learning ensures that the algorithms constantly get better, but there also has to be a rigorous layer of analysis to make sure these algorithms stay aligned with the company’s objectives and targets.
When viewed through this decision lens, it becomes clear that there are three major ways to create winning impact with Big Data.
1.The Battle of Business Rules – Who will create the most compelling and effective algorithms? Companies who have the capability to understand which data and which analytic insights will drive desired outcomes will create the foundation for cumulative learning and increasing advantage. Machines still need to be “taught” foundational business rules and the smartest teachers will win.
2.Decision Migration—Big Data’s analytics and processing aspects create the opportunity to transform episodic or fixed cycle decisions into algorithmic decisions, with potentially lucrative impacts for disrupters. In effect, this happened in segments of the advertising business where a large portion of traditional negotiated price (episodic/fixed cycle) ads were replaced by online ads driven by real-time auction pricing (algorithmic continuous). Another example can be found in the stock market with the advent of high frequency trading. Identifying which decisions can be migrated and how to do so will be another competitive arena.
3.Information Transformation – More rarely, we expect to see instances where new ways to play emerge in existing industries driven by companies that reset industry economics through by applying new analysis driven by Big Data capabilities. Featuring capabilities and cultures that are very different from their competitors, their strategic decisions are made differently and manifest themselves across the organisation. Two early examples are Harrah’s (now Caesars) in the gaming industry and Capital One in credit cards. In both cases, a CEO saw the transformative power of intensely data-driven decision processes to catalyse their businesses. The nature of this kind of transformation is complex relative to the other ways to win, but the pay-offs can be terabyte-big.
For Big Data to go from buzzword to providing value to your bottom line, the quickest route is to identify, manage and evolve the way your company uses data to make decisions and then focus on using Big Data to transform the key decisions that drive your business model.
Corey Yulinsky, a New York-based Partner in Booz $ Company’s Financial Services practice
Government Needs a 'Big Data First' Initiative
Tony Ayaz, AOL Gov, January 24, 2012
The White House's recently launched "Future First" initiative marks a milestone in the federal government's effort to invigorate the implementation of new technologies. As Federal CIO Steven VanRoekel begins to roll out new initiatives like "Shared Services First," agencies should ask themselves "What technology will help us better manage systems amidst the current data explosion?"
The answer lies in the ability to handle large volumes of machine-generated data, also known as big data. Agencies need to automate how they manage large volumes of machine data because the growth of data is outpacing human capacity to monitor and understand its relevance.
Machine data is the fastest growing, most complex and most valuable component of big data. This data comes from computers, applications, sensors, mobile devices or anything that is running within an IT infrastructure.
The ability to capture and analyze data from multiple sources is the core part of the big data challenge. This is a daunting task for many agencies made even more complex by recent budget cuts.
A "Big Data First" initiative should revolutionize the way government operates.
A successful "Big Data First" policy would require agencies implement IT architectures to have access to all IT data in real time. This would push agencies to leverage cutting edge technologies to reduce costs and deal with the recent data explosion effectively for improved security and operations.
However, before they make the transition, agencies to understand how to get the most value out of a seemingly endless pool of diverse data.
Making Sense of Machine Generated Data
According to recent reports from International Data Corporation (IDC), big data will earn its place as the next "must have" competency in 2012 as the volume of digital content grows to 2.7 zettabytes (ZB), up 48% from 2011.
Emerging big data technologies are taking the private sector by storm, and government needs to make sure it's not left behind. While a number of solution providers are gunning for a piece of the big data pie, agencies need to carefully select an offering that helps them best confront their unique big data challenges head on.
Some emerging open source big data technologies underscore the rapidly growing awareness and interest in solving the big data challenge. However, many of these open source technologies solve a portion of the problem and increase data management complexity.
The key to handling big data cost effectively is to deploy proven solutions that are architected for that need. One example is MapReduce technology. which our firm offers, but there are others.
These solutions map machine generated data from multiple data sources and leverage real time search technology across information silos thereby providing agencies with greater situational awareness. Moreover, leveraging a scalable infrastructure automates big data management and reduces long-term costs for data centers.
For example, if an analyst needs to investigate a rogue IP address he or she needs to retrace the transaction and analyze key transaction flows across many locations, sites and applications. Traditional data management solutions require pre-defined schemas to gain access to each relevant data source, site or user. However, the days of predefining schemas and receiving answers to previously determined questions are the ways of the past and do not address big data complexities. Agencies need an architecture that can analyze terabytes of data in real time by utilizing an IT search engine that instantly scans any type of machine data within in a single dashboard.
Continuous Monitoring, Compliance and Big Data
The most significant big data challenges agencies face include understanding the dynamic nature of cyber threats and meeting compliance goals. In order to effectively address security concerns, agencies are looking to continuous monitoring as the best line of defense.
However, complying with continuous monitoring standards like the Federal Information Security Management Act (FISMA) can prove challenging for many agencies because their internal operations models support information silos that separate IT operations from security and compliance functions.
These disparate environments make it nearly impossible to analyze all machine data with a single scalable solution. Big data technologies that leverage technologies like MapReduce architecture not only reduce complexity by providing real time visibility without reliance on relational databases and schemas, but they also reduce cost.
Independent organizations such as the Center for Regulatory Effectiveness (CRE) are taking notice of the benefits of big data technology. The CRE's report "FISMA Focus at the Center for Regulatory Effectiveness" calls for agencies to adopt a data-driven approach to cybersecurity so that federal IT managers can identify known and unknown cyber threats. This is a step in the right direction, but agencies can do more when it comes to continuous monitoring and compliance.
A "Big Data First" policy would set guidelines and best practices for agencies. The Administration needs to endorse this technological revolution of eliminating unnecessary information silos and recommend "best of breed" solutions to agencies. A "Big Data First" policy would modernize agency data centers and allow them to scale effectively to the increasing volumes of machine generated data.
Tony Ayaz is vice president at Splunk Federal.
Big Data: Google Widens Its Tracks
Privacy Changes to Combine Data on Users, Making Anonymity Harder to Keep
Julia Angwin, The Wall Street Journal, January 25, 2012
In a move that could make it harder for Google users to remain anonymous, Google Inc. said it would start combining nearly all the information it has on its users.
This could mean, for instance, that when users search via Google, the company will use their activities on sister sites like Gmail and YouTube to influence those users' search results. Google hasn't done that before.
Google's move—which was disclosed in a privacy policy that will take effect on March 1—is a sign of the fierce competition between Google and Facebook Inc. over personal data. Facebook has amassed an unprecedented amount of data about the lives of its more than 800 million members—information that is coveted by advertisers.
Google traditionally hasn't had the same amount of personal data about its users, and has kept much of its personal data separate. But as Facebook gears up for its planned initial public offering this spring, Google has amped up the competition.
Last year, Google launched its own social network, called Google+, and earlier this month Google started including data from Google+ in members' search results.
Google's latest move would allow the company to include insights from services such as Gmail and YouTube to search results as well.
This could effectively rewrite the relationship between users and the world's most-popular search engine.
Google has long treated users' search queries as sacrosanct—in part because they can contain very personal sensitive information—about topics such as health and finances.
In June, at The Wall Street Journal's All Things Digital conference, Google Executive Chairman Eric Schmidt said, "Google will remain a place where you can do anonymous searches. We're very committed to having you have control over the information we have about you. So, for example, if you want to continue to use Google and don't log in, and don't tell us who you are, that will continue to be true forever."
Mr. Schmidt's statement would remain true for people who aren't logged into a Gmail, Google+, YouTube, Android phone or any other Google account. But as Google's services become more ubiquitous and deeply linked, it could become more difficult for users to take Google up on that promise of anonymity.
"Google now watches consumers practically everywhere they go on the Web—and in real life, when using a mobile phone," said Christopher Soghoian, an independent privacy and security researcher in Washington D.C. "No single entity should be trusted with this much sensitive data."
Google said that it isn't collecting any new information, just combining it to provide better service to customers. For example, the company said that it could alert a user that he is going to be late to a meeting based on Google's analysis of the user's location, calendar and analysis of traffic on the road to the meeting.
"We'll treat you as a single user across all our products, which will mean a simpler, more intuitive Google experience," Alma Whitten, Google's director of privacy, product and engineering, wrote on the company's blog.
Google added that it would continue its policy of not combining user's personal information with data about their Web browsing collected by its DoubleClick advertising network.
The company last year signed a privacy agreement with the Federal Trade Commission. The settlement requires Google to ask users for permission before changing some of its privacy settings and requires the company to submit to privacy audits for 20 years.
This month, the company launched an advertising campaign touting its commitment to privacy.
Google until recently refrained from aggressively exploiting its own data about Internet users to show them online ads tailored to their interests, fearing a backlash. But the rapid emergence of rivals such as Facebook has caused it to change its policy over time.
In 2009 Google for the first time started collecting a new type of data about the websites people visit, and using it to track and show them ads across the Internet.
Last June, the company launched Google+, which was intended to rival Facebook, Twitter Inc. and other social-media companies whose users have willingly provided information about themselves.
With Tuesday's changes, Google is "setting the stage for one-upping" Facebook in terms of being able to better target online ads to website visitors based on what it knows about their interests, said Brian Kennish, a former Google programmer who runs Disconnect Inc., a firm that offers software to block Google and other companies that collect information about Web users.
Amir Efrati contributed to this article.
Write to Julia Angwin at julia.angwin@wsj.com
--------------------------------------------------
P Please consider the environment before printing this e-mail.
Julia Angwin, The Wall Street Journal, January 25, 2012
In a move that could make it harder for Google users to remain anonymous, Google Inc. said it would start combining nearly all the information it has on its users.
This could mean, for instance, that when users search via Google, the company will use their activities on sister sites like Gmail and YouTube to influence those users' search results. Google hasn't done that before.
Google's move—which was disclosed in a privacy policy that will take effect on March 1—is a sign of the fierce competition between Google and Facebook Inc. over personal data. Facebook has amassed an unprecedented amount of data about the lives of its more than 800 million members—information that is coveted by advertisers.
Google traditionally hasn't had the same amount of personal data about its users, and has kept much of its personal data separate. But as Facebook gears up for its planned initial public offering this spring, Google has amped up the competition.
Last year, Google launched its own social network, called Google+, and earlier this month Google started including data from Google+ in members' search results.
Google's latest move would allow the company to include insights from services such as Gmail and YouTube to search results as well.
This could effectively rewrite the relationship between users and the world's most-popular search engine.
Google has long treated users' search queries as sacrosanct—in part because they can contain very personal sensitive information—about topics such as health and finances.
In June, at The Wall Street Journal's All Things Digital conference, Google Executive Chairman Eric Schmidt said, "Google will remain a place where you can do anonymous searches. We're very committed to having you have control over the information we have about you. So, for example, if you want to continue to use Google and don't log in, and don't tell us who you are, that will continue to be true forever."
Mr. Schmidt's statement would remain true for people who aren't logged into a Gmail, Google+, YouTube, Android phone or any other Google account. But as Google's services become more ubiquitous and deeply linked, it could become more difficult for users to take Google up on that promise of anonymity.
"Google now watches consumers practically everywhere they go on the Web—and in real life, when using a mobile phone," said Christopher Soghoian, an independent privacy and security researcher in Washington D.C. "No single entity should be trusted with this much sensitive data."
Google said that it isn't collecting any new information, just combining it to provide better service to customers. For example, the company said that it could alert a user that he is going to be late to a meeting based on Google's analysis of the user's location, calendar and analysis of traffic on the road to the meeting.
"We'll treat you as a single user across all our products, which will mean a simpler, more intuitive Google experience," Alma Whitten, Google's director of privacy, product and engineering, wrote on the company's blog.
Google added that it would continue its policy of not combining user's personal information with data about their Web browsing collected by its DoubleClick advertising network.
The company last year signed a privacy agreement with the Federal Trade Commission. The settlement requires Google to ask users for permission before changing some of its privacy settings and requires the company to submit to privacy audits for 20 years.
This month, the company launched an advertising campaign touting its commitment to privacy.
Google until recently refrained from aggressively exploiting its own data about Internet users to show them online ads tailored to their interests, fearing a backlash. But the rapid emergence of rivals such as Facebook has caused it to change its policy over time.
In 2009 Google for the first time started collecting a new type of data about the websites people visit, and using it to track and show them ads across the Internet.
Last June, the company launched Google+, which was intended to rival Facebook, Twitter Inc. and other social-media companies whose users have willingly provided information about themselves.
With Tuesday's changes, Google is "setting the stage for one-upping" Facebook in terms of being able to better target online ads to website visitors based on what it knows about their interests, said Brian Kennish, a former Google programmer who runs Disconnect Inc., a firm that offers software to block Google and other companies that collect information about Web users.
Amir Efrati contributed to this article.
Write to Julia Angwin at julia.angwin@wsj.com
--------------------------------------------------
P Please consider the environment before printing this e-mail.
Press Release: Big Data? Big Problems!
Big Data? Big Problems! Everyone’s Talking about Big Data Analytics, but Nobody Acknowledges the Elephant in the Room: Skilled Knowledge Workers Are in Short Supply
Big data as a managed service? Evolving from buzzword to real, deployable solution
Press Release, The Wall Street Journal, January 23, 2012
STERLING, Va.--(EON: Enhanced Online News)--Today Customer Relationship Metrics, L.C. released the findings of a study indicating usage of the term “big data” has exploded online. The study was conducted by analysts at Customer Relationship Metrics using Nielsen McKinsey’s NM Incite technology, which collects user-generated content from over 180 million sites worldwide, including blogs, message boards, usenet groups, Twitter, Facebook and Video/Image sites (e.g., Youtube, Flickr).
“Ironically, use of the term big data grew significantly in mid-2011 when McKinsey & Co. issued its seminal research report Big data: The next frontier for innovation, competition, and productivity. The report warned of a growing shortage of talent to leverage big data and make decisions based on data trends.”
Virtually unheard of at the beginning of 2010, big data has quickly become one of the hottest buzzwords in IT circles. In the past three months, big data was the topic of discussion over 20,000 times per month in the press, blogs, and social networks, as measured by NM Incite. See accompanying chart.
Big Data? Big Problems!
But here’s the rub: even world-class enterprises are struggling with getting real value from big data, solely because knowledgeable workers are in short supply: those with the skills necessary to analyze and understand what the data is saying; translate the data into real business action that drives bottom-line results; and communicate recommendations to senior executives.
Dr. Jodie Monger, founder and president of Customer Relationship Metrics, said, “Right now, big data is nothing more than a buzzword. Everyone in IT knows that the enterprise cannot afford to overlook the massive data sets they create. They know that these data sets contain a plethora of information that can help them better serve their customers. But nobody knows how to actually reach this Holy Grail.”
Dr. Monger continued, “Ironically, use of the term big data grew significantly in mid-2011 when McKinsey & Co. issued its seminal research report Big data: The next frontier for innovation, competition, and productivity. The report warned of a growing shortage of talent to leverage big data and make decisions based on data trends.”
Big Problems? Big Solution!
So enterprises are caught in a jam: they need to analyze and act on data trends, but don’t have people who can do the job. Increasingly, these enterprises are outsourcing the job to Customer Relationship Metrics.
Dr. Monger continued, “Customer Relationship Metrics serves many of the most recognizable consumer brands on the planet. We help these companies dig deep into their data, spotting trends that emerge from daily interactions with customers through call centers, email dialogues, chat functions, and social media interactions.”
By focusing on data embedded within real customer interactions, companies can easily identify those service issues which lead to the most customer dissatisfaction. Once these problems are fixed, reputation grows and customer satisfaction increases organically.
Dr. Monger added, “Analyzing big data can be overwhelming. But we make it simple for customers by pointing our solutions at the most meaningful data sets that can deliver the most significant customer service results in the quickest timeframe possible. We eliminate blind alleys and avoid time and resource vampires, while making big data solutions easy to implement.”
Big Data as a Managed Service
Customer Relationship Metrics is a SaaS-based end-to-end big data solution. It includes data integration, software, and analytics that can be up and running within 60 days.
Dr. Monger concluded, “Deployment is where big-data-based BI solutions break down most frequently. Custom solutions and complex software development timelines mean delays, cost overruns, and intense frustration across the chain of command. By structuring our solution as a managed service, we deliver real value from big data and business intelligence in an abbreviated timeframe, with no significant capital costs. That’s a win/win for all involved."
For more information on Customer Relationship Metrics’ business intelligence solutions, please contact Jim Rembach at 336-288-8226.
ABOUT CUSTOMER RELATIONSHIP METRICS, L.C.
Customer Relationship Metrics, L.C., headquartered in Sterling, Virginia, is a provider of managed call center analytics and advisory services. Customer Relationship Metrics' business intelligence solutions are focused on delivering full turnkey Customer Experience Analytics, Speech Analytics and Operational Analytics programs that transform unstructured data into actionable business insights. Customer Relationship Metrics uses SaaS data collection and reporting tools combined with subject matter expertise to significantly lower the in-house total cost of ownership and skilled personnel gap. Founded in 1993, its CEO, Dr. Jodie Monger, invented post-call surveying with the award-winning External Quality Monitoring Program (EQM™) and its proprietary Survey Calibration Process that transcribes customer comments and uses them for analysis and dispositioning data as needed. She was the founding Associate Director of Purdue University's Center for Customer-Driven Quality.
For more information, visit their award-winning blog at http://www.metrics.net/blog, or their website at http://www.metrics.net.
Big data as a managed service? Evolving from buzzword to real, deployable solution
Press Release, The Wall Street Journal, January 23, 2012
STERLING, Va.--(EON: Enhanced Online News)--Today Customer Relationship Metrics, L.C. released the findings of a study indicating usage of the term “big data” has exploded online. The study was conducted by analysts at Customer Relationship Metrics using Nielsen McKinsey’s NM Incite technology, which collects user-generated content from over 180 million sites worldwide, including blogs, message boards, usenet groups, Twitter, Facebook and Video/Image sites (e.g., Youtube, Flickr).
“Ironically, use of the term big data grew significantly in mid-2011 when McKinsey & Co. issued its seminal research report Big data: The next frontier for innovation, competition, and productivity. The report warned of a growing shortage of talent to leverage big data and make decisions based on data trends.”
Virtually unheard of at the beginning of 2010, big data has quickly become one of the hottest buzzwords in IT circles. In the past three months, big data was the topic of discussion over 20,000 times per month in the press, blogs, and social networks, as measured by NM Incite. See accompanying chart.
Big Data? Big Problems!
But here’s the rub: even world-class enterprises are struggling with getting real value from big data, solely because knowledgeable workers are in short supply: those with the skills necessary to analyze and understand what the data is saying; translate the data into real business action that drives bottom-line results; and communicate recommendations to senior executives.
Dr. Jodie Monger, founder and president of Customer Relationship Metrics, said, “Right now, big data is nothing more than a buzzword. Everyone in IT knows that the enterprise cannot afford to overlook the massive data sets they create. They know that these data sets contain a plethora of information that can help them better serve their customers. But nobody knows how to actually reach this Holy Grail.”
Dr. Monger continued, “Ironically, use of the term big data grew significantly in mid-2011 when McKinsey & Co. issued its seminal research report Big data: The next frontier for innovation, competition, and productivity. The report warned of a growing shortage of talent to leverage big data and make decisions based on data trends.”
Big Problems? Big Solution!
So enterprises are caught in a jam: they need to analyze and act on data trends, but don’t have people who can do the job. Increasingly, these enterprises are outsourcing the job to Customer Relationship Metrics.
Dr. Monger continued, “Customer Relationship Metrics serves many of the most recognizable consumer brands on the planet. We help these companies dig deep into their data, spotting trends that emerge from daily interactions with customers through call centers, email dialogues, chat functions, and social media interactions.”
By focusing on data embedded within real customer interactions, companies can easily identify those service issues which lead to the most customer dissatisfaction. Once these problems are fixed, reputation grows and customer satisfaction increases organically.
Dr. Monger added, “Analyzing big data can be overwhelming. But we make it simple for customers by pointing our solutions at the most meaningful data sets that can deliver the most significant customer service results in the quickest timeframe possible. We eliminate blind alleys and avoid time and resource vampires, while making big data solutions easy to implement.”
Big Data as a Managed Service
Customer Relationship Metrics is a SaaS-based end-to-end big data solution. It includes data integration, software, and analytics that can be up and running within 60 days.
Dr. Monger concluded, “Deployment is where big-data-based BI solutions break down most frequently. Custom solutions and complex software development timelines mean delays, cost overruns, and intense frustration across the chain of command. By structuring our solution as a managed service, we deliver real value from big data and business intelligence in an abbreviated timeframe, with no significant capital costs. That’s a win/win for all involved."
For more information on Customer Relationship Metrics’ business intelligence solutions, please contact Jim Rembach at 336-288-8226.
ABOUT CUSTOMER RELATIONSHIP METRICS, L.C.
Customer Relationship Metrics, L.C., headquartered in Sterling, Virginia, is a provider of managed call center analytics and advisory services. Customer Relationship Metrics' business intelligence solutions are focused on delivering full turnkey Customer Experience Analytics, Speech Analytics and Operational Analytics programs that transform unstructured data into actionable business insights. Customer Relationship Metrics uses SaaS data collection and reporting tools combined with subject matter expertise to significantly lower the in-house total cost of ownership and skilled personnel gap. Founded in 1993, its CEO, Dr. Jodie Monger, invented post-call surveying with the award-winning External Quality Monitoring Program (EQM™) and its proprietary Survey Calibration Process that transcribes customer comments and uses them for analysis and dispositioning data as needed. She was the founding Associate Director of Purdue University's Center for Customer-Driven Quality.
For more information, visit their award-winning blog at http://www.metrics.net/blog, or their website at http://www.metrics.net.
Monday, January 23, 2012
Personalized Medicine-Humanity's Ultimate Big Data Challenge
Robert T Fassett, iHealth Connections, December 2011;1(2):90–5
Abstract
At the heart of personalized medicine lie big data, really big data—from rapidly accelerating genomics research, from deployments of electronic medical records, and soon from social networking, telemedicine, and the ‘Internet of Things’ remote sensors. The impact personalized medicine will have in transforming healthcare will depend not only on how well we gather and analyze these big data, but also on how effectively we transmit their derivative insights and interventions out to clinicians and ultimately their patients.
Acknowledgment: Editorial assistance was provided by Touch Briefings.
Support: The publication of this article was funded by Oracle Health Sciences.
Disclosure The author has no conflicts of interest to declare.
Correspondence: robert.fassett@oracle.com
We have reached an inflection point between the insular ‘sickcare’ non-system of the past and the collaborative, proactive, true ‘health and wellness’ system of the future. To overcome the inertia of our current ‘system’, disruptive forces are being applied—access reform, value-based reimbursement, evidence-based clinical guidelines, quality reporting, medical homes, and accountable care, among others.1,2
High-definition Healthcare
Another important vector for change has grown out of our massive collective investment in basic biomedical and clinical research. For example, the US alone has funded its National Institutes of Health (NIH) with $484 billion since 1950, with a current annual budget of over $30 billion.3 As we have come to better understand the phenotypic, genotypic, environmental, and lifestyle factors that determine our health, it has become clear that disease and wellness are inherently personal. Any two persons have 99.6 % of their DNA in common. In the remaining set of 24,000,000 base pairs that we each call our own lies humanity’s diversity, our individual predilection for disease, and the potential for truly personalized medicine.4
The US National Cancer Institute defines personalized medicine as “a form of medicine that uses information about a person’s genes, proteins, and environment to prevent, diagnose, and treat disease.”5 This is not to imply that, heretofore, the practice of medicine has been somehow impersonal. Hippocrates already recommended cold foods for ‘phlegmatic patients.’ Two millennia later, we understand that African-Americans respond differently to antihypertensives and prescribe accordingly. What is compelling about this new definition is its resolution. We are now capable of tailoring health and wellness at the molecular level—healthcare in its highest possible definition.6,7
In eight short years, we have progressed from a single human genome to the HapMap, and now to inexpensive whole-genome sequencing and the 1000 Genome Project.8 Genome-wide association studies have identified hundreds of genotype–disease linkages, some of which have strong clinical implications.9
We have begun to appreciate the non-linearity of the old DNA–RNA–protein central dogma and now see phenotype as the result of a complex network of interactions that include DNA structural modifications, novel transcriptional regulation via microRNA and short interfering RNA, post-translational modifications, etc.10 Astonishingly, we have vision into the transcriptome and proteome at the single-cell level.11 All this will soon result in the almost overwhelming growth of the fundamental substrate for personalized medicine: data.
Medicine’s Deep Space Objects
Individualizing treatment for a given patient is a truly daunting, data-driven task. It is not just a matter of wading through three billion base pairs to find a sequence variant that correlates with a particular disease. It is the multivariate ripple effects these polymorphisms have across the DNA–RNA–protein network, and then their interactions with the person’s environmental and lifestyle history, that must be understood. Given the magnitude of this endeavor, the resources committed, and the global cooperation that is needed, this is biology’s version of ‘big science.’
Finding a treatment based on a patient’s genes, proteins, and environment is essentially a signal-detection exercise. Gene defects (i.e., ‘signals’) that are relatively common and have a high penetrance—such as sickle cell anemia—are relatively easy to pick up. (Linus Pauling discovered the causative sickling protein sequence defect in 1949, four years before Watson and Crick determined the structure of DNA.) Gene defects that have a low prevalence and a low penetrance are much harder to detect and understand. They are the biologic equivalents of deep space objects. Many conditions, even common and deadly ones such as obesity, diabetes, and atherosclerosis, are thought to be influenced by many different sets of signals, some easy to detect and others that will require medicine’s equivalent of the Hubble space telescope.
This will ultimately require data volumes and manipulation techniques unprecedented in information science and technology. Detecting rare and variably expressed mutations and correlating them with fine-grained clinical observations and environmental factors in a large population will require massive amounts of high-resolution data. This may seem like a daunting thousand-mile march, but the longest road still lies ahead.
Abstract
At the heart of personalized medicine lie big data, really big data—from rapidly accelerating genomics research, from deployments of electronic medical records, and soon from social networking, telemedicine, and the ‘Internet of Things’ remote sensors. The impact personalized medicine will have in transforming healthcare will depend not only on how well we gather and analyze these big data, but also on how effectively we transmit their derivative insights and interventions out to clinicians and ultimately their patients.
Acknowledgment: Editorial assistance was provided by Touch Briefings.
Support: The publication of this article was funded by Oracle Health Sciences.
Disclosure The author has no conflicts of interest to declare.
Correspondence: robert.fassett@oracle.com
We have reached an inflection point between the insular ‘sickcare’ non-system of the past and the collaborative, proactive, true ‘health and wellness’ system of the future. To overcome the inertia of our current ‘system’, disruptive forces are being applied—access reform, value-based reimbursement, evidence-based clinical guidelines, quality reporting, medical homes, and accountable care, among others.1,2
High-definition Healthcare
Another important vector for change has grown out of our massive collective investment in basic biomedical and clinical research. For example, the US alone has funded its National Institutes of Health (NIH) with $484 billion since 1950, with a current annual budget of over $30 billion.3 As we have come to better understand the phenotypic, genotypic, environmental, and lifestyle factors that determine our health, it has become clear that disease and wellness are inherently personal. Any two persons have 99.6 % of their DNA in common. In the remaining set of 24,000,000 base pairs that we each call our own lies humanity’s diversity, our individual predilection for disease, and the potential for truly personalized medicine.4
The US National Cancer Institute defines personalized medicine as “a form of medicine that uses information about a person’s genes, proteins, and environment to prevent, diagnose, and treat disease.”5 This is not to imply that, heretofore, the practice of medicine has been somehow impersonal. Hippocrates already recommended cold foods for ‘phlegmatic patients.’ Two millennia later, we understand that African-Americans respond differently to antihypertensives and prescribe accordingly. What is compelling about this new definition is its resolution. We are now capable of tailoring health and wellness at the molecular level—healthcare in its highest possible definition.6,7
In eight short years, we have progressed from a single human genome to the HapMap, and now to inexpensive whole-genome sequencing and the 1000 Genome Project.8 Genome-wide association studies have identified hundreds of genotype–disease linkages, some of which have strong clinical implications.9
We have begun to appreciate the non-linearity of the old DNA–RNA–protein central dogma and now see phenotype as the result of a complex network of interactions that include DNA structural modifications, novel transcriptional regulation via microRNA and short interfering RNA, post-translational modifications, etc.10 Astonishingly, we have vision into the transcriptome and proteome at the single-cell level.11 All this will soon result in the almost overwhelming growth of the fundamental substrate for personalized medicine: data.
Medicine’s Deep Space Objects
Individualizing treatment for a given patient is a truly daunting, data-driven task. It is not just a matter of wading through three billion base pairs to find a sequence variant that correlates with a particular disease. It is the multivariate ripple effects these polymorphisms have across the DNA–RNA–protein network, and then their interactions with the person’s environmental and lifestyle history, that must be understood. Given the magnitude of this endeavor, the resources committed, and the global cooperation that is needed, this is biology’s version of ‘big science.’
Finding a treatment based on a patient’s genes, proteins, and environment is essentially a signal-detection exercise. Gene defects (i.e., ‘signals’) that are relatively common and have a high penetrance—such as sickle cell anemia—are relatively easy to pick up. (Linus Pauling discovered the causative sickling protein sequence defect in 1949, four years before Watson and Crick determined the structure of DNA.) Gene defects that have a low prevalence and a low penetrance are much harder to detect and understand. They are the biologic equivalents of deep space objects. Many conditions, even common and deadly ones such as obesity, diabetes, and atherosclerosis, are thought to be influenced by many different sets of signals, some easy to detect and others that will require medicine’s equivalent of the Hubble space telescope.
This will ultimately require data volumes and manipulation techniques unprecedented in information science and technology. Detecting rare and variably expressed mutations and correlating them with fine-grained clinical observations and environmental factors in a large population will require massive amounts of high-resolution data. This may seem like a daunting thousand-mile march, but the longest road still lies ahead.
Subscribe to:
Posts (Atom)