Big Data: The End of Privacy or a New Beginning?

Rubinstein, Ira. "Big Data: The End of Privacy or a New Beginning?" New York University, Information Law Institute (October 5, 2012).

From the abstract: “Big data” refers to novel ways in which organizations, including government and businesses, combine diverse digital data sets and then use statistics and other data mining techniques to extract from them both hidden information and surprising correlations. While big data promises significant economic and social benefits, it also raises serious privacy concerns. In particular, big data challenges the Fair Information Practices (FIPs) as embodied in various privacy laws including the EU Data Protection Directive. This past January, the European Commission released a proposal to reform and replace the Directive by adopting a new Regulation. In this paper, I argue that this Regulation relies too heavily on the discredited informed choice model, and therefore fails to fully engage with the coming big data tsunami. My contention is that when this advancing wave arrives, it will so overwhelm the core privacy principles of informed choice and data minimization on which the Directive rests that reform efforts alone will prove inadequate. Rather, an adequate response must combine legal reform with encouragement of new business models premised on consumer empowerment and supported by a personal data ecosystem. This new business model is important for two reasons: first, existing business models have proven time and again that privacy regulation is no match for them. Businesses inevitably collect and use more and more personal data, and while consumers realize many benefits in exchange, there is little doubt that businesses, not consumers, control the market in personal data with their own interests in mind. Second, a new business model, which I describe in this paper, promises to stand processing of personal data on its head by shifting control over both the collection and use of data from firms to individuals. This “control shift” — and this alone — stands a chance of making the FIPs efficacious by giving individuals the capacity to benefit from big data and hence the motivation to learn about and control how their data is collected and used, while also enabling businesses to profit from a new breed of services that are both data-intensive and imbued with privacy values." Read more

Harnessing Data as a New Source of Growth: Big Data Analytics and Policies

OECD Headquarters, Paris, France, October 22, 2012

Introduction
The confluence of several key socio-economic and technological trends is resulting in the generation of huge streams of data every day. The major trends include:

·       The increasing migration of social and economic activities on line: Social network site Facebook, for example, now counts over 900 million active participants around the world generating together more than 1 500 status updates every second about their interests and whereabouts. In 2011, e-commerce platform eBay collected data on more than 100 million active users including the 6 million new goods they offered every day.

·       The strong decline in the cost of data collection, storage, transportation, and processing: The average cost of consumer hard disk drives (HDDs) per gigabyte, for example, dropped on average by almost 40% per year between 1998 (USD 56 per gigabyte) and 2012 (USD 0.05 per gigabyte). In 1995, as another example, consumers in France paid USD 75 equivalent per month for a dial-up (56 Kb/s) connection, while in 2011 they paid the equivalence of USD 33 per month for a broadband (51 Mb/s) connection, which was almost 1 000 times faster.

·       The increasing deployment of “smart” ICT applications such as smart grids and smart transportations based on machine-to-machine (M2M) communication: Connecting one million homes to a smart grid may produce as much as 11 gigabytes of data per day. In order to accommodate for hourly readings through smart meters, a network with a minimum capacity of up to 1 Mbit/s dedicated to M2M communication is needed.

·       The continued expansion of mobile communication: In 2011, there were 780 million smart phones worldwide capable of collecting and transmitting geo-location data, which generated more than 600 petabytes (millions of gigabytes) of data every month. It is estimated that the global data traffic generated by mobile communication (including M2M enabled smart devices) will almost double every year to reach 11 exabyte (billions of gigabytes) per month by 2016.

The collection and exploitation of these large data flows through data analytics is leading to a shift towards a data-driven socio-economic model commonly referred to as “big data”. In this model, data is a core asset that provides a huge resource for new industries, processes, services and goods leading to significant competitive advantage. In business, for example, data analytics are increasingly being used in a wide number of operations ranging from optimising the value chain and manufacturing production to more efficiently using labour and improving customer relationships. It is estimated that firms, which adopt data-driven decision-making, for example, have output and productivity that is 5-6% higher than what would be expected given their other investments and information technology usage. These firms also perform better in terms of asset utilisation, return on equity and market value.

To unlock the potential of big data and data analytics (big data analytics), OECD countries need to ensure the development of coherent policies and practices around the collection, transportation, storage and use of data, most prominently in areas related to privacy protection. New data sources, new actors and the increasing ease with which personal data can be collected, linked and processed, seriously challenge the effective implementation of current frameworks for privacy protection. But the potential implications for policy also spill over into many other domains; including among others open access to data, intellectual property rights, competition, skills and employment, infrastructure, and measurement.

Objective
First launched in 2005, Technology Foresight Forums are an annual event organised by the OECD Committee for Information, Computer, and Communications Policy (ICCP) to help identify opportunities and challenges for the Internet Economy posed by technical developments. The 2012 Technology Foresight Forum will focus on the potential of big data analytics as a new source of growth, which could help generate significant economic and social benefits. It will put big data analytics in the context of emerging trends discussed at the last three Foresight Forums, namely mobile communications (2011), smart ICTs (2010), and cloud computing (2009), to highlight the confluence of trends leading towards a data-driven economy (see Figure below).

The confluence of major trends enabling data as a new source of growth

The 2012 Foresight Forum will then discuss the social and economic issues related to big data analytics that may put at risk fundamental values and warrant a review of current policy frameworks, most prominently those aimed at ensuring the protection of privacy, intellectual property, and competition. Other policy areas that will be addressed include health care, science and research, government administration and labour markets.

This year’s Foresight Forum will contribute to OECD’s ongoing horizontal project entitled New Sources of Growth: Intangible Assets (NSG) as well as to the follow-up horizontal project on Knowledge-Based Capital: Seizing the Benefits of New Sources of Growth, which will be launched in 2013. Both projects aim at (i) providing structured evidence of the economic value of intangible assets, including data, as a new source of growth, and (ii) improving understanding of current and emerging challenges for policies related to the increasing relevance of these intangible assets.

Modalities
The Foresight Forum represents a collaborative effort of policy makers, business, civil society, and the Internet technical community and thus provides opportunities to liaise with experts in the fields covered. To increase interactivity with a broader public, the Foresight Forum will be supported by various participative web technologies, including a webcast, and Twitter feeds. Further details will be announced closer to the event.

Agenda
The 2012 Foresight Forum will include two morning and three afternoon sessions: The first (morning) session will introduce data analytics in the context of the key technological and socio-economic trends, namely i) cloud computing; ii) smart ICT applications; and iii) the Internet of Things. The following session will then focus on the socio-economic implications of harnessing data as a new source for growth. The afternoon sessions will take a more in-depth look at specific areas that will include: i) science and research (including public health); ii) marketing and competition; and iii) public administration. Each of these sessions will be taken as a mean to discuss specific potential policy opportunities and challenges ranging from i) privacy and consumer protection; ii) intellectual property rights; iii) open access to data; iv) competition; and v) skills and employment. A concluding session will then highlight the main policy implications to be examined by the OECD in the context of its programme of work for 2013-14.

10 ways big data is remaking energy

Katie Fehrenbacher,  Gigaom,  Jan. 29, 2012, 9:30pm

One of the most obvious trends from the big smart grid conference DistribuTECH last week was how much analytics and big data tools will be used to try to remake energy in 2012, from curbing energy consumption, to reducing energy loss, to adding in more clean power to the grid. Here’s 10 ways that analytics and big data will start to shape the production and consumption of energy in the world:

1). Weather data: Having a finger on the pulse of constantly changing weather data on a micro and macro level can help utilities, building owners and consumers optimize their energy consumption habits and promote energy efficiency. Startup EnergyHub recently partnered with sensor network player Earth Networks to use weather data to make a more efficient form of demand response (utilities controlling power consumption). Other startups like EcoFactor, Opower and Tendril also use weather data as part of their energy behavioral analytics.

IBM has long sold a weather prediction service called Deep Thunder to municipalities, organizations and utilities, which use it to do things like tailor their services, change routes, or generate more or less power. I think weather data could some day provide a platform for some very important next generation services and applications for energy efficiency, much in the way that location data is used as a platform for a variety of services.

2). Cell phone data: Cell phones in our pockets are essentially palm-sized sensors and computers sending a constant stream of information to the cloud where companies could one day use that data to create energy efficiency and better energy products. And yes, a lot of that data is private information, but after that data is anonymized it can be used for the greater good of the community — particularly via the billions of cell phones in developing countries. A startup called Jana does research projects around cell phone data in developing countries, and looks to work with NGOs on programs to create better infrastructure, energy infrastructure and resources.

3). Connected thermostat data: One of the biggest trends from DistribuTECH this year was the overwhelming amount of smart thermostats that are now being sold and marketed. Companies can incorporate that thermostat data into data bases that can be used to promote energy efficiency. EcoFactor’s service remembers every time a home owner overrides the automated smart thermostat system and changes the personalized service to accommodate that manual override. Using 100,000 connected thermostats (which produce 5 billion data points each month) EnergyHub found some interesting statistics like folks in cold climates have a lower average heating temperature set point than households in warmer states.

4). Hadoop & energy databases: The open source data base tool Hadoop is well known — and oft used — in the computing worlds. But in the energy and utility worlds it’s quite rare. However, as the amount of energy data has started to rapidly grow from the smart grid, some companies are embracing Hadoop as a key way to manage energy info. Opower tells me it’s using Hadoop (and the company commercializing Hadoop, Cloudera) as an important way to manage its massive energy data streams. Likewise PJM has turned to Hadoop as a way to organize the energy data coming off of a synchophaser sensor project.

5). Clean power data: One of the main goals for the smart grid is to enable the addition of more variable clean power, which is far more unreliable than fossil fuels (the sun doesn’t shine and the wind doesn’t blow 24/7). Analytics crunching the data from a utilities’ energy supply and demand can help make clean power a little less variable, by being able to more accurately predict the environmental conditions, as well as more accurately assess demand from energy users.

6). Electric car data: Electric cars will by their nature be connected cars, using information technology to manage the vehicle charge and location. Utilities will be closely tracking the charging habits of electric car owners in order to make sure that the grid isn’t overloaded in some early adopter neighborhoods.

7). Power line sensors: One of the areas of low hanging fruit for the power grid is the simple task of helping utilities find blackouts more easily and be able to monitor and manage grid outages. That’s partly where sensor systems called synchophasers come in, which can in real time monitor the health of power lines, collecting multiple data streams per second. Expect all major networks to have synchophaser systems installed over the coming years.

8). Real estate data: Startups like First Fuel Software can use big data to make super accurate assessments about buildings and ways to reduce the energy consumption of buildings — without any extra hardware or monitoring software being installed at the building. Things like weather around the building, demographics of the people in the building, and the building’s historical energy consumption can be used to create an accurate projection. The best way to make a building more energy efficient is by getting as much data about the building;s energy use as possible.

9). Variable pricing: Some day when electricity is sold throughout the world at different prices dependent on supply and demand, massive data bases will be needed. This type of variable pricing is offered in some places in the world, but if it ever becomes ubiquitous it will help curb consumption, by offering high prices when energy is being over used.

10). Using behavioral analytics to curb energy consumption: Getting into the brains of energy users is the job of startups like Opower and Tendril (after it acquired Gr0unded Power.) Essentially these companies have collected data on consumers and demographics and they are using it to try to guess the best way to influence the consumer to do things like upgrade their home appliances and lights to more efficient ones.

Government Needs a 'Big Data First' Initiative

Tony Ayaz, AOL Gov,  January 24, 2012

The White House's recently launched "Future First" initiative marks a milestone in the federal government's effort to invigorate the implementation of new technologies. As Federal CIO Steven VanRoekel begins to roll out new initiatives like "Shared Services First," agencies should ask themselves "What technology will help us better manage systems amidst the current data explosion?"

The answer lies in the ability to handle large volumes of machine-generated data, also known as big data. Agencies need to automate how they manage large volumes of machine data because the growth of data is outpacing human capacity to monitor and understand its relevance.

Machine data is the fastest growing, most complex and most valuable component of big data. This data comes from computers, applications, sensors, mobile devices or anything that is running within an IT infrastructure.

The ability to capture and analyze data from multiple sources is the core part of the big data challenge. This is a daunting task for many agencies made even more complex by recent budget cuts.

A "Big Data First" initiative should revolutionize the way government operates.

A successful "Big Data First" policy would require agencies implement IT architectures to have access to all IT data in real time. This would push agencies to leverage cutting edge technologies to reduce costs and deal with the recent data explosion effectively for improved security and operations.

However, before they make the transition, agencies to understand how to get the most value out of a seemingly endless pool of diverse data.

Making Sense of Machine Generated Data

According to recent reports from International Data Corporation (IDC), big data will earn its place as the next "must have" competency in 2012 as the volume of digital content grows to 2.7 zettabytes (ZB), up 48% from 2011.

Emerging big data technologies are taking the private sector by storm, and government needs to make sure it's not left behind. While a number of solution providers are gunning for a piece of the big data pie, agencies need to carefully select an offering that helps them best confront their unique big data challenges head on.

Some emerging open source big data technologies underscore the rapidly growing awareness and interest in solving the big data challenge. However, many of these open source technologies solve a portion of the problem and increase data management complexity.

The key to handling big data cost effectively is to deploy proven solutions that are architected for that need. One example is MapReduce technology. which our firm offers, but there are others.

These solutions map machine generated data from multiple data sources and leverage real time search technology across information silos thereby providing agencies with greater situational awareness. Moreover, leveraging a scalable infrastructure automates big data management and reduces long-term costs for data centers.

For example, if an analyst needs to investigate a rogue IP address he or she needs to retrace the transaction and analyze key transaction flows across many locations, sites and applications. Traditional data management solutions require pre-defined schemas to gain access to each relevant data source, site or user. However, the days of predefining schemas and receiving answers to previously determined questions are the ways of the past and do not address big data complexities. Agencies need an architecture that can analyze terabytes of data in real time by utilizing an IT search engine that instantly scans any type of machine data within in a single dashboard.

Continuous Monitoring, Compliance and Big Data

The most significant big data challenges agencies face include understanding the dynamic nature of cyber threats and meeting compliance goals. In order to effectively address security concerns, agencies are looking to continuous monitoring as the best line of defense.

However, complying with continuous monitoring standards like the Federal Information Security Management Act (FISMA) can prove challenging for many agencies because their internal operations models support information silos that separate IT operations from security and compliance functions.

These disparate environments make it nearly impossible to analyze all machine data with a single scalable solution. Big data technologies that leverage technologies like MapReduce architecture not only reduce complexity by providing real time visibility without reliance on relational databases and schemas, but they also reduce cost.

Independent organizations such as the Center for Regulatory Effectiveness (CRE) are taking notice of the benefits of big data technology. The CRE's report "FISMA Focus at the Center for Regulatory Effectiveness" calls for agencies to adopt a data-driven approach to cybersecurity so that federal IT managers can identify known and unknown cyber threats. This is a step in the right direction, but agencies can do more when it comes to continuous monitoring and compliance.

A "Big Data First" policy would set guidelines and best practices for agencies. The Administration needs to endorse this technological revolution of eliminating unnecessary information silos and recommend "best of breed" solutions to agencies. A "Big Data First" policy would modernize agency data centers and allow them to scale effectively to the increasing volumes of machine generated data.

Tony Ayaz is vice president at Splunk Federal.

CTO Amazon: Data Without Limits

NEXT 2011 Conference

This year NEXT is all about Data Love. ...Data is the resource for the digital value creation and fuel for the economy. Today, data is what electricity has been for the industrial age…

Business developers, marketing experts and agency managers are faced with the challenge to create new applications out of the ever-growing data stream with added value for the consumer. In our data-driven economy, the consumer is in the focus point of consideration. Because his behaviour determines who wins, what lasts and what will be sold. Data is the crucial driver to develop relevant products and services for the consumer.

(http://nextconf.eu/next11/programme/)
Data Without Limits by Werner Vogels, CTO Amazon

Talk at http://video.nextconf.eu/video/1880845/data-without-limits

ABOUT THE SPEAKER:
Werner Vogels, Vice President & CTO at Amazon.com, is responsible for driving the company's technology vision.

Prior to joining Amazon, he worked as a researcher at Cornell University where he was a principal investigator in several research projects that target the scalability and robustness of mission-critical enterprise computing systems. He has held positions of VP of Technology and CTO in companies that handled the transition of academic technology into industry.

Werner holds a Ph.D. from the Vrije Universiteit in Amsterdam and has authored many articles for journals and conferences, most of them on distributed systems technologies for enterprise computing. He was named the 2008 CTO of the Year by Information Week for his contributions to making Cloud Computing a reality. For his unique style in engaging customers, media and the general public, he received the 2009 Media Momentum Personality of Award. 
See also

Stay Healthy: Big Data and Our Bodies  by David Rowan, Wired UK
http://video.nextconf.eu/video/1879024/stay-healthy-big-data-and-our

When there’s no such thing as too much information

BY STEVE LOHR, NYT, APR 23, 2011

 INFORMATION overload is a headache for individuals and a huge challenge for businesses. Companies are swimming, if not drowning, in wave after wave of data — from increasingly sophisticated computer tracking of shipments, sales, suppliers and customers, as well as e-mail, Web traffic and social-network comments. These Internet-era technologies, by one estimate, are doubling the quantity of business data every 1.2 years.

Yet the data explosion is also an enormous opportunity. In a modern economy, information should be the prime asset — the raw material of new products and services, smarter decisions, competitive advantage for companies, and greater growth and productivity.

Is there any real evidence of a "data payoff" across the corporate world? It has taken a while, but new research led by Erik Brynjolfsson, an economist at the Sloan School of Management at the Massachusetts Institute of Technology, suggests that the beginnings are now visible.

Mr. Brynjolfsson and his colleagues, Lorin Hitt, a professor at the Wharton School of the University of Pennsylvania, and Heekyung Kim, a graduate student at M.I.T., studied 179 large companies. Those that adopted "data-driven decision making" achieved productivity that was 5 to 6 percent higher than could be explained by other factors, including how much the companies invested in technology, the researchers said.

In the study, based on a survey and follow-up interviews, data-driven decision making was defined not only by collecting data, but also by how it is used — or not — in making crucial decisions, like whether to create a new product or service.

The central distinction, according to Mr. Brynjolfsson, is between decisions based mainly on "data and analysis" and on the traditional management arts of "experience and intuition."A 5 percent increase in output and productivity, he says, is significant enough to separate winners from losers in most industries.The companies that are guided by data analysis, Mr. Brynjolfsson says, are "harbingers of a trend in how managers make decisions." "And it has huge implications for competitiveness and growth," he adds. The research is not yet published, but it was presented at an academic conference this month.

The conclusion that companies that rely heavily on data analysis are likely to outperform others is not new. Notably, Thomas H. Davenport, a professor of information technology and management at Babson College, has made that point, and his most recent book, with Jeanne G. Harris and Robert Morison, is "Analytics at Work: Smarter Decisions, Better Results" (Harvard Business Press, 2010).

And companies like Google, whose search and advertising business is based on exploiting and organizing online information, are testimony to the power of intelligent data sifting.But the new research appears to be broader and to apply economic measurement to the impact of data-led decision making in a way not done before. "To the best of our knowledge," Mr. Brynjolfsson says, "this is the first quantitative evidence of the anecdotes we're been hearing about."

Mr. Brynjolfsson emphasizes that the spread of such decision making is just getting started, even though the data surge began at least a decade ago. That pattern is familiar in history. The productivity payoff from a new technology comes only when people adopt new management skills and new ways of working.

The electric motor, for example, was introduced in the early 1880s. But that technology did not generate discernible productivity gains until the 1920s. It took that long for the use of motors to spread, and for businesses to reorganize work around the mass-production assembly line, the efficiency breakthrough of its day.

The story was much the same with computers. By 1987, the personal computer revolution was more than a decade old, when Robert M. Solow, an economist and Nobel laureate, dryly observed, "You can see the computer age everywhere but in the productivity statistics."

It was not until 1995 that productivity in the American economy really started to pick up. The Internet married computing to low-cost communications, opening the door to automating all kinds of commercial transactions. The gains continued through 2004, well after the dot-com bubble burst and investment in technology plummeted.

The technology absorption lag accounts for the delayed productivity benefits, observes Robert J. Gordon, an economist at Northwestern University."It's never pure technology that makes the difference," Mr. Gordon says. "It's reorganizing things — how work is done. And technology does allow new forms of organization."

Since 2004, productivity has slowed again. Historically, Mr. Gordon notes, productivity wanes when innovation based on fundamental new technologies runs out. The steam engine and railroads fueled the first industrial revolution, he says; the second was powered by electricity and the internal combustion engine. The Internet, according to Mr. Gordon, qualifies as the third industrial revolution — but one that will prove far more short-lived than the previous two.  "I think we're seeing hints that we're running through inventions of the Internet revolution," he says.

STILL, the software industry is making a big bet that the data-driven decision making described in Mr. Brynjolfsson's research is the wave of the future. The drive to help companies find meaningful patterns in the data that engulfs them has created a fast-growing industry in what is known as "business intelligence" or "analytics" software and services. Major technology companies — I.B.M., Oracle, SAP and Microsoft — have collectively spent more than $25 billion buying up specialist companies in the field.

I.B.M. alone says it has spent $14 billion on 25 companies that focus on data analytics. That business now employs 8,000 consultants and 200 mathematicians. I.B.M. said last week that it expected its analytics business to grow to $16 billion by 2015.

"The biggest change facing corporations is the explosion of data," says David Grossman, a technology analyst at Stifel Nicolaus. "The best business is in helping customers analyze and manage all that data."  

How new Internet standards will finally deliver a mobile revolution

The McKinsey Quarterly
APRIL 2011 • Bengi Korkmaz, Richard Lee, and Ickjin Park

As the Web experience evolves, smartphones may soon live up to their name, and every business’s mobile strategy will grow in importance.

An arcane-sounding change with potentially significant implications for consumers and businesses is under way on the Web: the shift to a new generation of HTML,¹ the programming standard that underpins the Internet. Senior executives, regardless of industry, should take note; like the exponential growth of device-specific applications, this evolution of HTML will further boost the power of mobile devices, accelerating changes in the way people consume content and the potential use of smartphones and tablets as both a marketing platform and a productivity tool.

The next generation of the Internet standard essentially will allow programs to run through a Web browser rather than a specific operating system. That means consumers will be able to access the same programs and cloud-based content from any device—personal computer, laptop, smartphone, or tablet—because the browser is the common platform. This ability to work seamlessly anytime, anywhere, on any device could change consumer behavior and shift the balance of power in the mobile-telecommunications, media, and technology industries. It will create opportunities and present challenges. This article seeks to provide a primer on these changes for senior executives, who may feel the effects of the move toward “Web-centricity” much sooner than they think.

Web-centricity

In some ways, the evolution of mobile technology resembles the battle among PC makers in the 1980s. While we today take it for granted that Microsoft’s Windows operating system underpins hardware from countless manufacturers, it wasn’t always that way. Remember the operating systems that powered the Commodore 64, the biggest-selling PC of all time, or the Apple II? Before the emergence of Microsoft’s DOS and then Windows, PC users faced a tough decision about which technology to adopt, because that determined the games and utilities they could use, as well as the general usefulness of their computers. The same occurs today with mobile devices. Users must weigh the hardware and software merits and commit themselves to a technology, whether it’s a device from manufacturers such as Apple or Research in Motion, the ever-increasing array of tablets and smartphones running Google’s Android operating system, or, soon, offerings from Nokia running on Microsoft’s Windows Phone 7 operating system.

The next generation of HTML, known as HTML5, may narrow these differences between mobile devices. HTML5, the most significant evolution yet in Web standards, is designed to allow programs to run through a Web browser, complete with video and other multimedia content that today require plug-in software and other work-arounds. In theory, this will make the browser a universal computing platform: without leaving it, users could do everything from editing documents to accessing social networks, watching movies, playing games, or listening to music. Not only would any device with a Web browser have these capabilities, but consumers would also have access to all content stored remotely “in the cloud,” independent of locations and devices.

That’s the first reason Web-centricity holds particular promise for mobile devices. The second is that it helps overcome the relatively weak processing power of smartphones and tablets compared with PCs and laptops. It’s partly this lack of horsepower that has fuelled the explosive growth in applications (or “apps”) to optimize the performance of specific devices: the average smartphone user now spends more than 11 hours a month using apps, more time than either Web browsing or talking, according to a March 2011 study by research firm Zokem. HTML5 has the potential to improve the mobile experience—its specifications enable browsers to locally store 1,000 times more data than they currently do, so users can work when offline—writing e-mails, for example—and their devices will automatically update when a network becomes available. What’s more, programs and applications run faster because complex processing tasks are handled by network servers, although mobile-network capacity must go on growing to deal with heavier data demands.

 

        
        Winning the Web standards battle         
                 

Of course, not all programs are suited to running through browsers, nor is HTML5 the first would-be universal platform to emerge: Sun Microsystems (purchased by Oracle in 2010) promised that with its Java language, programmers could “write once, run anywhere.” Things haven’t worked out that way. And there’s never a guarantee that one kind of standard will prevail (see sidebar, “Winning the Web standards battle”).

The rate at which developers are writing apps and consumers buying them is dizzying, and ingrained behavior can be hard to change. Web-centricity may raise security fears among users because programs are no longer installed on specific devices and because data are stored remotely. And there could be fragmentation issues with both the standard and the browsers—after all, existing ones, such as Google’s Chrome, Microsoft’s Internet Explorer, and Mozilla’s Firefox, don’t all treat the current standard, HTML4, the same way.²

Despite these possible headwinds, the number of HTML5 Web sites is increasing by the day. Hardware manufacturers are lining up behind HTML5, and the development community is undertaking efforts to safeguard data in the cloud at a very fast pace. We therefore estimate that more than 50 percent of all mobile applications will switch to HTML5 within three to five years—and the rate of transition could be considerably higher and faster. No matter how quickly the shift occurs, it will affect both consumers and businesses significantly.

Consumer impact

Consider a simple task many consumers currently use mobile devices for: reading news headlines. Today, that requires accessing a specific Web site—often a sluggish exercise in frustration—or separately installing an application on every device used and, for those that charge a fee, paying each time. With Web-centricity, a single application can theoretically be accessed from any device through a browser—pay once and you’re done. And because all content is stored in the cloud, billing information and preferences can be seamlessly shared and accessed, and all devices remain in sync. A consumer can start reading an article on a tablet and then switch to a laptop, picking up where she left off. In a more advanced example, she could start an instant-messaging or video-chat conversation on her desktop computer and continue it on her smartphone. The bottom line for consumers: Web-centricity represents a major step toward genuinely “smart” devices that offer the same simple, relevant, and personalized experience everywhere.

Industry impact

These changes to consumer behavior may affect the economics of industries ranging from telecommunications and media to technology and even advertising. As Web stores selling applications that can be used across devices proliferate, for example, cutthroat competition may leave ad agencies reminiscing wistfully about the days when they could claim up to 40 percent of every dollar of mobile-advertising revenue. Consider, briefly, the implications for the following players in a world where content is everywhere and the relative importance of operating systems and Web browsers for creating and distributing programs and applications is shifting.

Software developers. Application developers currently pay a fee of up to 30 percent to device makers, telecommunications operators, or operating-system developers whenever an application is sold to a consumer. In a Web-centric world, developers can avoid these intermediaries: not only can the same application be sold across all devices but anyone can set up a Web store and sell directly to users. Google, for instance, is already charging application developers a distribution fee of about 5 percent through its Chrome Web store.³ In addition, the emergence of an open platform will probably motivate bigger enterprise software companies to introduce—and quickly—mobile-based programs for managing customer relationships, marketing, and supply chains.

Telecom operators. Web-centricity may be a double-edged sword for telecom players. On the one hand, it will spur demand for mobile-Internet services, create opportunities for operators as consumers seek applications that work across multiple devices, and loosen the grip of native app stores. On the other hand, there’s no guarantee that operators can make money with new apps, the likely surge in data traffic will require significant investments in network infrastructure, and operators may face increased competition from companies offering Web-based mobile-voice and -video services.

Content providers. Web-centricity should provide revenue and savings opportunities for content providers. On the revenue side, the ease with which consumers can access Web-centric content on the go should stimulate their interest in more relevant, timely material. Moreover, the seamlessness with which consumers can access HTML5 content across devices could create more opportunities for providers, such as television and movie studios, to offer consumers programming directly or to work through aggregators such as Apple’s iTunes. Finally, advertising could support additional mobile content. Fragmented mobile platforms today make it hard for online publishers to manage ad inventories across a broad range of users. Advanced features such as consumer targeting and measurement may migrate to the mobile-Web environment. Of course, this development will no doubt attract entrants and intensify competition, making the new environment as challenging as it is dynamic.

Savings, a secondary benefit, come from avoiding the cost of converting an application from one platform to another (today, typically around 50 percent of the original development cost). Newspapers and magazines, for example, should be able to create content once and deliver it seamlessly across multiple devices, lowering production costs and increasing reach.

Device makers. Web-centricity will probably make consumers more “device agnostic,” and that will in turn reduce the ability of players to control an ecosystem of developers and could accelerate the commoditization of mobile devices. The shift does, however, create opportunities. Manufacturers will be able to better and more easily integrate software and hardware experiences within and across devices. They can try to develop compelling cross-device applications and speed up the push to make synchronizing and storing data across devices easier. Finally, they have some control (along with operators) in choosing the default set of Web-centric services and applications embedded in devices.

What it means for senior executives

Consumer uses propel many innovations associated with Web-centricity. Yet it could ultimately provide a range of benefits for companies as information technology moves to Web-centric platforms and away from the current hard-wired infrastructure and applications. These are enterprise-level issues, and any CEO who isn’t confident that the organization is grappling with them should start pushing the senior team to understand their importance.

The CMO

The emergence of the “m-dot revolution”⁴—the increasingly strong tendency of consumers to use mobile devices to access company and product information—will have its greatest impact on chief marketing officers. Many companies are already experimenting with innovative smartphone applications; Volkswagen, for instance, has released a popular racing game for the iPhone. Companies will be able to continue taking advantage of the enhanced power of mobile Web browsers to create compelling experiences directly for users. In addition, CMOs will need to push their teams to develop compelling mobile-advertising strategies that go well beyond merely inserting ads into applications, as many do today. HTML5 should create opportunities to use video advertising more often, for example, and the development of robust mobile capabilities may spur the evolution of marketing tactics such as the monitoring of shopping activity to deliver real-time, location-specific coupons.

The CIO

Web-centricity puts additional pressure on organizations to invest in corporate cloud infrastructure. Chief information officers should, for example, prepare for the day when consumers, employees, and suppliers all communicate and interact through the use of mobile devices that run Web applications. This phenomenon will not only extend the reach of the enterprise but also place a premium on analytics and possibly improve the competitiveness of companies that can exploit the new information and interactions a Web-centric environment provides. 

CIOs will have to decide whether costs can be cut and productivity increased by introducing rich applications both horizontally, across industries (for example, enterprise customer-relationship-management systems such as Salesforce.com), and vertically, within industries (say, mobile electronic medical records in health care or smartphone-based claims processing in insurance). Web-centricity also promises smaller productivity improvements, such as allowing users to store content locally for later uploading. Employees will therefore be able to work without being connected to the Internet—for instance, when they’re on airplanes.

The CEO

From the perspective of the chief executive officer, Web-centricity should be part of a broader imperative to elevate the importance of mobile marketing in corporate strategy. CEOs will need a response when, as must inevitably happen, they are asked how their companies are dealing with the m-dot revolution, which introduces a mobile element into everything from commerce to advertising to public relations. What’s needed is not just the coordination of mobile initiatives from functional offices, however. CEOs must take a big-picture approach to the collective implications of Web-centricity, the way it redefines a company’s interactions with employees and customers, and the challenges and opportunities it presents.

Of course, Web-centricity will require spending money to make money. Organizations will have to make IT investments, particularly for cloud-based computing and mobile platforms. Employees, especially in sales and operations, will need training in the art and science of mobility if companies are to maximize cost savings and productivity improvements. Yet Web-centricity also promises to make the mobile-Internet experience more open, complex, and dynamic. It may change the way consumers and enterprises behave. Even if companies don’t understand the technical aspects of this transition, they must master the technology’s potential and possible ramifications.

About the Authors

Bengi Korkmaz is an associate principal in McKinsey’s Istanbul office; Richard Lee is a principal in the Seoul office, where Ickjin Park is an associate principal.

Finance: Elusive information (FT today)

BY TOM BRAITHWAITE, FT, February 15 2011 22:25




It was Friday August 15 2008 and a senior official at the US Federal Reserve in Washington wrestled with a thorny problem: he wanted to know what was happening inside Lehman Brothers but was afraid to ask.

Pat Parkinson, now the Fed's top bank supervisor, was trying to find out which companies had derivatives contracts with Lehman as he gauged how severe the impact would be if the investment bank collapsed. But colleagues in New York told him that just requesting the data would be "a huge negative signal" for the bank's prospects and they were "very reluctant" to do anything that might "spook the market".

Lehman's implosion the following month was not the only recent instance where a calamitous lack of decent data has plagued financial markets. Others range from the European bank stress tests carried out last year, which officials admit relied on information "polluted by accounting", to the US stock market "flash crash" on May 6 that left the Securities and Exchange Commission floundering for answers.

But for the first time in decades there is a growing movement to rebuild the creaking data architecture that underpins modern finance.

Regulators' need to understand the crisis provided the impetus. When Lehman fell in September 2008, not only did institutions not know their rivals' exposure to Lehman, or to other problem areas such as subprime mortgages; they were sometimes unable to map their own with any speed. In the vortex, stock prices collapsed, liquidity dried up and investors and bankers ran scared.

"Everyone's going, 'what do I hold that is Lehman?'" says Mike Atkin, head of the Enterprise Data Management Council, a group of banks, information technology companies and regulators. "Wait a minute ... what is Lehman? Lehman isn't one entity – it's 10,000 entities. We don't know what our exposure is because we're not sure what Lehman is."

Lowdown on the OFR ● The Office of Financial Research, set up by the DoddFrank financial reform act last year to improve the quality and analysis of US data, has wide-ranging powers enabling it to compel institutions to provide information. ● Lewis Alexander, interim head and former senior Citigroup economist, has "a few million" dollars from the Federal Reserve for staffing. By 2012 the OFR will be funded by a tax on big banks. ● Based in the Treasury, the OFR will have an independent chair who reports to Congress. The White House has approached candidates.

Soon afterwards, John Liechty, a Pennsylvania State University professor, started asking regulators which of them collected data on the financial system as a whole. "I figured somebody would have the data so they could piece together the financial system and work out where the risk is. They said: 'Nobody's got it. Some is there and unshared. Other parts are not there at all.' I said: 'This is dangerous; it's crazy.'"

For years, individual statisticians, technology specialists and economists from regulators, financial institutions and academia had warned of the dangers. For years, they were dismissed as Jeremiahs and a root-and-branch reform of the data networks underpinning the financial system was rejected by the industry and regulators, which saw big costs and limited benefits.

John Geanakoplos, a Yale University professor, blames the Fed for not using data well, sometimes because of bureaucratic blockages, in one instance because officials balked at paying $400,000 for mortgage information, and sometimes because of a philosophical belief in self-correcting markets. In a recent presentation to the European Central Bank, he took aim at the "Greenspan-Bernanke doctrine" that "denied that there are bubbles, or that they could recognise one if they saw it".

In less damning terms, Keith Saxton, IBM's London-based global director of financial markets, points to the same problem. "Most of the data that the regulators and the central banks collect are what I call quite traditional," he says. "They have a view of 'The Bank' and everyone assumed that because that bank was healthy maybe there wasn't a problem with the system. It has turned out that the data they had about that bank weren't granular enough to be accurate: these guys can't get to the cash flow of an instrument."



But across the world, the evangelical geeks are gaining ground. In the US, Jack Reed, the Democratic senator from Rhode Island, took up the cause and managed to slip a new early-warning agency – the Office of Financial Research – into the mammoth Dodd-Frank financial reform bill that passed Congress last summer.

The OFR, which is being incubated in the Treasury before it gains independence, has begun work on standardising the components of every significant financial transaction. Eventually it will collect data, pull it all into a supercomputer and analyse the results, with the ultimate aim of spotting bubbles before they burst.

In a rare moment for Washington, an idea that was not proposed by the executive branch and without a big bloc of business support became law. A loose coalition of advocates managed to persuade one lawmaker, who sold the initiative to his colleagues in the face of scepticism from existing agencies. "Within the government there was a hesitancy to create an agency that was independent," says Mr Reed. The senator, an army veteran, says he wants the OFR to act like a "red team" – the military term for a group of soldiers charged with probing the weaknesses of their own comrades.

Words of change from previously sceptical bureaucrats are borne out by actions. The Fed does now buy the expensive granular mortgage data that Prof Geanakoplos highlighted. New rules will force more derivatives through clearing houses, allowing Mr Parkinson to grab data from fewer sources without alerting the market.

But the real data zealots think that regulators should go much further: the ultimate prize is a "dashboard" of the whole financial system. While anyone can view a snapshot of the stock market, the OFR and sister agencies in Europe and Asia would have the same visibility over darker parts of finance, allowing them to test various scenarios on Wall Street at the push of a button. The trouble is, at the moment, this is science fiction; it is impossible to create. "When something starts to go awry we go, 'what am I holding?' and I need to know it down to the loan level so I can run it against a scenario," says Mr Atkin.

"If Ford goes bankrupt what happens to the homes in Detroit? Well, I don't know how many homes in Detroit I've got in my mortgage-backed security because I can't unravel the bloody thing."

Francis Gross, head of external statistics at the Frankfurt-based ECB, points to the same issue: "The basic assumption is you have these vast pools of data that are quite homogeneous so that a spade is a spade – and that's where the problem comes."

The reason is that the building blocks of so-called "reference data" are not standard. As banks have grown by acquisition they have acquired thousands of legacy systems spread across different businesses – they do not have standard ways of recording data even within the group. So plotting relationships across the financial system is almost impossible. "The trading desk may book a transaction as Deutsche Bank and code that as 'DB'," says Lew Alexander, the Treasury official in charge of setting up the OFR. "When it gets to accounting, 'DB' may mean Dresdner Bank."


It is these non-standard "identifiers", for companies and for instruments, that Mr Gross and Mr Alexander are now trying to reconcile on both sides of the Atlantic. Unravelling garbled transactions costs the industry hundreds of millions of dollars a year in people and IT but during the boom years it always seemed too fiddly to fix.

One banker says: "We do a transaction and we get a lot of breaks and reconciliations ... Generally over 90 per cent of them are due to inconsistency of reference data, not due to misunderstanding between parties. If we all had the same reference data we would revolutionise how operations are done in our firms." He adds: "Logistically it's impossible to do unless it's imposed."

Mr Gross, who is leading calls for international standards of reference data, says regulators must be in the driving seat or it will be "ask[ing] cats to herd themselves".

The US now has the structure to start work, although even after Mr Reed managed to get the OFR through Congress, a handful of powerful Republicans continue to oppose it. Karl Rove, an adviser to former president George W. Bush, and Richard Shelby, the senior Republican on the Senate banking committee, both think it reeks of big government.

"I believe that the Democrats' new Office of Financial Research will not only fail to detect systemic threats and asset price bubbles in the future, it may threaten the civil liberties and privacy of Americans, waste billions of dollars of taxpayer resources and lull markets into the false belief that this new government power will protect the financial system from risk," says Mr Shelby.

Some Wall Street executives are worried about disclosing their trading positions to anyone, even regulators policing the system for systemic risk. Says the banker who has watched the OFR closely and supports it: "There is a concern about people giving up their positions ... There was a time in 1905 when people didn't report their income to the [Internal Revenue Service]. I think it will become: you do a transaction, you report it to the OFR."

From the other end of the political spectrum, liberal Democrats in Congress are worried the Treasury and the Fed are paying lip service to the OFR and will not give it the tools to be intrusive enough. They think scepticism among senior officials endures.

It is certainly a never-ending struggle. Forty years before Mr Parkinson grappled with Lehman, one of his antecedents extolled the virtues of technology for market supervision. In 1968, a year in which the New York Stock Exchange had to close for days at a time because paper records of trades were so out of hand, Manuel Cohen, chairman of the SEC, boasted that his agency was now using "its own computer" to monitor markets.

"We are able to provide a measure of protection to investors that theretofore had been virtually impossible due to budget and 'manpower' limitations," he said. "But our techniques in this area are not as fully developed as they will be."
..........................................
Information technology
How innovation has come to mean different things on different coasts.

In recent decades, a curious paradox has hung over the American economy. On the west coast, a gaggle of entrepreneurial companies, filled with some of the country's brightest brains, has been scrambling to track data in the smartest and most innovative way, writes Gillian Tett.

Companies such as Google, Amazon and Facebookare now able to monitor what consumers and businesses are doing around the world in real time. They can track everything from book purchases to friendship links and the consumption of breakfast cereal.

But on the east coast, another collection of highly talented brains has been delivering a very different form of innovation. Wall Street has produced a plethora of products and processes that has made the financial system more complex and (often) more opaque.

But while bankers have used cutting-edge computer technology to, say, develop ultra-fast automatic trading strategies, the data-handling innovations developed on the west coast have been slow to move east.

As recently as six years ago, traders in the credit default swaps market, for example, were still conducting deals by fax. Banks' back offices were not standardised and regulators could not collect data from them in anything resembling a timely manner.

Beyond banking, many other parts of the financial world went almost entirely untracked by regulators, who remain behind the technological curve.

The question that hangs over the Office of Financial Research, being set up as part of reforms to the sector, is whether these different west and east coast worlds can now meet – and apply Silicon Valley-style innovation to the financial system as a whole. Can the techniques that allow Facebook to aggregate data on online friends in a flash be used to track derivatives trades, say?

Optimists argue that the answer is yes, given the extraordinary strides in computing power that have already occurred. Officials linked to the OFR have started talking to companies such as IBMabout how to transplant innovations in the non-financial world into a coherent form of data collection in finance.

But pessimists retort that financial companies have little incentive to co-operate; after all, opacity has on the whole served Wall Street well, enabling traders to enjoy fat profits.
Either way, the really big question is whether the type of entrepreneurial, innovative drive that inspires Silicon Valley can be transplanted to the state sector.

"If you really wanted to revolutionise [data collection], you should ask somebody like Google to run it, and pay them properly," observes one senior banker, only partly in jest.
Right now, however, that prospect seems even harder to imagine than a world where the OFR starts to fly.

Copyright The Financial Times Limited 2011. Print a single copy of this article for personal use. Contact us if you wish to print more to distribu