Data is used for everything, from working out which films a streaming service recommends you watch, to whether or not you are eligible for a loan.

The growth of data has created a huge opportunity to increase understanding of consumers. As former Hewlett-Packard chief executive Carly Fiorina puts it, “The goal is to turn data into information and information into insight.”

But as the AI era continues, ever more data will be collected, posing legal, ethical, budgetary and even environmental challenges for businesses and governments.

Whose data?

According to a 2015 IBM study, 90% of all existing data was collected in the previous two years. And amid this massive growth comes concern about confidentiality.  

For instance, in the UK the data of at least 1 million patients was transferred to AI provider Deep Mind in an attempt to better understand kidney ailments, while Google collected data on iPhone users’ web browser histories then sold it to advertisers.

Developments like this have raised fundamental questions: who owns data? What can they do with it?

Regulation has been slow to protect people’s rights in some countries, and there has been intense debate about what rules cover data once it crosses national borders.

Global privacy

The European Union and the United States had a long-standing agreement known as Safe Harbour, but in 2015 the EU’s Court of Justice declared the agreement invalid after revelations the US’s National Security Agency had spied on EU citizens’ data.

A replacement, Privacy Shield, came into force in 2017 and aimed to tighten the obligations of US corporates with access to EU data.

The US currently has a mish-mash of federal and state laws, but no country-wide regulatory framework, which some suggest ill serves individuals.

mixed signals

Globally there is a mixed picture on data privacy. Singapore, for example, in 2012 introduced a Personal Data Protection Act that aimed to provide a baseline standard of protection, such as consent and an explanation of why data was being collected and stored.

In 2017, China’s cyber security law, although largely aimed at increasing national security, also introduced comprehensive national privacy regulation.

India introduced Privacy Rules in 2011 requiring corporates collecting, processing and storing personal data to comply with certain procedures, such as ensuring they have obtained consent for reasonable use of information.

Perhaps the biggest regulatory change is coming from the EU, which in May 2018 will see its General Data Protection Regulation come into force. This is an all-embracing framework that aims to put individuals in control of their own data.

European standards  

Aims of the EU regulation include ensuring data is collected lawfully, with consent, for specified, explicit and legitimate purposes, and is not kept for longer than needed.

People will have the right to demand to be sent their data and know how it has been used whenever they want, and to demand it be removed from company systems.

Some business leaders see this an opportunity and a threat: the one being that it will focus companies on greater service personalisation, while the other that if a company offends the public, mass demands to delete data could divert resources from core money-making operations.

Crucially, the regulations demand that businesses storing EU citizens’ data comply with the rules even if they are not based in the EU.

A fine problem?

Fines for serious data breaches, such as loss of commercially sensitive information, could lead to levies equalling 4% of global turnover or €20 million, whichever is greater.

Experts say that adherence to the regulation could become central to trade deals with other blocs and states, and could be a way of forcing those with laxer standards to improve their own regulations.

Meanwhile, companies that interact with data on a corporate’s behalf will also have to prove they are rule-compliant – blaming a contractor will not be a legitimate excuse in the event of a hack.

Mines of information

The collection of data on a vast scale poses other problems. Increasingly companies must ensure they delete old or unwanted data regularly, that publicly collected information is suitably anonymised, and that data is secure.

For security, some business use hybrid storage methods, keeping crucial information in servers sheltered from internet access while less sensitive material is hosted on cloud platforms, but this is not cheap.

The electricity needed to run a large data centre, particularly its cooling systems, can be the same as for decent-sized town, which adds to corporate bills, while applications such as algorithms for driverless cars will demand more real-time analysis and delivery of information over the internet.

So despite all the opportunities that data brings for companies – it also brings a regulatory, environmental and cash cost that can only really go one way – up.