AI raises ethical questions as well as a need for more transparency from companies and more regulations from governments

BearingPoint as the partner of the ESCP “Retailing 4.0" Chair is committed to contributing to building bridges between the academic world and companies. As part of this engagement, the BearingPoint Retail, Luxury and CPG team mentored the writing of a series of articles of ESCP MSc students and shared insights on major retail stakes. Enjoy your reading !

The impact on data protection

Data is the key driver for everything, and companies are fighting for it in order to provide a seamless and personalized omnichannel journey. Nevertheless, it would be a naïve assumption to think that their intentions stop there. This raises ethical questions as well as a need for more transparency from companies and more regulations from governments.

To answer those questions, let’s first define what Freedom of clients is. AI will impact dramatically the right to freedom and will influence the way people get access to and find information online. Online intermediaries - on top of all social media platforms and search engines - increasingly use AI systems to control information that users engage with, in opaque and inscrutable ways. While some of their uses, such as spam filters or suggested items for online shopping for instance, may seem harmless, others can have more serious repercussions. Tech companies are mainly using Big data to send us ads and guide us in our choices when it comes to selecting a product. Thus, these targeted ads are limiting our choices – put in other words: our freedom.

But what are the most concerning issues that we face with AI and ML (Machine Learning)?
Sharing and collecting data are AI and ML’s original function. This fact puts internet users in a vulnerable position as they cannot know which data is being shared and/or collected. On the other hand, the use of AI in cyber security tools helps to improve our ability to identify and respond to threats.

Privacy concerns are emerging as companies are increasingly introducing consumers’ and suppliers’ data into advanced AI-powered algorithms to create new and sensitive pieces of information, without the knowledge of consumers and employees involved.

Data appropriation can be considered a form of exploitation, as companies use data to create value without providing people with comparable compensation. As mentioned by The Guardian in a 2016 article, Oracle and MIT Technology Review argue that a major reason for the success of companies such as Amazon is that they all use “data as an asset” and profit from that. Thus, the data broker industry is able to make $200bn in revenues each year. When taking this figure into consideration, could it really be recognized as fair to the customer?


The revenue the data broker industry is able to make each year.

What if they resell their collection of data for political goals or surveillance organizations, just like happened with the Facebook scandal in 2018, where the personal data of internet users were collected and analysed for the U.S. election candidates?

What if data is failed to be protected? To give another example - that is commonly less known - Marriott announced in late November 2018 a leak in their data center, affecting the data of about 327 million guests. Among the sensitive information, there were names, passport numbers, email addresses, phone numbers, payment card numbers, and payment card expiration dates.

The rise of techniques such as video surveillance, facial recognition, behaviour analysis etc., by public authorities and private companies such as Alibaba hinder freedom of expression and could be for governments a tool to implement predictive policies that citizens didn’t ask for.

To name an example that sounds like an episode of Black Mirror, the Chinese government has created a social credit score (SCS). Such a system creates concerns about individual choice, rights and freedom by inciting citizens who behave “right” in the society, to enjoy benefits such as priority access to public housing, travel visas and job promotions. Citizens are practically forced to comply with regulations in an uncommon way. However, at the same time this SCS leaves the world wondering about its reliability and also, more privileged citizens are induced to engage in corrupt activities and pay to have their score increased.

This leads us to the opposing point of views regarding the subject: the EU’s and China’s. The EU backs AI regulation while China and the US favour technology. Indeed, the race for innovation determines the efforts to safeguard basic rights…

As a matter of fact, China’s Government is really incentivizing companies to use AI and ML as they are also using it to control the population. Nevertheless, few policy advisers now recommend the Chinese to introduce a regulatory framework for AI and enhance technology used by regulators to strengthen industry-wide supervision, as technology could get out-of-hand some days.

Alibaba, for instance, is now coping with security matters. The e-business website will likely face spill-over concerns over security as it looks to become a hardware and software powerhouse with the launch of its Hanguang 800 AI chip, but the thing that will prove to be a valuable proposition will be its ties to China.

What about Europe? Would we allow private companies to do such things?

In our case, the EU aims to regulate AI ethics. The EU data processing regulations limits the potential for AI in Europe. Therefore, the European companies are not able – at the moment – to use the technology to its full potential.

By 2018, 26 countries had established national strategies. Although many of these mention ethics, they are often only a general statement on the need to preserve rights.

As with data and privacy regulation, the EU is continuing to develop rules on AI. The guidelines issued by the European Commission in April and supported by high-level experts follow the idea of a "trustworthy AI".

Moreover, they provide clear ethical principles and a checklist to be used when developing AI systems. The EU's GDPR, affects the use of AI in at least three ways: by limiting the collection and use of data, restricting automated decision making and increasing compliance costs and risks. However, we can wonder If the EU will want to reform the GDPR in order to not fall behind other countries, such as the United States and China, in the development and use of AI.

Recommendations :

The reasons why Alibaba was so successful in China comes from the fact that it was able to build an ecosystem which includes various activities such as C2C, B2C, financial services and payments with AliExpress, Taobao, Tmall, logistics, travel agency and brick-and-mortar stores with Hema. By analysing Alibaba’s path, we can clearly see that its success comes from its ability to make customers stick to it.

However, it is also easy for Alibaba to, at some point, be displaced by another ecosystem player; which is basically what happened to eBay. If the Chinese retailer wants to continue thriving, it will have to keep investing in AI and data to automate more decisions, cutting costs without negatively impacting the customer experience, getting obsessed with what the customer wants and anticipating future disruptions as it has done so far. The trends we are now seeing in China will for sure come to Europe and the U.S., even though customers there are more concerned about their data being collected and used.

To keep staying relevant, retailers will also have to be where the customers most need them and will have to consider unconventional digitised and AI-empowered point of sales. Alibaba is already ahead with that; it has transformed restrooms into a retail opportunity. Other retailers might follow next.

Retailers will face the choice of having to compete on automation, convenience, speed and low prices or on expertise, creativity, curation (and convenience as well). Looking at 2020, we have reasons to think that the second set of characteristics will be the winning one. In fact, AI allows employees to move away from tasks that can be automated (e.g. pricing, product assortment) and rather focus on enhancing the relationship between the brand and its customers.


ML and AI are great tools that offer extensive support to enhance customer experience, and also holistically boost companies’ growth. The more AI and ML are integrated in the in-store technologies, the more retailers will experience a positive financial impact. In fact, personalized offerings and optimised assortments will increase sales and cut down waste. At the same time, customers’ conversion rate will increase and retailers will exploit customers’ willingness to pay for individual products and increase their basket size. In-store employees will be more skilled and better trained, since most of the basic tasks will be performed with the help of AI tools. Consequently, their job will shift to higher value and more customer-facing activities. Taking as reference a research done by McKinsey, there are reasons to believe that in the future, partially automated stores will double today's stores’ profitability. This is due to minimizing the workforce in-store, better inventory management, optimized supply chain and enhanced customer experience leading to higher sales volumes.

We can also expect a rise in the trend of AI-fueled outsourcing, meaning that AI-empowered platforms like Google Duplex will be able to find the product or service that fits the customer best, get the best deal with the seller and then book an appointment or directly purchase the goods.

When looking at the 2020 trends coming from China, live streaming seems to be the most promising. This trend began in 2014 when Taobao and Mogujie started to sell apparel to female shoppers. This year, most of China’s ecommerce platforms have launched - or are planning to launch - similar live stream functions to keep up with Alibaba, meanwhile this trend is going global. Korea and Japan were some of the first countries to do it, and Alibaba later brought the practice to Russia and Southeast Asia. Today it is using structural data and AI still has not been included. We have reasons to believe that this is the future and that when AI will be incorporated into live streaming, this will become a ground-breaking approach for retailers to sell.

However, rules and regulations have to be laid out and business practices have to be strictly scrutinized in order to protect customers’ freedom. We do not know whether the advantages brought by AI, ML and the related data collection counterbalance the privacy concerns. However, we have the impression that the average citizen is not well informed on the subject and that as a consequence, his/her information can be easily exploited. In fact, if people knew that these technologies do not only imply getting better and personalized customer experiences, but actually also involve other aspects of their lives - such as their jobs or insurance contracts - then they would be much more careful about what they share and whom they share it with. 

If we would get to a very advanced use of AI, would this really be the life citizens of the world are asking for? Are we ready to give up our freedom and intimacy just to get a better customer experience? Do we really want to give our most sensitive personal information to big multinational corporations? These are the questions we urgently need to ask ourselves before living in a world we did not ask for.

Authors: Philippe Lesage, Grégoire Baizeau, Solène Bongrand-Mas de Trehoult , Judith Garniron, Greta Rivoira, Shakti Vel Ramesh


  • Sources
    • Rathod, Atman, et al. “Machine Learning Applications in Retail: 6 Real World Examples from Market Leaders.” Data Driven Investor, 18 July 2019,
    • “The Amazing Ways Chinese Tech Giant Alibaba Uses Artificial Intelligence And Machine Learning.” Bernard Marr,
    • Contributor, V. (n.d.). 4 Ways AI is Revolutionizing Personalization and Customer Experience in Retail. Retrieved July 01, 2020, from
    • “Artificial Intelligence Resolves Paradox of Consumer Choice.” Zenith, 1 May 2018,
    • “At Alibaba, Artificial Intelligence Is Changing How People Shop Online.”, 22 Jan. 2019,
    • “The Future of Retail.” TrendWatching,
    • Matveeva, Sophia. “Five Trends Driving The Future Of Retail.” Forbes, Forbes Magazine, 31 Oct. 2019,
    • About the author(s) Praveen Adhi is a partner in McKinsey’s Chicago office. “A Transformation in Store.” McKinsey & Company,
    • Article 19. “Privacy and Freedom of Expression In the Age of Artificial Intelligence”. April 2018. PDF file. Retrieved from
    • Yu, Eileen. “Alibaba Heralds 'Data Intelligence' Era, but Likely Faces Security Concerns over Chinese Ties.” ZDNet, ZDNet, 25 Sept. 2019,
    • “Data Privacy Risks to Consider When Using AI.” FM Magazine, 1 Feb. 2020,
    • Faggella, Daniel. “AI in China - Recent History, Strengths and Weaknesses of the Ecosystem.” Emerj, Emerj, 5 Apr. 2019,
    • “Alibaba, Big Data, and the End of E-Commerce.” Lynn Huang,
    • Mark Esposito Professor of Business & Economics at Harvard University and Grenoble École de Management, et al. “What Is Machine Learning?” The Conversation, 10 Apr. 2020,,or%20to%20make%20accurate%20predictions.
    • Grothaus, Michael. “How Our Data Got Hacked, Scandalized, and Abused in 2018.” Fast Company, Fast Company, 12 Dec. 2018,
    • Department, Published by Statista Research, and Apr 29. “China: Online Shoppers 2020.” Statista, 29 Apr. 2020,
    • Murrell, Audrey. “Pushing The Ethical Boundaries Of Big Data: A Look At China's Social Credit Scoring System.” Forbes, Forbes Magazine, 31 July 2018,
    • “Big Data in Healthcare: Privacy Is Major Ethical Concern.” Relias Media - Continuing Medical Education Publishing,
    • Faggella, Daniel. “AI in China - Recent History, Strengths and Weaknesses of the Ecosystem.” Emerj, Emerj, 5 Apr. 2019,
    • “The Guide to Machine Learning in Retail.” The Guide to Machine Learning in Retail | Tryolabs Resources, 21 Aug. 2018,
    • Jeffrey Ding. “Deciphering China’s AI Dream”. 2018. Centre for the Governance of AI, Future of Humanity Institute, University of Oxford [PDF file]. Retrieved from
    • China Institute for Science and Technology Policy at Tsinghua University. “China AI Development Report 2018”. July 2018. PDF file. Retrieved from:
    • Lifelong Learners. “Retail 2020 | 5 Technologies that will change the way you shop”. Dec 4, 2017. [Video file]. Retrieved from:
    • Alibaba Group. “Alibaba's "New Retail" Explained”. Mar 30, 2018. [Video file]. Retrieved from:
    • maybe add "recommendations based on selling rules only advantaging the companies (e.g. products that companies want to sell, upselling, etc.), monetization of customers data (trade between companies of your data)" here
    • add scoring of customers based on unethical rules (e.g. diseases), data leakage (e.g. bank accounts, pass codes, …), etc. here?
Toggle location