References
— David Danks & Alex John London (2017). Algorithmic Bias in autonomous Systems. In Proc. IJCAI 2017. Online: https://www.andrew.cmu.edu/user/ddanks/papers/IJCAI17-AlgorithmicBias-Distrib.pdf
— ACM (2017) Online: https://www.acm.org/binaries/content/assets/public-policy/2017_joint_statement_algorithms.pdf
— FATML Online: https://www.fatml.org/resources/principles-for-accountable-algorithms
— The IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems. Ethically Aligned Design: A Vision for Prioritizing Human Well-being with Autonomous and Intelligent Systems, Version 2. IEEE, 2017. Online: http://standards.ieee.org/develop/indconn/ec/autonomous_systems.html
14 Algorithms and markets: a need for regulation?
Heike Schweitzer
Freie Universität Berlin
14.1 Statement
Three main developments characterize the ongoing digitalization of the economy: (1) The increasing amount and importance of automatically generated data for the creation of new products and services and for the organization of all economic activities; (2) the development of ever more sophisticated algorithms to make economic use of that data; and (3) the rise of new business models based on data and its systematic analysis and use.
These three developments fundamentally change the way we communicate, interact socially and act (and are treated) on the market. As a consequence, the existing social and legal order has come under pressure: Its ability to ensure the fundamental values of our societies – among them private autonomy, privacy and a sophisticated control of private and public power – can no longer be taken for granted.
- As a reaction to the new explosion of data, data analysis capabilities – and possibly also “data concentration” – we have to rethink the informational order as it has existed in the past. This includes a rethinking of data protection rules as a new legal infrastructure for the functioning of markets, a rethinking of the role and responsibility of information intermediaries and the like.
- The ever more widespread use of “autonomous agents” challenges our concepts of agency, responsibility and liability. We have to rethink the rules on decision-making, discuss the legal demands we want to place on decision-making by autonomous agents and the preconditions for attributing such decisions to natural or legal persons.
- To what extent do we want to bind autonomous decision-making software to anti-discrimination rules and/or other ethical standards?
- If decision-making by self-learning algorithm resembles more some sort of “intuitive”/unconscious decision making than a conscious/rational decision-making (Mireille Hildebrandt): To what extent do we require a rational justification of decisions based on transparent criteria such that their fairness can be subjected to legal control? When / how do we want to impose such requirements on private actors, as opposed to state actors?
- We have to understand how the use of algorithms affects markets and analyze whether adjustments to the existing rules – in particular: data protection, fair trading and competition rules – are needed to safeguard their well-functioning and fairness. To make things more complicated, these three challenges are not separate, but deeply interlinked. We have to understand and handle the interdependency of orders (information order, decision making systems, market order). The goal must be a reconceptualization of these orders and their adaptation to the new challenges of a digital society and economy in a way that preserves the fundamental values that we continue to regard as the basis of our democratic societies and social market economies.
14.2 Research questions
The focus here is on how the use of algorithms affects markets and whether, due to such effects, new needs for regulations arise.
In the currently ongoing debates, the focus is on three potential risks associated with the exponential growth of the use of algorithms in the marketplace:
(a) Algorithms can increase market transparency – will they thereby facilitate coordination among suppliers as well as buyers? Under what conditions? Product homogeneity and a small number of actors in the market are familiar factors. But in an algorithm-driven market environment, do we have to expect collusion also where product heterogeneity, innovation and heterogeneity of preferences prevail, and where products and services become more and more individualised? Do we need to adjust existing competition rules to deal with the risk of “algorithmic collusion”, namely abandon the distinction between “independent decision-making” and collusion, so as to cover parallel behavior also? (b) Algorithms can facilitate price discrimination among different buyers, possibly even enabling some sellers to approximate perfect price discrimination. Is this a challenge to the well-functioning of markets that the law should address? Does it threaten to undermine the general trust in the fairness of market-functioning? So far, the main task algorithms perform in the platform economy is to find the right match between heterogeneous products and heterogeneous preferences. In this regard, they “discriminate” according to preferences. There is a worry, though, that based on detailed personal profiles and an increased algorithmic understanding of “situational” power algorithms will offer the same products at different prices to different consumers. Will this do harm to the well-functioning of markets – despite the fact that, arguably, output will be maximized? What are the distributional consequences? What are the consequences for trust in markets? Can buyers self-defend against such uses of algorithms? Is there a risk of a costly “arms’ race”? Is there a need for transparency (a duty to disclose arguably already follows from unfair trading law) or for more intrusive regulation? Does competition law provide a layer of protection? In the “old world”, active consumers were meant to provide protection also to the lazy consumers. In a world characterized by personalization, this may no longer hold. Should competition law react by narrowing market definitions and expanding its concept of market power to cases of situational power? Or should private law react by specifying its concept of “bonos mores”? Can data protection law contribute to the solution of the problem, and if so: how? (c) Which principles apply to the use of algorithms by digital information intermediaries – and in particular: by digital information intermediaries with some degree of market power? The explosion of information needs to a new importance of information intermediaries that ensure an efficient matching of parties. What is the effect of these intermediaries on market functioning? The intransparency of matching algorithms may significantly decrease the risk of collusion between sellers in markets – all the more, since information intermediaries should generally not be interested in such collusion. Also, the presence of information intermediaries should lower the risk of consumer exploitation – at least to the extent that they offer a meaningful product and price comparison. New risks can arise when intermediaries themselves possess some degree of market power – either on the business side or on the consumer side; and all the more, if the intermediaries are vertically integrated. Do we need a generalized principle of “digital intermediary neutrality” to be implemented into the relevant ranking algorithms? or a principle that outlaws the algorithmic priorization of vertically integrated offers? A fiduciary duty of personal butlers (as a specialized version of information intermediary / agent) vis-à-vis a consumer using it? Are digital information intermediaries under an obligation to explain the ranking of their offers – in order to effectively outlaw discrimination and/or self-priorization? (d) Primarily, markets are meant to ensure an efficient allocation of resources. In order to perform this function, a degree of trust in the well-functioning and fairness of these markets must exist. In the presence of algorithm-driven markets: Do we need new rules for ensuring such trust? Do we need to expand the existing anti-discrimination rules as
they apply also to algorithmic decision-making? Do we need new rules to avoid consumer exploitation in the light of potentially new degrees of information asymmetry?