This paragraph caught my attention, as I’ve been trying to shift the discussion from surveillance to debt capitalism.

What does this algorithm-industrial complex look like and who is involved?

Perhaps our first glimpse of the catastrophic impact of algorithms on civil society was robodebt, deemed unlawful by the Federal Court in a blistering assessment describing it as a “massive failure in public administration” of Australia’s social security scheme.

Very well said.

The algorithm-industrial complex is characterised by a power and skills distortion: public sectors gutted of skills, and the influence of and outrageous expenditure on outsourcing, tech, and consultants.

…what defines the algorithm-industrial complex is the emergence of policy which can only be executed via algorithms.

That’s not quite right, since humans have used algorithms for thousands of years, but I get the point.

One of Khwãrezm’s most famous residents was Muhammad ibn Mūsa al-Khwarizmī, an influential 9th century scholar, astronomer, geographer, and mathematician known especially for his contributions to the study of algebra. Indeed, the latinization of his name, which meant ‘the native of Khwãrezm’ in Persian, gave English the word algorithm. He wrote a book in Arabic about Hindu-Arabic numerals; the Latin translation of the book title was Algoritmi de numero Indorum (in English Al-Khwarizmi on the Hindu Art of Reckoning).

Although the word algorithm can be traced to this man’s name in ancient Baghdad, it’s really the even more ancient Babylonians who started using algorithms 4,000 years ago, as computer scientist and mathematician Donald E. Knuth wrote in Ancient Babylonian Algorithms, Communications of the ACM 15, no. 7 (July 1972) 671-77:

One of the ways to help make computer science respectable is to show that is deeply rooted in history, not just a short-lived phenomenon. Therefore it is natural to turn to the earliest surviving documents which deal with computation, and to study how people approached the subject nearly 4000 years ago. […] The calculations described in Babylonian tablets are not merely the solutions to specific individual problems; they are actually general procedures for solving a whole class of problems.

The problem today thus isn’t someone or something following instructions, its how people centralize instructions to be brittle and dictatorial (concentrate wealth through high exit barriers) instead of making them flexible to embrace the compromise of representative democracy.

Hitler’s devout followers are not much different from a descendant of his followers who sets up companies of self-serving algorithms. So the real danger is a policy dependent on an intentionally monopolistic (fascist) model of proprietary technology, such as Palantir.

David Hume warned of exactly this in the 1700s, which is very late when you think about how old algorithms really are.