Thinking Complete

During my last few years working as an AI researcher, I increasingly came to appreciate the distinction between what makes science successful and what makes scientists successful. Science works because it has distinct standards for what types of evidence it accepts, with empirical data strongly prioritised. But scientists spend a lot of their time following hunches which they may not even be able to articulate clearly, let alone in rigorous scientific terms - and throughout the history of science, this has often paid off. In other words, the types of evidence which are most useful in choosing which hypotheses to prioritise can differ greatly from the types of evidence which are typically associated with science. In particular, I’ll highlight two ways in which this happens. First is scientists thinking in terms of concepts which fall outside the dominant paradigm of their science. That might be because those concepts are too broad, or too philosophical, or too interdisciplinary. For e...

Jebari and Lundborg have recently published an article entitled Artificial superintelligence and its limits: why AlphaZero cannot become a general agent . It focuses on the thorny issue of agency in superintelligent AIs. I’m glad to see more work on this crucial topic; however, I have significant disagreements with their terminology and argumentation, as I outline in this reply. Note that it was written rather quickly, and so might lack clarity in some places, or fail to convey some nuances of the original article. I welcome comments and further responses. Their paper runs roughly as follows: Jebari and Lundborg first discuss the belief–desire model for intentional action, under which agents act in ways that they believe will bring about their desires. They then distinguish between different degrees of generality that agents can have: “general agency is, loosely speaking, the ability to act in a diverse set of situations.” They consider thermostats to be very specialised agents, and do...

I recently finished reading The Technology Trap , by Carl Frey. The book attempts to do two things: chronicle the role of technology in economic progress throughout history, and argue that automation in our own era parallels the first seven decades of the industrial revolution, during which the wealth from mechanisation failed to reach most citizens, leading to a populist backlash. I particularly enjoyed the first component, because until now I’ve read much less about the industrial revolution than I should have. That also means that I’m not qualified to evaluate the book’s accuracy. However, it had interesting discussions of: The technological prowess of the Romans, and why they were held back from industrialising both because of their slave-based economy, and also because of an implicit dismissal of the private economy. The development of some surprisingly important technologies during the Middle Ages, such as wind- and water-mills, better ways of accessing horsepower (via im...

It was very striking to me to contrast the two recent successes of OpenAI: one, OpenAI Five , beating some of the best humans at a complex game in a sophisticated virtual environment; and the other, Dactyl , fumblingly manipulating blocks in ways that children master at young ages. This is not to diminish how much of an achievement Dactyl is - no other reinforcement learning system has come close to this sort of performance in a physical task. But it does show that the real world is very complicated , compared with even our most advanced virtual worlds. To be fair, the graphics and physics engines used to render videos are becoming very good (and as movies show, practically indistinguishable from real life when enough work is put in). Audio generation is worse, except on human voices, which are now very convincing - but background sounds aren't a crucial component of our environment anyway. The biggest experiential difference between current simulations and the real world seems to...

High-level analysis of reinforcement learning Building machines that learn and think like people . Lake et al. think that human learners are doing something fundamentally different from machine learners: we carry out tasks in the context of many years of related tasks, whereas ML systems cannot. So, they ask, "how do we bring to bear rich prior knowledge to learn new tasks and solve problems so quickly?" The core ingredients they identify are intuitive models of physics and psychology, which exist from infanthood, and the capacity for rapid building of mental models which can be used for classification, prediction, communication, explanation, and composition. The skill of model-building can be distinguished from pattern recognition, which the authors suggest makes up much of the recent progress in deep learning. In particular, notions of composition, causation and hierarchy which are central to models may be weak or non-existent in pattern recognition. Further, they note ...

This blog often concerns itself with death, but not so much with that other unyielding constant of life, taxes. Inspired by a conversation with my friend Will, I’ve decided to rectify that imbalance. Apart from complaints, what is there to say about taxes? Economic theory says they should: Be very difficult to evade. Not disincentivise the production of things we actually want. Be progressive - ideally in percentage terms, but at least in absolute terms. A poll tax (a flat tax on all adults) meets criteria 1 and 2, since I doubt people will take it into account much when having children. But it fails dramatically on 3 - it's a fixed flat sum, which means it can't raise much money without badly hurting the poor. A consumption tax like VAT (value added tax) discourages consumer spending, which may well be good for people's happiness and the environment - but still hurts the poor if implemented badly (because it increases the cost of goods by a fixed percentage). A good...

...you can download machine learning papers, which is almost the same thing, and I have so many papers on my to-read list. I like being able to write summaries of the most interesting ones, but when I hit 200 open tabs of interesting papers, I realised that the backlog was only going to keep piling up. So I've spent the day skimming as many as I could. There are many I'd like go to back and reread in more detail, but for now the summaries below will have to do. Deep learning theory Approximation by superpositions of a sigmoidal function . The original paper showing that sufficiently-wide one-hidden-layer neural networks can approximate any function. Why does unsupervised pre-training help deep learning?  Another classic paper, offering an explanation for why unsupervised pre-training improves performance: apparently it's an "unusual form of regularisation". (How widely is it still used?) Do deep nets really need to be deep?  Ba and Caruna show that shall...

Dota 2 and Starcraft 2 are two of the most popular and complex esports; professional gamers train for years to master them. But how hard are they for AIs? DeepMind is working on the latter . OpenAI has already created a system ( OpenAI Five ) which is able to beat strong amateurs at a simplified version of the former, and they'll be challenging professionals in August. By their own account, they were surprised that their standard reinforcement learning approach worked so well, because of the game's complexity: Counting every fourth frame, as OpenAI Five does, Dota games last on average 20,000 "moves", compared with 40 for chess and 150 for go. Unlike chess and go, Dota is a partial-information game in which players need to infer what their enemies are doing. The range of actions in Dota is very large, with many of them varying continuously. Apparently, around 1000 are valid at each timestep (although I'm not quite sure what OpenAI are referring to as an actio...

Can the brain do backpropagation? - Hinton, 2016 In this talk, Hinton rebuts four arguments which neuroscientists have used to argue that the brain is not learning using backpropagation. Most human learning is unsupervised, without the explicit loss functions usually used in backpropagation. Hinton argues that error signals can be derived in unsupervised contexts using many different methods: reconstructing input signal (like autoencoders); comparing local predictions with contextual predictions; learn a generative model (wake-sleep algorithm); use a variational autoencoder; generative adversarial learning. Neurons don't send real numbers, but rather binary spikes. Hinton argues that this is a form of regularisation which actually makes the brain more effective. Any real number can be converted into a probability of a binary firing signal; the effects of doing so are similar to those of using dropout. Such strong regularisation is necessary because the brain has arou...

My views on intelligence have shifted recently after listening to an episode of the Rationally Speaking podcast where Dr Herculano-Houzel talked about her book The Human Advantage . She made a number of fascinating points based on her pioneering research on measuring the number of neurons in various species' brains: The brains of primates have higher neuron density than the brains of other mammals, particularly in the cerebral cortex, which is largely responsible for higher-level abstract thought. Amongst primates, number of neurons is roughly proportional to body mass. Apes are the main exception to the latter rule; their diets aren't calorie-rich enough for them to support brains as large as the trend would suggest. Humans, by contrast, fit right on the trend line: our innovations such as using tools and fire to cook food allowed us to obtain more calories and therefore grow bigger brains than other apes (bipedalism also helped, reducing the energy cost of walking by a ...

Is implementing a model of intelligence like the one which I outlined in my last essay easy or hard? How surprised should we be if we learn that it won't be achieved in the next 200 years? My friend Alex and I have very different priors for these questions. He's a mathematician, and constantly sees the most intelligent people in the world bashing their minds against problems which are simple and natural to pose, but whose solutions teeter on the edge of human ability (e.g. Fermat's last theorem), and where any tiny error can invalidate a proof. Many took hundreds of years to solve, or are still open. I'm a computer scientist, and am therefore based in a field which has blossomed in less than a century. It's a field where theoretical problems naturally cluster into complexity classes which are tightly linked, so that solutions to one problem can easily be transformed into solutions for others. We can alter Turing machines in many ways - adding extra tapes, making...