Generalist AI beyond Deep Learning
Table of Contents
- Deep learning is bruteforcing the solution
- Brain vs computer comparision don't look in opposite direction: How may brains are needed to simulate MacOS.
1. What is Intelligence?
is the ability to construct a path through space of computable functions. aka. Intelligence is the ability to make models.
We are agent that interact with the system and control it. To control a system requires counterfactual reasoning. This means, a model of how the world works is required.
How to construct a space of computable functions that is tractable?
Deep Learning is the only thing that works at scale. It finds arbitary functions for big data, within reasonable time.
How does deep learning work? It is differentiable computing. Represent functions such that they form a continuous space.
It represents all functions by real numbers.
Alternate to this is representing functions is by using boolean operators or finite automata.
At the base level of physics, things are discrete. This might be because language itself has to discrete. Those languages that assume that the bottom most region is continuous result into contradiction.
Equivalence between continuous mathematics and discrete mathematics has been shown.
Lisp interpreter as a diophantine equation with 17k variables. 'The complete arithmetization of EVAL' [Algorithmic Information Theory - 1987.pdf] shows that continuous mathematics can run lisp interpreter.
From discrete logical units, people build Continuous Deep Learning Algorithms/Models.
The methods of Neuroscience couldn't reverse engineer a microprocessor which is even simpler that the brain. This means the tools or the methodology of neuroscience is not sophisticated enought to compeletely understand the brain. Could a Neuroscientist Understand a Microprocessor? - Eric Jonas, Konrad Paul Kording
References
- https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1005268 (Reverse Engineer the Brain) (Generalist AI beyond Deep Learning)
