“Fuzzy pattern matching” as the low layer, the abstract logic reasoning as the high layer. Lifelong learning is essential here. o1 or o3 might perform program synthesis in natural language CoT or latent space.
The mathematical tradition in exact science has emphasized the idea of predicting the behavior of systems by doing things like solving mathematical equations. But what computational irreducibility implies is that out in the computational universe that often won’t work, and instead the only way forward is just to explicitly run a computation to simulate the behavior of the system. — https://writings.stephenwolfram.com/2017/05/a-new-kind-of-science-a-15-year-view/
Our computational systems operate far above theoretical energy minimums, with vast room for improvement before hitting fundamental physical constraints. Mathematical and quantum limits define the boundaries of what’s possible, but we are nowhere near approaching these theoretical limits. Landauer’s Limit: current computing systems operate approximately one billion times above the theoretical minimum energy required for computation. Godel’s Incompleteness: mathematical systems have inherent limitations that prevent complete knowledge of all mathematical truths. Turing’s halting problem: certain classes of algorithmic problems are fundamentally unsolvable through computation. Quantum uncertainty: the universe contains inherently unpredictable elements that cannot be captured through discrete mathematics. Irreducible complexity: some problems lack elegant mathematical solutions and must retrain their full complexity. https://www.youtube.com/watch?v=pNKjfTDg1gE
For if meaningful general predictions are to be possible, it must at some level be the case that the system making the predictions be able to outrun the system it is trying to predict. — A New Kind of Science, Page-741
While this is exciting for now, and will be for the foreseeable future, the big question I have is whether or not there will be a sigmoid curve of intelligence or continued exponentials. Surely there must be theoretical limits to intelligence, or at least useful intelligence? Turing’s Halting Problem and Gödel’s Incompleteness Theorem combined with problems like quantum uncertainty and irreducible complexity certainly seem to indicate that there are ceilings of accurate, useful intelligence. Beyond a certain point “theory will only get you so far.” — https://x.com/DaveShapi/status/1885288880259154319
Evolution of robots is bounded by physical world, while software is only bounded by compute power. However, software is bounded by its own complexity to some extent.
Are Biological Systems More Intelligent Than Artificial Intelligence?