Any for form of discovery exercise can be phrased as an optimization problem (and therefore have an objective function). So, optimizing after objective functions is not inherently non-generalizing.

You misrepresented the whole field of ML based on an arbitrary definition of intelligence. What’s more, you seem to believe systematic generalization is somehow a fundamentally different approach to AI. Hint: systematic generalization refers to a learning algorithm’s ability to extrapolate learned behavior to unseen situations that are distinct but semantically similar to its training data.

Optimizing after an objective function is not inherently against the idea of generalization. It’s just that the optimization step is not the whole picture. An intelligent agent must also develop its own preferences to sample new data from the environment. It is true that backprop is not efficient for real time applications such as reinforcement learning (and hopefully in the future there will be more specialized optimization algorithms), but it is not fundamentally obstructive. What you can optimize with one algorithm, you can optimize with another, as long as both can navigate the same loss landscapes.

Hinton is clueless about the brain, I’m sorry to say. The cortex uses massive feedback pathways for both learning and top-down attention purposes. However, feedback in the brain does not propagate error signals for gradient learning a la DL. It generates success signals using a winner-take-all mechanism. The signals are used not to modify weights, but to strengthen synaptic connections until they become permanent. It’s called STDP, a form of Hebbian learning.

Furthermore, the brain does not optimize objective functions, a learning approach that is inherently and hopelessly non-generalizing. The brain discovers context, which is temporal at its core. Spike timing is essential to context. Thus generalized intelligence is context-bound. The work of Hinton and the rest of the DL community, while valuable to automation applications, is irrelevant to AGI. One man’s opinion, of course.

Feedforward is a simplified architecture assumed for the actual complex brain architecture which forms feedback (as opposed to feedforward) and loop structures.

In real-world, the precise details of the system is rarely know. BP won’t work if the activation function is not differentiable. The alternative Forward-forward algorithm proposed by Geoffrey Hinton, it seems at the moment, can be applied only to a small subset of problems as it has generalisation and scaling-up issues. Hopefully, it will get resolved soon.

By contrast, with either FF or DL, perception is impossible without recognition, i.e., without prior learned representations of the objects to be perceived. Automatic universal invariance is also non-existent. This is a fatal flaw if AGI is the goal.

Back-propagation is merely a symptom of a much bigger problem in mainstream AI: the notion that learning consists of optimizing an objective function. Function optimization is the opposite of generalization. The brain can perceive anything without optimization. I believe that AGI research should focus exclusively on systematic generalization. That’s where almost all the research money should go in my opinion.

In conclusion, I’m afraid that Geoffrey Hinton is just spinning his wheels. Same with the rest of the DL community. Fortunately, a few researchers are working on systematic generalization. The best times are still ahead of us.