Localized Neural learning vs backpropagation [D]

I’ve recently started exploring machine learning by building neural networks from scratch and studying the underlying mathematics. I come from a physics and mathematics background, and as I’ve become more familiar with backpropagation, I’ve grown increasingly interested in alternative learning mechanisms, particularly predictive coding, Hebbian learning, and Oja’s rule.
What fascinates me is the possibility of networks learning through locally available information rather than relying on gradients propagated from a global loss function. I’m aware that backpropagation remains the dominant approach for training deep neural networks, and I’m trying to understand whether these alternative methods offer meaningful practical advantages or are primarily interesting from a theoretical and biological perspective.
There are a few questions I’ve been thinking about:
1. What are the actual advantages of local learning over backpropagation?
I understand the conceptual differences, but I’m struggling to identify situations where local weight updates would meaningfully outperform conventional gradient-based optimization. Are there particular computational constraints, architectures, or environments where locality offers an advantage in efficiency, adaptability, or scalability?
2. Could local learning be particularly useful for unsupervised learning and discovering latent structure?
This is probably what interests me most. If a network updates its weights based on local correlations or prediction errors, could it develop representations that differ meaningfully from those learned through conventional backpropagation?
I understand that backpropagation is an optimization method rather than something that inherently limits which patterns a network can discover. However, I’m curious whether the inductive biases of certain local learning rules might make them better suited to identifying particular types of structure in high-dimensional, unlabeled data.
For example, I’m involved in experimental biophysics research and have been thinking about molecular dynamics, where we might want to identify hidden collective variables, conformational states, or relationships between molecular interactions. Could local learning provide advantages in discovering these representations, or would the choice of objective function and architecture matter far more than the learning rule itself?
3. What about continual or online learning?
Could locally updating networks be particularly useful in environments where data arrives continuously and the underlying distribution changes over time? I’m curious whether these approaches offer any inherent advantages in adapting to new information while preserving previously learned representations, especially compared with online backpropagation.
4. How mature is research into alternatives to backpropagation?
Have approaches like predictive coding, Hebbian learning, equilibrium propagation, and other biologically inspired methods been extensively investigated and generally found less effective, or are there still meaningful open problems where they could be competitive?
I’m also aware that certain predictive coding formulations can approximate backpropagation, which makes me wonder how fundamentally different some of these approaches actually are.
I’m relatively new to this field, so I’m not approaching this with the assumption that biologically inspired methods are inherently superior, or that backpropagation needs to be replaced. I’m more interested in understanding whether alternative learning rules could be particularly well suited to specific problems, especially those involving unsupervised representation learning, complex dynamical systems, or constrained computational environments.
I’d really appreciate any perspectives from people working in this area, particularly empirical comparisons, limitations I might be overlooking, or papers and research directions worth exploring.