In current ML systems, where is the main bottleneck: dataset quality or model architecture improvements? [D]

A lot of recent progress in ML appears to come from scaling existing architectures rather than introducing fundamentally new ones.

At the same time, there’s increasing emphasis on dataset quality, curation, and synthetic data pipelines.

In practice, I’m trying to understand how this tradeoff looks in real systems:

How much effort is typically spent on data cleaning and filtering vs model design??

Whether dataset quality improvements still yield larger gains compared to architectural changes??

How synthetic data is affecting training stability and generalization in practice??

In many applied settings, it seems like data constraints become the limiting factor before architecture does, but I’m not sure if that’s broadly true across domains.