The Monoculture of Machine Learning
Machine-learning scientists like catchy paper titles. Whether it is a confident Attention is All You Need, a rather philosophical Everything is Connected, or a Tolkienesque Tree! I am no Tree! I am a Low Dimensional Hyperbolic Embedding, there is always a lot of fun in conferences. One less fun aspect of this is a strong belief in a monoculture. There is only a single thread that is pursued. These days, it is all about transformers—and rightly so, because this architecture does seem to be highly useful.
Yet, in the quest for the philosophic mercury or the Holy Grail, the community is wont to ignore or even actively suppress other research directions. This is dangerous because we still lack an understanding of our monoculture—and in the real world, having only a single crop is not a long-term strategy to keep the soil fertile. I believe that machine learning would be better off with more diversity. Diversity in opinion, diversity in research agendas, and, not to put too fine a point on it, diversity in backgrounds. We need engineers and tamers of large models as much as we need theoreticians and bridge builders. Otherwise, the monoculture will eventually perish.
Here’s to more diversity, until next time!