A Method(ology) to the Madness

Tags: musings, research

« Previous post: Machine Learning Needs a Langlands … — Next post: Getting started with a Ph.D. »

Words mean things and as writers¹, we should make sure that they mean the right things. Lewis Carroll expressed this marvellously:

“When I use a word,” Humpty Dumpty said, in rather a scornful tone, “it means just what I choose it to mean—neither more nor less.” “The question is,” said Alice, “whether you can make words mean so many different things.”

One of the words that seems to mean too many things is the word methodology. Many research papers in machine learnings devote an entire section to their methodology, so surely, the term must be important. Most authors use the term to refer to the collection of their methods, so you find phrases² such as ‘Our methodology uses X’ or ‘Our framework has $n$ methodological components’. In fact, this is wrong, at least considering the original meaning of the term:

Methodology is the systematic, theoretical analysis of the methods applied to a field of study. It comprises the theoretical analysis of the body of methods and principles associated with a branch of knowledge. Typically, it encompasses concepts such as paradigm, theoretical model, phases and quantitative or qualitative techniques.

Most research papers, however, do not perform such a theoretical analysis. In fact, most research papers primarily describe a new method, a set of methods, a framework, or something similar³. This is not the same as a methodology! So, in which context would the use of the word be appropriate, then? Suppose you are writing a paper that discusses the proper use of algorithms for machine learning—when to use them, how to interpret their results, and so on. In this case, you would be writing a methodology paper. To put it differently: if you are describing and analysing how to do research in your field, you are probably working on a methodological paper. In all other cases, do not use methodology.

Of course, as language changes, some pundits might not see the harm in adding this additional meaning to the word. I object to this for the reason that the purpose of scientific writing should be striving for clarity, first and foremost. If you use methodology as a more ‘mature’ or ‘complex’ term than method, you are making your writing harder to understand; in particular for non-native speakers. It is particularly jarring for me to see reviewers commenting on a ‘problem with the methodological contributions’, and admittedly, I often feel the urge to write back and say that the paper indeed does not have any of these contributions, so there could not possibly be any problems with them. So far, I refrained from such childish behaviour, and I would not suggest that complaining about this becomes the new trend. However, if you do write a methodology paper, forget all about this post and go ahead—I think machine learning as a discipline might need such papers to grow and mature.

In any case: happy writing, until next time!