Tags: academia, research, musings

« Previous post: Remembering My Grandfather — Next post: Why Write? »

When I started my Ph.D., I found writing papers to be a somewhat daunting task. With this guide, I want to offer a brief perspective on the most relevant aspects of writing.

Structure

The standard—one could almost say ‘canonical’—structure of a paper looks like this:

1. Abstract
2. Introduction
3. Background
4. Related Work
5. Methods
6. Experiments
7. Discussion

Let us look at all of these!

Caveat lector: most of what is about to follow should be treated as a guideline, laced with my personal preferences. Develop your own style and always feel free to deviate. Remember: writing should also be fun!

1. Abstract

The abstract provides a short, succinct, pithy summary of your work. While the length of abstracts can vary, I find 4–6 sentences to be a good length to aim for. This is short enough to read quickly while long enough to provide some details about your work.

The purpose of the abstract is to state what you are doing (or trying to do) and why this is relevant. When possible, add both qualitative and quantitative information about the solution (i.e. mention what method(s) you used, potentially outlining some of the gains and observations).

An abstract is great if it helps someone without prior exposure to your paper to at least briefly understand what is going on there and, moreover, makes that person want to read your paper. Think of it as a TL;DR for your work.

2. Introduction

The introduction sets the stage for all the players in your paper: it introduces the methods, and provides the backdrop for the problem you set out to solve. Take some time to motivate why this is a problem worth solving. Mention how you solve it, i.e. what contributions you make in the paper, and how you know that it is a good solution (mentioning some overall improvements in experiments, for instance).

Many ML papers use some kind of ‘Contributions’ list; my personal preference is to use a \paragraph for this and expand on the individual items via an inparaenum, i.e. an enumeration that is set within a paragraph of text.¹

In terms of length, I prefer to have the abstract and introduction together take up about 1 page of a single-column paper format or about 1 column of a double-column paper format. If you have more concepts or results to explain, aim for a longer introduction.

A very nice trend is to have an ‘Overview’ or ‘Hero’ figure, i.e. some explanatory diagram or pipeline of what is going on in your paper on either the first or the second page. This can make it much easier for readers to understand what you are doing in your paper.

3. Background

This section is all about laying down some notation as well as an explanation of most of the concepts. Ideally, even if I am not an expert in the precise subject matter of your paper, I should at least get a rough idea of what you are doing later on, what your methods are relying on, and—potentially—what your theorems might be. Think of this section as the foundation upon which your method is built.

There is no ideal length here, but I would aim for something that is at least shorter than your ‘Methods’ section.

4. Related Work

This section is all about discussing the work of others, and—crucially–contextualising it. It is not enough to just list as many papers as you can here, but you should also provide some details as to their relevance or their limitations. Unfortunately, this section is often written last, and it shows. Avoid phrasal templates of the form ‘Smith et al. do X. Miller et al. do Y’. Rather tell your readers why existing work is great but that does not solve your specific problem. A great ‘Related Work’ section helps readers a lot, in particular if they want to understand your specific research domain somewhat better.

The ‘Related Work’ section is often positioned before describing your own method, even though that creates some logical inconsistencies. Try to follow the conventions of your field here.

As for length, aim for a full column in a double-column format, or for about three quarters of a page in a single-column format. That should be plenty. Some papers manage to integrate this section into other sections; if you can pull it off—in particular if you are working in a new area—this can be very refreshing and make your paper stand out. Of course, you need to make sure that you give credit to those that came before you.

5. Methods

This is where you can have the most fun, and it is typically the section that is easiest to write. If you are not experienced in writing papers, this is the perfect section to start with! Describe your model, your algorithms, your theoretical insights. All jargon and technical terms that you use should be defined in the ‘Background’ section. If appropriate, you can conclude this section by a discussion of limitations of your method—it is clear that any method has limitations, so do not be shy in pointing them out.

Ideally, a skilled graduate student in your field should be able to build their own version of your method by just reading the ‘Methods’ section. Hence, make sure to describe any specific modifications that you did to standard methods (notice that specific pre-processing steps that pertain to your data should best be mentioned in another section, viz. the ‘Experiments’ section).

The ‘Methods’ section will be perused by expert readers and reviewers, so make sure that the descriptions are accessible, precise, and consistent. Get your colleagues involved, in particular those with more experience, to get an idea of whether the description in the section is sufficient to facilitate reproducibility.

The length of the ‘Methods’ section depends a lot on the type of paper that you are writing. With a mix of theory and experiments, you might have to use up to four full pages of content to describe algorithms and their properties. In more applied papers, the ‘Methods’ section is typically shorter.

6. Experiments

The bread and butter of machine learning research and many other disciplines, the ‘Experiments’ section is where you can describe what types of phenomena you can study with your new method, and what insights you obtained. Typically, this section should have a subsection or paragraph in which you detail the experimental setup. The setup includes any pre-processing steps and data sets you picked, as well as your choice of hyperparameters, data splits, and so on.

Afterwards, you should describe the individual experiments in detail. In machine learning papers, this commonly requires some tables or performance curves. Make sure to have the following aspects covered:

• A comparison with other methods (typically, if something is mentioned in the ‘Related Work’ section, you should compare to it).
• A description in terms of statistical measures of dispersion, such as standard deviations for performance plots or confidence intervals. In particular when comparing a large set of methods, you should all your inference tasks multiple times to ensure that you get an understanding of the performance.
• A set of ablation studies where you analyse the performance of individual parts of your model. The idea is to verify that each part of your method actually contributes something relevant to the overall performance; you do not want to find out later on that, say, a random projection would have worked just as well than your highly-trained neural network.

The experiments can be followed by individual discussions of the ramifications, or you can collect them in a single ‘Discussion’ section at the end (see below). If you have not already analysed the limitations of your method in the ‘Methods’ section, do it as a part of this section here.

Again, the length of the ‘Experiments’ section depends very much on the type of paper you are writing. I would suggest to have one full page as an absolute minimum for most machine learning papers, with some papers having sections that rival the ‘Methods’ section in length. This is probably one of these areas where the conference culture deviates the most from journals, since in journal papers—especially in the biomedical domain—most of the space is taken up by experiments and discussions, and methods are typically confined to supplementary materials.

7. Discussion

Much like a great rug, the ‘Discussion’ section ties your paper together. Summarise and conclude why the problem you solved is relevant, why your method is useful, and so on. Try to give a succinct summary and contextualisation of the observations you made before. It is also custom practice to discuss future work, so make sure to think about how your method could be extended. This is your opportunity to provide some new ideas for readers—use it!

This section is typically half a page or approximately a full column, but much shorter and pithy ‘Discussion’ sections are often seen in the wild.

General Advice and Anti-Patterns to Avoid

Having been writing and reviewing papers for some time, I like to believe that I have picked up on some anti-patterns or common mistakes. Again, these are geared towards a conference paper, but I will try to keep them as general as possible.

• If you do not know where to start writing, draft the ‘Methods’ section first, and keep the ‘Introduction’ and ‘Conclusion’ until the very last moment. Often, it is easier to describe the what and how, rather than the overall why.
• Do not make the introduction just ‘rehash’ what you wrote in the abstract.
• The introduction should avoid too much jargon or, if jargon needs to be used, provide a brief definition. Ideally, every reader should get a good idea of what you are about to accomplish in the paper.
• Make sure to credit previous contributions correctly. We are all standing on the shoulders of giants, and it is part of our scientific commitment to honesty to always acknowledge those who came before us.
• Do not overemphasise your contributions. Or, to put it more poetically, do not let your ego write cheques that your brain cannot cash. We are all proud of our methods and achievements—and rightly so—but we should also operate with a large degree of humility. A particular bad anti-pattern that I am seeing is what I like to call the ’neglected baseline:’ essentially, you train your baseline methods once, namely at the start of the project, then you never touch them again. By contrast, your own method is trained and re-trained all the time. Naturally, this will result in a large competitive advantage of your method, so make sure to train your baselines in the strongest fashion possible whenever you can!
• Make sure to have every section include at least something ‘intuitive’ to say to readers that might be lost in the details of your method. In particular when your paper is heavy on theory, it can be helpful to take a step back every once in a while and tell a reader why a certain result is important or what its ramifications are.
• Iterate, iterate, iterate. Nature abhors a vacuum, and human beings abhor an empty page. Just write something in each section and improve it later on. A good method can also be to describe every section with a set of paragraphs, which you then subsequently extend and improve.
• Be consistent with the tenses you use, and aim for preferring the present tense when referring to the work at hand.
• Omit needless words (but maybe not all of them, since the pacing of a text still matters). At the very least, avoid ‘weasel words’ such as ‘quite’ or ‘very’.²
• Invest in the writing early on. We do not have to write literary masterpieces, even though, as evidenced by books like The Oxford Book of Modern Science Writing, papers do not have to be dry and boring. However, it is my observation that our community tends to be a little bit lax when it comes to the writing part; this is problematic because a well-written paper will make it so much easier for your readers to understand and use your method.
• Be bold and compress your paper. In line with omitting needless words, you may also find that you can reformulate and simplify certain sections.
• Check your bibliography carefully. Always search for the right version of a paper. Google Scholar has a tendency to give you weirdly-formatted citations of arXiv paper. Do make an effort and find out whether a paper was actually published at a conference or in a journal and cite it appropriately.³

Pet Peeves

Here are some of my personal pet peeves. I cannot claim in good conscience that there a good reasons for any of them, but if we are co-authoring a paper, you can make me very happy by checking out this list.

• Prefer the active voice over the passive voice at all times.
• ‘On the other hand’ needs to be proceeded by ‘On the one hand.’ Consider using ‘By contrast’ instead.
• Use simple words. Why write ‘utilise’ when ‘use’ would do?
• Methodology often does not mean what you think it means. Admittedly, language is changing and all that jazz, but I cannot shake the feeling that people are using this to make their method sound smarter. ‘No, sir, this paper does not contains methods. It contains methodologies!’
• In general, it can pay off to write with a thesaurus or a dictionary if you are not a native speaker. Make sure that you use the right words, and resist the ‘false friends’ of your native language.
• Please, please, please learn how to properly add citations. If you are using natbib, you can use \citep for parenthetical citations, and \citet for in-text citations. For biblatex, the commands are \parentcite and \textcite, respectively. Do not use bracketed citations as nouns unless mandated by an editor. For instance, ‘This method is great for graphs [1].’ is acceptable, while ‘See [1] for more details’ is not. Why should you care? I believe that the semantic intent of a citation matters; moreover, if you follow this rule, you can easily swap citation styles for different journals and never have to worry about your citations looking weird!
• Always use the right type of quotes. It is easiest to just use the csquotes package and use the \enquote command. If you are quoting manually, use ``word'' or `word' to ensure that the quotes are typeset correctly.⁴

Final Words of Advice

All of this might seem daunting. Always ignore everything that makes no sense to you at a specific stage and focus on having fun. I really mean it—writing up your project should be a joyous experience! It is so rewarding to be able to share your view with a greater community. Keep on writing until you have found your own voice and your own style, and best of luck!