Reading research papers is a fundamental task for any academic or professional researcher. A typical researcher will likely spend hundreds of hours every year reading papers. However, learning to efficiently read a paper is a critical but rarely taught skill.
Important
The ultimate goal when approaching a manuscript is to understand the scientific contributions the authors are making. This is not an easy task and may require going over the paper several times. To master this process, one must abandon the novice habit of reading sequentially from beginning to end. Instead, an expert employs a structured, multi-pass methodology combined with a rigorously critical mindset.
1. The Core Intellectual Mindset
Before dissecting the structure of a paper, a reader must adopt the correct psychological framework. Reading a research paper must be a critical process.
The Principle of Suspicion
You should not assume that the authors are always correct; instead, be suspicious. Critical reading involves asking appropriate questions: Are they solving the right problem? Are there simple solutions the authors do not seem to have considered?
Furthermore, you must evaluate the empirical rigor: did they gather the right data to substantiate their argument, and did they appear to gather it in the correct manner? Did they interpret the data in a reasonable manner?
However, tearing down a paper is easy; building upon it requires deeper intellect. Reading creatively involves harder, more positive thinking. You must ask: What are the good ideas in this paper, and do these ideas have other applications or extensions that the authors might not have thought of? Can they be generalized further?
2. The Three-Pass Methodology
The most efficient way to tackle complex literature is the “three-pass” approach. The key idea is that you should read the paper in up to three passes, instead of starting at the beginning and plowing your way to the end. Each pass accomplishes specific goals and builds upon the previous pass.
Pass 1: The Bird’s-Eye View
The first pass is a quick scan to get a bird’s-eye view of the paper. This pass should take about five to ten minutes.
Execution steps:
- Carefully read the title, abstract, and introduction.
- Read the section and sub-section headings, but ignore everything else.
- Read the conclusions.
- Glance over the references, mentally ticking off the ones you’ve already read.
The 5 Cs of the First Pass
At the end of the first pass, you should be able to answer the five Cs:
- Category: What type of paper is this (e.g., measurement, analysis of an existing system, research prototype)?
- Context: Which other papers is it related to, and which theoretical bases were used?
- Correctness: Do the assumptions appear to be valid?
- Contributions: What are the paper’s main contributions?
- Clarity: Is the paper well written?
Using this information, you may choose not to read further if the paper doesn’t interest you, or if the authors make invalid assumptions.
Pass 2: Grasping the Content
In the second pass, read the paper with greater care, but ignore details such as proofs. The second pass should take up to an hour. You may want to skip over all the equations the first time through the body of the paper.
Execution steps:
- Analyze Visuals: Look carefully at the figures, diagrams and other illustrations in the paper; pay special attention to graphs. Are the axes properly labeled, and are results shown with error bars so that conclusions are statistically significant?
- Track Context: Remember to mark relevant unread references for further reading. The references are very important when you are researching a topic, as they point you to related research.
Note
After this pass, you should be able to grasp the content of the paper and summarize its main thrust, with supporting evidence, to someone else. If the paper remains incomprehensible due to unfamiliar terminology, unproven assertions, or sheer fatigue, you may choose to set it aside, return to it later, or persevere to the third pass.
Pass 3: Virtual Re-implementation
To fully understand a paper, particularly if you are a reviewer, requires a third pass. The key to the third pass is to attempt to virtually re-implement the paper: that is, making the same assumptions as the authors, re-create the work. This pass can take about four or five hours for beginners, and about an hour for an experienced reader.
This pass requires great attention to detail, where you should identify and challenge every assumption in every statement. By comparing this re-creation with the actual paper, you can easily identify not only a paper’s innovations, but also its hidden failings and assumptions.
Note
At the end of this pass, you should be able to reconstruct the entire structure of the paper from memory.
3. Active Note-Taking and Synthesis
Reading must leave an intellectual paper trail. Many people cover the margins of their copies of papers with notes. If you have questions or criticisms, write them down so you do not forget them. You should underline key points the authors make and mark the data that is most important or that appears questionable.
The Distillation Test
After the first read-through, try to summarize the paper in one or two sentences. Almost all good research papers try to provide an answer to a specific question. If you can succinctly describe a paper, you have probably recognized the question the authors started with and the answer they provide.
Finally, to truly gauge the scientific merit, you must compare the paper to other works in the area. Scientific contributions can take on many forms: some papers offer new ideas, others implement ideas and show how they work, and others bring previous ideas together and unite them under a novel framework. Knowing other work in the area can help you to determine which sort of contribution a paper is actually making.
4. Scaling Up: Conducting a Literature Survey
When entering an unfamiliar field, paper reading skills are put to the test. To survey a new domain efficiently:
- Use an academic search engine such as Google Scholar or CiteSeer and some well-chosen keywords to find three to five recent papers in the area.
- Do one pass on each paper to get a sense of the work, then read their related work sections to find a pointer to a recent survey paper.
- If no survey exists, find shared citations and repeated author names in the bibliography to identify the key papers and researchers in that area.
- Go to the websites of the key researchers and see where they’ve published recently, which will help you identify the top conferences in that field.