## Summary This post outlines my approach to academic paper reviewing, from initial screening and detailed reading to final assessment, with emphasis on constructive feedback. ## Initial Screening I reject reviews from journals on my blacklist (e.g., Epidemics due to profit model) without reading the abstract. For other papers, I read the abstract and immediately look for code and a preprint. If these aren't available, I typically reject the review, citing this reason. ## Review Process If I proceed with reviewing, I follow this structured approach: 1. **First Pass**: Read the first and last paragraphs of both the introduction and discussion. I create a structured document with the paper's section headers and add initial notes. 2. **Issue Collection**: Based on the first pass, I compile preliminary major and minor issues sections. 3. **Detailed Reading**: I then read the paper in full order: - Introduction - Methods - SI Methods - **Code Review** (this can be a scary process) - Results - SI Results - Discussion 4. After each section, I pause to update my major and minor issues list. 5. **Finalisation**: I review my compiled issues and write a brief summary. These components form my final review. If I encounter numerous significant problems during this process, I may stop and note this in my review summary. I generally aim for this process to take no more than 3 hours. ## Philosophy I try to focus on helping improve rather than blocking publication. If I sense that my feedback isn't helping or authors are resistant to changes (after a few rounds), I acknowledge this and move on. This can be tricky to be mindful of. I view reviewing as voluntary work—finding an approach that works for you is important, and you don't need to do it if you don't want to. ## Using Language Models in the Review Process I haven't integrated LLMs into my review workflow yet, but I'm exploring several approaches: 1. **Conversational Analysis**: Loading my structured notes and issues into an LLM's context to have an interactive conversation about the paper's strengths and weaknesses as I develop my review. 2. **Text Simplification**: I currently use LLMs to help break down verbose or complex text into bullet points. This is particularly helpful as someone with dyslexia, as it makes dense academic writing more accessible without changing the content. ## Applying This Approach to General Paper Reading I use a similar but more informal approach when reading papers for my own knowledge. This typically results in structured notes organised by paper section, capturing key points and potential issues as I read. These notes become a valuable resource for future reference and help distill complex information into more digestible formats. Typically I only do the first few steps with most papers I read and its an exciting time when I think its worth going the whole way to give something detailed read. ## Resources For general guidance, I recommend the "Ten Simple Rules for Reviewers" paper: https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.0020110 (I like these ten simple rules papers more generally) I sometimes make my paper notes public (though apparently not for a few years). Here is an example: [[Analysis of 2.1 million SARS-CoV-2 genomes identifies mutations associated with transmissibility]] There are a few more in [[Papers]] as well of varying completeness.