2023

Time is Encoded in the Weights of Finetuned Language Models
_{Kai Nylund, Suchin Gururangan, Noah A. Smith}
_{in submission // [paper] [code]}

SILO Language Models: Isolating Legal Risk in a Nonparametric Datastore
_{Sewon Min^*, Suchin Gururangan^*, Eric Wallace, Hannaneh Hajishirzi, Noah A. Smith, Luke Zettlemoyer}
_{in submission // [paper] [code]}
_{^*Equal contribution}

Scaling Expert Language Models with Unsupervised Domain Discovery
_{Suchin Gururangan^*, Margaret Li^*, Mike Lewis, Weijia Shi, Tim Althoff, Noah A. Smith, Luke Zettlemoyer}
_{in submission // [paper] [code]}
_{^*Equal contribution}

Editing Models with Task Arithmetic
_{Gabriel Ilharco, Marco Tulio Riberio, Mitchell Wortsman, Suchin Gururangan, Ludwig Schmidt, Hannaneh Hajishirzi, Ali Farhadi}
_{ICLR 2023 // [paper] [code]}

2022

lo-fi: distributed fine-tuning without communication
_{Mitchell Wortsman, Suchin Gururangan, Shen Li, Ali Farhadi, Ludwig Schmidt, Michael Rabbat, Ari S. Morcos}
_{TMLR // [paper]}

M2D2: A Massively Multi-Domain Language Modeling Dataset
_{Machel Reid, Victor Zhong, Suchin Gururangan, Luke Zettlemoyer}
_{EMNLP 2022 // [paper] [code]}

Whose Language Counts as High Quality? Measuring Language Ideologies in Text Data Selection
_{Suchin Gururangan, Dallas Card, Sarah K. Dreier, Emily K. Gade, Leroy Wang, Blarry Wang,Luke Zettlemoyer, and Noah A. Smith}
_{EMNLP 2022 // [paper] [code]}

Nearest Neighbor Zero-Shot Inference
_{Weijia Shi, Julian Michael, Suchin Gururangan, and Luke Zettlemoyer}
_{EMNLP 2022 // [paper] [code]}

Branch-Train-Merge: Embarrassingly Parallel Training of Expert Language Models
_{Margaret Li^*, Suchin Gururangan^*, Tim Dettmers, Mike Lewis, Noah A. Smith, and Luke Zettlemoyer}
_{in submission // [paper] [code]}
_{^*Equal contribution}

Time Waits for No One! Analysis and Challenges of Temporal Misalignment
_{Kelvin Luu, Daniel Khashabi, Suchin Gururangan, Karishma Mandyam, and Noah A. Smith}
_{NAACL 2022 // [paper] [code]}

DEMix Layers: Disentangling Domains for Modular Language Modeling
_{Suchin Gururangan, Mike Lewis, Ari Holtzman, Noah A. Smith, and Luke Zettlemoyer}
_{NAACL 2022 // [paper] [model code] [data code]}

2021

All That’s ‘Human’ Is Not Gold: Evaluating Human Evaluation of Generated Text
_{Elizabeth Clark, Tal August, Sofia Serrano, Nikita Haduong, Suchin Gururangan, and Noah A. Smith}
_{ACL 2021 // [paper]}
_{🔥 Outstanding Paper Award 🔥}

Expected Validation Performance and Estimation of a Random Variable’s Maximum
_{Jesse Dodge, Suchin Gururangan, Roy Schwartz, Dallas Card, and Noah A. Smith}
_{EMNLP Findings 2021 // [paper]}

Detoxifying Language Models Risks Marginalizing Minority Voices
_{Albert Xu, Eshaan Pathak, Eric Wallace, Suchin Gururangan, Maarten Sap, and Dan Klein}
_{NAACL 2021 // [paper]}

2020

RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models
_{Sam Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A. Smith}
_{EMNLP Findings 2020 // [paper] [code] [demo]}
_{Press: [Wired] [IEEE] [GeekWire][Nature]}

Don’t Stop Pretraining: Adapt Language Models to Domains and Tasks
_{Suchin Gururangan, Ana Marasović, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A. Smith}
_{ACL 2020 // [paper] [code]}
_{🔥 Honorable Mention for Best Overall Paper 🔥}

2019

Variational Pretraining for Semi-supervised Text Classification
_{Suchin Gururangan,Tam Dang, Dallas Card, and Noah A. Smith}
_{ACL 2019 // [paper] [code]}

Show Your Work: Improved Reporting of Experimental Results
_{Jesse Dodge, Suchin Gururangan, Roy Schwartz, Dallas Card, and Noah A. Smith}
_{EMNLP 2019 // [paper] [code]}
_{Press: [Wired]}
_{Basis for the Reproducibility Checklist of major NLP conferences}

Emergent coordination underlying learning to reach to grasp with a brain-machine interface
_{with many authors 🙂}
_{Journal of Neurophys 2019 // [paper]}

2018

Annotation Artifacts in Natural Language Inference Data
_{Suchin Gururangan^*, Swabha Swayamdipta^*, Omer Levy, Roy Schwartz, Samuel Bowman, and Noah A. Smith}
_{NAACL 2018 // [paper]}
_{^*Equal contribution}

2014

Analysis of Graph Invariants in Functional Neocortical Circuitry Reveals Generalized Features Common to Three Areas of Sensory Cortex
_{Suchin Gururangan, Alex Sadovsky and Jason Maclean}
_{PLOS Comp Bio 2014 // [paper]}