Long-Context Understanding

GPT-4 Technical Report

S. McKinney, Todor Markov, Jacob Menick, I. Sutskever, N. Keskar, Jan Leike, Alec Radford, Miles Brundage, Heewoo Jun, Christopher Berner, Greg Brockman, Brooke Chan, David Farhi, Scott Gray, J. Pachocki, Michael Petrov, Henrique Pondé de Oliveira Pinto, Szymon Sidor, John Schulman, Wojciech Zaremba, Irwan Bello, Chris Hallacy, Sandhini Agarwal, Girish Sastry, Pamela Mishkin, Gretchen Krueger, Barret Zoph, Christine McLeavey, Lama Ahmad, Sam Altman, Tarun Gogineni, Johannes Heidecke, A. Paino, Ashley Pantuliano, Giambattista Parascandolo, Michelle Pokrass, Elizabeth Proehl, Bob Rotsted, P. Tillet, Hannah Wong, Shengjia Zhao, Adam Perelman, Aditya Ramesh, Akila Welihinda, Gabriel Bernadett-Shapiro, Leo Gao, Ashvin Nair, David Dohan, Chelsea Voss, Shibani Santurkar, Brandon Houghton, Arvind Neelakantan, Mark Chen, Jerry Tworek, Mikhail Pavlov, Alethea Power, Mo Bavarian, Clemens Winter, F. Such, Igor Babuschkin, S. Balaji, Evan Morikawa, Mira Murati, Peter Welinder, Bob McGrew, Reiichiro Nakano, Pranav Shyam, Eric Sigler, Ma-teusz Litwin, Benjamin Chess, J. Kiros, OpenAI Josh Achiam, Steven Adler, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Shyamal Anadkat, Red Avila, Valerie Balcom, Paul Baltescu, Haim-ing Bao, J. Belgum, Jake Berdine, Lenny Bogdonoff, Oleg Boiko, Made-laine Boyd, Anna-Luisa Brakman, Tim Brooks, Kevin Button, Trevor Cai, Rosie Campbell, Andrew Cann, Brittany Carey, Chelsea Carlson, Rory Carmichael, Che Chang, Fotis Chantzis, Derek Chen, Sully Chen, Ruby Chen, Jason Chen, Chester Cho, Casey Chu, Hyung Won Chung, Dave Cummings, Jeremiah Currier, Yunxing Dai, Cory Decareaux, Thomas Degry, Noah Deutsch, Damien Deville, Arka Dhar, Steve Dowling, Sheila Dunning, Adrien Ecoffet, Atty Eleti, Tyna Eloundou, L. Fedus, Niko Felix, Sim'on Posada Fishman, Juston Forte, Is-abella Fulford, Elie Georges, C. Gibson, Vik Goel, Gabriel Goh, Raphael Gontijo-Lopes, Jonathan Gordon, Morgan Grafstein, Ryan Greene, Joshua Gross, S. Gu, Yufei Guo, Jesse Han, Jeff Harris, Yuchen He, Mike Heaton, Chris Hesse, Alan Hickey, W. Hickey, Peter Hoeschele, Kenny Hsu, Shengli Hu, Xin Hu, Joost Huizinga, Shantanu Jain, Shawn Jain, Joanne Jang, Angela Jiang, Roger Jiang, Haozhun Jin, Denny Jin, Shino Jomoto, B. Jonn, Tomer Kaftan, Lukasz Kaiser, Ali Kamali, I. Kanitscheider, Tabarak Khan, Logan Kilpatrick, Jong Wook Kim, Christina Kim, Yongjik Kim, Hendrik Kirchner, Matthew Knight, Daniel Kokotajlo, Lukasz Kondraciuk, Andrew Kondrich, Aris Konstantinidis, Kyle Kosic, Vishal Kuo, Michael Lampe, Ikai Lan, Teddy Lee, Jade Leung, Daniel Levy, Chak Li, Rachel Lim, Molly Lin, Stephanie Lin, Theresa Lopez, Ryan Lowe, Patricia Lue, A. Makanju, Kim Malfacini, Sam Manning, Yaniv Markovski, Bianca Martin, Katie Mayer, Andrew Mayne, Paul McMillan, Jake McNeil, David Medina, Aalok Mehta, Luke Metz, An-drey Mishchenko, Vinnie Monaco, Daniel P. Mossing, Tong Mu, O. Murk, David M'ely, Rajeev Nayak, Richard Ngo, Hyeonwoo Noh, Ouyang Long, Cullen O'Keefe, Joe Palermo, J. Parish, Emy Parparita, Alexandre Passos, Andrew Peng, Filipe de Avila Belbute Peres, Michael Pokorny, Vitchyr H. Pong, Tolly Powell, Boris Power, Raul Puri, Jack W. Rae, Cameron Raymond, Francis Real, Kendra Rimbach, Carl Ross, Henri Roussez, N. Ryder, M. Saltarelli, Ted Sanders, Heather Schmidt, David Schnurr, Daniel Selsam, Kyla Sheppard, Toki Sherbakov, Jessica Shieh, Sarah Shoker, Maddie Simens, Jordan Sitkin, Katarina Slama, Ian Sohl, Benjamin Sokolowsky, Yang Song, Natalie Staudacher, Natalie Summers, Jie Tang, N. Tezak, Madeleine Thompson, Amin Tootoonchian, Elizabeth Tseng, Preston Tuggle, Nick Turley, Juan Felipe Cer'on Uribe, Andrea Vallone, Arun Vijayvergiya, Carroll L. Wainwright, Justin Jay Wang, Alvin Wang, Ben Wang, Jonathan Ward, Jason Wei, CJ Weinmann, Jiayi Weng, Lilian Weng, Matt Wiethoff, Dave Willner, Samuel Wolrich, Lauren Workman, Sherwin Wu, Jeff Wu, Michael Wu, Kai Xiao, Tao Xu, Sarah Yoo, Kevin Yu, Qim-ing Yuan, Rowan Zellers, Chong Zhang, Marvin Zhang, Tianhao Zheng, Juntang Zhuang, William Zhuk•Tue Mar 14 2023

GPT-4, a large-scale, multimodal model which can accept image and text inputs and produce text outputs, is developed, a Transformer-based model pre-trained to predict the next token in a document which exhibits human-level performance on various professional and academic benchmarks.

21639 0

Paper Graph

Long-Context Understanding | State-of-the-Art

Benchmarks

Libraries

Datasets

Subtasks

Most implemented papers

GLM-130B: An Open Bilingual Pre-trained Model

Content

GPT-4 Technical Report

Judging LLM-as-a-judge with MT-Bench and Chatbot Arena

FABLES: Evaluating faithfulness and content selection in book-length summarization

LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding

LooGLE: Can Long-Context Language Models Understand Long Contexts?

Long-context LLMs Struggle with Long In-context Learning

InternLM2 Technical Report

Ada-LEval: Evaluating long-context LLMs with length-adaptable benchmarks