natural-language-processing-3

Legal Reasoning

3260 papers • 126 benchmarks • 313 datasets

This task has no description! Would you like to contribute one?

(Image credit: Papersgraph)

Benchmarks

These leaderboards are used to track progress in legal-reasoning-3

Trend

Dataset

Best Model

Actions

LegalBench (Issue-spotting)

LegalBench (Rule-recall)

Libraries

i

Use these libraries to find legal-reasoning-3 models and implementations

hazyresearch/legalbench

3 papers 272

Datasets

No datasets available.

Subtasks

No subtasks available.

Most implemented papers

DISC-LawLLM: Fine-tuning Large Language Models for Intelligent Legal Services

Wei Chen, Shengbin Yue, Siyuan Wang, Bingxuan Li, Chenchen Shen, Shujun Liu, Yuxuan Zhou, Yao Xiao, Song Yun, Wei Lin, Xuanjing Huang, Zhongyu Wei•Tue Sep 19 2023

Quantitative and qualitative results on DISC-Law-Eval demonstrate the effectiveness of the system in serving various users across diverse legal scenarios, and enhances models' ability to access and utilize external legal knowledge.

136

Content

0

Paper Graph

Causality and Responsibility for Formal Verification and Beyond

Hana Chockler•Thu Aug 25 2016

The applications of causality to formal verification, namely, explanation of counterexamples, refinement of coverage metrics, and symbolic trajectory evaluation are described.

9 0

Paper Graph

Passing the Brazilian OAB Exam: Data Preparation and Some Experiments

P. Delfino, Bruno Cuconato, E. Haeusler, Alexandre Rademaker•Wed Dec 13 2017

The construction of a new data set is described and some preliminary experiments on it, treating the problem of finding the justification for the answers to questions as a baseline performance measure against which to evaluate future improvements.

16 0

Paper Graph

Designing normative theories for ethical and legal reasoning: LogiKEy framework, methodology, and tool support

Christoph Benzmüller, X. Parent, Leendert van der Torre•Sun Mar 24 2019

Off-the-shelf theorem provers and model finders for HOL are assisting the LogiKEy designer of ethical intelligent agents to flexibly experiment with underlying logics and their combinations, with ethico-legal domain theories, and with concrete examples---all at the same time.

54 0

Paper Graph

LegalBench: Prototyping a Collaborative Benchmark for Legal Reasoning

Neel Guha, Julian Nyarko, Christopher R'e, Daniel E. Ho•Mon Sep 12 2022

How IRAC-a framework legal scholars use to distinguish different types of legal reasoning-can guide the construction of a Foundation Model oriented benchmark is described and a seed set of 44 tasks built according to this framework is presented.

23 0

Paper Graph

LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models

Christopher Ré, Neel Guha, Daniel E. Ho, Julian Nyarko, D. Rockmore, Nils Holzenberger, Faiz Surani, Joel Niklaus, Peter Henderson, Adam Chilton, Aditya Narayana, Alex Chohlas-Wood, Austin M. K. Peters, Brandon Waldon, Diego A. Zambrano, Dmitry Talisman, E. Hoque, F. Fagan, Galit Sarfaty, Gregory M. Dickinson, Haggai Porat, Jason Hegland, Jessica Wu, Joe Nudell, John J. Nay, Jonathan H. Choi, K. Tobia, M. Hagan, Megan Ma, Michael A. Livermore, Nikon Rasumov-Rahe, Noam Kolt, Sean Rehaag, Sharad Goel, Shangsheng Gao, Spencer Williams, S. Gandhi, Tomer Zur, Varun J. Iyer, Zehua Li•Sat Aug 19 2023

To enable cross-disciplinary conversations about LLMs in the law, it is shown how popular legal frameworks for describing legal reasoning correspond to LegalBench tasks, thus giving lawyers and LLM developers a common vocabulary.

314 0

Paper Graph

A Comprehensive Evaluation of Large Language Models on Legal Judgment Prediction

Yixin Cao, Xiang Wang, Tat-Seng Chua, Ruihao Shui•Tue Oct 17 2023

Practical baseline solutions based on LLMs are designed and an intriguing paradox wherein an IR system surpasses the performance of LLM+IR due to limited gains acquired by weaker LLMs from powerful IR systems is presented.

48 0

Paper Graph

Modeling Legal Reasoning: LM Annotation at the Edge of Human Agreement

Rosamond Thalken, Edward H. Stiglitz, David M. Mimno, Matthew Wilkens•Thu Oct 26 2023

A highly complex task that is challenging even for humans: the classification of legal reasoning according to jurisprudential philosophy is considered, and it is found that generative models perform poorly when given instructions equal to the instructions presented to human annotators through the codebook.

21 0

Paper Graph

Can ChatGPT Perform Reasoning Using the IRAC Method in Analyzing Legal Scenarios Like a Lawyer?

Lay-Ki Soon, Terry Yue Zhuo, Lizhen Qu, Xiaoxi Kang, Adnan Trakic, Patrick Charles Emerton, Genevieve Grant•Sun Oct 22 2023

A novel corpus consisting of scenarios pertain to Contract Acts Malaysia and Australian Social Act for Dependent Child is constructed and the first empirical assessment of ChatGPT for IRAC analysis is conducted in order to understand how well it aligns with the analysis of legal professionals.

20 0

Paper Graph

TMID: A Comprehensive Real-world Dataset for Trademark Infringement Detection in E-Commerce

Tongxin Hu, Zhuang Li, Xin Jin, Lizhen Qu, Xin Zhang•Thu Dec 07 2023

This study introduces TMID, a novel dataset to detect trademark infringement in merchant registrations sourced directly from Alipay, one of the world's largest e-commerce and digital payment platforms, and offers a thorough collection of legal rules and merchant and trademark-related contextual information with annotations from legal experts.

2 0

Paper Graph

Adding a benchmark result helps the community track progress.