Passing the Brazilian OAB Exam: Data Preparation and Some Experiments
Published in
International Conference on Legal Knowledge and...(2017)External Links:
TL
TL;DR
The construction of a new data set is described and some preliminary experiments on it, treating the problem of finding the justification for the answers to questions as a baseline performance measure against which to evaluate future improvements.
Abstract
In Brazil, all legal professionals must demonstrate their knowledge of the law and its application by passing the OAB exams, the national bar exams. The OAB exams therefore provide an excellent benchmark for the performance of legal information systems since passing the exam would arguably signal that the system has acquired capacity of legal reasoning comparable to that of a human lawyer. This article describes the construction of a new data set and some preliminary experiments on it, treating the problem of finding the justification for the answers to questions. The results provide a baseline performance measure against which to evaluate future improvements. We discuss the reasons to the poor performance and propose next steps.
References19 items
1
Passing a USA National Bar Exam: a First Corpus for Experimentation
2
An Intuitionisticaly based Description Logic
3
OpenWordNet-PT: An Open Brazilian Wordnet for Reasoning
4
Intuitionistic Logic and Legal Ontologies
5