natural-language-processing-5

Method name prediction

3260 papers • 126 benchmarks • 313 datasets

This task has no description! Would you like to contribute one?

(Image credit: Papersgraph)

Benchmarks

These leaderboards are used to track progress in method-name-prediction-10

Trend

Dataset

Best Model

Actions

CodeSearchNet

Libraries

i

Use these libraries to find method-name-prediction-10 models and implementations

mdrafiqulrabin/tnpa-generalizability

3 papers 31

Datasets

CodeSearchNet

Subtasks

No subtasks available.

Most implemented papers

Testing Neural Program Analyzers

Mohammad Amin Alipour, Md Rafiqul Islam Rabin, Ke Wang•Sat Aug 24 2019

In a preliminary experiment on a neural model recently proposed in the literature, it is found that the model is very brittle, and simple perturbations in the input can cause the model to make mistakes in its prediction.

19

Content

mdrafiqulrabin/SIVAND

2 papers 10

0

Paper Graph

Embedding Java Classes with code2vec: Improvements from Variable Obfuscation

E. Frank, Rhys Compton, Panos Patros, Abigail M. Y. Koay•Sun Apr 05 2020

The effect of obfuscating variable names during training of a code2vec model is investigated to force it to rely on the structure of the code rather than specific names and a simple approach to creating class-level embeddings by aggregating sets of method embeddeddings is considered.

68 0

Paper Graph

Understanding neural code intelligence through program simplification

Mohammad Amin Alipour, Md Rafiqul Islam Rabin, Vincent J. Hellendoorn•Sun Jun 06 2021

The proposed approach, Sivand, uses simplification techniques that reduce the size of input programs of a CI model while preserving the predictions of the model, and is broadly applicable across many model architectures and problem domains.

69 0

Paper Graph

Memorization and Generalization in Neural Code Intelligence Models

Mohammad Amin Alipour, Md Rafiqul Islam Rabin, Aftab Hussain, Vincent J. Hellendoorn•Tue Jun 15 2021

This work evaluates the memorization and generalization tendencies in neural code intelligence models through a case study across several benchmarks and model families by leveraging established approaches from other fields that use DNNs, such as introducing targeted noise into the training dataset.

48 0

Paper Graph

Extracting Label-specific Key Input Features for Neural Code Intelligence Models

Md Rafiqul Islam Rabin•Sun Feb 13 2022

The code intelligence (CI) models are often black-box and do not offer any insights on the input features that they learn for making correct predictions. This opacity may lead to distrust in their prediction and hamper their wider adoption in safety-critical applications. In recent, the program reduction technique is widely being used to identify key input features in order to explain the prediction of CI models. The approach removes irrelevant parts from an input program and keeps the minimal snippets that a CI model needs to maintain its prediction. However, the state-of-the-art approaches mainly use a syntax-unaware program reduction technique that does not follow the syntax of programs, which adds significant overhead to the reduction of input programs and explainability of models. In this paper, we apply a syntax-guided program reduction technique that follows the syntax of input programs during reduction. Our experiments on multiple models across different types of input programs show that the syntax-guided program reduction technique significantly outperforms the syntax-unaware program reduction technique in reducing the size of input programs. Extracting key input features from reduced programs reveals that the syntax-guided reduced programs contain more label-specific key input features and are more vulnerable to adversarial transformation when renaming the key tokens in programs. These label-specific key input features may help to understand the reasoning of models' prediction from different perspectives and increase the trustworthiness to correct classification given by CI models.

0 0

Paper Graph

Syntax-guided program reduction for understanding neural code intelligence models

Mohammad Amin Alipour, Md Rafiqul Islam Rabin, Aftab Hussain•Fri May 27 2022

A syntax-guided program reduction technique that considers the grammar of the input programs during reduction that is faster and provides smaller sets of key tokens in reduced programs is applied.

24 0

Paper Graph

Assessing Project-Level Fine-Tuning of ML4SE Models

Egor Bogomolov, T. Bryksin, Sergey Zhuravlev, Egor Spirin•Mon Jun 06 2022

This work develops a framework to assess quality improvements that models can get after fine-tuning for the method name prediction task on a particular project, and shows that per-project fine- tuning can greatly improve the models' quality as they capture the project's domain and naming conventions.

8 0

Paper Graph

Evaluation of Generalizability of Neural Program Analyzers under Semantic-Preserving Transformations

Mohammad Amin Alipour, Md Rafiqul Islam Rabin•Tue Apr 14 2020

A large-scale evaluation of the generalizability of two popular neural program analyzers using seven semantically-equivalent transformations of programs to provide the initial stepping stones for quantifying robustness in neural program Analyzers.

20 0

Paper Graph

Contrastive Code Representation Learning

P. Abbeel, Joseph E. Gonzalez, Ion Stoica, Tianjun Zhang, Ajay Jain, Paras Jain•Wed Jul 08 2020

Contracode is proposed: a contrastive pre-training task that learns code functionality, not form, and improves summarization and TypeScript type inference accuracy by 2 to 13 percentage points over competitive baselines.

172 0

Paper Graph

On the generalizability of Neural Program Models with respect to semantic-preserving program transformations

Mohammad Amin Alipour, Lingxiao Jiang, Md Rafiqul Islam Rabin, Ke Wang, Nghi D. Q. Bui, Yijun Yu•Thu Jul 30 2020

The results show that even with small semantically preserving changes to the programs, these neural program models often fail to generalize their performance, and suggest that Neural program models based on data and control dependencies in programs generalize better than neural program model based only on abstract syntax trees (ASTs).

105 0

Paper Graph

Adding a benchmark result helps the community track progress.