This is our ARTS test set, an enriched test set to probe Aspect Robustness of ABSA.

Last update: Nov 16, 2022

Related tags

Overview

This is the repository for our 2020 paper "Tasty Burgers, Soggy Fries: Probing Aspect Robustness in Aspect-Based Sentiment Analysis".

Data

We provide a Aspect Robustness Probing test set for SemEval 2014 Aspect-Based Sentiment Analysis (ABSA).

Our new enriched test sets are at data/arts_testset
Our AspectSet mentioned in the paper Section 2.3 (Table 4) is provided in data/aspectset

Data Generation Process

We generate our new probing test set by three automatic strategies:

RevTgt (sentence with a red background): Reverse the sentiment of the Target aspect.
RevNon (sentence with a green background): Reverse the sentiment of the Non-target aspect.
AddDiff (sentence with a blue background): Add new aspects with Different sentiment.

Aspect Probing Results

We probed nine ABSA models (as mentioned in our paper).

Their outputs on SemEval 2014 are in the output folder.

How to Use Our Code

If you have a new ABSA dataset, you can run our code to generate you own aspect robustness probing test set.

python code/main.py -dataset_name laptop

More Questions

If you have more questions, please feel free to submit a GitHub issue.

You might also like...

codes for paper Combining Dynamic Local Context Focus and Dependency Cluster Attention for Aspect-level sentiment classification

DLCF-DCA codes for paper Combining Dynamic Local Context Focus and Dependency Cluster Attention for Aspect-level sentiment classification. submitted t

15 Aug 30, 2022

MemStream: Memory-Based Anomaly Detection in Multi-Aspect Streams with Concept Drift

MemStream Implementation of MemStream: Memory-Based Anomaly Detection in Multi-Aspect Streams with Concept Drift . Siddharth Bhatia, Arjit Jain, Shivi

61 Dec 2, 2022

Code and dataset for ACL2018 paper "Exploiting Document Knowledge for Aspect-level Sentiment Classification"

Aspect-level Sentiment Classification Code and dataset for ACL2018 [paper] ‘‘Exploiting Document Knowledge for Aspect-level Sentiment Classification’’

146 Nov 29, 2022

This project is a re-implementation of MASTER: Multi-Aspect Non-local Network for Scene Text Recognition by MMOCR

This project is a re-implementation of MASTER: Multi-Aspect Non-local Network for Scene Text Recognition by MMOCR，which is an open-source toolbox based on PyTorch. The overall architecture will be shown below.

82 Nov 17, 2022

This is the code for ACL2021 paper A Unified Generative Framework for Aspect-Based Sentiment Analysis

This is the code for ACL2021 paper A Unified Generative Framework for Aspect-Based Sentiment Analysis Install the package in the requirements.txt, the

108 Dec 23, 2022

Fuzzing JavaScript Engines with Aspect-preserving Mutation

DIE Repository for "Fuzzing JavaScript Engines with Aspect-preserving Mutation" (in S&P'20). You can check the paper for technical details. Environmen

190 Dec 11, 2022

Enhancing Aspect-Based Sentiment Analysis with Supervised Contrastive Learning.

Enhancing Aspect-Based Sentiment Analysis with Supervised Contrastive Learning. Enhancing Aspect-Based Sentiment Analysis with Supervised Contrastive

7 Dec 16, 2021

A sample pytorch Implementation of ACL 2021 research paper "Learning Span-Level Interactions for Aspect Sentiment Triplet Extraction".

Span-ASTE-Pytorch This repository is a pytorch version that implements Ali's ACL 2021 research paper Learning Span-Level Interactions for Aspect Senti

10 Dec 6, 2022

PyTorch implementation of our Adam-NSCL algorithm from our CVPR2021 (oral) paper "Training Networks in Null Space for Continual Learning"

Adam-NSCL This is a PyTorch implementation of Adam-NSCL algorithm for continual learning from our CVPR2021 (oral) paper: Title: Training Networks in N

34 Dec 21, 2022

Comments

The version of allennlp?

Recently I am reading your article and trying to run your code, but I always encounter all kinds of problems with allennlp. What is the version of allennlp?

opened by BD-MF 4
How to flip the polarities for those opinion words with original 'neutral' labels?

With respect to the opinion polarity modification, I find that in the article you placed most emphasis on how to operate on those opinion words with 'negative' or 'positive' labels. But I do notice in your dataset that the 'neutral' polarities can be flipped into other labels. And this part is not mentioned in the paper.

If using the same method, e.g., looking up WordNet for an antonym as its replacement, how to determine the sentiment polarity for this antonym when its origin is a 'neutral' term?

So my technical question is: How did you flip the polarities for those opinion words with original 'neutral' labels?

opened by scofield7419 3
Augmented training/developing sets available?

Hi,

Solid and fantastic work!

You presented the augmented test set. I am wondering if you constructed the corresponding training/developing sets as well? If yes, will they be available?

opened by scofield7419 2
Question about the tradeoff between robustness with generalization in ABSA task

Great work and solid experiment 🎉 In the paper table 10b, column 'Adv→N' shows that adversarial training could improve the model performance in the ARTS test dataset. My question is have you test the effect of adversarial training for the original test dataset e.g 'Adv→Ori‘. So that we could see after training on hard examples, how the generalization gap changes. And whether there is a tradeoff between robustness with generalization in ABSA task.

opened by Opdoop 2

This is our ARTS test set, an enriched test set to probe Aspect Robustness of ABSA.

Related tags

Overview

Data

Data Generation Process

Aspect Probing Results

How to Use Our Code

More Questions

You might also like...

codes for paper Combining Dynamic Local Context Focus and Dependency Cluster Attention for Aspect-level sentiment classification

MemStream: Memory-Based Anomaly Detection in Multi-Aspect Streams with Concept Drift

Code and dataset for ACL2018 paper "Exploiting Document Knowledge for Aspect-level Sentiment Classification"

This project is a re-implementation of MASTER: Multi-Aspect Non-local Network for Scene Text Recognition by MMOCR

This is the code for ACL2021 paper A Unified Generative Framework for Aspect-Based Sentiment Analysis

Fuzzing JavaScript Engines with Aspect-preserving Mutation

Enhancing Aspect-Based Sentiment Analysis with Supervised Contrastive Learning.

A sample pytorch Implementation of ACL 2021 research paper "Learning Span-Level Interactions for Aspect Sentiment Triplet Extraction".

PyTorch implementation of our Adam-NSCL algorithm from our CVPR2021 (oral) paper "Training Networks in Null Space for Continual Learning"

Comments

The version of allennlp?

How to flip the polarities for those opinion words with original 'neutral' labels?

Augmented training/developing sets available?

Question about the tradeoff between robustness with generalization in ABSA task

Owner

Automatically download the cwru data set, and then divide it into training data set and test data set

Source Code for our paper: Understand me, if you refer to Aspect Knowledge: Knowledge-aware Gated Recurrent Memory Network

Code for our paper Aspect Sentiment Quad Prediction as Paraphrase Generation in EMNLP 2021.

Code implementation of "Sparsity Probe: Analysis tool for Deep Learning Models"

Python implementation of Lightning-rod Agent, the Stack4Things board-side probe

Attention Probe: Vision Transformer Distillation in the Wild

Pytorch implementation of ICASSP 2022 paper Attention Probe: Vision Transformer Distillation in the Wild

Official implementation of "Open-set Label Noise Can Improve Robustness Against Inherent Label Noise" (NeurIPS 2021)

Voxel Set Transformer: A Set-to-Set Approach to 3D Object Detection from Point Clouds (CVPR 2022)

Code for the paper "MASTER: Multi-Aspect Non-local Network for Scene Text Recognition" (Pattern Recognition 2021)