This is our ARTS test set, an enriched test set to probe Aspect Robustness of ABSA.

Overview

This is the repository for our 2020 paper "Tasty Burgers, Soggy Fries: Probing Aspect Robustness in Aspect-Based Sentiment Analysis".

Data

We provide a Aspect Robustness Probing test set for SemEval 2014 Aspect-Based Sentiment Analysis (ABSA).

Data Generation Process

We generate our new probing test set by three automatic strategies:

  • RevTgt (sentence with a red background): Reverse the sentiment of the Target aspect.
  • RevNon (sentence with a green background): Reverse the sentiment of the Non-target aspect.
  • AddDiff (sentence with a blue background): Add new aspects with Different sentiment.

method_illustration.png

Aspect Probing Results

We probed nine ABSA models (as mentioned in our paper).

  • Their outputs on SemEval 2014 are in the output folder.

How to Use Our Code

If you have a new ABSA dataset, you can run our code to generate you own aspect robustness probing test set.

python code/main.py -dataset_name laptop

More Questions

If you have more questions, please feel free to submit a GitHub issue.

You might also like...
codes for paper Combining Dynamic Local Context Focus and Dependency Cluster Attention for Aspect-level sentiment classification
codes for paper Combining Dynamic Local Context Focus and Dependency Cluster Attention for Aspect-level sentiment classification

DLCF-DCA codes for paper Combining Dynamic Local Context Focus and Dependency Cluster Attention for Aspect-level sentiment classification. submitted t

MemStream: Memory-Based Anomaly Detection in Multi-Aspect Streams with Concept Drift
MemStream: Memory-Based Anomaly Detection in Multi-Aspect Streams with Concept Drift

MemStream Implementation of MemStream: Memory-Based Anomaly Detection in Multi-Aspect Streams with Concept Drift . Siddharth Bhatia, Arjit Jain, Shivi

Code and dataset for ACL2018 paper "Exploiting Document Knowledge for Aspect-level Sentiment Classification"

Aspect-level Sentiment Classification Code and dataset for ACL2018 [paper] ‘‘Exploiting Document Knowledge for Aspect-level Sentiment Classification’’

This project is a re-implementation of MASTER: Multi-Aspect Non-local Network for Scene Text Recognition by MMOCR
This project is a re-implementation of MASTER: Multi-Aspect Non-local Network for Scene Text Recognition by MMOCR

This project is a re-implementation of MASTER: Multi-Aspect Non-local Network for Scene Text Recognition by MMOCR,which is an open-source toolbox based on PyTorch. The overall architecture will be shown below.

This is the code for ACL2021 paper A Unified Generative Framework for Aspect-Based Sentiment Analysis

This is the code for ACL2021 paper A Unified Generative Framework for Aspect-Based Sentiment Analysis Install the package in the requirements.txt, the

Fuzzing JavaScript Engines with Aspect-preserving Mutation

DIE Repository for "Fuzzing JavaScript Engines with Aspect-preserving Mutation" (in S&P'20). You can check the paper for technical details. Environmen

 Enhancing Aspect-Based Sentiment Analysis with Supervised Contrastive Learning.
Enhancing Aspect-Based Sentiment Analysis with Supervised Contrastive Learning.

Enhancing Aspect-Based Sentiment Analysis with Supervised Contrastive Learning. Enhancing Aspect-Based Sentiment Analysis with Supervised Contrastive

A sample pytorch Implementation of ACL 2021 research paper "Learning Span-Level Interactions for Aspect Sentiment Triplet Extraction".

Span-ASTE-Pytorch This repository is a pytorch version that implements Ali's ACL 2021 research paper Learning Span-Level Interactions for Aspect Senti

PyTorch implementation of our Adam-NSCL algorithm from our CVPR2021 (oral) paper "Training Networks in Null Space for Continual Learning"

Adam-NSCL This is a PyTorch implementation of Adam-NSCL algorithm for continual learning from our CVPR2021 (oral) paper: Title: Training Networks in N

Comments
  •  The version of allennlp?

    The version of allennlp?

    Recently I am reading your article and trying to run your code, but I always encounter all kinds of problems with allennlp. What is the version of allennlp?

    opened by BD-MF 4
  • How to flip the polarities for those opinion words with original 'neutral' labels?

    How to flip the polarities for those opinion words with original 'neutral' labels?

    With respect to the opinion polarity modification, I find that in the article you placed most emphasis on how to operate on those opinion words with 'negative' or 'positive' labels. But I do notice in your dataset that the 'neutral' polarities can be flipped into other labels. And this part is not mentioned in the paper.

    If using the same method, e.g., looking up WordNet for an antonym as its replacement, how to determine the sentiment polarity for this antonym when its origin is a 'neutral' term?

    So my technical question is: How did you flip the polarities for those opinion words with original 'neutral' labels?

    opened by scofield7419 3
  • Augmented training/developing sets available?

    Augmented training/developing sets available?

    Hi,

    Solid and fantastic work!

    You presented the augmented test set. I am wondering if you constructed the corresponding training/developing sets as well? If yes, will they be available?

    opened by scofield7419 2
  • Question about the tradeoff between robustness with generalization in ABSA task

    Question about the tradeoff between robustness with generalization in ABSA task

    Great work and solid experiment 🎉 In the paper table 10b, column 'Adv→N' shows that adversarial training could improve the model performance in the ARTS test dataset. My question is have you test the effect of adversarial training for the original test dataset e.g 'Adv→Ori‘. So that we could see after training on hard examples, how the generalization gap changes. And whether there is a tradeoff between robustness with generalization in ABSA task.

    opened by Opdoop 2
Owner
PhD in NLP and Causality. Affiliated with Max Planck Institute, Germany & research intern at Amazon AI. Supervised by Prof Bernhard Schoelkopf.
null
Automatically download the cwru data set, and then divide it into training data set and test data set

Automatically download the cwru data set, and then divide it into training data set and test data set.自动下载cwru数据集,然后分训练数据集和测试数据集

null 6 Jun 27, 2022
Source Code for our paper: Understand me, if you refer to Aspect Knowledge: Knowledge-aware Gated Recurrent Memory Network

KaGRMN-DSG_ABSA This repository contains the PyTorch source Code for our paper: Understand me, if you refer to Aspect Knowledge: Knowledge-aware Gated

XingBowen 4 May 20, 2022
Code for our paper Aspect Sentiment Quad Prediction as Paraphrase Generation in EMNLP 2021.

Aspect Sentiment Quad Prediction (ASQP) This repo contains the annotated data and code for our paper Aspect Sentiment Quad Prediction as Paraphrase Ge

Isaac 39 Dec 11, 2022
Code implementation of "Sparsity Probe: Analysis tool for Deep Learning Models"

Sparsity Probe: Analysis tool for Deep Learning Models This repository is a limited implementation of Sparsity Probe: Analysis tool for Deep Learning

null 3 Jun 9, 2021
Python implementation of Lightning-rod Agent, the Stack4Things board-side probe

Iotronic Lightning-rod Agent Python implementation of Lightning-rod Agent, the Stack4Things board-side probe. Free software: Apache 2.0 license Websit

null 2 May 19, 2022
Attention Probe: Vision Transformer Distillation in the Wild

Attention Probe: Vision Transformer Distillation in the Wild Jiahao Wang, Mingdeng Cao, Shuwei Shi, Baoyuan Wu, Yujiu Yang In ICASSP 2022 This code is

Wang jiahao 3 Oct 31, 2022
Pytorch implementation of ICASSP 2022 paper Attention Probe: Vision Transformer Distillation in the Wild

Attention Probe: Vision Transformer Distillation in the Wild Jiahao Wang, Mingdeng Cao, Shuwei Shi, Baoyuan Wu, Yujiu Yang In ICASSP 2022 This code is

IIGROUP 6 Sep 21, 2022
Official implementation of "Open-set Label Noise Can Improve Robustness Against Inherent Label Noise" (NeurIPS 2021)

Open-set Label Noise Can Improve Robustness Against Inherent Label Noise NeurIPS 2021: This repository is the official implementation of ODNL. Require

Hongxin Wei 12 Dec 7, 2022
Voxel Set Transformer: A Set-to-Set Approach to 3D Object Detection from Point Clouds (CVPR 2022)

Voxel Set Transformer: A Set-to-Set Approach to 3D Object Detection from Point Clouds (CVPR2022)[paper] Authors: Chenhang He, Ruihuang Li, Shuai Li, L

Billy HE 141 Dec 30, 2022
Code for the paper "MASTER: Multi-Aspect Non-local Network for Scene Text Recognition" (Pattern Recognition 2021)

MASTER-PyTorch PyTorch reimplementation of "MASTER: Multi-Aspect Non-local Network for Scene Text Recognition" (Pattern Recognition 2021). This projec

Wenwen Yu 255 Dec 29, 2022