Syllable-level Input Defence for Low-resource Text via Pseudo-perplexity Purification

Abstract

Deployed low-resource language text classification models face security threats from adversarial attacks, and retraining them incurs
high costs. Therefore, we have been studying and building an adversarial attack and defence system for low-resource language text. Given that
it is the basis for the system, we put forward a model-agnostic, syllable-level input-level defence based on pseudo-perplexity detection and
mask reconstruction. By estimating external masked language models, we can know if there is a disturbance. Then, the suspicious syllables
are corrected in a multi-dimensional restricted candidate space to modify the text and reduce the pseudo-perplexity of the whole sentence as
much as possible. Therefore, the input is clean and the parameters of the victim model do not change. Based on the experiment results of the
TNCC-title and TU_SA datasets, the proposed defence can be stably restored from all kinds of attacks, and it has a small change rate and high
semantic fidelity. A Practical and Deployable Approach to Enhance the Robustness of Low-Resource Language Text Classification Systems is
Presented in this paper.