Work

Text Substitution in Natural Language Processing (NLP)

Year: 2025

Type: article

Abstract: This article explores the often-overlooked yet critical step of text substitution in natural language processing (NLP). Covering contractions, HTML/XML cleaning, unit standardisation, and domain-specific preprocessing, it demonstrates how substitution enhances model efficiency and semantic clarity. Through practical code, it shows how consistent representation improves tokenisation and machine interpretation. The session bridges theory and practice, offering both foundational principles and research-oriented challenges in creating customised pipelines for varied NLP tasks, from general corpus cleaning to physics education systems like AINSTEIN.

Source: Zenodo (CERN European Organization for Nuclear Research)

DOI: 10.5281/zenodo.15768482

Author: Majumdar, Partha

Language: English

Topic: Natural Language Processing Techniques

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

OpenAlex ID: W6893338498

FWCI:

Cites:

Cited by:

Related to: