Work

Efficient and effective learning of text representations

Year: 2023
Type: article
Abstract: Text representation learning has played a pivotal role in enabling a wide range of Natural Language Processing (NLP) tasks that involve the processing, analysis, and utilization of human-generated textual data.These representations are typically obtained by transforming raw texts into vectors via deep neural networks (e.g., Transformers [215]).Recent advances in Large Language Models (LLMs) [20, 160] have demonstrated the great potential of learning generic text representations applicable across a wide spectrum of applications.This success is underpinned by two critical factors: (1) the utilization of extensive text data for training LLMs in both pretraining and fine-tuning, and (2) the scaling up of LLMs to encompass tens or even hundreds of billions of parameters.Consequently, training LLMs entails substantial costs, including the acquisition of massive labeled data and the infrastructure required to support these large models.Motivated by these challenges, my doctoral research seeks to develop efficient and effective methods for learning text representations, spanning several key subtopics: < l a t e x i t s h a 1 _ b a s e 6 4 = " g 7 y w
Author: Meng, Yu
Language: English
Open Access status: green
OpenAlex ID: W7220802998
FWCI:
Cites: 161
Cited by:
Related to: 10