Year: 2015
Type: dissertation
Abstract: This thesis describes a semi-supervised approach to improving statistical dependency parsing using word clusters as features in MaltParser, a data-driven transition-based dependency parser.We experiment with using different clustering features for generating clusters, using the minibatch variant of the K-means algorithm.The clusters are used as a source of additional information in an expanded feature model used by the MaltParser system.A baseline parser is applied to unlabeled text, generating a dependency representation of the text.K-means is then used for generating word clusters based on this dependency representation.We then re-train the parser with information on these clusters by using a cluster-informed feature model, having all the baseline feature functions, but expanded with addition cluster feature functions, making this an instance of semi-supervised learning.We report significantly improved parsing results when using a cluster-informed parser compared to the baseline parser, using both in-domain and out-of-domain data.iii List of Figures2.1 Constituent parse tree example of sentence from Reuters . .2.2 Dependency graph example of sentence from Reuters . . . .2.3 CoNLL format example of sentence from Reuters . . . . . .2.4 ConLL file example annotated with cluster labels . . . . . . .3.1 Hierarchical clustering example of lemmas from Reuters . .3.2 Brown clustering example . . . . . . . . . . . . . . . . . . . .6.1 Dependency clustering features example . . . . . . . . . . .6.2 Path-to-root-pos clustering features example . . . . . . . . .6.3 Maltparser training example . . . . . . . . . . . . . . . . . . .6.4 MaltParser parsing example example . . . . . . . . . . . . .6.5 Clustering example of web data -part 1 . . . . . . . . . . . .6.6 Clustering example of web data -part 2 . . . . . . . . . . . .v vi List of Tables 2.1 MaltParser feature models used in this thesis . . . . . . . . .5.1 Statistics for clustering data sets . . . . . . . . . . . . . . . . .5.2 Statistics for SANCL data sets for parsing . . . . . . . . . . .5.3 Statistics for Wall Street Journal split . . . . . . . . . . . . . .5.4 Parsing scores comparing gold vs auto Pos tags . . . . . . . .6.1 Most frequent lemmas from the Reuters corpus . . . . . . . .6.2 Most frequent lemmas from the All SANCL corpus . . . . .6.3 Parsing scores for assessing impact from non-deterministic clustering 50,000 lemmas . . . . . . . . . . . . . . . . . . . . .6.4 Development results on parsing WSJ 22 from clustering Reuters on path-to-root-pos features . . . . . . . . . . . . . .6.5 Development results on parsing Weblogs from clustering Reuters on path-to-root-pos features . . . . . . . . . . . . . .6.6 Development results on parsing Emails from clustering Reuters on path-to-root-pos features . . . . . . . . . . . . . .6.7 Held-out parsing results based on clustering Reuters on path-to-root-pos features . . . . . . . . . . . . . . . . . . . . .6.8 Development results on parsing WSJ 22 from clustering Reuters on dependency features . . . . . . . . . . . . . . . . .6.9 Development results on parsing Weblogs from clustering Reuters on dependency features . . . . . . . . . . . . . . . . .6.10 Development results on parsing Emails from clustering Reuters on dependency features . . . . . . . . . . . . . . . . .6.11 Held-out parsing results based on clustering Reuters on dependency features . . . . . . . . . . . . . . . . . . . . . . . .6.12 Comparing held-out results between dependency and pathto-root-pos geatures when clustering Reuters . . . . . . . . .6.13 Number of clustering features based on unlabeled SANCL data for clustering . . . . . . . . . . . . . . . . . . . . . . . . .6.14 Development results on parsing Weblogs from clustering indomain on path-to-root-pos features . . . . . . . . . . . . . .6.15 Development results on parsing Emails from clustering indomain on path-to-root-pos features . . . . . . . . . . . . . .6.16 Held-out parsing results with Form_simple model based on clustering in-domain on path-to-root-pos feature . . . . . . .vii 6.17 Held-out parsing results with Form_all model based on clustering in-domain on path-to-root-pos features form-all .6.18 Comparing held-out parsing with Form_simple and Form_all models based on clustering in-domain on path-root-pos features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6.19 Comparing held-out parsing results based on clustering Reuters and in-domain on path-root-pos features . . . . . . .6.20 Clustering features statistics for unlabeled SANCL data using dependency features . . . . . . . . . . . . . . . . . . . . .6.21 Development results on parsing Weblogs based on clustering in-domain on dependency features . . . . . . . . . . . . . .6.22 Development results on parsing Emails based on clustering in-domain on dependency features . . . . . . . . . . . . . . . .6.23 Held-out parsing results with the Form_simple model based on clustering in-domain on dependency features form-simple 6.24 Held-out parsing results with the Form_all model based on clustering in-domain on dependency features . . . . . . . . . .6.25 Comparing held-out parsing results based on clustering indomain on dependency features . . . . . . . . . . . . . . . . .6.26 Comparing held-out parsing results based on clustering Reuters and in-domain on dependency features . . . . . . . .6.27 Comparing all Reuters and in-domain held-out results . . .6.28 Comparing held-out parsing results based on clustering all SANCL domains concatenated . . . . . . . . . . . . . . . . .6.29 Comparing All SANCL and in-domain held-out-results . . .6.30 Parsing scores for assessing impact from non-deterministic clustering comparing 10,000 and 50,000 lemmas . . . . . . .6.31 Comparing held-out parsing results based on clustering 10,000 and 50,000 lemmas from Reuters on dependency features 6.32 Comparing held-out parsing results based on clustering 10,000 and 50,000 lemmas from in-domain based on dependency features. . . . . . . . . . . . . . . . . . . . . . . . . . . . .6.33 Comparing held-out parsing results based on clustering 10,000 and 50,000 lemmas from all SANCL domains concatenated . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6.34 Comparing held-out parsing results based on K-means and agglomerative clustering on 5,000 lemmas from Reuters . . .6.35 Comparing our held-out parsing results with parsing based on Brown clusters (Øvrelid, Skjærholt) with gold PoS tags .6.36 Comparing our held-out parsing results with parsing based on Brown clusters (Øvrelid, Skjærholt) with auto PoS tags .viii
Author: Jostein Lien
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W2544444394