Proceedings of the 22nd Conference on Computational Natural Language Learning 2018
DOI: 10.18653/v1/k18-1001
|View full text |Cite
|
Sign up to set email alerts
|

Embedded-State Latent Conditional Random Fields for Sequence Labeling

Abstract: Complex textual information extraction tasks are often posed as sequence labeling or shallow parsing, where fields are extracted using local labels made consistent through probabilistic inference in a graphical model with constrained transitions. Recently, it has become common to locally parametrize these models using rich features extracted by recurrent neural networks (such as LSTM), while enforcing consistent outputs through a simple linear-chain model, representing Markovian dependencies between successive… Show more

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...
2
1
1

Citation Types

0
7
0

Year Published

2019
2019
2022
2022

Publication Types

Select...
4

Relationship

1
3

Authors

Journals

citations
Cited by 4 publications
(7 citation statements)
references
References 16 publications
0
7
0
Order By: Relevance
“…We train a variety of citation field extraction models including one based on RoBERTa [Liu et al, 2019]. We show that this model trained only on the UMass CFE dataset matches state-ofthe-art results [Thai et al, 2018]. We then show that training the BERT-based model on our large automatically generated dataset drastically improves the results, outperforming the state of the art approach by 1.2 points of F1, a 24.48% relative reduction in error.…”
Section: Introductionmentioning
confidence: 87%
“…We train a variety of citation field extraction models including one based on RoBERTa [Liu et al, 2019]. We show that this model trained only on the UMass CFE dataset matches state-ofthe-art results [Thai et al, 2018]. We then show that training the BERT-based model on our large automatically generated dataset drastically improves the results, outperforming the state of the art approach by 1.2 points of F1, a 24.48% relative reduction in error.…”
Section: Introductionmentioning
confidence: 87%
“…Low-rank structure has been explored in both HMMs [Siddiqi et al, 2009], a generalization of PCFGs called weighted tree automata [Rabusseau et al, 2015], and conditional random fields [Thai et al, 2018]. The reduced-rank HMM [Siddiqi et al, 2009] has at most 50 states, and relies on spectral methods for training.…”
Section: Related Workmentioning
confidence: 99%
“…We extend the low-rank assumption to neural parameterizations, which have been shown to be effective for generalization [Kim et al, 2019, Chiu andRush, 2020], and directly optimize the evidence via gradient descent. Finally, Thai et al [2018] do not take advantage of the low-rank parameterization of their CRF potentials for faster inference via low-rank matrix products, a missed opportunity. Instead, the low-rank parameterization is used only as a regularizer, with the full potentials instantiated during inference.…”
Section: Related Workmentioning
confidence: 99%
“…LDCRF has been shown to outperform HMM, CRF and HCRF on several sequence labeling tasks Sun et al, 2008). Thai et al (2018) recently proposed a very similar model called Embedded-State Latent CRFs. They claim to factorize the log potential as the novelty over a LDCRF, however such factorization is not reflected in their model structure and mathematical descriptions.…”
Section: Literature Reviewmentioning
confidence: 99%