RT Conference Proceedings
T1 An analysis of biomedical tokenization : problems and strategies
A1 Cruz Díaz, Noa Patricia
A1 Maña López, Manuel Jesús
AB Choosing the right tokenizer is a non-trivial task, especially in the biomedical domain, where it poses additional challenges, which if not resolved means the propagation of errors in successive Natural Language Processing analysis pipeline. This paper aims to identify these problematic cases and analyze the out-put that, a representative and widely used set of tokenizers, shows on them. This work will aid the decision making process of choosing the right strategy according to the down-stream application. In addition, it will help developers to create accurate tokenization tools or improve the existing ones. A total of 14 problematic cases were described, show-ing biomedical samples for each of them. The outputs of 12 tokenizers were provided and discussed in relation to the level of agreement among tools
YR 2015
FD 2015
LK http://hdl.handle.net/10272/11951
UL http://hdl.handle.net/10272/11951
LA eng
NO Cruz Díaz, N.P., Maña López, M.J.: "An analysis of biomedical tokenization : problems and strategies". En: Sixth International Workshop on Health Text Mining and Information Analysis (Louhi), pages 40–49,. Lisbon, Portugal, 17 September 2015
DS Repositorio Institucional de la Universidad de Huelva
RD 14 jul 2026