BioCreative - Goals

Critical Assessment of Information Extraction in Biology - data sets are available from Resources/Corpora and require registration.

Background

Goals [2003-12-31]

Assessment of state of the art: One of the aims of the BioCreAtIvE challenge is to determine the state of the art for a given task in biomedical text mining and information extraction. This can be achieved if a considerable number of participants from a given community participates and the provided results of each system is evaluated by domain experts using well defined evaluation metrics. It is thus assured that a variety of methods are assessed using a common high quality data set.
Monitor improvements: By providing some sense of continuity in some of the tasks, improvements of different approaches can be monitored, and more successful techniques/strategies compared to previous attempts can be identified.
Biological meaningful tasks: The tasks posed at the BioCreAtIvE challenge should have practical relevance to facilitate efficient information access for the biology community and not focus merely on computational linguistics/NLP aspects. Thus the possibility of integrating some of the participating systems into useful applications, e.g. to link a given database entry to literature derived information is taken into account while defining the tasks.
Gold standard datasets: The development of suitable, large enough training and test data from large literature collections is especially demanding, consisting in a time-consuming process which requires expert domain knowledge. Therefore one of the aims of BioCreAtIvE is also to produce high quality 'Gold Standard' data sets for the development and evaluation of existing as well as new biomedical text mining systems.
Point out demands by biologists: The BioCreAtIvE challenge evaluation also serves to promote important interactions between the biologists/biological database community and the biomedical IE/text mining community. Needs and demands of the first community are pointed out, and possible solutions and applications which address those demands are developed by the second community.
Promote the development of new strategies: This challenge evaluation also tries to promote the development of new/complementary strategies by the participating teams to solve a given task. In fact, in case of the first BioCreAtIvE challenge (task 2) a variety of different strategies were adopted to solve the same problem.
Explore evaluation standards: Also the consideration of the most suitable evaluation strategies for each of the post tasks are explored.
Reinforcing community: The field of biomedical text mining is highly interdisciplinary, and the BioCreAtIvE challenge is an important event which facilitates the exchange of experiences of groups with different perspectives/background.