Archive for the ‘Curation’ Category
Friday, April 19th, 2013
Preliminary evaluation of the CellFinder literature curation pipeline for gene expression in kidney cells and anatomical parts by Mariana Neves, Alexander Damaschun, Nancy Mah, Fritz Lekschas, Stefanie Seltmann, Harald Stachelscheid, Jean-Fred Fontaine, Andreas Kurtz, and Ulf Leser. (Database (2013) 2013 : bat020 doi: 10.1093/database/bat020)
Abstract:
Biomedical literature curation is the process of automatically and/or manually deriving knowledge from scientific publications and recording it into specialized databases for structured delivery to users. It is a slow, error-prone, complex, costly and, yet, highly important task. Previous experiences have proven that text mining can assist in its many phases, especially, in triage of relevant documents and extraction of named entities and biological events. Here, we present the curation pipeline of the CellFinder database, a repository of cell research, which includes data derived from literature curation and microarrays to identify cell types, cell lines, organs and so forth, and especially patterns in gene expression. The curation pipeline is based on freely available tools in all text mining steps, as well as the manual validation of extracted data. Preliminary results are presented for a data set of 2376 full texts from which >4500 gene expression events in cell or anatomical part have been extracted. Validation of half of this data resulted in a precision of ∼50% of the extracted data, which indicates that we are on the right track with our pipeline for the proposed task. However, evaluation of the methods shows that there is still room for improvement in the named-entity recognition and that a larger and more robust corpus is needed to achieve a better performance for event extraction.
Database URL: http://www.cellfinder.org/.
Another extremely useful data curation project.
Do you get the impression that curation projects will continue to be outrun by data production?
And that will be the case, even with machine assistance?
Is there an alternative to falling further and further behind?
Such as abandoning some content (CNN?) to simply forever go uncurated? Or the same to be true for government documents/reports?
I am sure we all have different suggestions for what data to dump alongside the road to make room for the “important” stuff.
Suggestions on solutions other than simply dumping data?
Posted in Curation, Data, Entity Resolution, Named Entity Mining | No Comments »
Tuesday, March 19th, 2013
Digital Curation Bibliography: Preservation and Stewardship of Scholarly Works, 2012 Supplement by Charles W. Bailey, Jr.
From the webpage:
In a rapidly changing technological environment, the difficult task of ensuring long-term access to digital information is increasingly important. The Digital Curation Bibliography: Preservation and Stewardship of Scholarly Works, 2012 Supplement presents over 130 English-language articles, books, and technical reports published in 2012 that are useful in understanding digital curation and preservation. This selective bibliography covers digital curation and preservation copyright issues, digital formats (e.g., media, e-journals, research data), metadata, models and policies, national and international efforts, projects and institutional implementations, research studies, services, strategies, and digital repository concerns.
It is a supplement to the Digital Curation Bibliography: Preservation and Stewardship of Scholarly Works, which covers over 650 works published from 2000 through 2011. All included works are in English. The bibliography does not cover conference papers, digital media works (such as MP3 files), editorials, e-mail messages, letters to the editor, presentation slides or transcripts, unpublished e-prints, or weblog postings.
The bibliography includes links to freely available versions of included works. If such versions are unavailable, italicized links to the publishers' descriptions are provided.
Links, even to publisher versions and versions in disciplinary archives and institutional repositories, are subject to change. URLs may alter without warning (or automatic forwarding) or they may disappear altogether. Inclusion of links to works on authors' personal websites is highly selective. Note that e-prints and published articles may not be identical.
The bibliography is available under a Creative Commons Attribution-NonCommercial 3.0 Unported License.
Supplement to “the” starting point for research on digital curation.
Posted in Archives, Bibliography, Curation, Data Preservation, Digital Library, Librarian/Expert Searchers | No Comments »
Monday, March 18th, 2013
The results are in – inorganics are out!
From the ChEMBL-og blog which “covers the activities of the Computational Chemical Biology Group at the EMBL-EBI in Hinxton.
From the post:
A few weeks ago we ran a small poll on how we should deal with inorganic molecules – not just simple sodium salts, but things like organoplatinums, and other compounds with dative bonds, unusual electronic states, etc. The results from you were clear, there was little interest in having a lot of our curation time spent on these. We will continue to collect structures from the source journals, and they will be in the full database, but we won’t try and curate the structures, or display them in the interface. They will be appropriately flagged, and nothing will get lost. So there it is, democracy in action.
So for ChEMBL 16 expect fewer issues when you try and load our structures in your own pipelines and systems.
Just an FYI that inorganic compounds are not being curated at ChEMBL.
If you decide to undertake such work, contacting ChEMBL to coordinate collection, etc., would be a good first step.
Posted in Cheminformatics, Curation | No Comments »
Monday, March 18th, 2013
Crowdsourced Chemistry Why Online Chemistry Data Needs Your Help by Antony Williams. (video)
From the description:
This is the Ignite talk that I gave at ScienceOnline2010 #sci010 in the Research Triangle Park in North Carolina on January 16th 2010. This was supposed to be a 5 minute talk highlighting the quality of chemistry data on the internet. Ok, it was a little tongue in cheek because it was an after dinner talk and late at night but the data are real, the problem is real and the need for data curation of chemistry data online is real. On ChemSpider we have provided a platform to deposit and curate data. Other videos will show that in the future.
Great demonstration of the need for curation in chemistry.
And of the impact that re-usable information can have on the quality of information.
The errors in chemical descriptions you see in this video could be corrected in:
- In an article.
- In a monograph.
- In a webpage.
- In an online resource that can be incorporated by reference.
Which one do you think would propagate the corrected information more quickly?
Documents are a great way to convey information to a reader.
They are an incredibly poor way to store/transmit information.
Every reader has to extract the information in a document for themselves.
Not to mention that data is fixed, unless it has incorporated information by reference.
Funny isn’t it? We are still storing data as we did when clay tablets were the medium of choice.
Isn’t it time we separated presentation (documents) from storage/transmission (data)?
Posted in Cheminformatics, Crowd Sourcing, Curation | No Comments »
Monday, March 11th, 2013
Studying PubMed usages in the field for complex problem solving: Implications for tool design by Barbara Mirel, Jennifer Steiner Tonks, Jean Song, Fan Meng, Weijian Xuan, Rafiqa Ameziane. (Mirel, B., Tonks, J. S., Song, J., Meng, F., Xuan, W. and Ameziane, R. (2013), Studying PubMed usages in the field for complex problem solving: Implications for tool design. J. Am. Soc. Inf. Sci.. doi: 10.1002/asi.22796)
Abstract:
Many recent studies on MEDLINE-based information seeking have shed light on scientists’ behaviors and associated tool innovations that may improve efficiency and effectiveness. Few, if any, studies, however, examine scientists’ problem-solving uses of PubMed in actual contexts of work and corresponding needs for better tool support. Addressing this gap, we conducted a field study of novice scientists (14 upper-level undergraduate majors in molecular biology) as they engaged in a problem-solving activity with PubMed in a laboratory setting. Findings reveal many common stages and patterns of information seeking across users as well as variations, especially variations in cognitive search styles. Based on these findings, we suggest tool improvements that both confirm and qualify many results found in other recent studies. Our findings highlight the need to use results from context-rich studies to inform decisions in tool design about when to offer improved features to users.
From the introduction:
For example, our findings confirm that additional conceptual information integrated into retrieved results could expedite getting to relevance. Yet—as a qualification—evidence from our field cases suggests that presentations of this information need to be strategically apportioned and staged or they may inadvertently become counterproductive due to cognitive overload.
Curated data raises its ugly head, again.
Topic maps curate data and search results.
Search engines don’t curate data or search results.
How important is it for your doctor to find the right answers? In a timely manner?
Posted in Curation, Interface Research/Design, Searching, Usability, Users | No Comments »
Saturday, March 9th, 2013
Research Data Symposium – Columbia.
Posters from the Research Data Symposium, held at Columbia University, February 27, 2013.
Subject to the limitations of the poster genre but useful as a quick overview of current projects and directions.
Posted in Archives, Curation, Data Management, Data Science, Librarian/Expert Searchers, Library | No Comments »
Friday, March 1st, 2013
Shedding Light on the Dark Data in the Long Tail of Science by P. Bryan Heidorn. (P. Bryan Heidorn. “Shedding Light on the Dark Data in the Long Tail of Science.” Library Trends 57.2 (2008): 280-299. Project MUSE. Web. 28 Feb. 2013. .)
Abstract:
One of the primary outputs of the scientific enterprise is data, but many institutions such as libraries that are charged with preserving and disseminating scholarly output have largely ignored this form of documentation of scholarly activity. This paper focuses on a particularly troublesome class of data, termed dark data. “Dark data” is not carefully indexed and stored so it becomes nearly invisible to scientists and other potential users and therefore is more likely to remain underutilized and eventually lost. The article discusses how the concepts from long-tail economics can be used to understand potential solutions for better curation of this data. The paper describes why this data is critical to scientific progress, some of the properties of this data, as well as some social and technical barriers to proper management of this class of data. Many potentially useful institutional, social, and technical solutions are under development and are introduced in the last sections of the paper, but these solutions are largely unproven and require additional research and development.
From the article:
In this paper we will use the term dark data to refer to any data that is not easily found by potential users. Dark data may be positive or negative research findings or from either “large” or “small” science. Like dark matter, this dark data on the basis of volume may be more important than that which can be easily seen. The challenge for science policy is to develop institutions and practices such as institutional repositories, which make this data useful for society.
Dark Data = Any data that is not easily found by potential users.
A number of causes are discussed, not the least of which is our old friend, the Tower of Babel.
A final barrier that cannot be overlooked is the Digital Tower of Babel that we have created with seemingly countless proprietary as well as open data formats. This can include versions of the same software products that are incompatible. Some of these formats are very efficient for the individual applications for which they were designed including word processing, databases, spreadsheets, and others, but they are ineffective to support interoperability and preservation.
As you know already, I don’t think the answer to data curation, long term, lies in uniform formats.
Uniform formats are very useful but are domain, project and time bound.
The questions always are:
“What do we do when we change data formats?”
“Do we dump data in old formats that we spent $$$ developing?”
“Do we migrate data in old formats, assuming anyone remembers the old format?”
“Do we document and map across old and new formats, preparing for the next ‘new’ format?”
None of the answers are automatic or free.
But it is better to make in informed choice than a default one of letting potentially valuable data rot.
Posted in Curation, Librarian/Expert Searchers, Library | No Comments »
Friday, March 1st, 2013
Looking out for the little guy: Small data curation by Katherine Goold Akers. (Akers, K. G. (2013), Looking out for the little guy: Small data curation. Bul. Am. Soc. Info. Sci. Tech., 39: 58–59. doi: 10.1002/bult.2013.1720390317)
Abstract:
While big data and its management are in the spotlight, a vast number of important research projects generate relatively small amounts of data that are nonetheless valuable yet rarely preserved. Such studies are often focused precursors to follow-up work and generate less noisy data than grand scale projects. Yet smaller quantity does not equate to simpler management. Data from smaller studies may be captured in a variety of file formats with no standard approach to documentation, metadata or preparation for archiving or reuse, making its curation even more challenging than for big data. As the information managers most likely to encounter small datasets, academic librarians should cooperate to develop workable strategies to document, organize, preserve and disseminate local small datasets so that valuable scholarly information can be discovered and shared.
A reminder that for every “big data” project in need of curation, there are many more smaller, less well known projects that need the same services.
Since topic maps don’t require global or even regional agreement on ontology or methodological issues, it should be easier for academic librarians to create topic maps to curate small datasets.
When it is necessary or desired to merge small datasets that were curated with different topic map assumptions, new topics can be created that merge the data that existed in separate topic maps.
But only when necessary and at the point of merging.
To say it another way, topic maps need not anticipate or fear the future. Tomorrow will take care of itself.
Unlike “now I am awake” approaches, that must fear the next moment of consciousness will bring change.
Posted in Curation, Librarian/Expert Searchers, Library | No Comments »
Sunday, January 20th, 2013
An overview of the BioCreative 2012 Workshop Track III: interactive text mining task Cecilia N. Arighi, et. al. (Database (2013) 2013 : bas056 doi: 10.1093/database/bas056)
Abstract:
In many databases, biocuration primarily involves literature curation, which usually involves retrieving relevant articles, extracting information that will translate into annotations and identifying new incoming literature. As the volume of biological literature increases, the use of text mining to assist in biocuration becomes increasingly relevant. A number of groups have developed tools for text mining from a computer science/linguistics perspective, and there are many initiatives to curate some aspect of biology from the literature. Some biocuration efforts already make use of a text mining tool, but there have not been many broad-based systematic efforts to study which aspects of a text mining tool contribute to its usefulness for a curation task. Here, we report on an effort to bring together text mining tool developers and database biocurators to test the utility and usability of tools. Six text mining systems presenting diverse biocuration tasks participated in a formal evaluation, and appropriate biocurators were recruited for testing. The performance results from this evaluation indicate that some of the systems were able to improve efficiency of curation by speeding up the curation task significantly (∼1.7- to 2.5-fold) over manual curation. In addition, some of the systems were able to improve annotation accuracy when compared with the performance on the manually curated set. In terms of inter-annotator agreement, the factors that contributed to significant differences for some of the systems included the expertise of the biocurator on the given curation task, the inherent difficulty of the curation and attention to annotation guidelines. After the task, annotators were asked to complete a survey to help identify strengths and weaknesses of the various systems. The analysis of this survey highlights how important task completion is to the biocurators’ overall experience of a system, regardless of the system’s high score on design, learnability and usability. In addition, strategies to refine the annotation guidelines and systems documentation, to adapt the tools to the needs and query types the end user might have and to evaluate performance in terms of efficiency, user interface, result export and traditional evaluation metrics have been analyzed during this task. This analysis will help to plan for a more intense study in BioCreative IV.
Curation is an aspect of topic map authoring, albeit with the latter capturing information for later merging with other sources of information.
Definitely an article you will want to read if you are designing text mining as part of a topic map solution.
Posted in Annotation, Bioinformatics, Curation, Text Mining | 1 Comment »
Wednesday, January 16th, 2013
Research Data Curation Bibliography (version 2) by Charles W. Bailey.
From the introduction:
The Research Data Curation Bibliography includes selected English-language articles, books, and technical reports that are useful in understanding the curation of digital research data in academic and other research institutions. For broader coverage of the digital curation literature, see the author's Digital Curation Bibliography: Preservation and Stewardship of Scholarly Works,which presents over 650 English-language articles, books, and technical reports.
The "digital curation" concept is still evolving. In "Digital Curation and Trusted Repositories: Steps toward Success," Christopher A. Lee and Helen R. Tibbo define digital curation as follows:
Digital curation involves selection and appraisal by creators and archivists; evolving provision of intellectual access; redundant storage; data transformations; and, for some materials, a commitment to long-term preservation. Digital curation is stewardship that provides for the reproducibility and re-use of authentic digital data and other digital assets. Development of trustworthy and durable digital repositories; principles of sound metadata creation and capture; use of open standards for file formats and data encoding; and the promotion of information management literacy are all essential to the longevity of digital resources and the success of curation efforts.
This bibliography does not cover conference papers, digital media works (such as MP3 files), editorials, e-mail messages, interviews, letters to the editor, presentation slides or transcripts, unpublished e-prints, or weblog postings. Coverage of technical reports is very selective.
Most sources have been published from 2000 through 2012; however, a limited number of earlier key sources are also included. The bibliography includes links to freely available versions of included works. If such versions are unavailable, italicized links to the publishers' descriptions are provided.
Such links, even to publisher versions and versions in disciplinary archives and institutional repositories, are subject to change. URLs may alter without warning (or automatic forwarding) or they may disappear altogether. Inclusion of links to works on authors' personal websites is highly selective. Note that e prints and published articles may not be identical.
An archive of prior versions of the bibliography is available.
If you are a beginning library student, take the time to know the work of Charles Bailey. He has consistently made a positive contribution for researchers from very early in the so-called digital revolution.
To the extent that you want to design topic maps for data curation, long or short term, the 200+ items in this bibliography will introduce you to some of the issues you will be facing.
Posted in Archives, Curation, Data Preservation, Librarian/Expert Searchers, Library | No Comments »
Saturday, January 12th, 2013
The Xenbase literature curation process by Jeff B. Bowes, Kevin A. Snyder, Christina James-Zorn, Virgilio G. Ponferrada, Chris J. Jarabek, Kevin A. Burns, Bishnu Bhattacharyya, Aaron M. Zorn and Peter D. Vize.
Abstract:
Xenbase (www.xenbase.org) is the model organism database for Xenopus tropicalis and Xenopus laevis, two frog species used as model systems for developmental and cell biology. Xenbase curation processes centre on associating papers with genes and extracting gene expression patterns. Papers from PubMed with the keyword ‘Xenopus’ are imported into Xenbase and split into two curation tracks. In the first track, papers are automatically associated with genes and anatomy terms, images and captions are semi-automatically imported and gene expression patterns found in those images are manually annotated using controlled vocabularies. In the second track, full text of the same papers are downloaded and indexed by a number of controlled vocabularies and made available to users via the Textpresso search engine and text mining tool.
Which curation workflow will work best for your topic map activities will depend upon a number of factors.
What would you adopt, adapt or alter from the curation workflow in this article?
How would you evaluate the effectiveness of any of your changes?
Posted in Bioinformatics, Curation, Literature | No Comments »
Friday, November 30th, 2012
Techmeme founder: Give me human editors and the New York Times by Jeff John Roberts.
From the post:
At the event in New York, which was hosted by media company Outbrain, Rivera explained to Business Insider’s Steve Kovach why algorithms will never be able to curate as effectively as humans.
“A lot of people who think they can go all the way with the automated approach fail to realize a news story has become obsolete,” said Rivera, explaining that an article can be quickly superseded even if it receives a million links or tweets.
This is why Rivera now relies on human editors to shepherd the headlines that bubble up and swat down the inappropriate ones. He argues any serious tech or political news provider will always have to do the same.
Rivera is also not enthused about social-based news platforms — sites like LinkedIn Today or Flipboard that assemble news stories based on what your friends are sharing on social media. Asked if Techmeme will offer a social-based news feed, Rivera said don’t count on it.
“People like to go to the New York Times and look at what’s on the front page because they have a lot of trust in what editors decide and they know other people read it. We want to do the same thing,” he said. “There’s value in being divorced from your friends … I’d rather see what’s on the front of the New York Times.”
Are you trapped in a social media echo chamber?
Escape with the New York Times.
I first saw this in a tweet by Peter Cooper.
Posted in Curation, Human Cognition | 1 Comment »
Monday, November 26th, 2012
Collaborative biocuration—text-mining development task for document prioritization for curation by Thomas C. Wiegers, Allan Peter Davis and Carolyn J. Mattingly. (Database (2012) 2012 : bas037 doi: 10.1093/database/bas037)
Abstract:
The Critical Assessment of Information Extraction systems in Biology (BioCreAtIvE) challenge evaluation is a community-wide effort for evaluating text mining and information extraction systems for the biological domain. The ‘BioCreative Workshop 2012’ subcommittee identified three areas, or tracks, that comprised independent, but complementary aspects of data curation in which they sought community input: literature triage (Track I); curation workflow (Track II) and text mining/natural language processing (NLP) systems (Track III). Track I participants were invited to develop tools or systems that would effectively triage and prioritize articles for curation and present results in a prototype web interface. Training and test datasets were derived from the Comparative Toxicogenomics Database (CTD; http://ctdbase.org) and consisted of manuscripts from which chemical–gene–disease data were manually curated. A total of seven groups participated in Track I. For the triage component, the effectiveness of participant systems was measured by aggregate gene, disease and chemical ‘named-entity recognition’ (NER) across articles; the effectiveness of ‘information retrieval’ (IR) was also measured based on ‘mean average precision’ (MAP). Top recall scores for gene, disease and chemical NER were 49, 65 and 82%, respectively; the top MAP score was 80%. Each participating group also developed a prototype web interface; these interfaces were evaluated based on functionality and ease-of-use by CTD’s biocuration project manager. In this article, we present a detailed description of the challenge and a summary of the results.
The results:
“Top recall scores for gene, disease and chemical NER were 49, 65 and 82%, respectively; the top MAP score was 80%.”
indicate there is plenty of room for improvement. Perhaps even commercially viable improvement.
In hindsight, not talking about how to make a topic map along with ISO 13250, may have been a mistake. Even admitting there are multiple ways to get there, a technical report outlining one or two ways would have made the process more transparent.
Answering the question: “What can you say with a topic map?” with “Anything you want.” was, a truthful answer but not a helpful one.
I should try to crib something from one of those “how to write a research paper” guides. I haven’t looked at one in years but the process is remarkably similar to what would result in a topic map.
Some of the mechanics are different but the underlying intellectual process is quite similar. Everyone who has been to college (at least of my age), had a course that talked about writing research papers. So it should be familiar terminology.
Thoughts/suggestions?
Posted in Authoring Topic Maps, Bioinformatics, Biomedical, Curation, Genomics, Searching | No Comments »
Thursday, November 22nd, 2012
eGIFT: Mining Gene Information from the Literature by Catalina O Tudor, Carl J Schmidt and K Vijay-Shanker.
Abstract:
Background
With the biomedical literature continually expanding, searching PubMed for information about specific genes becomes increasingly difficult. Not only can thousands of results be returned, but gene name ambiguity leads to many irrelevant hits. As a result, it is difficult for life scientists and gene curators to rapidly get an overall picture about a specific gene from documents that mention its names and synonyms.
Results
In this paper, we present eGIFT (http://biotm.cis.udel.edu/eGIFT webcite), a web-based tool that associates informative terms, called iTerms, and sentences containing them, with genes. To associate iTerms with a gene, eGIFT ranks iTerms about the gene, based on a score which compares the frequency of occurrence of a term in the gene’s literature to its frequency of occurrence in documents about genes in general. To retrieve a gene’s documents (Medline abstracts), eGIFT considers all gene names, aliases, and synonyms. Since many of the gene names can be ambiguous, eGIFT applies a disambiguation step to remove matches that do not correspond to this gene. Another additional filtering process is applied to retain those abstracts that focus on the gene rather than mention it in passing. eGIFT’s information for a gene is pre-computed and users of eGIFT can search for genes by using a name or an EntrezGene identifier. iTerms are grouped into different categories to facilitate a quick inspection. eGIFT also links an iTerm to sentences mentioning the term to allow users to see the relation between the iTerm and the gene. We evaluated the precision and recall of eGIFT’s iTerms for 40 genes; between 88% and 94% of the iTerms were marked as salient by our evaluators, and 94% of the UniProtKB keywords for these genes were also identified by eGIFT as iTerms.
Conclusions
Our evaluations suggest that iTerms capture highly-relevant aspects of genes. Furthermore, by showing sentences containing these terms, eGIFT can provide a quick description of a specific gene. eGIFT helps not only life scientists survey results of high-throughput experiments, but also annotators to find articles describing gene aspects and functions.
Website: http://biotm.cis.udel.edu/eGIFT
Another lesson for topic map authoring interfaces: Offer domain specific search capabilities.
Using a ****** search appliance is little better than a poke with a sharp stick in most domains. The user is left to their own devices to sort out ambiguities, discover synonyms, again and again.
Your search interface may report > 900,000 “hits,” but anything beyond the first 20 or so are wasted.
(If you get sick, get something that comes up in the first 20 “hits” in PubMed. Where most researchers stop.)
Posted in Curation, Genomics, Search Engines, Search Interface, Search Requirements, Text Mining | No Comments »
Thursday, November 22nd, 2012
Developing a biocuration workflow for AgBase, a non-model organism database by Lakshmi Pillai, Philippe Chouvarine, Catalina O. Tudor, Carl J. Schmidt, K. Vijay-Shanker and Fiona M. McCarthy.
Abstract:
AgBase provides annotation for agricultural gene products using the Gene Ontology (GO) and Plant Ontology, as appropriate. Unlike model organism species, agricultural species have a body of literature that does not just focus on gene function; to improve efficiency, we use text mining to identify literature for curation. The first component of our annotation interface is the gene prioritization interface that ranks gene products for annotation. Biocurators select the top-ranked gene and mark annotation for these genes as ‘in progress’ or ‘completed’; links enable biocurators to move directly to our biocuration interface (BI). Our BI includes all current GO annotation for gene products and is the main interface to add/modify AgBase curation data. The BI also displays Extracting Genic Information from Text (eGIFT) results for each gene product. eGIFT is a web-based, text-mining tool that associates ranked, informative terms (iTerms) and the articles and sentences containing them, with genes. Moreover, iTerms are linked to GO terms, where they match either a GO term name or a synonym. This enables AgBase biocurators to rapidly identify literature for further curation based on possible GO terms. Because most agricultural species do not have standardized literature, eGIFT searches all gene names and synonyms to associate articles with genes. As many of the gene names can be ambiguous, eGIFT applies a disambiguation step to remove matches that do not correspond to this gene, and filtering is applied to remove abstracts that mention a gene in passing. The BI is linked to our Journal Database (JDB) where corresponding journal citations are stored. Just as importantly, biocurators also add to the JDB citations that have no GO annotation. The AgBase BI also supports bulk annotation upload to facilitate our Inferred from electronic annotation of agricultural gene products. All annotations must pass standard GO Consortium quality checking before release in AgBase.
Database URL: http://www.agbase.msstate.edu/
Another approach to biocuration. I will be posting on eGift separately but do note this is a domain specific tool.
The authors did not set out to create the universal curation tool but one suited to their specific data and requirements.
I think there is an important lesson here for semantic authoring interfaces. Word processors offer very generic interfaces but consequently little in the way of structure. Authoring annotated information requires more structure and that requires domain specifics.
Now there is an idea, create topic map authoring interfaces on top of a common skeleton, instead of hard coding interfaces as users “should” use the tool.
Posted in Bioinformatics, Biomedical, Curation, Genomics, Text Mining | No Comments »
Monday, November 19th, 2012
Accelerating literature curation with text-mining tools: a case study of using PubTator to curate genes in PubMed abstracts by Chih-Hsuan Wei, Bethany R. Harris, Donghui Li, Tanya Z. Berardini, Eva Huala, Hung-Yu Kao and Zhiyong Lu.
Abstract:
Today’s biomedical research has become heavily dependent on access to the biological knowledge encoded in expert curated biological databases. As the volume of biological literature grows rapidly, it becomes increasingly difficult for biocurators to keep up with the literature because manual curation is an expensive and time-consuming endeavour. Past research has suggested that computer-assisted curation can improve efficiency, but few text-mining systems have been formally evaluated in this regard. Through participation in the interactive text-mining track of the BioCreative 2012 workshop, we developed PubTator, a PubMed-like system that assists with two specific human curation tasks: document triage and bioconcept annotation. On the basis of evaluation results from two external user groups, we find that the accuracy of PubTator-assisted curation is comparable with that of manual curation and that PubTator can significantly increase human curatorial speed. These encouraging findings warrant further investigation with a larger number of publications to be annotated.
Database URL: http://www.ncbi.nlm.nih.gov/CBBresearch/Lu/Demo/PubTator/
Presentation on PubTator (slides, PDF).
Hmmm, curating abstracts. That sounds like annotating subjects in documents doesn’t it? Or something very close. 
If we start off with a set of subjects, that eases topic map authoring because users are assisted by automatic creation of topic map machinery. Creation triggered by identification of subjects and associations.
Users don’t have to start with bare ground to build a topic map.
Clever users build (and sell) forms, frames, components and modules that serve as the scaffolding for other topic maps.
Posted in Bioinformatics, Curation, Literature, Text Mining | No Comments »
Sunday, November 18th, 2012
Opportunities for text mining in the FlyBase genetic literature curation workflow by Peter McQuilton. (Database (2012) 2012 : bas039 doi: 10.1093/database/bas039)
Abstract:
FlyBase is the model organism database for Drosophila genetic and genomic information. Over the last 20 years, FlyBase has had to adapt and change to keep abreast of advances in biology and database design. We are continually looking for ways to improve curation efficiency and efficacy. Genetic literature curation focuses on the extraction of genetic entities (e.g. genes, mutant alleles, transgenic constructs) and their associated phenotypes and Gene Ontology terms from the published literature. Over 2000 Drosophila research articles are now published every year. These articles are becoming ever more data-rich and there is a growing need for text mining to shoulder some of the burden of paper triage and data extraction. In this article, we describe our curation workflow, along with some of the problems and bottlenecks therein, and highlight the opportunities for text mining. We do so in the hope of encouraging the BioCreative community to help us to develop effective methods to mine this torrent of information.
Database URL: http://flybase.org
Would you believe that ambiguity is problem #1 and describing relationships is another one?
The most common problem encountered during curation is an ambiguous genetic entity (gene, mutant allele, transgene, etc.). This situation can arise when no unique identifier (such as a FlyBase gene identifier (FBgn) or a computed gene (CG) number for genes), or an accurate and explicit reference for a mutant or transgenic line is given. Ambiguity is a particular problem when a generic symbol/ name is used (e.g. ‘Actin’ or UAS-Notch), or when a symbol/ name is used that is a synonym for a different entity (e.g. ‘ras’ is the current FlyBase symbol for the ‘raspberry’ gene, FBgn0003204, but is often used in the literature to refer to the ‘Ras85D’ gene, FBgn0003205). A further issue is that some symbols only differ in case-sensitivity for the first character, for example, the genes symbols ‘dl’ (dorsal) and ‘Dl’ (Delta). These ambiguities can usually be resolved by searching for associated details about the entity in the article (e.g. the use of a specific mutant allele can identify the gene being discussed) or by consulting the supplemental information for additional details. Sometimes we have to do some analysis ourselves, such as performing a BLAST search using any sequence data present in the article or supplementary files or executing an in-house script to report those entities used by a specified author in previously curated articles. As a final step, if we cannot resolve a problem, we email the corresponding author for clarification. If the ambiguity still cannot be resolved, then a curator will either associate a generic/unspecified entry for that entity with the article, or else omit the entity and add a (non-public) note to the curation record explaining the situation, with the hope that future publications will resolve the issue.
One of the more esoteric problems found in curation is the fact that multiple relationships exist between the curated data types. For example, the ‘dppEP2232 allele’ is caused by the ‘P{EP}dppEP2232 insertion’ and disrupts the ‘dpp gene’. This can cause problems for text-mining assisted curation, as the data can be attributed to the wrong object due to sentence structure or the requirement of back- ground or contextual knowledge found in other parts of the article. In cases like this, detailed knowledge of the FlyBase proforma and curation rules, as well as a good knowledge of Drosophila biology, is necessary to ensure the correct proforma field is filled in. This is one of the reasons why we believe text-mining methods will assist manual curation rather than replace it in the near term.
I like the “manual curation” line. Curation is a task best performed by a sentient being.
Posted in Curation, Genomics, Text Mining | No Comments »
Sunday, November 4th, 2012
Manual Gene Ontology annotation workflow at the Mouse Genome Informatics Database by Harold J. Drabkin, Judith A. Blake and for the Mouse Genome Informatics Database. Database (2012) 2012 : bas045 doi: 10.1093/database/bas045.
Abstract:
The Mouse Genome Database, the Gene Expression Database and the Mouse Tumor Biology database are integrated components of the Mouse Genome Informatics (MGI) resource (http://www.informatics.jax.org). The MGI system presents both a consensus view and an experimental view of the knowledge concerning the genetics and genomics of the laboratory mouse. From genotype to phenotype, this information resource integrates information about genes, sequences, maps, expression analyses, alleles, strains and mutant phenotypes. Comparative mammalian data are also presented particularly in regards to the use of the mouse as a model for the investigation of molecular and genetic components of human diseases. These data are collected from literature curation as well as downloads of large datasets (SwissProt, LocusLink, etc.). MGI is one of the founding members of the Gene Ontology (GO) and uses the GO for functional annotation of genes. Here, we discuss the workflow associated with manual GO annotation at MGI, from literature collection to display of the annotations. Peer-reviewed literature is collected mostly from a set of journals available electronically. Selected articles are entered into a master bibliography and indexed to one of eight areas of interest such as ‘GO’ or ‘homology’ or ‘phenotype’. Each article is then either indexed to a gene already contained in the database or funneled through a separate nomenclature database to add genes. The master bibliography and associated indexing provide information for various curator-reports such as ‘papers selected for GO that refer to genes with NO GO annotation’. Once indexed, curators who have expertise in appropriate disciplines enter pertinent information. MGI makes use of several controlled vocabularies that ensure uniform data encoding, enable robust analysis and support the construction of complex queries. These vocabularies range from pick-lists to structured vocabularies such as the GO. All data associations are supported with statements of evidence as well as access to source publications.
Semantic uniformity is achievable, in a limited enough sphere, provided you are willing to pay the price for it.
It has a high rate of return over less carefully curated content.
The project is producing high quality results, although hampered by a lack of resources.
My question is whether a similar high quality of results could be achieved with less semantically consistent curation by distributed contributors?
Harnessing the community of those interested in such a resource. And refining those less semantically consistent entries into higher quality annotations.
Pointers to examples of such projects?
Posted in Annotation, Bioinformatics, Curation, Ontology | No Comments »
Tuesday, October 16th, 2012
Data Curation in the Networked Humanities by Michael Ullyot.
From the post:
These talks are the first phase of Encoding Shakespeare: my SSHRC-funded project for the next three years. Between now and 2015, I’m working to improve the automated encoding of early modern English texts, to enable text analysis.
This post’s three parts are brought to you by the letter p. First I outline the potential of algorithmic text analysis; then the problem of messy data; and finally the protocols for a networked-humanities data curation system.
This third part is the most tentative, as of this writing; Fall 2012 is about defining my protocols and identifying which tags the most text-analysis engines require for the best results — whatever that entails. (So I welcome your comments and resource links.)
A project that promises to touch on many of the issues in modern digital humanities. Do review and contribute if possible.
I have a lingering uneasiness with the notion of “data curation.” With the data and not curation part.
To say “data curation” implies we can identify the “data” that merits curation.
I don’t doubt we can identify some data that needs curation. The question being is it the only data that merits curation?
We know from the early textual history of the Bible that the text was curated and in that process, variant traditions and entire works were lost.
Just my take on it but rather than “data curation,” with the implication of a “correct” text, we need semantic curation.
Semantic curation attempts to preserve the semantics we see in a text, without attempting to find the correct semantics.
Posted in Curation, Humanities, Literature | No Comments »
Wednesday, July 25th, 2012
The Case for Curation: The Relevance of Digest and Citator Results in Westlaw and Lexis by Susan Nevelow Mart and Jeffrey Luftig.
Abstract:
Humans and machines are both involved in the creation of legal research resources. For legal information retrieval systems, the human-curated finding aid is being overtaken by the computer algorithm. But human-curated finding aids still exist. One of them is the West Key Number system. The Key Number system’s headnote classification of case law, started back in the nineteenth century, was and is the creation of humans. The retrospective headnote classification of the cases in Lexis’s case databases, started in 1999, was created primarily although not exclusively with computer algorithms. So how do these two very different systems deal with a similar headnote from the same case, when they link the headnote to the digesting and citator functions in their respective databases? This paper continues an investigation into this question, looking at the relevance of results from digest and citator search run on matching headnotes in ninety important federal and state cases, to see how each performs. For digests, where the results are curated – where a human has made a judgment about the meaning of a case and placed it in a classification system – humans still have an advantage. For citators, where algorithm is battling algorithm to find relevant results, it is a matter of the better algorithm winning. But no one algorithm is doing a very good job of finding all the relevant results; the overlap between the two citator systems is not that large. The lesson for researchers: know how your legal research system was created, what involvement, if any, humans had in the curation of the system, and what a researcher can and cannot expect from the system you are using.
A must read for library students and legal researchers.
For legal research, the authors conclude:
The intervention of humans as curators in online environments is being recognized as a way to add value to an algorithm’s results, in legal research tools as well as web-based applications in other areas. Humans still have an edge in predicting which cases are relevant. And the intersection of human curation and algorithmically-generated data sets is already well underway. More curation will improve the quality of results in legal research tools, and most particularly can be used to address the algorithmic deficit that still seems to exist where analogic reasoning is needed. So for legal research, there is a case for curation. [footnotes omitted]
The distinction between curation, human gathering of relevant material and aggregation, machine gathering of potentially relevant material looks quite useful.
Curation anyone?
I first saw this at Legal Informatics.
Posted in Aggregation, Curation, Legal Informatics, LexisNexis, Westlaw | No Comments »
Wednesday, June 13th, 2012
Network of data visualization references by Nathan Yau.
From the post:
Developer Santiago Ortiz places Delicious tags for visualization references in a discovery context. There are two views. The first is a network view with tags and resources as nodes. A fisheye effect is used to zoom in on nodes and make the more readable. Mouse over a tag, and the labels for related resources get bigger, and likewise, mouse over a resource, and the related tags get bigger.
The second view lets you compare resources. In the network view, select two ore more resources, and then click on the bottom button to compare the selected.
On the left hand side, top, you will see:
- blogs
- studios
- people
- tools
- books
I had to select one of those before getting the option to switch to the second view.
The graph view seems to move too quickly but that may just be me.
I am sure there is a “big data” view of visualization but I find this more limited view quite useful.
As a matter of fact, I suspect finding sub-communities that share semantics is going to be more of a growth area than “big data.” To be sure, you may start with “big data” but you will quickly boil it down to “small data” that is both useful and relevant to your user community.
Small enough for machine-assisted curation no doubt. Where the curation is the value-add resulting in a product.
Posted in BigData, Curation, Graphics, Visualization | No Comments »
Wednesday, May 9th, 2012
GATE Teamware: Collaborative Annotation Factories
From the webpage:
Teamware is a web-based management platform for collaborative annotation & curation. It is a cost-effective environment for annotation and curation projects, enabling you to harness a broadly distributed workforce and monitor progress & results remotely in real time.
It’s also very easy to use. A new project can be up and running in less than five minutes. (As far as we know, there is nothing else like it in this field.)
GATE Teamware delivers a multi-function user interface over the Internet for viewing, adding and editing text annotations. The web-based management interface allows for project set-up, tracking, and management:
- Loading document collections (a “corpus” or “corpora”)
- Creating re-usable project templates
- Initiating projects based on templates
- Assigning project roles to specific users
- Monitoring progress and various project statistics in real time
- Reporting of project status, annotator activity and statistics
- Applying GATE-based processing routines (automatic annotations or post-annotation processing)
I have known about the GATE project in general for years and came to this site after reading: Crowdsourced Legal Case Annotation.
Could be the basis for annotations that are converted into a topic map, but…, I have been a sysadmin before. Maintaining servers, websites, software, etc. Great work, interesting work, but not what I want to be doing now.
Then I read:
Where to get it? The easiest way to get started is to buy a ready-to-run Teamware virtual server from GATECloud.net.
Not saying it will or won’t meet your particular needs, but, certainly is worth a “look see.”
Let me know if you take the plunge!
Posted in Annotation, Cloud Computing, Collaboration, Corpora, Corpus Linguistics, Curation, Entity Resolution, Linguistics | No Comments »
Wednesday, April 13th, 2011
7th international Digital Curation Conference
Call for papers where topics include:
- Lessons learned from the inter-disciplinary use of open data: examples of enablers, barriers and success stories
- Curation of mixed data collections, with open and sensitive or private content
- Gathering evidence for benefits of data sharing
- Building capacity for the effective management, sharing and reuse of open data
- Scale issues in the management of sensitive data
- Tensions between maintaining quality and openness
- Linked data, open data, closed data and provenance
- Technical and organisational solutions for data security
- Developing new metrics for open data
- Ethical issues and personal data
- Legislation and open data
Submission deadline: 25 July 2011
Conference:
5 – 7 December 2011
Marriott Royal Hotel, Bristol, UK
Posted in Conferences, Curation | No Comments »