We have just released a new version of the synthetic tree, along with new versions of the APIs.
The biggest change in this version is that we have completely replaced the synthesis method used to produce the tree. We are still using neo4j to serve the tree, but have moved synthesis out of the graph database and into a make-based pipeline that uses a C++ library. This new method is improves efficiency, reproducibility, and allows us to more clearly connect input sources with edges in the tree. In addition to support statements, the new pipeline also produces conflict statements about the inputs that do not support a given edge (we are working to get these displayed on the properties panel for each node).
You can view the new version here and read the release notes. We want to particularly highlight the self-documenting nature of the new method. Primary credit for the new method goes to Mark Holder and Ben Redelings – paper coming soon!
The largest change is the node IDs. We have previously mentioned issues with node stability, and in this version of the tree & APIs, we use either Open Tree Taxonomy IDs for taxa nodes or mrca statements for non-taxa nodes rather than unstable neo4j node IDs which will transfer (or fail gracefully) for new versions of the tree. We have also made input and output parameters more stable across methods.
We also make public the verbose subtree format that we use to build the tree browser – rather than simply a newick string, you can obtain the tree with all provenance information, including support and conflict.
All v2 methods should continue to work, but we plan to deprecate the v2 methods in June 2016.
OpenTree, FuturePhy and Arbor jointly held the first round of clade workshops in Gainesville at the end of February. There were three taxon-focused groups taking part, studying barnacles, beetles, and catfish – each with a very diverse set of participants. Expertise in the room included taxonomy, systematics, ecology, phylogenetic methods, bioinformatics, genomes, ontologies, and scientific illustration (to name a few). While each group had different goals for progress in understanding the biology of their taxon of interest, each group required a unified tree merging taxonomic and phylogenetic information for their clade. Sounds like a job for OpenTree! In advance of the workshop, we created tree collections (ranked lists of published trees in OpenTree) for each clade, and completed the beta version of our new synthesis algorithm. While there was only limited curation of new studies in the lead up to the meeting, during the workshop participants imported more than 40 new published phylogenies into the OpenTree database and curated tree collections. Not only will this burst of skilled curation improve accuracy of the synthetic tree in the future, we were able to use our new rapid synthesis method to produce on-the-fly custom synthesis trees for each clade collection during the workshop. By reviewing these synthetic trees and updating the input trees and rankings, participant groups were able to simultaneously achieve a better understanding of the relationships in their clade of interest, and of the OpenTree synthesis procedure. These clade synthetic trees were an efficient and reproducible methods for providing a unified view of taxon relationships which could then be compared to publications and to expert-curated supertrees produced by grafting existing trees.
On the last day, we asked each group to list the top features they want from OpenTree. This list included:
- Better conflict visualization – between published input trees, the synthetic tree of life, and the Open Tree Taxonomy
- Ways to summarize / visualize the annotations file created along with the synthetic tree (this file includes information about sources that support & conflict with each edge).
- Provide a method for proposing new taxa for tips in a phylogeny that cannot be mapped to OTT (and for collecting supporting information about these new taxa)
- Include branch length and / or time information in the synthetic tree
- More fine-grained control over synthesis (be able to mask part of an input tree, suppress poorly-supported branches)
Many thanks to all of the participants, and in particular to Nico Cellinese and Rob Guralnick for local logistics, and to the University of Florida Informatics Institute for hosting. It was an enjoyable and productive meeting for the OpenTree crew, and hopefully for all the attendees!
We are excited to publish the first draft of the Open Tree of Life in PNAS:
Scientists have used gene sequences and morphological data to construct tens of thousands of evolutionary trees that describe the evolutionary history of animals, plants, and microbes. This study is the first, to our knowledge, to apply an efficient and automated process for assembling published trees into a complete tree of life. This tree and the underlying data are available to browse and download from the Internet, facilitating subsequent analyses that require evolutionary trees. The tree can be easily updated with newly published data. Our analysis of coverage not only reveals gaps in sampling and naming biodiversity but also further demonstrates that most published phylogenies are not available in digital formats that can be summarized into a tree of life.
This is only a first draft, and there are plenty of places where the tree does not represent what we know about phylogenetic relationships. We can improve this tree through incorporation of new taxonomic and phylogenetic data. Our data store of trees (which contains many more trees than are included in the draft tree of life) is also a resource for other analyses. If you want to contribute a published tree for synthesis (or for analyses of coverage, conflict, etc), you can upload it through our curation interface.
Other pages and links:
- supplemental doc with details about methods
- Dryad data package when you can download the taxonomy and the tree
- infographics about the tree of life
- interactive tree browser
- roundup of news coverage
Many thanks to all of the people that provided data, discussion, review, curation, and code and of course to NSF Biology for funding this work!
Currently, OpenTree has two different types of node IDs. Taxonomy (OTT) IDs are assigned to named nodes when we construct a taxonomy release, and phylogenetic node IDs are assigned by the treemachine neo4j graph database for nodes that do not align to an OTT ID (i.e. nodes added due to phylogenetic resolution). The OTT IDs are fairly stable over time, but the neo4j node IDs are definitely not stable, and the same neo4j ID may point to a completely unrelated node in future versions of the graph.
This system is problematic because we expose both types of IDs in the APIs (and also in URLs for the tree browser). The lack of neo4j node stability therefore affects API calls that use nodeIDs, browser bookmarks to nodes in the synthetic tree, and feedback left by users about specific nodes in the tree (see feedback issue #63 and treemachine issue #183). The OTT IDs are problematic as well: it is not straightforward to document when we reuse an existing OTT ID, mint a new ID, or delete an existing ID, when going from one version of the taxonomy version to the next.
At our recent face-to-face meeting, we discussed a proposal for a node identifier registry and are looking for feedback. We don’t intend this system to be a universally-used set of node definitions (i.e. we aren’t trying making a PhyloCode registry). We want a lightweight system that prevents exposure of unstable nodeIDs through the APIs to clients (including our own web application) and provides some measure of predictability. Feeedback on this proposal would be greatly appreciated.
- be able to use the same node ID definitions across OTT and the synthetic tree
- transparency about when we re-use a nodeID from a previous version of tree or taxonomy (or not)
- users get an error when using a node ID from a previous version where there is no current node that fits that definition
- fixing errors (such as moving a snail found in a worm taxon to its proper location) should not involve massive numbers of ID changes
- generation of node definitions based on a given taxonomy must be automated and efficient
- application of node definitions to an existing tree / taxonomy must be automated and efficient
Develop a lightweight registry of node definitions based on the structure of the OpenTree taxonomy. For each new version of the taxonomy and synthetic tree, use the registry to decide when to re-use existing node IDs and when to register a new definition + ID.
Leaf nodes will be assigned IDs during creation of OTT based on name (together with enough taxonomic context to separate homonyms).
The definition of the ID for a non-leaf node will include a list of IDs for nodes that are descendents of the intended clade, a list that are excluded from being descendents, and (optionally) a taxonomic name.
Definitions would never be deleted from the registry, although not all definitions will be used in any given tree / taxonomy.
- How many descendant and excluded nodes to include in the definitions: The definition needs some specificity but also can’t assume a complete list due to future addition of new species. Perhaps, for example, four descendants and three exclusions would be a decent compromise between one and thousands?
- How to choose the specific nodes in the lists of descendants and exclusions: Should be ‘popular’ (should occur in as many sources as possible) and informative (if T has children T1 and T2 then at least one definition descendant should be taken from T1, and at least one from T2). Excluded nodes should be ‘near misses’ rather than arbitrarily chosen.
- What to do when >1 node meets the definition: Add an option of adding constraints to the registered definition in order to remove the ambiguity while preserving the ID.
- What to do when >1 definition matches a node: Ambiguous assignments can be resolved either by the addition of constraints, or by the creation of new ids.
- Modification / versioning of definitions: If we add constraints to a definition (for example, to resolve ambiguity), does this mint a new ID or version the existing definition?
The challenges to completing the Tree of Life and integrating data layers (NSF GoLife goals) are huge and vary across clades. Some groups have a nearly-complete tree but lack publicly available data layers, whereas other groups lack phylogenetic resolution or the resources to support tree / data integration. Partnering with Open Tree of Life and Arbor Workflows, FuturePhy will support a series of clade-based workshops to identify and solve specific challenges in tree of life synthesis and data layer integration.
RFP: 2 page proposals to fund small workshops and/or hackathons on completing the tree of life and integrating data layers for specific clades.
Proposal deadline: Nov. 1, 2015
Meeting dates: Feb 20-23
26-28, 2016 *note changed dates!*
Location: Gainesville, University of Florida
Participants per workshop: 10 maximum funded (virtual attendees possible)
Contacts: email@example.com (FuturePhy), firstname.lastname@example.org (OpenTree), email@example.com (Arbor)
The full call for participation and a link to a proposal template is available at the FuturePhy website.
Have questions about this or future workshops? Attend our webinar Thursday, September 17 at 1 pm EDT. See details on how to connect.
A little known side element to the Open Tree of Life project is the “Edu Tree of Life,” an interactive educational experience to engage the public. Nearing completion, our goal with this website has been to educate young students as well as the general public on topics surrounding evolution and phylogenetic trees. Our approach is to visually inform and engage users with colorful and entertaining animation, interactive features, and contextualization of facts and figures.
Our educational site is composed of three unique, interactive views of the ToL:
1) A “Big Picture” tree provides a zoomed-out timeline perspective of life’s history on earth and explains key elements of the tree of life using a stylized, graphic visualization. This ‘macro’ view presents the evolutionary history of Earth, starting from the creation of our planet and spanning all the way to present day. As the user moves up the timeline, the tree ‘grows’ in front of them revealing historical information; each new screen also offers a detailed explanation of one of several core concepts surrounding evolution. Video explanations containing animations live narrators explain each of these core concepts. Along with the videos, ‘pop-up’ information boxes also offer information.
- A macro View
- Key Concepts
- Timeline of Life
- Chaptered Format/Parallax Scrolling
- Videos and Animation
The core concepts we explore are:
- The Origin of Life
- The Three Domains of Life
- Common Ancestors
- Lateral/Horizontal Gene Transfer & Genes
2) The page titled ““Categorizing Life on Earth” is a mid-sized view of life, a data-driven interactive tree with a focus on the groupings of species (clades). This Tree uses a sampling of data to illustrate hierarchy with a familiar ‘tree’ structure that employs branching lines of evolution. It pulls images from Phylopic and data from EOL for descriptions. A user can expand and contract nodes to view clades they find interesting. Still to come: we are exploring ways to illustrate LGT and are working on connecting nodes back via their common ancestry, so that clicking any two nodes will show you a visualization of how those species are connected through the whole tree of life.
- Mid-sized view of major clades
- Data-driven interactive
- Shows Common Ancestry, Phylogeny and Clades
- Species groupings ending in Clades
3) The “Explore Species” page is our ‘micro view’ of species on Earth. This interactive spinning wheel allows a user to select any of about 180 species to learn about. The 180 species were chosen as exemplary based on many factors: some were chosen for their relative familiarity with the general public, but many were chosen due to specific scientific breakthroughs associated with them. Many were the first species within their field of study to be gene-sequenced, some are keystone species with important evolutionary relatives, and others have strange or unique characteristics worthy of mention.
The information offered for each species includes an image (when available), scientific and common names, the major domain within which the species resides, and then a brief description of the species. This was achieved using the Encyclopedia of Life’s online API, which allowed us to pull information and other resources off of their site to show on ours. As a way of opening an educational portal between the two, any species you click on in the Wheel of Life can also be visited on its parent page at the Encyclopedia of Life, where much more information about all species can be found. We hope that this partnership will prove very fruitful for bringing in casual interest and turning it into a burning passion for evolutionary science and history. Even if we only end up with a few more zoologists, we’ll be happy.
- Micro View
- Exemplary/representative species
- Connects to EoL API, a gateway for further learning
- Catalogue of interesting species.
- Some info on Major Domains.
- Fun, introductory look into species and their connections.
We welcome your comments. —John Allison and Karl Gude
This is the first in a series of posts about several phylogeny initiatives newly-funded by NSF focused on both technical and community aspects of phylogeny. Plenty of potential for mutually beneficial work with OpenTree, and we are excited to help.
First up… FuturePhy!
FuturePhy is an NSF-sponsored, three-year program of conferences, workshops and hackathons on the Tree of Life. The project aims to promote novel, integrative data analyses and visualization, interdisciplinary syntheses of phylogenetic sciences, and cross-cutting uses of phylogenetics to develop and address new research questions and applications.
The first phase of this mission is critical: to bring together a broad community of people from diverse backgrounds who are active in phylogenetics research, who use the tree of life in research or education, who will benefit in applied or practical ways from a comprehensive tree of life, or who come from a background that offers new perspectives on defining, addressing or transcending key challenges in phylogenetics.
Help accelerate progress in all aspects of phylogenetics research by joining FuturePhy today. Diverse opportunities will be available to attend FuturePhy sessions in person or virtually, and to link FuturePhy to existing projects and initiatives.
- We invite you to participate in the project in several ways:
Register on futurephy.org. Scientists from all aspects of the phylogenetic sciences, educators, members of the tree-using community, and others interested in phylogenetics are welcome.
- Take the community survey and let FuturePhy what workshop and hackathon topics they should fund.
- Contribute to the discussion forum on futurephy.org. This is the best way to log your interest and contribute ideas.
- Send email at firstname.lastname@example.org with ideas or comments
- Tweet to the FuturePhy community: @FuturePhy
- Comment in the FuturePhy phylobabble thread