Principle #5 scope - automated validation

[beckyjackson]

FP 5 - Scope

Automated checks:

1. Does the ontology have a scope (domain)?
2. Is it orthogonal to other OBO Foundry ontologies?

Mechanism:

Some ontologies use the domain property in the YAML. If the value is missing from the YAML, this check fails.

Checking orthogonality will be a bit more difficult. Some of the domains are not very specific (e.g., VO has domain: health). We can provide an info message for now if the domain overlaps with another ontology's domain. Unless we have a CV for domain, though, it will be hard to implement.

A scope/domain annotation property for the ontology header would be good to have as well. Possible in IAO?

[nataled]

From the EWG discussion on this:

Check to see if term labels reproduce those found in other ontologies without using original IDs. BioPortal mapping does this. Main idea here is to determine if the scope is too wide. This would be a quick first pass to find potential issues, and will need supplemental manual checking. Do the terms stay within the stated scope of the ontology?

Related issue: #547 (wherein links to ontologies 'domain' fail)

[beckyjackson]

This would be an interesting comparison @nataled - do you think this would be appropriate as part of the dashboard, or perhaps through another report? We've got a lot of information in the dashboard already, and I wouldn't want anything to get lost. I'd personally rather see this as a different report, but I could be convinced otherwise.

[nataled]

I think we'd need to figure out how to do this by hand, just to get an idea of how it should (or if it could) work. To my knowledge we have yet to try it for an ontology review. The Bioportal mapping will only show external ontologies with the same label. We were thinking that if the ontology being reviewed had minted terms that are also found in other ontologies, there could be at least an overlap of terms, and possibly a too-broad scope.

I'm not yet familiar with the ins and outs of ROBOT to be able to answer the question of dashboard vs other report. Perhaps others will have an opinion. Personally, I favor approaches that make things clear over those that capture everything in one place, if those are the choices.

[jamesaoverton]

@nataled This is a good idea. ROBOT only works with one or a few ontologies at a time, and to check this properly we'd have to have all the OBO terms in some sort of database or triplestore. We should do that eventually, and it will be useful for lots of other things. I haven't worked with Bioportal mapping, but I doubt that they'd appreciate us requesting mappings for tens-of-thousands of terms at a time.

[nataled]

If I'm not mistaken, BioPortal just does it. I looked into it once upon a time (when looking at a new ontology), and did find it helpful. Not sure how automated it could be as presented. Here is the relevant mappings for OBI: http://bioportal.bioontology.org/ontologies/OBI/?p=mappings. As you will see, it is done on an ontology-by-ontology basis.

[dosumis]

Current implementation should be modified to take taxon into account (as specified in ontology metadata). It is currently incorrectly flagging ontologies as having overlapping scope where they apply to non-overlapping taxons.

[matentzn]

Reading this: Registry ontologies should cover non-overlapping domains from here. I thought the whole point of "domain" was to group related ontologies; all disease, or phenotype ontologies.. Now, from the "i want my ontology to be totally green on the dashboard"-perspective, it becomes more of an exercise in "changing the metadata so that it sounds like the scope is unique", or am I mistaken? Is their any chance to re-evaluate the purpose of the domain: or has this ship sailed?

[jamesaoverton]

You're right @matentzn. I underestimated the power of the checkmark.

At this point, I think it's valuable to have a small number of broad categories such as "disease", "phenotype", and "experiment" that let us group ontologies. If a blue "i" discourages this, then we should revise the test.

Longer term, we're hoping that the top-level term(s) of each ontology will cleanly point to OBO Core. Then we'll have a hierarchy that we can use to distinguish scopes.

Medium term, we could use the taxon hierarchy to distinguish scopes for many ontologies.

@beckyjackson: Please drop the orthogonality test until we can do better.