Changes

Page history
Wiki describing the snapshot of the Ontology Debugger at Tag... authored by Wolfgang Schmid's avatar Wolfgang Schmid 
Wiki describing the snapshot of the Ontology Debugger at Tag https://git-ainf.aau.at/interactive-KB-debugging/debugger/commits/v_1_0_0_SNAPSHOT_20160628
Hide whitespace changes
Inline Side-by-side
onto-debugger.md
View page @ 0f850ebf
# Protégé Debugger Plug-In # Ontology Debugger Plug-In for Protégé
# IMPORTANT NOTE: # IMPORTANT NOTE:
This plug-in is still in development! Very probably you will experience faulty and unexpected behaviour such as deadlocks, etc. Please use our issue tracking system to report any bugs you observe to us. We will try to remove them as soon as possible. Thanks! This plug-in is still in development! Very probably you may experience faulty and unexpected behaviour such as deadlocks, exceptions etc. You may report any bugs you experience in our [issue tracking system](https://git-ainf.aau.at/interactive-KB-debugging/debugger/issues). We will try to fix them as soon as possible. Thanks!
# Installation # Installation
These steps are necessary in order to run the plug-in: These steps are necessary in order to run the plug-in:
* Download the most recent version of Protégé Desktop from [http://protege.stanford.edu/](http://protege.stanford.edu/) and follow the installation instructions. Note that the Debugger Plug-In is not compatible with Protégé version 4 and below. * Download the most recent version of Protégé Desktop from [http://protege.stanford.edu/](http://protege.stanford.edu/) and follow the installation instructions. Note that the Debugger Plug-In is not compatible with Protégé version 4 and below.
* Download the most recent [Protégé Debugger Plug-In](/uploads/14b0041bfa33146eb10283d795b745de/org.exquisite.protege-1.0-SNAPSHOT.jar) and copy it into the plug-ins subfolder of your Protégé 5 desktop client. If your Protégé client is running, please restart it to load the Plug-In. You should see now a new tab called Exquisite Debugger! * Download the most recent [Protégé Ontology Debugger Plug-In](/uploads/c48430be5a83d1a6763f733451b4c3c1/org.exquisite.protege-1.0-SNAPSHOT.jar) and copy it into the ```plugins``` subfolder of your Protégé 5 desktop client. If your Protégé client is already running, then you have to restart the client to load the plug-In. You should see now a new menu entry named ```Ontology Debugger``` as shown here: ![ExquisiteDebuggerTab](/uploads/36b3ac39574518abc3b27b8e8f15fe47/ExquisiteDebuggerTab.PNG)
#### Solver plug-ins are required # How to use the Ontology Debugger Plug-In in Protégé
Our plug-in requires at least one reasoner. Protege 5 already comes with a pre-installed HermiT solver, but you can also install other reasoners like FaCT++ or Pellet. Installing additional reasoner plug-ins in Protégé is done by selecting *File->Check for plug-ins...*. After installing additional reasoner plug-ins you will have to restart Protégé. To demonstrate the features of our Debugger Plug-In, let us load the Koala ontology. Select ```File->Open from URL... ``` and enter the URL http://protege.stanford.edu/ontologies/koala.owl.
![ExquisiteDebuggerTab](/uploads/a25cb9d39e0523e0b9ddfdd83928e50e/ExquisiteDebuggerTab.PNG) ### The Ontology Debugger
# How to use the Protégé Debugger Plug-In Once you have loaded the *Koala ontology*, you can open the *Ontology Debugger* by selecting ```Ontology Debugger->Open Ontology Debugger Tab``` in the menu. You may also activate the tab by selecting it in ```Window->Tabs```. The initial layout of the *Ontology Debugger* should look similar to this:
To demonstrate the features of our Debugger Plug-In, let us load the Koala ontology (*File->Open from URL... * and select http://protege.stanford.edu/ontologies/koala.owl)!
Once the ontology is loaded you can open the *Exquisite Ontology Debug Tab* (*Exquisite Debugger -> Open Exquisite Ontology Debug Tab*). ![debugger](/uploads/c76aeb8c4b28b0842b9062c37efc52d3/debugger.PNG)
#### The Exquisite Ontology Debugging Tab The Ontology Debugger can be seen as divided into three main sections where each one is responsible for the presentation and manipulation of different kind of data during the debugging session of inconsistent and/or incoherent ontologies. In the following we will describe the sections and their meaning.
The Exquisite Ontology Debugging Tab is split up into four differen views: the diagnoses view which enables the user to calculate and view the diagnoses of the current diagnoses model. A conflicts view that lists the minimal conflict set. A view for the diagnoses model and one view for the testcases. These two views consist of an explicit declaration of axioms (the _diagnoses model_) that are possibly faulty (we name them the _knowledge base_), axioms which are correct (or the _background knowledge_) as well as entailed and not entailed axioms, which must or must not be concluded from the diagnoses model. Additionally we also list positive and negative test cases. ##### Diagnosesmodel / Testcases
![debugger](/uploads/0379f2d5d842519e812a506ca5d1c2bd/debugger.PNG) The left section is responsible to present the the currently used __Diagnosismodel__ consisting of __Correct Axioms__ (also called the _Background Knowledge_) which represent axioms that are asserted by the expert to be correct, then the __Possibly Faulty Axioms__ (the _Knowledge Base_ or _KB_) which represents the set of axioms that might be erroneous and thus be the causing for the inconsistencies / incoherencies.
Please note currently when loading an ontology the debugger automatically assigns simple axioms to the list of correct axioms (also called background knowledge). In the separate __Testcases__-Tab the test cases from a debugging session are shown. There are two possible kinds of test cases: _Entailed Testcases_ represent test cases that must be entailed in the active ontology and _Non Entailed Testcases_ represent test cases that must not be entailed in the currently active ontology. In the following image we see two test cases that have been answered by an experted which have to be entailed in the _Koala_ ontology.
# Diagnoses ![testcases](/uploads/b10cd0a725fc5f874d8630de49109423/testcases.PNG)
Next to the overview of the diagnoses model the Exquisite Ontology Debugging Tab's main functionality is to search for diagnoses.
Before we do calculate a set of diagnoses for the loaded ontology, we first have to configure which reasoner we want to use for the diagnoses search. Therefore, select *Reasoner* and select a preferred reasoner. For this example we are using the HermiT reasoner. ##### Diagnoses / Conflicts
#### Options for diagnoses The mid-section shows us the diagnoses and conflicts that are calculated during an ontology debugging session. The debugging session itself can be started by clicking on the __Start Debugging__ button. Once a debugging session is started, diagnoses are calculated and one or more queries are computed, which the expert has to answer in order to find the error in the ontology. Each set of diagnoses are based on a minimal conflicts set which will be shown in the __Conflicts__ tab.
After selction of an appropriate reasoner, let us see what option we do have for the diagnoses search.
Select *Exquisite Debugger->Options*.
![options_diagnoses](/uploads/cfcaf35eade5adf9674103113493aebe/options_diagnoses.PNG) Before we can start a debugging session we have to select a reasoner. Protégé 5 already comes with a pre-installed HermiT solver, but you can also install other reasoners like FaCT++ or Pellet. Installing additional reasoner plug-ins in Protégé is done by selecting ```File->Check for plug-ins...```. Protégé has to be restarted once you have installed additional reasoner plug-ins.
* As diagnoses engine we can choose either between Inv-QuickXPlain, HS-Tree (default) and HS-DAG. The first algorithm computes diagnoses directly, i.e. without computation of minimal conflicts, and must be used if you want to find only a few diagnoses very fast. The other two algorithms are based on computation of conflicts and are preferable if you want to use the interactive version of our debugger. If you start a debugging session (__Start Debugging__) using he HermiT solver and given the default settings you will probably see as set of such diagnoses: ![diagnoses](/uploads/5c4bc9fe3e2ec9ad490329b11d5ca479/diagnoses.PNG)
* Next we can choose how many diagnoses we want to calculate at most (default is 9). Please note that the higher the number the more time is required to calculate depending on the complexity of the ontology.
* If you want to debug an incoherent ontology, the process quite often can be speed up by reducing the incoherency to inconsistency -- by adding a new individual to every unsatisfiable class -- and by replacing the initial ontology with a set of "star" modules. Both options are selected by default.
#### The search for diagnoses ##### Queries / Answers
Once you set up the preferences, you can start the search by clicking the *Search Diagnoses* button. Note that the process might take a while for large ontologies. Currently we are working on a design of a window informing you about the progress of the search. Given the session from above, the right section will show us the following question to be answered by an expert: ```Marsupials DisjointWith Peron``` and ```Koala SubClassOf isHardWorking value false```. In our example we will define both axioms as __Entailed__. Acknowledging this decision by pressing __Commit__ shall list both in the __Answers__ section as well as in thelist of __Entailed Testcases__ in our left section. ![queries](/uploads/c24bda03d842df06a67d1fb6b14941f8/queries.PNG)
In our example we will get these diagnoses: Since we still have no solution the next set of diagnoses and queries are calculated. If we continue answering the questions we should finally find a diagnosis corresponding to our test cases and which represents the possible fault in the ontology.
![diagnoses](/uploads/46f03e27ccdb5c895f68ea30e538d459/diagnoses.PNG) ![finished_session](/uploads/814d5a767a42468350c5b5a29b928275/finished_session.PNG)
_Please note_: when you want to select another reasoner for diagnoses calculation you have to choose *Exquisite Debugger->Options* once to make the reasoner selection effective (known bug). # Preference settings for the Ontology Debugger Plug-In in Protégé
Also note that the measures of each diagnoses is currently not implemented correctly. The above debugging session was using the default settings and the HermiT solver. You can change these settings in ```Ontology Debugger->Options```
# Queries ### Preferences for Diagnosis calculation
Using the diagnoses we can interview the expert if some axioms are entailed or not entailed. This interaction is possible with the *Exquisite Interactive Debugging Tab*
#### The Exquisite Ontology Interactive Debugging Tab ![pref_diag](/uploads/b086b2ff1cc8d637f81e69cac91eba87/pref_diag.PNG)
**Please note that some functionality in this tab is not implemented yet.**
Open the Exquisite Ontology Interactive Debug Tab (*Exquisite Debugger -> Open Exquisite Ontology Interactive Debug Tab*).
We should still see the diagnoses from our previous step. So we can ask the expert by pressing the button *Get Query*. But before we do that let us check which options and preferences are possible for query computation. ##### Engine Types
##### Options for Querying As diagnoses engine we can choose either between Inv-QuickXPlain, HS-Tree (default) and HS-DAG. The first algorithm computes diagnoses directly, i.e. without computation of minimal conflicts, and must be used if you want to find only a few diagnoses very fast. The other two algorithms are based on computation of conflicts and are preferable if you want to use the interactive version of our debugger.
Select *Exquisite Debugger->Options*
![options_queris](/uploads/38acca6a7fdda1f01cfa0e926b549825/options_queris.PNG) ##### Diagnoses Calculation
We can define the minimal and maximal number of queries to compute. If the maximal number is smaller than the minimal, maximal equals minimal. Next we can choose how many diagnoses we want to calculate at most (default is 9). Please note that the higher the number the more time is required to calculate depending on the complexity of the ontology.
Next we can define if we want to enrich the queries, set the sort criterion, a requirements measurement and some thresholds (a detailed description of these options will follow soon).
### Ask the expert If you want to debug an incoherent ontology, the process quite often can be speed up by reducing the incoherency to inconsistency -- by adding a new individual to every unsatisfiable class -- and by replacing the initial ontology with a set of "star" modules. The first option is selected by default. __DO NOT ENABLE THE STAR MODULE OPTION__ as this may cause deadlocks!
If we have set the options and did get some diagnoses, we can press the *Get Query* button to generate a query.
![queries](/uploads/fb14daaa272427520aa75e25060502ce/queries.PNG)
The expert can now tell, if the axiom is entailed (by clicking on the + sign) or not entailed (click on the - sign). The question mark sign is currently not implemented, additionally, the adding of entailed and not entailed axioms to the diagnoses model is not yet implemented. In a future (soon), we will be able to get the next query according to this submission, if the expert/user wants to. ### Preferences for Query computation
All this functionality will be implemented in near future.
\ No newline at end of file ![pref_query](/uploads/bbc5e8e9c0685fb3a1cf06961cf25731/pref_query.PNG)
##### Query Computation
We can define the minimal and maximal number of queries to compute. If the maximal number is smaller than the minimal, maximal equals minimal. The default for both is 1. Next we can define if we want to enrich the queries. This is enabled by default.
##### Measurements
The the sort criterion is necessary for prioritizing diagnoses in query computation. You can choose between MinCard (default), MinSum and MinMax. Next you can define Requirements Measurements such as Entropy (default), Split in Half and RIO. can either bea MinCard which preferes those , a requirements measurement and some thresholds. A detailed description of these options will follow.
##### Thresholds
Depending on the above selection of the Requirements Measurement the threshold values can be set (Default for Entropy: 0.5, Cardinality: 0 and Cautious 0.4). A detailed description of these parameters will follow soon.
### Preference measures
![pref_measures](/uploads/2f32abe48474f7b37a9724138a7977a5/pref_measures.PNG)
These measures set measures to the diagnoses.
The user can choose between Cardinality (default) which simply count the number of axioms per diagnoses, EqualCosts which equalizes all diagnoses and Syntax which calculates the diagnoses measures according to the occurrence of keywords in the axioms.
\ No newline at end of file