| Date: | 2023, June 13 |
| Time: | 11:30 a. m. |
| Place: | G29-018 |
| Author: | Lespin, Daniel |
| Title: | Varying Ontology Knowledge Inclusion in Transformer Pre-Training for Poison Prediction |
Conventional methods for assessing chemical safety face challenges due to the requirements of a large number of chemicals, time, and resources, emphasizing the necessity for more effective predictive models. The ontology pre-trained Transformer network achieves improvements in performance, robustness, and interpretability with an additional pre-training step using a subset of the ChEBI ontology data for molecular multi-label classification [1]. However, it is unclear to what extent the domain knowledge of the ontology is responsible for the achievements of the Transformer model and whether further inclusion of domain knowledge during training leads to additional improvements. Therefore, this work evaluates the performance, robustness, and attention of two opposing models implementing the ontology pre-training. In particular, one model utilizes expanded data by incorporating additional classes of the ChEBI ontology during the pre-training, whereas the other model uses false domain knowledge derived from a distorted version of the same ontology structure. We expect the model with additional knowledge to significantly outperform the opposing model in all assessed aspects. Hence, providing an improved model for chemical safety assessment and reinforcing the effectiveness of the ontology pre-training approach.
[1] Martin Glauer et al. “Ontology Pre-training for Poison Prediction”. In: arXiv preprint arXiv:2301.08577 (2023).