Predicting Transcription Factor Binding Sites using Transformer based Capsule Network

Prediction of transcription factor binding sites (TFBSs) is important for understanding how the transcription factors (TFs) regulate gene expression and how this regulation can be modulated in gene therapy. Although in the past few years there have been some significant works which have addressed this issue, there are ample spaces of improvement. In this regard, a transformer based capsule network viz. DNABERT-Cap is put forth to predict transcription factor binding sites mining ChIP-seq datasets. DNABERT-Cap is built on top of DNABERT which is a bidirectional encoder pre-trained with a large number of genomic DNA sequences, empowered with a capsule layer responsible for the final prediction. The proposed model builds a strong predictor for the prediction of TFBSs by utilising the joint optimization of features from DNABERT and capsule layer, along with convolutional and bidirectional long-short term memory layers. In order to evaluate the efficiency of DNABERT-Cap, benchmark ChIP-seq datasets of five cell lines viz. A549, GM12878, Hep-G2, H1-hESC and Hela, available in the ENCODE repository are used. The results show that the average area under the receiver operating characteristic curve (AUC) score exceeds 0.91 for all such five cell lines. DNABERT-Cap is also compared with existing state-of-the-art deep learning based predictors viz. DeepARC, DeepTF, CNN-Zeng and DeepBind, and is seen to outperform those predictors.