The interplay between transcription factors (TFs) and regulatory elements (REs) drives gene transcription, forming gene regulatory networks (GRNs). Advances in single-cell technologies now enable simultaneous measurement of RNA expression and chromatin accessibility, offering unprecedented opportunities for GRN inference at single-cell resolution. However, heterogeneity across omics layers complicates regulatory feature extraction. We present scTFBridge, a multi-omics deep generative model for GRN inference. scTFBridge disentangles latent spaces into shared and specific components across omics layers. By integrating TF-motif binding knowledge, scTFBridge aligns shared embeddings with specific TF regulatory activities, enhancing biological interpretability. Using explainability methods, scTFBridge computes regulatory scores for REs and TFs, enabling robust GRN inference. It consistently outperformed baseline methods in both cis- and trans-regulation inference tasks. Our results demonstrate that scTFBridge can uncover cell-type-specific susceptibility genes and distinct regulatory programs, offering new insights into gene regulation mechanisms at single-cell resolution.