Accurate prediction of the International Roughness Index (IRI) is critical for road safety and maintenance decision. This study proposes a novel Physics Aware-Informer (PA-Informer) model that integrates the efficiency of the Informer structure with physics constraints derived from partial differential equations (PDEs). The model addresses two key challenges: (1) performance degradation in short-sequence scenarios, and (2) lack of physics constraints in conventional data-driven models. By embedding residual PDEs to link IRI with influencing factors such as temperature, precipitation, and joint displacement, and introducing a dynamic weighting strategy for balance between data-driven and physics-informed losses, the PA-Informer achieves robust and accurate predictions. Experimental results, based on data from four climatic regions in China, demonstrate its superior performance. The model achieves an MSE of 0.0165 and R² of 0.962 with input window length of 30 weeks, and an MSE of 0.0152 and R² with input window length of 120 weeks. Its accuracy is superior to other models, and the stability of the model when the input window length changes is far better than that of other models. Sensitivity analysis highlights joint displacement and internal stress as the most influential features, with stable sensitivity coefficients (Sp ≈ 0.89 and Sp ≈ 0.81). These findings validate the PA-Informer as a reliable and scalable tool for pavement performance prediction under diverse conditions, offering significant improvements over other IRI prediction models.