The ability to discriminate modulation frequencies is important for speech intelligibility because speech has amplitude and frequency modulations. Neurophysiological responses assessed by envelope following responses (EFRs) significantly decline at faster amplitude modulation frequencies (AMF) in older subjects. A typical assumption is that a decline in EFRs will necessarily result in corresponding perceptual deficits. To test this assumption, we investigated young and aged Fischer-344 rats’ behavioral AMF discrimination abilities and compared to their EFRs. A modified version of prepulse inhibition (PPI) of acoustic startle reflex (ASR) was used to obtain behavioral performance. A PPI trial contains pulses of sinusoidal AM (SAM) at 128 Hz presented sequentially, a SAM prepulse with different AMF and a startle-eliciting-stimulus. To account for hearing threshold shift or age-related synaptopathy, stimulus levels were presented at 10-dB lower or match to the aged peripheral neural activation (using auditory brainstem response wave I amplitude). When AMF differences and modulation depths were large, young and aged animals’ behavioral performances were comparable. Aged animals’ AMF discrimination abilities declined as the AMF difference or the modulation depth reduced, even compared to the young with peripheral matching. Young animals showed smaller relative decreases in EFRs with reduced modulation depths. The correlation of EFRs and AM perception was identified to be more consistent in young animals. The overall results revealed larger age-related deficits in behavioral perception compared to EFRs, suggesting additional factors that affect perception despite smaller degradation in neural responses. Hence, behavioral and physiological measurements are critical in unveiling a more complete picture on the auditory function.