The activity of neural populations typically encodes more information about sensory or motor variables than can be captured by point estimates of the variables. We present and compare two approaches to quantifying this additional or ancillary information and its relationship to uncertainty: the mutual information between activity and estimation error, and the Fisher information loss, which can be interpreted in terms of curvature in information geometry. We show that deviations from Gaussianity of estimation errors, including the long tails frequently observed in human behavioural tasks, are an expected corollary of the presence of ancillary information. However, populations with similar distributions of estimation error can differ substantially in their ancillary information content depending on the noise characteristics. For a given population tuning and noise model, our results quantify an upper bound on the information about uncertainty that can be obtained from population activity alone: behaviour demonstrating knowledge in excess of this bound would indicate access to a separate source of information about uncertainty. Finally, we contrast the effects of external noise and decreasing internal signal strength on ancillary information and the Gaussianity of errors. Our work directly relates knowledge about uncertainty to non-Gaussianity in sensory estimates, and establishes a coherent theoretical foundation for investigating the basis of metacognition in neural population activity.