This survey paper presents a comprehensive review of convolution operations in Convolutional Neural Networks (CNNs). We trace the evolution of convolutions from their mathematical foundations to their implementation in modern deep learning architectures. We examine various types of convolutions, their properties, and their applications across different domains, including image processing, audio analysis, natural language processing, and scientific applications. Additionally, we discuss the biological inspiration behind CNNs, key architectural innovations, and emerging trends in the field. This paper serves as a resource for researchers and practitioners interested in understanding the fundamental building blocks of CNNs and their continuing evolution.