The rapid integration of unmanned aerial vehicles (UAVs) into next-generation wireless systems demands seamless and reliable handover (HO) mechanisms to ensure continuous connectivity. However, frequent topology changes, high mobility, and dynamic channel variations make traditional HO schemes inadequate for UAV-assisted 6G networks. This paper presents a comprehensive review of existing HO optimization studies, emphasizing artificial intelligence (AI) and machine learning (ML) approaches as enablers of intelligent mobility management. The surveyed works are categorized into three main scenarios: non-UAV HOs, UAVs acting as aerial base stations, and UAVs operating as user equipment, each examined under traditional rule-based and AI/ML-based paradigms. Comparative insights reveal that while conventional methods remain effective for static or low-mobility environments, AI- and ML-driven approaches significantly enhance adaptability, prediction accuracy, and overall network robustness. Emerging techniques such as deep reinforcement learning and federated learning (FL) demonstrate strong potential for proactive, scalable, and energy-efficient HO decisions in future 6G ecosystems. The paper concludes by outlining key open issues and identifies future directions toward hybrid, distributed, and context-aware learning frameworks for resilient UAV-enabled HO management.