The antigenic evolution of human seasonal influenza viruses is primarily driven by single amino acid substitutions immediately adjacent to the receptor binding site in the hemagglutinin (HA) protein. The ability to predict these substitutions would allow vaccine strains to be selected with an understanding of likely future antigenic variation. Here, we estimate the effect of HA substitutions on viral fitness using measurements of convergent evolution in a large phylogeny. We show that the substitutions which have historically caused major antigenic changes in H3N2 influenza viruses were nearly always one of few substitutions near the HA receptor binding site estimated to be under positive selection in sequences collected before the antigenic transition, based on convergent acquisition of the substitution in multiple independent lineages. Furthermore, this signal predates the establishment of the major clade containing the antigenic substitution by more than one year, so is highly informative for prospective prediction.