On The Origin of Species by Charles Darwin Chapter 9 Page 28

In the same family there may be a genus, as Dianthus, in which very many species can most readily be crossed; and another genus, as Silene, in which the most persevering efforts have failed to produce between extremely close species a single hybrid. Even within the limits of the same genus, we meet with this same difference; for instance, the many species of Nicotiana have been more largely crossed than the species of almost any other genus; but Gartner found that N. acuminata, which is not a particularly distinct species, obstinately failed to fertilise, or to be fertilised, by no less than eight other species of Nicotiana. Many analogous facts could be given.

No one has been able to point out what kind or what amount of difference, in any recognisable character, is sufficient to prevent two species crossing.