This page deals with myths related to Darwin and his family.
Darwin's family intermarried only between the Wedgwoods, Galtons, and Huxleys, in the eugenics inspired belief that their descendants would be supermen.
Extremely common in the old days, and somewhat even today, prominent families typically married other prominent families. Three generations prior to Charles Darwin there was a lot of inbreeding between the Darwins and Wedgwoods. Darwin feared his children would inherit some sort of genetic deformability due to this, not in spite of it. The Huxleys did not mix into the Darwin-Wedgwood family until the time of Darwin's grandchildren, when his grandson married Angela Huxley. At this time his grandchildren and their children married into many other families, surnames such as: Raverat; Keynes; Conford; Williams; Barlow; Worsley; Chapman; Massingberd; Pease.
There's absolutely no evidence of a planned system of interbreeding amongst the families. The Darwin-Wedgwood interbreeding was a result of a very common occurrence of the desire to keep wealth and property in the family in the 19th century, but as times changed, that view changed. As I noted above, Darwin feared inbreeding very much, and it's extremely silly to believe he'd make an exception for the Huxleys.
Francis Galton, cousin of Charles Darwin, promoted eugenics and had his family interbreed with another family, and this resulted in ironic deformities.
Interestingly enough, Galton invented the term eugenics in 1883. Galton proposed that individuals with good genetic backgrounds breed with individuals with the same backgrounds, as to improve the genetic stock even more - he even suggested economic incentives to promote breeding.
Francis Galton had no children and thus no decedents, and thus no inbreeding, and thus no deformities.