Familicide is an Epic level necromancy spell. It kills every living blood relative of the victim (though not the victim itself, adding to the depravity of the spell since they are forced to live through their deaths), regardless of how distantly related they may be from the victims. There seem to be limits, such that the spell cannot cause an extinction, but they are not elaborated on beyond that. Furthermore, the power of those subjected to the spell seems to be irrelevant: the one known casting killed the entire Draketooth family line, and 1/4 of the black dragon species.
The spell requires a living (or sentient un-living) target, and direct physical contact between the caster and the target. The spell's range is measured in thousands of miles, if not limitless, (though still limited to one plane of existence). The spell first kills anyone who is directly related to the target, then it kills anyone directly related to those it had just killed. The victim is left totally unharmed.
The precise mechanics of the spell are as follows: 
Step 1: Kill everyone with the original target's blood. This is a simple yes/no effect: Is a creature (the secondary target) related by blood to the original target at all, in any way? If yes, kill it. If no, move on. Number of generations or percentage of blood or direction doesn't matter.
Step 2: Kill everyone who shares blood with any of the people killed in Step 1. Think of it as killing everyone descended from (or siblings to) any and all still-living ancestors of each secondary target. So if Penelope had a grandfather on one side and a great-grandmother on the other side who were still alive, every person who could trace their blood back to either of those people would be dead, because Penelope's daughter carries both of their bloods. If a person can only trace their blood through (say) Penelope's already-dead great-great-great-grandfather, then they're safe. Thus cousins and second-cousins and the like are all dead, but more distant genetic relations are not.
It was used by Vaarsuvius when he/she had his/her soul spliced with three evil arcane-caster's souls (including Haerta Bloodsoak, the source of the spell) on the mother of the dragon they had killed on the Order of the Stick's quest for the starmetal, killing the entire bloodline of the dragon. Vaarsuvius estimates that casting the spell just once on the dragon wiped out 1/4 of the black dragon population.
The spell had an unintended consequence: Girard Draketooth is revealed to have been 1/4 black dragon, and as such his entire family was killed.. and as his family propogated by seducing random outsiders and absconding with the resulting child, this extended the secondary effect to countless others - including Tarquin's wife.
It should be noted that under average circumstances, the parameters of "living blood relative" and "all living relatives one step removed" would have a far less devastating(though still appaling) effect; using the spell on an average NPC would affect perhaps several hundred individuals in total - their immediate family and related families in their immediate vicinity.
Vaarsuvius' casting of the spell upon a subject such as an ancient dragon exponentially extended the effect despite the low fecundity of dragons, especially as the propensity of both dragons and humans to breed outside their species is well documented. The purposeful fecundity of the Draketooth clan extended it in a similar manner. It is possible that Vaarsuvius killed an measurable percentage of the planet's population with a single spell.