Mad Targaryens

"Babies or no, theirs was the same cursed blood that flowed within the Mad King's veins. They were 'dragon-spawn', and couldn't be allowed to survive. What would they grow to be, loyal subjects?...Was it a crime to put an end to a family of lunatics born of incest?"

- Robert Baratheon In A Song of Ice and Fire novels and its TV series adaptation, the Mad Targaryens refers an extreme and villanious non-official faction within House Targaryens, given to the extremely psychopathic family members of House Targaryen, due to the marriage between its close reletives.

History
House Targaryen carries the trait for insanity in its bloodline. Over three centuries of heavy inbreeding, marrying brother to sister whenever possible to "keep the bloodline pure," resulted in many of the medical problems seen with incest, particularly mental instability.

The most prominent example of the Targaryen madness was the last Targaryen king, Aerys II, who subsequently became known as "the Mad King". Aerys suffered from some sort of paranoid schizophrenia, hearing voices in his head which weren't real and which told him to burn people alive because they were all plotting against him.

Other Targaryens displayed highly bizarre personalities, though they were not strictly speaking "insane". They didn't have hallucinations or hear voices, but some had aggressive, sociopathic personalities, such as Aerion Brightflame - older brother of Maester Aemon. Eventually, Aerion succumbed to the delusional belief that if he drank wildfire it would transform him into a dragon: he was wrong. The effects worsened with each generation of compounded inbreeding. By the later centuries of their rule, it was joked that when a new Targaryen was born the gods would flip a coin to decide if they would be a brilliant statesman or insane.

Generally, if a Targaryen had multiple sons and one of them was clearly mentally unstable, they would try to skip over him in the line of succession in favor of a more stable younger son. Therefore, fewer of the early Targaryen kings had mental problems, because if a king did have a mentally unstable son he wouldn't be allowed to succeed his father. However, during the last century of their rule the Targaryen madness became increasingly common, making it more and more difficult to avoid having a mentally unstable son inherit the throne.

Known mad Targaryens

 * King Maegor the Cruel was perhaps the first mad Targaryen in Westros history. He usurped the throne from his nephew and promptly decapitated the one Archmaester who protested. As king, Maegor turned to brutal tactics to suppress the Faith of the Seven, even riding on his dragon Balerion to burn down a Sept with all worshipers inside, using archers to pick off stragglers.
 * King Baelor the Blessed was overzealously obsessed with religion and purity, to the point that he starved himself into an early grave because he believed that food is of this world, and the material world is sinful.
 * Prince Rhaegel, older brother of King Maekar, had massive delusions and was known to randomly take off his clothes and dance naked through the halls of the Red Keep.
 * Prince Aerion Brightflame, son of King Maekar and nephew of the also-mad Prince Rhaegel. Aerion killed himself drinking wildfire, believing it would turn him into a dragon.
 * King Aerys the Mad, nephew of Aerion. The infamous Mad King. Had violent visual and auditory hallucinations, as well as a near permament feeling of persecution, which drove him to execute anyone who slighted him out of fear that they were conspiring against him. Tried to blow up King's Landing with wildfire, believing he would rise from the ashes as a dragon... and it eventually killed him.
 * Viserys, the Beggar King, second son of Aerys II, had delusions of greatness and would have frequent, violent outbursts of jealous anger towards his sister, Daenerys.

Madness in House Blackfyre
"All the same, his descendants continued to threaten the Targaryens until half a century later, when Barristan the Bold slew Maelys the Monstrous on the Stepstones, extinguishing Daemon's line."

- Varys Members from the infamous House Blackfyre, which originated from Daemon Blackfyre, a bastard of Aegon IV Targaryen, were possibly affected by the madness with Targaryen blood, so that they rebelled four times to ursurp the Iron Throne. Maelys Blackfyre, known as "Maelys the Monstruous", who is the last Blackfyre ursurper, was also discribed as an extreme and unpleasant savage.

Trivia

 * Even though House Targaryen was filled with madness, savage and disgrace it brought, this does not mean that all Targaryens were as a rule mentally unstable, as seen with how  Daen erys Targaryen  is mentally stable while her own brother  Viserys  is highly disturbed. Daenerys's oldest brother  Rh aegar  was considered a great man by all who knew him, and  Maeste r Aemon  became a wise and revered advisor to the  Ni ght's Watch  for many decades.
 * Daenerys is not thought to be insane, though her merciless treatment of slave masters in Meereen is a dangerous hint that she may not have escaped from the Targaryen madness. Even so, her actions were ultimately against a group who were mistreating others (seeing this as an echo of how she was mistreated), and she thought she was helping to secure freedom for the slaves of the region.
 * However, Barristan Selmy notes that the Mad King also genuinely believed his actions were just, even though others knew he was a madman, which compels Daenerys to become more merciful.