Conquest was added in the Binding of Isaac: Wrath of the Lamb. In the Bible, Conquest is actually the first Horseman to appear, while Pestilence is a Horseman existing only in popular culture, coming from a different translation of the Greek word 'Thanatos', often meaning Death instead (the canon four Horsemen, thus, are Conquest, War, Famine and Death).
Conquest's entry in the secrets is called "The Forgotten Horseman", because of that very reason.
Death is the only horseman specifically given a name in The Book of Revelation.
The use of the Book of Revelations item to summon a horseman for the current floor is reference to the fact that the book of the Bible in which the horsemen are described is Revelation.
None of the Harbingers have a death animation.
Under normal circumstances, Conquest and the Headless Horseman are 2 of only 5 bosses (the others being Krampus, The Fallen, and Baby Plum if spared) who drop or can drop active items upon defeat.
The horses used by the Harbingers resemble stick horses.