Taking items dropped by Krampus will not remove the cross.
Taking the boss's item affected by Satanic Bible will not remove the cross.
When playing as Tainted Lazarus, the holder must enter the Devil room to trigger the cross; it will not work if you transform inside. The character with the inverted cross must be the one to travel to the next floor.
Glowing Hour Glass: +1 damage given by Redemption will persist after time leap to the upper floor, as well as giving another activated Redemption after time leap. The soul heart given by Redemption will only persist if you time leap to the upper floor before Redemption finishes its animation. This will allow to take the Redemption damage buff and soul hearts several times per floor. However, if you can use Glowing Hour Glass two times in a row, in order to get 2 soul hearts, you'll need to leave the boss room before going down the floor the second time.
The pickup quote is taken from the last verse of the Lord's Prayer: "And lead us not into temptation, but deliver us from evil.'"