No such conveyance or charge shall be deemed fraudulent in favor of a subsequent purchaser, who has actual or legal notice thereof at the time of his purchase, unless it appears that the grantee in such conveyance or person to be benefited by such charge was privy to the fraud intended.