Topical use of aloe vera gel is reported safe (toxicity is minimal). There have been no reports of adverse effects following maternal use or via milk ingestion. Do not use on nipples, as it should not be ingested by baby (see below).
Oral use of the aloe vera latex is strongly discouraged; can cause severe gastric irritation. Per the classic King's American Dispensatory, "Administered to nursing mothers it will purge the sucking child." Children, nursing moms and pregnant women should not use this (internally) at all.