Peanut butter to jelly is fine because the jelly is refrigerated.

Jelly to peanut butter could not be fine depending on if you refrigerate your peanut butter or not.

If you don’t refrigerate your peanut butter then don’t do jelly to peanut butter.