Birds usually feed during the day and do not take the whole fruit. Deer like tender new shoots and not the fruit. Porcupine saw off whole branches. Raccoons feed at night and love plums but leave a mess of pits and partially eaten fruit on the ground behind. Humans can pick all the fruit clean or do some damage to branches when done in the middle of the night. Almost always they do not stop to eat the fruit so do not leave the pits behind.
Online Resources:
Wildlife Damage Inspection Decision Tree