Ants aren't really your problem. They are just attracted to the moisture in the holes, which were caused by other insects. A likely culprit is the tomato fruitworm, also known as the corn earworm. Bt (Bacillus thuringensis) is a nontoxic biological control that you can apply to the plants. You may want to visit the eXtension website for additdional information on controlling garden pests.