Can cows over eat - yes. Will they over eat forage such as hay - depends. I assume that when you say over-eat, you mean they eat more than they need to meet their nutrient needs. So this depends on what stage of production they are in, lactating or gestating. If the hay is high quality that means that it is highly digestible and it will be digested rapidly and moved through the digestive system quickly. If the nutrient concentration is such that it meets the cow's requirements, cows with unlimited access to this high quality hay, will eat more than they need. Over-eating a forage will likely not hurt the cow, but will increase feed costs.
Cows over-eating grains is not a good situation. This will usually result in acidosis, founder, reduction in performance, and sometimes death of the animal.
