In the cooler months, mice look for places to spend the winter in relative warmth, protected from the elements, and in close proximity to a food supply. It would not be unusual to find them in compost and is not an indication that you are doing anything right or wrong, relative to maintaining a compost pile/bin.
You can discourage their presence by employing various strategies including,
1. placing compost in a secure bin to prevent access
2. avoiding putting food in the compost
3. frequently turning over the compost to disrupt their nesting. Be careful when you do this, and avoid breathing any dust from the dried urine or excrement as it is possible to contract a serious illness from mice caused by the Hantavirus. Read more about mice and Hantavirus at:
Hanta virus
4. implementing mouse control. Learn more about mouse control by visiting FAQ 877 (FAQ 877).
Read about backyard composting at:
Backyard Composting