There are two things right off the bat that may be a large part of your problem: the north slope and partial shade. Most wildflowers want full sun to profusely bloom and produce seed. Depending on the angle of your slope, it may also be contributing to the problem if it's steep enough to cause seed to wash away.

The decline of your flower patch could be an indication that the original planting took well, but current conditions are discouraging its ongoing success. It may also be the case that over the years, the shade is increasing as the plants have grown larger.

So you have a nontrivial problem to solve if you are determined to have a wildflower bed in this area. Shade is your enemy. Is it possible for you to trim up some of the trees or shrubs shading the wildflower garden? This could give your flowers more of the sun they love. Remember that wildflowers will always be struggling in a situation where they are not receiving at least eight hours of full sun. Any less light makes the plants spindly, prone to flopping over, and less likely to produce viable seed.

Not knowing if your "wildflower bucket" contained native flower/grass seed, it's unclear whether the mix you bought was annuals only or also perennials. If you try again, be sure the mix you select includes perennials. Once established, they would help hold the soil in place for the seeds of the annuals. It's also a good idea in the fall to mulch mow the area after most of the plants have died and dried out. This helps distribute seed, and the chopped up dead plant material provides a light mulch that also helps keep the seed in place.