The grinder is going to make the most difference in your coffee, once you start with good FRESH beans.
You need to buy a grinder, you need to buy a good grinder. A GOOD grinder. Expect to spend several hundred dollars. My favorite hand grinder was $330......I gave another $250 hand grinder to my son because it didnt taste as good as my favorite one. The price isnt indicative of anything, but the fact is that different grinders actually taste different.
My advice....would be to get a $8 plastic hario v60 funnel, filters, a pour kettle, a small scale..... And you will be well on your way to making excellent coffee.
Aeropress......is extremely fiddly. Extremely. You want to know why there's a thousand different aeropress recipes out there? Because its FIDDLY, and very seldom are people satisfied with the results. Note, I'm not saying you can't make decent coffee with it,..... But it is far from the simplest, most reliable way to make good coffee. It's just " another " way to make coffee, of which there are many. I have one, dont use it. I have a clever dripper.....use it now and then....but not much. Hario v60 gets lions share 7 days a week... Don't need complicated procedures, recipes you have to time, and half a dozen parts to wash after making a cup of coffee.........
Look for a decent local roaster near you to buy some fresh roasted beans from. FRESH is the key. There's nothing wrong with mail ordering but it will take several days to get it....at best. By 2weeks...coffee loses half its flavor.....even by 1 week its much different than 1-2 days off roast. I roast my own weekly....because staying in fresh coffee any other way is a hassle. When I first got into it I drove by a local roaster it was just a mile or two out of my way on my way home from work........ But having to go by there once a week...or not have coffee tomorrow morning.. got to be real hassle. Ive used coffee subscription services to have new things to try occassionally for variety....but bulk of my coffee i drink i roast. I keep several beans on hand and decide on spur of moment what I'm going to roast next , how im going to roast it, and how much. I have a strong preference for only washed coffees, and of that I prefer Central Americans for everyday coffee.