My thinking is that the university and Athens would be a magnet for the surrounding area in all things: sports, concerts, theater, movies, shopping and restaurants. Unless Chauncey and Coolville have become hot spots. So, if we don't draw well the level of interest must not be that great. I certainly see the HS sports conflict sometimes.
Your premise is that Athens County is the totality of where the fans are from. Other towns are out there within 45 minutes and Columbus is a huge market to draw potential fans from. If you are thinking Athens County as a limitation for sponsorships and donations I can see that point but there is enough population around to fill up a major D1 arena.
Unlike football where OU is asking for fans to show up at a high level FCS stadium, a 13,000 seat arena is like having 65,000 seats in football. The expectation for many is the NFL 60,000+ seat experience is what football is about. Thats something the town doesn't have the infrastructure to support and competing against programs in the region that are 60,000+.