When people start abusing your issue tracker to get support, then obviously your other support channels are not accessible enough.
Make sure you have a separate support system which is both easy to find and easy to use.
When I would have a problem with your software, a text file called
contributing.md would not be the first place to look. In fact I would likely not look at it at all, because I don't want to contribute. I want to ask a question. When you are lucky, I might look at a text file called
README or whatever documents I find in a sub-directory called
/doc/. But very likely the first place I would look would be your website, and when the only interactive feature I find there is the issue tracker, I would see no other choice.
Mailing lists and IRC channels are nice, but they don't help when they aren't accessible from the web browser. With either medium I have no idea if they are even active.
Do you have a mailing list archive which is searchable from the web and which can be easily found? That would allow people to see if the mailing list is active and if support questions are welcome there. They could also use the search function to check if their question is already answered which saves you a lot of time handling duplicate questions (a problem with a medium like IRC which has no persistence at all).
What about a web-based support forum? Did you consider creating one?