That's what the big shouty all-caps warranty disclaimer, that many people seem to dislike, is doing. Here's an example from the MIT license:
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
It's actually two sentences; the first is the warranty disclaimer - I don't guarantee this software does what you think it does - and the second part is the limitation of liability - I'm not responsible for whatever happens when you use the software. Usually it's the first sentence that is controversial; it usually comes up whenever a software flaw causes significant damage and some people start calling for programmers to be held responsible. The situation that the second sentence addresses does come up too, though. If you pay attention to news and politics, you can probably find examples of where a product is used unethically, but people want to hold the manufacturers of that product responsible. Think about examples where the product is perceived to facilitate unethical use - hacking/cracking, copyright infringement, trading of illegal goods etc.
Therefore you should use standard open source licenses, which are vetted by real lawyers and may have also been tested in court. Such licenses usually have limitations of liability in them, to provide some protection for you.
Examples
I've already listed the MIT example, but it also exists in BSD (similar wording, also all caps), GPLv3 (clause 16, also all caps), Apache (clause 8, surprisingly not in all caps), MPL (clause 7, not in all caps but amusingly highlighted in yellow).