Is it fine to open source a virus, exploit or any other potential dangerous software (if used in the wrong way)?
If so, are there any limitations/circumstances when you can't? Does it depends on the country law?
Open Source Stack Exchange is a question and answer site for people organizing, marketing or licensing open source development projects. It only takes a minute to sign up.Sign up to join this community
In some countries it is illegal to develop viruses and other malware. It usually does not matter if open- or closed source.
When you are in such a country, you can of course release your software under an open source license anonymously to avoid legal repercussions. But considering that you have no way to legally defend your license without giving up your anonymity and facing charges yourself, your license terms can not be enforced. Also, the people who would violate your license are likely also criminals according to far more jurisdictions (developing malware might not be illegal everywhere, but using it to intentionally steal data, sabotage systems etc. likely is), so they have lots of other reasons to not get identified.
That means any malware you publish is effectively public domain.
Otherwise, there is nothing which stops you. One of the most important aspects of the OSI open source definition and the FSF free software definition is that there must be no restrictions on what purposes the software can be used for. This includes illegal activity (both OSI and FSF have denied their approval for software which had terms like "this software should not be used for evil").
When or whether to reveal an exploit is a security question which has been debated for a long time. See the wikipedia article on Computer Security which discusses "full disclosure" versus "coordinated disclosure" and "Non-disclosure". I've seen "coordinated disclosure" also called "responsible disclosure", but according to the cited article that term has fallen out of favor.
If the exploit is publicly known, or once the target has had time to react to the vulnerability (CERT Disclosure Policy allows 45 days but I've had as little as 0 days) then there shouldn't be any reason not to publish the code that exhibits the exploit as "open source" or otherwise. Note that CERT doesn't publish exploits; they only publish descriptions of vulnerabilities to exploits.
So in answer to your question, yes it's fine to publish the exploit once the target project has had time to develop and distribute a patch. I agree with CERT that 45 days is plenty.
When you write a virus, exploit, or other potentially dangerous software, there are ethical and legal considerations.
The problems don't become larger if you publish the software under an open source license rather than another license.
The issues around disclosure in general are well described in Glenn Randers-Pehrsons answer, but the licensing doesn't change that.