19

Other than having someone manually reading all of the code, how can you demonstrate an open-source project's reputability? i.e. it is not a virus/malware/whatever.

I'm not concerned with the efficiency of the program. I just need to be able to demonstrate that the program is exactly what it says it is and is thus not a security risk to install.

  • 2
    And how do you demonstrate that the program is exactly what it says it is, generally? I think that it isn't much different with open-source. The difference is that if you have time, you CAN manually examine the source. That's not an option with closed-source. – cubuspl42 Aug 20 '15 at 10:51
  • @cubuspl42 Well, assuming that the binaries you have were actually built from the same source codes, which isn't necessarily the case. And of course, you can always check the binaries themselves, though it's a bit more expensive than looking through the sources, most of the time. – Luaan Aug 20 '15 at 11:18
  • When it comes to some lone-developer somewhere, of whom no one has heard of, I think its just easiest to look at other peoples feedback/reviews/comments concerning his project. – G.Rassovsky Sep 3 '15 at 8:49
8

The same way as you know any other program is not actually a virus/malware/etc.

You might trust organization x not to publish malware pretending to be something else. You might trust that it's a well known product that wouldn't contain malware. You might perform static analysis.

This is no different than with propriety software.

7

Not right now, but in the future you'll be able to look for a badge from the just-announced Linux Foundation's Badge Program. The Best Practices Badge is a secure open source development maturity model. Projects having a CII (Core Infrastructure Initiative) Best Practices Badge will showcase the project's commitment to security.

At the moment there's a call for feedback on the criteria for awarding the Best Practices Badge, at Github.

Right now you could use those proposed Best Practices criteria to make your own evaluation of any open-source project.

Also, right now you might look for a digital signature (e.g., file.tar.gz.asc, file.exe.asc) accompanying the distribution, and actually verify it if one is present. That helps ensure that no middleman has tampered with the software before you received it. Even if you don't verify it, you might find the fact that it is present comforting; see this XKCD.com cartoon.

  • 3
    Seems like a perfectly secure virus or trojan would still be able to get badges ;) – Michael Schumacher Aug 22 '15 at 8:46
6

In my experience, this is typically done with various analysis tools.

The first phase is a standard virus check on all of the files to ensure that they are clean. This is true for any software, including open-source as well as commercial packages.

The next step is the use of an approved static analysis tool that includes searching for security vulnerabilities. I'm aware that some projects use HP's Fortify Static Code Analyzer, but others exist as well. Some are even designed to scan binaries. Note that these tools aren't perfect, but they can provide a higher level insight into the quality and any possible issues with an application.

Ultimately, if you're integrating an open source (or any package) into your software application, you should work with the customer to determine what is acceptable. There could be accepted or trusted sources, libraries or packages that have undergone deeper analysis, and so forth. It does depend a lot on who the end user is and what their security requirements are.

3

I think that your use of the term 'reputability' is important here.

Open Source projects, like commercial products, have reputations. Many organizations make decisions based on reputation. Once in a while, as in Heartbleed, the results are not good.

As others have said, demonstrating that a program 'does what it says it does' is a requirement for a 'provable' program. For a conventional programming language, that's solving the (unsolvable) halting problem. There are some very specialized languages/compilers used for certain critical security and reliability applications that can prove the program's behavior; I've never heard of any open source application written using one; F6 used them.

So, you have to fall back on common sense. What kind of community produces the product? What are their coding standard and review procedures? What do their tests look like? How long have they been around? How widely used are they. In short, well, 'reputation'.

2

I just need to be able to demonstrate that the program is exactly what it says it is and is thus not a security risk to install.

Almost all open source licenses include language along these lines:

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE

Basically, with open source software there is typically no guarantee that the program is what it says it is or that it's free from malware.

Trying to claim that your software can be trusted will not get you very far when the license text conflicts with your claim.

Some open source projects will provide the software under two licenses:

  • one that is open source
  • another that requires a payment and does have a warranty guaranteeing the software is "fit for a particular purpose"

The way you guarantee software is good, is my having a person (or company) read the source code and say that the software is good. Involving payment adds extra legal weight on top of whatever credibility this person/company has.

2

How trustworthy a project is may be reflected in its reputation, but the existing user base simply not care beyond the first question of

Does it even try to do what it says on the tin? Reviews and other evidence of an active user base such as community mailing lists will give you a hand in this area. Some critical thinking about whether you trust the reviews may be required. Inclusion in downstream packages such as a Linux distribution would be a stronger indication in this area.

In case you care more than the current users you could ask a few more questions:

Does the installer/binary only include only what's in the code repository? The most effective way to ensure that nothing unpleasant got injected during packaging is to build from source. If that's too high a hurdle to jump, get the installer from a trusted packager (like you Linux distro) or the project's own site.

Is the code clean/securely built? If you can't or won't review the code base in full you may be able to get a piece of software to do it for you (Fortify, Coverity, ...). This may still not be practical for many people. In which case the next best thing is evidence that the project cares about security. If the project has signed up with the Coverity Scan initiative they have at least looked for issues in the code base, even if the issues have not been addressed. Quick response to security issues raised by community and evidence of code review are also good signs.

1

F-Droid does exactly that.

The F-Droid team:

  • Receives applications
  • Actually read the whole source code
  • Discard your project if it contains any binary blob or suspect code

If evaluation was successful, your app is then added to the official F-Droid repository with the following metadata:

  • Whether the app reports anything about the user (very strict, includes even what some might consider harmless: crash reports, service authentication)
  • Whether the app contains any form of advertising
  • etc

Only for Android projects unfortunately.

0

If you want to do this with a static analysis, the answer is that you can't. It is impossible to proof this for code in the general case; if you could, you could solve the Halting Problem - which is mathematically impossible.
A static analysis might find it if there are problems but can never prove that there aren't any.

The best you can do is allow other people to run their virus scanners on it. Basically by opening up the sources you're already allowing them to do just that.

If people are doubtful about your project, they can run it inside a VM and see what happens. Although this is not entirely a guarantee; there is at least one vulnerability VENOM, CVE-2015-3456 that allowed code to escape the VM (privilege escalation).

Technically speaking, to put an attack in a new Open Source project is counterproductive. To pull it off, you would need an attack that isn't in the databases of the malware scanners yet - a zero-day exploit. And by putting it in an Open Source project, you're effectively publishing it. So you might as well publish your zero-day exploit in the reputable way, and make a more positive name for yourself.

  • 3
    You may not be able to make a guarantee with static analysis tools, but some do detect security vulnerabilities, especially if you download source files and build the libraries yourself instead of prebuild binaries. We use HP Fortify on some projects, which includes not only scans of our code for customers, but scans of externally developed software. It will identify unsafe function calls, SQL injection, and a few other problems. They may have false positives and false negatives, but it's better than just a virus scanner. – Thomas Owens Aug 19 '15 at 14:43
  • WRT the last paragraph, I've heard a few experienced developers make the claim that the behavior introduced by the Heartbleed vulnerability was so obviously wrong that it had to have been put there on purpose, but no one noticed because 1) very few people were reading OpenSSL's code to begin with and 2) the implementation of this obviously wrong behavior was extremely obscure and obfuscated. Without going into the question of whether or not this specific allegation is true, do you agree that that sort of thing could successfully happen in a major open-source project? – Mason Wheeler Aug 19 '15 at 19:26
  • @MasonWheeler It could happen, but a major OS project won't just allow anyone's contributions. If it was deliberate, it required a high amount of patience and skill to build up the required trust. – S.L. Barth - Reinstate Monica Aug 19 '15 at 19:34
  • I'd almost say that the redundancy in the Heartbeat spec should have been so obvious people should have noticed it. A security protocol violating single-point-of-truth? – Random832 Aug 20 '15 at 2:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.