Coverity open source project static code security analyser analyzed the sourcecode of Linux, FreeBSD, glibc, PgSQL, Nginx etc etc, what about ask them to analyze Gambit's sourcecode too?
Dear Marc and list, Coverity is an automated static sourcecode analysis tool that scans open source projects such as Linux, FreeBSD, glibc, PostgreSQL, Nginx and OpenSMTPD from the OpenBSD team <http://scan.coverity.com/all-projects.html> . As I got it originates from some project at Stanford and the open source project scanning is free courtesy of sponsoring from the US Department of Homeland Security<http://www.prnewswire.com/news-releases/lamp-software-stack-more-reliable-than-baseline-open-source-software---coverity-analysis-for-dhs-finds-55241072.html> . The general feedback is very positive: Published reports<http://scan.coverity.com/report/> and for instance the OpenSMTPD guys tell that they had good use of it<https://www.poolp.org/> : Bret then asked me if I had ever tried the scanner at Coverity and told me I should try to get our codebase scanned with it. I recalled seeing a summary of issues uncovered by their scanner on Apache a while ago and it was very impressive, so I asked to register OpenSMTPD and was happy when they confirmed I could use their scanner without having to sign any NDA, terms of services or random lawyer documents [rare and appreciable] :-) .. We didn't find anything really scary, probably due to the fact we already cleaned and removed scary code detected by Clang, but we did find lots of little issues that we wouldn't spot just from reading, even several times. Amongst the issues, we found a descriptor leak, several small memory leaks, possible double-free, possible double-close, possible double-fclose, use-after-free, various missing bzero, dead code and null-derefs. Many of these would lead to a crash on OpenBSD but the reason we didn't experience them is that they happened in error code paths for the most part. We have fixed about 40 issues, there are 40 still, with several part of external code (mail.local, aldap, etc) and some we believe are false positives but require further reading with a clear mind. The scanner manages to find errors which makes you wonder if they really are, until you realize that it does a far better job at unrolling the code ;-) This teaches us two lessons: find a way to better test error paths, run the scanner from Coverity regularly ! What is to win is a third party security certification, which would be a really valuable thing to have. And, couldn't go worse than that they'd flag false positive or alike. OpenSMTPD's authors say that it took *no* time from them to to do the scan in terms of configuration, it was all about making the account and exchanging some email. What do you say about submitting <http://scan.coverity.com/start/> Gambit? If you would want to do this would be cool. Also I could do the emailing needed for it. Please tell know what you think. Best regards, Mikael
Afficher les réponses par date
I'm curious whether it will go batshit with the tagged pointers. :D I did a trial period with Coverity a long time ago, and it found plenty of stuff in that codebase and was nice for not having a lot of false positives (though it definitely had some). But I have trouble imagining it grokking this kind of generated C and doing much with it. But I'd love to see what it says. Sent from my iPhone On Feb 5, 2013, at 4:02 PM, Mikael <mikael.rcv@gmail.com> wrote:
Dear Marc and list,
Coverity is an automated static sourcecode analysis tool that scans open source projects such as Linux, FreeBSD, glibc, PostgreSQL, Nginx and OpenSMTPD from the OpenBSD team.
As I got it originates from some project at Stanford and the open source project scanning is free courtesy of sponsoring from the US Department of Homeland Security.
The general feedback is very positive: Published reports and for instance the OpenSMTPD guys tell that they had good use of it:
Bret then asked me if I had ever tried the scanner at Coverity and told me I should try to get our codebase scanned with it. I recalled seeing a summary of issues uncovered by their scanner on Apache a while ago and it was very impressive, so I asked to register OpenSMTPD and was happy when they confirmed I could use their scanner without having to sign any NDA, terms of services or random lawyer documents [rare and appreciable] :-)
..
We didn't find anything really scary, probably due to the fact we already cleaned and removed scary code detected by Clang, but we did find lots of little issues that we wouldn't spot just from reading, even several times.
Amongst the issues, we found a descriptor leak, several small memory leaks, possible double-free, possible double-close, possible double-fclose, use-after-free, various missing bzero, dead code and null-derefs. Many of these would lead to a crash on OpenBSD but the reason we didn't experience them is that they happened in error code paths for the most part. We have fixed about 40 issues, there are 40 still, with several part of external code (mail.local, aldap, etc) and some we believe are false positives but require further reading with a clear mind. The scanner manages to find errors which makes you wonder if they really are, until you realize that it does a far better job at unrolling the code ;-)
This teaches us two lessons: find a way to better test error paths, run the scanner from Coverity regularly !
What is to win is a third party security certification, which would be a really valuable thing to have.
And, couldn't go worse than that they'd flag false positive or alike.
OpenSMTPD's authors say that it took *no* time from them to to do the scan in terms of configuration, it was all about making the account and exchanging some email.
What do you say about submitting Gambit?
If you would want to do this would be cool. Also I could do the emailing needed for it. Please tell know what you think.
Best regards, Mikael
_______________________________________________ Gambit-list mailing list Gambit-list@iro.umontreal.ca https://webmail.iro.umontreal.ca/mailman/listinfo/gambit-list
Mm. To start with, Gambit has a good number of klocs of original C code too. Brgds 2013/2/5 Jason Felice <jason.m.felice@gmail.com>
I'm curious whether it will go batshit with the tagged pointers. :D
I did a trial period with Coverity a long time ago, and it found plenty of stuff in that codebase and was nice for not having a lot of false positives (though it definitely had some). But I have trouble imagining it grokking this kind of generated C and doing much with it.
But I'd love to see what it says.
Sent from my iPhone
On Feb 5, 2013, at 4:02 PM, Mikael <mikael.rcv@gmail.com> wrote:
Dear Marc and list,
Coverity is an automated static sourcecode analysis tool that scans open source projects such as Linux, FreeBSD, glibc, PostgreSQL, Nginx and OpenSMTPD from the OpenBSD team<http://scan.coverity.com/all-projects.html> .
As I got it originates from some project at Stanford and the open source project scanning is free courtesy of sponsoring from the US Department of Homeland Security<http://www.prnewswire.com/news-releases/lamp-software-stack-more-reliable-than-baseline-open-source-software---coverity-analysis-for-dhs-finds-55241072.html> .
The general feedback is very positive: Published reports<http://scan.coverity.com/report/> and for instance the OpenSMTPD guys tell that they had good use of it<https://www.poolp.org/> :
Bret then asked me if I had ever tried the scanner at Coverity and told me I should try to get our codebase scanned with it. I recalled seeing a summary of issues uncovered by their scanner on Apache a while ago and it was very impressive, so I asked to register OpenSMTPD and was happy when they confirmed I could use their scanner without having to sign any NDA, terms of services or random lawyer documents [rare and appreciable] :-)
..
We didn't find anything really scary, probably due to the fact we already cleaned and removed scary code detected by Clang, but we did find lots of little issues that we wouldn't spot just from reading, even several times.
Amongst the issues, we found a descriptor leak, several small memory leaks, possible double-free, possible double-close, possible double-fclose, use-after-free, various missing bzero, dead code and null-derefs. Many of these would lead to a crash on OpenBSD but the reason we didn't experience them is that they happened in error code paths for the most part. We have fixed about 40 issues, there are 40 still, with several part of external code (mail.local, aldap, etc) and some we believe are false positives but require further reading with a clear mind. The scanner manages to find errors which makes you wonder if they really are, until you realize that it does a far better job at unrolling the code ;-)
This teaches us two lessons: find a way to better test error paths, run the scanner from Coverity regularly !
What is to win is a third party security certification, which would be a really valuable thing to have.
And, couldn't go worse than that they'd flag false positive or alike.
OpenSMTPD's authors say that it took *no* time from them to to do the scan in terms of configuration, it was all about making the account and exchanging some email.
What do you say about submitting <http://scan.coverity.com/start/> Gambit?
If you would want to do this would be cool. Also I could do the emailing needed for it. Please tell know what you think.
Best regards, Mikael
_______________________________________________ Gambit-list mailing list Gambit-list@iro.umontreal.ca https://webmail.iro.umontreal.ca/mailman/listinfo/gambit-list
participants (2)
-
Jason Felice -
Mikael