[Lisa_seminaires] [Lisa_labo] Fwd: Connectionists: The ReScience journal

Hovig Bayandorian hovigbay at yahoo.com
Sam 5 Sep 22:54:39 EDT 2015


John Ioannidis - Reproducibility of scientific results
|   |
|   |  |   |   |   |   |   |
| John Ioannidis - Reproducibility of scientific results |
|  |
| View on www.youtube.com | Preview by Yahoo |
|  |
|   |


also see
https://en.wikipedia.org/wiki/Ben_GoldacreBad Pharma - Wikipedia, the free encyclopedia

|   |
|   |  |   |   |   |   |   |
| Bad Pharma - Wikipedia, the free encyclopediaBad Pharma: How Drug Companies Mislead Doctors and Harm Patients is a book by British physician and academic Ben Goldacre about the pharmaceutical industry, i... |
|  |
| View on en.wikipedia.org | Preview by Yahoo |
|  |
|   |

 


     On Saturday, September 5, 2015 9:13 PM, David Warde-Farley <d.warde.farley at gmail.com> wrote:
   

 I would argue that it's a social problem that is at least partly solved by enforcing community norms on archival publication venues, i.e. Reject anything that doesn't have compelling results on known public benchmarks (if there's other less reproducible stuff too, fine).If it's just people posting on arXiv, there's little to be done about it. We are better off having those unpublished reports in public view than not, though they are clearly a less compelling and less trustworthy form of evidence.On Sep 5, 2015 8:05 PM, "Yoshua Bengio" <yoshua.umontreal at gmail.com> wrote:

Yes. It's sad.
2015-09-05 19:44 GMT-04:00 Alex Lamb <alex6200 at gmail.com>:

A lot of the papers that I've read recently use a private training set and a public testing set (for example, the Google and Facebook face recognition papers).  In areas where the largest datasets are privately held, research is probably going to continue to be impossible to fully reproduce.  
On Sat, Sep 5, 2015 at 7:03 PM, David Krueger <david.scott.krueger at gmail.com> wrote:

To what extent can this be remedied by using aws or similar?On Sep 5, 2015 7:00 PM, "David Warde-Farley" <d.warde.farley at gmail.com> wrote:

Very encouraging to see this happening and that other people are
concerned about it.

I would add that reproducibility in machine learning looks simple
compared to other scientific domains, but looks are deceiving. Every
"simple Python script" is built upon a broad and deep tower of library
dependencies, leading to an exponential number of ways that your
computing environment can conspire against you (nevermind hardware
differences...).

On Sat, Sep 5, 2015 at 6:35 PM, Yoshua Bengio
<yoshua.umontreal at gmail.com> wrote:
>
> Very interesting!
> Reproducibility is VERY weak in the machine learning community, and needs to
> be improved.
>
> ---------- Forwarded message ----------
> From: Nicolas P. Rougier <Nicolas.Rougier at inria.fr>
> Date: 2015-09-03 8:57 GMT-04:00
> Subject: Connectionists: The ReScience journal
> To: Connectionists group <connectionists at cs.cmu.edu>
>
>
>
> It's our great pleasure to announce the creation of "ReScience" which is a
> peer-reviewed journal that targets computational research and encourages the
> explicit replication of already published research, promoting new and
> open-source implementations in order to ensure that the original research is
> reproducible.
>
> To achieve such a goal, the whole editing chain is radically different from
> any other traditional scientific journal. ReScience lives on GitHub where
> each new implementation is made available together with comments,
> explanations and tests. Each submission takes the form of a pull request
> that is publicly reviewed and tested in order to guarantee that any
> researcher can re-use it.
>
> Students are strongly encourage to submit to ReScience. Even if the
> publishing model is a bit different from other academic journals, this will
> give them a first experience at peer-reviewed publishing where they have to
> use a rigorous and scientific approach.
>
>         • More on the journal website:
> https://github.com/ReScience/ReScience/wiki
>         • Current issue:
> https://github.com/ReScience/ReScience/wiki/Current-Issue
>         • FAQ:
> https://github.com/ReScience/ReScience/wiki/Frequently-Asked-Questions
>         • Follow us on twitter (@ReScienceEds):
> https://twitter.com/rescienceeds
>
> And if you're familiar with Git and GitHub, you can also become a reviewer:
> just contact us.
>
>
> Konrad Hinsen & Nicolas Rougier
>
>
> _______________________________________________
> Lisa_labo mailing list
> Lisa_labo at iro.umontreal.ca
> https://webmail.iro.umontreal.ca/mailman/listinfo/lisa_labo
>
_______________________________________________
Lisa_labo mailing list
Lisa_labo at iro.umontreal.ca
https://webmail.iro.umontreal.ca/mailman/listinfo/lisa_labo


_______________________________________________
Lisa_labo mailing list
Lisa_labo at iro.umontreal.ca
https://webmail.iro.umontreal.ca/mailman/listinfo/lisa_labo




_______________________________________________
Lisa_labo mailing list
Lisa_labo at iro.umontreal.ca
https://webmail.iro.umontreal.ca/mailman/listinfo/lisa_labo





_______________________________________________
Lisa_labo mailing list
Lisa_labo at iro.umontreal.ca
https://webmail.iro.umontreal.ca/mailman/listinfo/lisa_labo

  
-------------- section suivante --------------
Une pièce jointe HTML a été nettoyée...
URL: http://webmail.iro.umontreal.ca/pipermail/lisa_seminaires/attachments/20150906/fb401966/attachment-0001.html 


Plus d'informations sur la liste de diffusion Lisa_seminaires