CI is essential for an efficient development process (such as validation of pull requests) and to exercice atypical build configurations to avoid the accumulation of bitrot. Gambit still uses Travis and AppVeyor because those were the “obvious” free services when CI was added to the repo. There are more options now, but always some limitations for free services. It is important that CI cover the main operating systems.
I’m open to changing the CI service, but I dont’ have much experience outside of Travis and Appveyor to make an informed decision. I hope others with more experience (and time) can look into this and propose a solution.
Here is my wish list of features:
1) Support for linux, macOS, and Windows is a minimum. Can other OSes and variants of linux be supported too? Can other processor architectures be added (ARM, risc-V, …)?
2) Must be free and offer fast feedback. Ideal would be to host the CI service locally so we have more control over the resources allocated. We have a few machines in our lab that could be used for this including multi-core linux machines (one with 64 cores!), a macOS machine with 6 cores, and a raspberry pi 4. The script githooks/onpush.sh was intended for this but is currently suffering from bitrot… (can the CI service operations be part of the CI tests???? :-)
3) It would be good to integrate the Gambit forensics benchmarking tool so we can also keep track of performance (http://udem-dlteam.github.io/gambit-forensics/). This is still a WIP… don’t believe the numbers shown yet.
4) Having a centralized CI configuration file or directory would be good to easily add new configurations.
5) As many backends as possible should be tested: - C backend is clearly a must - universal backend, at least the JS target which is the most useful currently - native backends (x86, ARM, risc-v)
On a related subject, the Gambit bootstrap process is causing issues lately. This suggests to me that I have to change my (our?) habit of directly committing to master and instead only work with pull requests that can be accepted into master when the CI indicates that the bootstrap hasn’t broken. That will make for a smoother build experience for the Gambit users.
Marc
On Nov 2, 2019, at 5:41 AM, Lassi Kortela lassi@lassi.io wrote:
What is the main developers' stance on CI?
Currently a Linux build is done on Travis and a Windows build on AppVeyor.
I would recommend migrating to newer CI services:
I've have an excellent experience with Cirrus CI. They offer an extremely simple, no-frills experience; the UI has only the information you need. They support building on arbitrary Docker containers (which means it's easy to build on any Linux distro). They also have Windows, MacOS and FreeBSD. All of these can be driven from one YAML file.
I can also recommend GitLab CI which is similarly easy to use. Like Cirrus, they easily support Docker containers for Linux builds. I'm not sure how to do MacOS, Windows or BSD on it; you may have to bring your own server and install your own build runner on it. GitLab CI has more features than Cirrus but I'm not sure whether we need them (yet). It may be that GitLab CI can only be used on GitLab-hosted repos; not sure about this. So if we host Gambit on GitHub, there would have to be a hook that auto-updates a GitLab mirror as well.
Circle CI also gets a thumbs up from Göran Weinholt who has done complex CI for Scheme. It has Linux, Windows and Mac; not sure about the details. I have no personal experience with this service.
Separately from the choice of CI service, there's the question of how to handle bootstrapping. Browsing Gambit's recent Travis logs, some builds fail because they expect a `make modules` run. How often does the CI break due to bootstrapping issues, and is there an easy solution to them?
It might be worth considering some "releases between releases" to ease bootstrapping for people who actively follow Gambit to make it easier to build the latest master. Would this make sense? I don't really grasp the details of what I'm talking about here.
Gambit-list mailing list Gambit-list@iro.umontreal.ca https://mailman.iro.umontreal.ca/cgi-bin/mailman/listinfo/gambit-list