Hi Christian,
The idea of using two separate repositories for source and generated source is interesting. I would like to bring this to git mailing list, they may provide insightul comments for your idea or even other approaches.
Background for Git people: gambit-c was previously stored in mercurial. The main source is in gambit-c (a Scheme implementation or a Lisp dialect). *.scm files generate *.c, which will be compiled by gcc as usual. Both *.scm and generated *.c are now stored in mercurial. Gambit-C maintainers have recently decided to move to Git.
On 10/15/08, Christian Jaeger christian@pflanze.mine.nu wrote:
I wonder whether it would be a good idea, and good occasion to realize it, to move source files and generated files into separate repositories and 'link' those together using the git submodule feature.
Expected advantages:
- no clutter when looking through the history (can possibly be mitigated
by constraining git log, git diff etc. to the non-generated paths only, although I don't think this is possible (cleanly) with the current directory structure); the same holds true for using "git format-patch" (one wouldn't usually want to include the generated files in diffs sent to the mailing list)
- when merging branches, there will usually be no need to deal with
merge conflicts in the generated files (one would just regenerate them instead)
- [especially for files being generated not by Gambit itself (for
example "configure"),] the files can be regenerated by differing [external] software versions without having to deal with those superfluous changes in the source repository.
By still committing the generated files--to a different submodule--Gambit can still be updated through Git alone, and the possible advantage of tracking the generated files to see the effects of changes in compiler sources can still be had.
Expected disadvantages:
- all generated files need to reside in a separate directory structure;
e.g. the file $BASEDIR/lib/_io.c would have to be at a place like $BASEDIR/build/lib/_io.c instead, where build/ is the submodule taking all generated files; since the "configure" file is expected to reside at the toplevel, I guess this would require that "make update" copies it from $BASEDIR/build/configure to $BASEDIR/configure (assuming that one cannot use a symlink because of portability reasons).
- to commit the generated files, a separate step is necessary ("cd
$otherrepo; git commit -a", or maybe easier create a "make commit_generated" make target?)
- to make this work with the "source" repository residing at the
toplevel, the Git superproject repository (of which the "source" and "build" repositories are submodules) would need to reside in a non-standard directory, like $BASEDIR/.gitsuperproject/ instead of the usual .git/, and using the GIT_DIR environment variable to access it, although this can probably be handled by make targets (i.e. "make update" would set GIT_DIR=$BASEDIR/.gitsuperproject when calling "git submodule update").
- there may be some cases to flesh out; like, should "make update"
really call "git submodule update" (which simply sets the submodules to the reference given by the superproject, throwing away changes done by the user in the submodules (they can be recovered from the git reflog, but may still be a surprise)) or should it run "git pull" in each submodule instead?
I thought I'd bring this up now because if package maintainers need to adapt some things anyway, that may be a good time to do it now. (There's even the possibility to split the converted Mercurial repository into the source + build parts in retrospect now, which won't be possible anymore later on (without changing the sha1 sums of the whole Git history with the associated breakage of existing clones), although that may not be important.)
I'm willing to help in the effort, although I don't know the build tools (autoconf and make) and their use in the setup well, so I would probably be quite a bit lost when doing it alone.
Christian.