[gambit-list] Separating generated files? (Re: Mercurial -> git)

Nguyen Thai Ngoc Duy pclouds at gmail.com
Wed Oct 15 10:54:26 EDT 2008


Hi Christian,

The idea of using two separate repositories for source and generated
source is interesting. I would like to bring this to git mailing list,
they may provide insightul comments for your idea or even other
approaches.

Background for Git people: gambit-c was previously stored in
mercurial. The main source is in gambit-c (a Scheme implementation or
a Lisp dialect). *.scm files generate *.c, which will be compiled by
gcc as usual. Both *.scm and generated *.c are now stored in
mercurial. Gambit-C maintainers have recently decided to move to Git.

On 10/15/08, Christian Jaeger <christian at pflanze.mine.nu> wrote:
> I wonder whether it would be a good idea, and good occasion to realize
>  it, to move source files and generated files into separate repositories
>  and 'link' those together using the git submodule feature.
>
>  Expected advantages:
>
>  - no clutter when looking through the history (can possibly be mitigated
>  by constraining git log, git diff etc. to the non-generated paths only,
>  although I don't think this is possible (cleanly) with the current
>  directory structure); the same holds true for using "git format-patch"
>  (one wouldn't usually want to include the generated files in diffs sent
>  to the mailing list)
>
>  - when merging branches, there will usually be no need to deal with
>  merge conflicts in the generated files (one would just regenerate them
>  instead)
>
>  - [especially for files being generated not by Gambit itself (for
>  example "configure"),] the files can be regenerated by differing
>  [external] software versions without having to deal with those
>  superfluous changes in the source repository.
>
>  By still committing the generated files--to a different
>  submodule--Gambit can still be updated through Git alone, and the
>  possible advantage of tracking the generated files to see the effects of
>  changes in compiler sources can still be had.
>
>  Expected disadvantages:
>
>  - all generated files need to reside in a separate directory structure;
>  e.g. the file $BASEDIR/lib/_io.c would have to be at a place like
>  $BASEDIR/build/lib/_io.c instead, where build/ is the submodule taking
>  all generated files; since the "configure" file is expected to reside at
>  the toplevel, I guess this would require that "make update" copies it
>  from $BASEDIR/build/configure to $BASEDIR/configure (assuming that one
>  cannot use a symlink because of portability reasons).
>
>  - to commit the generated files, a separate step is necessary ("cd
>  $otherrepo; git commit -a", or maybe easier create a "make
>  commit_generated" make target?)
>
>  - to make this work with the "source" repository residing at the
>  toplevel, the Git superproject repository (of which the "source" and
>  "build" repositories are submodules) would need to reside in a
>  non-standard directory, like $BASEDIR/.gitsuperproject/ instead of the
>  usual .git/, and using the GIT_DIR environment variable to access it,
>  although this can probably be handled by make targets (i.e. "make
>  update" would set GIT_DIR=$BASEDIR/.gitsuperproject when calling "git
>  submodule update").
>
>  - there may be some cases to flesh out; like, should "make update"
>  really call "git submodule update" (which simply sets the submodules to
>  the reference given by the superproject, throwing away changes done by
>  the user in the submodules (they can be recovered from the git reflog,
>  but may still be a surprise)) or should it run "git pull" in each
>  submodule instead?
>
>  I thought I'd bring this up now because if package maintainers need to
>  adapt some things anyway, that may be a good time to do it now. (There's
>  even the possibility to split the converted Mercurial repository into
>  the source + build parts in retrospect now, which won't be possible
>  anymore later on (without changing the sha1 sums of the whole Git
>  history with the associated breakage of existing clones), although that
>  may not be important.)
>
>  I'm willing to help in the effort, although I don't know the build tools
>  (autoconf and make) and their use in the setup well, so I would probably
>  be quite a bit lost when doing it alone.
>
>  Christian.
-- 
Duy



More information about the Gambit-list mailing list