[lammps-users] Can't push a pruned history to github (#2987)

I attempted to work with a pruned history as suggested in 2987 for a
work around to the 100MB large blob issue

and discovered this also doesn't work with github as they don't let
you push pruned histories

$ git push test
...
To github.com:twhitehead/lammps-new.git
! [remote rejected] develop -> develop (shallow update not allowed)
error: failed to push some refs to 'github.com:twhitehead/lammps-new.git'

Thanks! -Tyson

Tyson,

For the people that read only this email exchange (and I had asked to move it here, because I think there are others reading here that might be interested to know how to address this situation) and not the discussion on github, let me prefix this with some background.

you are asking about this, because you want to create a private git repository of the LAMMPS repository on github for maintaining custom branches, yet that is not possible because of size limitations imposed by github for committing large files (and there have been a few that are over the limit a long time ago and then some more recent that trigger a warning).

after doing some research on my own and also based on my experience with maintaining my own private LAMMPS git repository (at the time when LAMMPS was still maintained with subversion and the official git repository could not be merged to) I have come to the following conclusions.

  • there is no real solution currently. we cannot “prune” the git history of the LAMMPS repository as that would change all commits beginning with the first deleted commit and then diverge the official repository from everybody else.
  • you can skip making the repository private and create a fork instead (github does not allow to switch forks from public to private)
  • you could ask the github admins to make an exception
  • you can try a different provider (i recall having no problems importing the - already inflated - LAMMPS repo into bitbucket, for example).
    for submitting pull requests, you can then create a public fork on github. I am keeping a public and a private repo on bitbucket alive to this day and in sync with the github repo for more than 5 years now.
  • you can set up a “bare” git repository on a (cloud? or local) server that you and others can access with ssh keys (but without the benefit of issues, pull requests etc.)
  • you can have a private repo on github that you seed with source files from a current snapshot, i.e. that will have an entirely different git history. you should still be able to import from the public repo through rebasing merges, but the git history will be different and you’ll have to make sure that you follow the changes from upstream consistently. if i understand correctly, this is not much of a limitation, because you will not be able to submit a meaningful pull request from a private, non-forked repository to the LAMMPS repo anyway. so you would also have to create a public fork where you create a feature branch for the expressed purpose of creating a pull request from a suitable diff/patch from the private repo. i have done something similar for multiple years and it was feasible, but requires some discipline.

my personal preference would be to just go with the second option and use a public repo, since I believe a) that we are not in a field, where there are lots of people just waiting to “steal” somebody else’s work and derive a significant profit/benefit from it and b) most scientific software is almost like it is encrypted without detailed documentation/tutorials and the knowledge/experience of the developers that wrote it. but I also acknowledge that not everybody shares my view on this.

So, if somebody else has an idea for an alternative that is not mentioned here, please speak up and share it. as mentioned before, I am certain that there are more people than just Tyson that would love to see a proper solution to this issue rather than just some workarounds with limitations.

cheers,
axel.