GIT - HowTo's about "repositories"

mail

How to make a Git repository read-only ?

Situation

I have developed scripts on a server and versioned these with Git locally. This went fine until I had time to make this a little cleaner : At that point, on the initial server, scripts are overwritten by newer versions and there is no point in versioning them locally anymore since they are dynamically generated by already-versioned code. On the other hand, I don't want to lose the "old" development history, so I won't delete the local .git directory. My solution is then to make the repository read-only by refusing commits.

Details

To do so, I'll have to create a pre-commit hook :

Solution

Remember you can't paste TABs in terminals running GNU screen (), use SPACEs instead.
rootOfGitRepo='/path/to/gitRepository'
gitScript="$rootOfGitRepo/.git/hooks/pre-commit"

cat << ENDOFSCRIPT > "$gitScript"
#!/usr/bin/env bash

cat << ENDOFMESSAGE


	No more commits, sorry.


ENDOFMESSAGE
exit 1
ENDOFSCRIPT
chmod +x "$gitScript"
cat "$gitScript"
"$gitScript"; echo $?
mail

How to untrack files and reclaim disk space ?

Let's try this :

Script :

#!/usr/bin/env bash

workDir=$(mktemp --tmpdir -d tempDir_XXXXXXXX)
zerosFile="$workDir/zeros"
otherFile="$workDir/other"

getDirSize() {
	directory=$1
	message=$2
	echo -en "\t==>\tDirectory size [$message] : "
	du -sh $1
	}

countOccurrencesInGitLog() {
	fullPathToFile=$1
	trackedFile=$(basename "$1")
	echo -en "\t==>\tNUMBER OF OCCURRENCES OF '$trackedFile' IN 'git log' : "
	git log --name-status | grep -c "$trackedFile"
	}

# init
cd "$workDir"
git init
getDirSize "$workDir" 'Empty Git repository'

# create and commit "$otherFile"
echo 'this is an other file' > "$otherFile"
git a "$otherFile"
git co -m 'this is an other file'
getDirSize "$workDir" "After creating + committing the 'other' file"

# create and commit "$zerosFile"
dd if=/dev/zero bs=1K count=100 of="$zerosFile"

getDirSize "$workDir" "'zeros' file created but not committed yet"
git a "$zerosFile"
git co -m 'Lots of zeros !!! (initial commit)'
getDirSize "$workDir" "'zeros' file committed"

# append data to "$zerosFile" + commit
for i in {1..5}; do
	echo "hello world ($i)" >> "$zerosFile"
	git a "$zerosFile"
	git co -m "hello world ($i)"
	getDirSize "$workDir" "After hello world ($i) in 'zeros' file"
done

# cleaning the git repo
git gc
getDirSize "$workDir" "After 'git gc' (1)"

# untracking "$zerosFile"
git rm --cached "$zerosFile"
git co -m "rm --cached 'zeros'"
getDirSize "$workDir" "After untracking the 'zeros' file + commit"

# what about the git log (1) ?
countOccurrencesInGitLog "$zerosFile"

# cleaning the git repo
git gc
getDirSize "$workDir" "After 'git gc' (2)"

# removing the 'zeros' file
rm "$zerosFile"
getDirSize "$workDir" "After 'rm' of 'zeros' file"

# cleaning the git repo
git gc
getDirSize "$workDir" "After 'git gc' (3)"

# what about the git log (2) ?
countOccurrencesInGitLog "$zerosFile"

# cleaning before leaving
cd ..
[ -d "$workDir" ] && rm -rf "$workDir"

Output :

Initialized empty Git repository in /tmp/tempDir_FWkkVbkH/.git/
	==>	Directory size [Empty Git repository] : 35K	/tmp/tempDir_FWkkVbkH
[master (root-commit) 599b54f] this is an other file
 1 file changed, 1 insertion(+)
 create mode 100644 other
	==>	Directory size [After creating + committing the 'other' file] : 50K	/tmp/tempDir_FWkkVbkH
100+0 records in
100+0 records out
102400 bytes (102 kB, 100 KiB) copied, 0.000523495 s, 196 MB/s
	==>	Directory size ['zeros' file created but not committed yet] : 150K	/tmp/tempDir_FWkkVbkH
[master daa0106] Lots of zeros !!! (initial commit)
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 zeros
	==>	Directory size ['zeros' file committed] : 155K	/tmp/tempDir_FWkkVbkH
[master bc46b8c] hello world (1)
 1 file changed, 0 insertions(+), 0 deletions(-)
	==>	Directory size [After hello world (1) in 'zeros' file] : 162K	/tmp/tempDir_FWkkVbkH
[master 8ceab4f] hello world (2)
 1 file changed, 0 insertions(+), 0 deletions(-)
	==>	Directory size [After hello world (2) in 'zeros' file] : 168K	/tmp/tempDir_FWkkVbkH
[master d35a527] hello world (3)
 1 file changed, 0 insertions(+), 0 deletions(-)
	==>	Directory size [After hello world (3) in 'zeros' file] : 174K	/tmp/tempDir_FWkkVbkH
[master 7db7977] hello world (4)
 1 file changed, 0 insertions(+), 0 deletions(-)
	==>	Directory size [After hello world (4) in 'zeros' file] : 182K	/tmp/tempDir_FWkkVbkH
[master 6cd929a] hello world (5)
 1 file changed, 0 insertions(+), 0 deletions(-)
	==>	Directory size [After hello world (5) in 'zeros' file] : 188K	/tmp/tempDir_FWkkVbkH
Counting objects: 21, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (19/19), done.
Writing objects: 100% (21/21), done.
Total 21 (delta 6), reused 0 (delta 0)
	==>	Directory size [After 'git gc' (1)] : 153K	/tmp/tempDir_FWkkVbkH
rm 'zeros'
[master d68e9bd] rm --cached 'zeros'
 1 file changed, 0 insertions(+), 0 deletions(-)
 delete mode 100644 zeros
	==>	Directory size [After untracking the 'zeros' file + commit] : 156K	/tmp/tempDir_FWkkVbkH
	==>	NUMBER OF OCCURRENCES OF 'zeros' IN 'git log' : 9
Counting objects: 22, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (14/14), done.
Writing objects: 100% (22/22), done.
Total 22 (delta 6), reused 21 (delta 6)
	==>	Directory size [After 'git gc' (2)] : 154K	/tmp/tempDir_FWkkVbkH
	==>	Directory size [After 'rm' of 'zeros' file] : 53K	/tmp/tempDir_FWkkVbkH
Counting objects: 22, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (14/14), done.
Writing objects: 100% (22/22), done.
Total 22 (delta 6), reused 22 (delta 6)
	==>	Directory size [After 'git gc' (3)] : 53K	/tmp/tempDir_FWkkVbkH
	==>	NUMBER OF OCCURRENCES OF 'zeros' IN 'git log' : 9

What we can remember from this experience :

  1. an empty Git repository is about 100KB
  2. after a few commits, git gc helps shrinking the repository
  3. untracking a "big" file having some history but without deleting it (--cached mode) actually adds some data to the repository
  4. after git rm --cached fileIDontWantToTrackAnymore, the full history of fileIDontWantToTrackAnymore is still visible
  5. after git rm --cached, git gc does not make miracles
  6. the most efficient way to save space is to (mv|rm) fileIDontWantToTrackAnymore
  7. right after this again, git gc is helpless
  8. it is now impossible to checkout fileIDontWantToTrackAnymore since it's unknown to Git
  9. fileIDontWantToTrackAnymore still appears in git log, even though it's been untracked and deleted
    1. How to make fileIDontWantToTrackAnymore disappear from history ? Read this
    2. Is it relevant ?
mail

How to remove a file from the Git history ?

The solution below is a condensed / incomplete + updated version of an other article :

Quick answer (source):

git filter-repo --invert-paths --path fileToRemove
  • You may sometimes need the --force flag.
  • Depending on circumstances, Git _may_ not display special characters the way the shell does. For instance, é in a file name is displayed as \303\251 by Git (which is the corresponding Unicode value).
    Filenames with spaces or accented letters must be :
    • properly quoted
    • OR have special characters (spaces and accented letters) properly escaped

Detailed answer

For an automated way to do this (and more), see my removeFileFromGitHistory project on GitHub.
mail

How to extract part of a Git repository into a new independent repository ?

Situation

I have a Git repository like : 
.
├── dir1
├── dir2
│   └── myScript
├── dir3
└── .git
This was fine until today, but now I'd like myScript to live in its own distinct repository.

Details

The concept is to :
  1. clone the existing repository into a new place. This will become the new "myScript" repository
  2. remove everything from it which is not related to myScript itself
This can be done with git filter-branch git filter-repo.

Solution

This is not the complete solution : after running the steps below, there will be some work left not described here. But I guess you'll get the idea anyway
  1. let's play in a temporary directory so that we won't break anything :
    workDir='/run/shm/testGit'; sourceRepo='path/to/source/repo/'; mkdir -p "$workDir" && cd "$workDir" && git clone "$sourceRepo" .
  2. do the job :
  3. clean to retry :
    cd .. && [ -d "$workDir" ] && rm -Rf "$workDir"
What's left to do now you have your distinct "sub-repo" :
  1. move it out of its temporary location
  2. reset the remotes
  3. in the source repository : delete stuff you just extracted or you'll have duplicate contents and it'll become a nightmare

Alternate solution

This solution relies on git filter-branch and is now deprecated. It's there for "historical" reference only.
  1. find the root of the repository
  2. clone it :
    existingRepoRootDir='/root/of/existing/repository'; newRepoDir='/path/to/new/repository'; mkdir -p "$newRepoDir" && git clone "$existingRepoRootDir" "$newRepoDir"
    Cloning into '/path/to/new/repository'...
done.
  3. then enter the clone and keep only the dir2 stuff :
    cd "$newRepoDir" && git filter-branch --subdirectory-filter dir2 --prune-empty -- --all
    Rewrite dd39933a75594ac29d780de568e872fa22e1a480 (45/62) (1 seconds passed, remaining 0 predicted)
Ref 'refs/heads/master' was rewritten
Ref 'refs/remotes/origin/master' was rewritten
WARNING: Ref 'refs/remotes/origin/master' is unchanged
  4. The 2 previous steps as a big one-liner :
    existingRepoRootDir='/root/of/existing/repository'; relativePathToDirectoryToKeep='dir2'; newRepoDir='/path/to/new/repository'; mkdir -p "$newRepoDir" && git clone "$existingRepoRootDir" "$newRepoDir" && cd "$newRepoDir" && git filter-branch --subdirectory-filter "$relativePathToDirectoryToKeep" --prune-empty -- --all
  5. Don't forget the final clean up :
    • in this new repository : reset the remotes
    • in the source repository : delete stuff you just extracted (or you'll have duplicate contents and it'll become a nightmare )
mail

How to "get" all branches from a remote ?

Using the vague word "get" on purpose because I still don't know whether its a matter of pull, fetch, or anything else .

  • when you git clone a repository, you get everything, including branches.
  • they are not listed by git branch until you explicitly git checkout myBranch
  • to get further changes made on the original repository, you'll have to explicitly checkout + pull every branch (view short example).

Full demonstration :

  1. create a repository with several branches :
    baseDir='/run/shm/playingWithGit'; repoDir="$baseDir/repo"; mkdir -p "$repoDir"; cd "$repoDir"; git init; echo 'hello world' > myFile; git add myFile; git commit myFile -m 'initial commit (on master)'; for i in {1..3}; do branchName="branch$i"; git checkout -b "$branchName"; echo "hello from '$branchName'" >> myFile; git commit myFile -m "commit from '$branchName'"; git checkout master; done; git branch
    Initialized empty Git repository in /dev/shm/playingWithGit/repo/.git/
[master (root-commit) 2d79b06] initial commit (on master)
 1 file changed, 1 insertion(+)
 create mode 100644 myFile
Switched to a new branch 'branch1'
[branch1 407687e] commit from 'branch1'
 1 file changed, 1 insertion(+)
Switched to branch 'master'
Switched to a new branch 'branch2'
[branch2 8680357] commit from 'branch2'
 1 file changed, 1 insertion(+)
Switched to branch 'master'
Switched to a new branch 'branch3'
[branch3 4188c2c] commit from 'branch3'
 1 file changed, 1 insertion(+)
Switched to branch 'master'
  branch1
  branch2
  branch3
* master
  2. view branches content :
    for branch in $(git branch | tr -d '* '); do git checkout "$branch"; cat myFile; echo; done
    Switched to branch 'branch1'
hello world
hello from 'branch1'

Switched to branch 'branch2'
hello world
hello from 'branch2'

Switched to branch 'branch3'
hello world
hello from 'branch3'

Switched to branch 'master'
hello world
  3. create a clone and try to get everything from the remote :
    cloneDir="$baseDir/clone"; mkdir -p "$cloneDir"; cd "$cloneDir"; git clone "$repoDir" .
    Cloning into '.'...
done.
    git branch; cat myFile
    * master
hello world
    for branch in branch1 branch2 branch3; do git checkout "$branch"; cat myFile; echo; done
    Branch 'branch1' set up to track remote branch 'branch1' from 'origin'.
Switched to a new branch 'branch1'
hello world
hello from 'branch1'

Branch 'branch2' set up to track remote branch 'branch2' from 'origin'.
Switched to a new branch 'branch2'
hello world
hello from 'branch2'

Branch 'branch3' set up to track remote branch 'branch3' from 'origin'.
Switched to a new branch 'branch3'
hello world
hello from 'branch3'
    git branch
      branch1
  branch2
* branch3
  master
    All branches are actually there .
  4. make some changes on the remote repository :
    cd "$repoDir" && for branch in master branch1 branch2 branch3; do git checkout "$branch"; echo "I made this commit on branch '$branch' after the 'clone'" >> myFile; git add myFile; git commit -m "another commit on '$branch'"; done
    Already on 'master'
[master 28b51d1] another commit on 'master'
 1 file changed, 1 insertion(+)
Switched to branch 'branch1'
[branch1 0ff4301] another commit on 'branch1'
 1 file changed, 1 insertion(+)
Switched to branch 'branch2'
[branch2 fcab109] another commit on 'branch2'
 1 file changed, 1 insertion(+)
Switched to branch 'branch3'
[branch3 fcf5830] another commit on 'branch3'
 1 file changed, 1 insertion(+)
  5. view branches content : 
    Switched to branch 'branch1'
hello world
hello from 'branch1'
I made this commit on branch 'branch1' after the 'clone'

Switched to branch 'branch2'
hello world
hello from 'branch2'
I made this commit on branch 'branch2' after the 'clone'

Switched to branch 'branch3'
hello world
hello from 'branch3'
I made this commit on branch 'branch3' after the 'clone'

Switched to branch 'master'
hello world
I made this commit on branch 'master' after the 'clone'
  6. will I get these ?
    cd "$cloneDir" && git fetch
    remote: Enumerating objects: 20, done.
remote: Counting objects: 100% (20/20), done.
remote: Compressing objects: 100% (8/8), done.
remote: Total 12 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (12/12), done.
From /run/shm/playingWithGit/repo
	4188c2c..fcf5830  branch3	-> origin/branch3
	407687e..0ff4301  branch1	-> origin/branch1
	8680357..fcab109  branch2	-> origin/branch2
	2d79b06..28b51d1  master	-> origin/master
    git branch
      branch1
  branch2
* branch3
  master
  7. view branches content again : 
    Switched to branch 'branch1'
Your branch is behind 'origin/branch1' by 1 commit, and can be fast-forwarded.
  (use "git pull" to update your local branch)			thank you, Git 
hello world
hello from 'branch1'

Switched to branch 'branch2'
Your branch is behind 'origin/branch2' by 1 commit, and can be fast-forwarded.
  (use "git pull" to update your local branch)
hello world
hello from 'branch2'

Switched to branch 'branch3'
Your branch is behind 'origin/branch3' by 1 commit, and can be fast-forwarded.
  (use "git pull" to update your local branch)
hello world
hello from 'branch3'

Switched to branch 'master'
Your branch is behind 'origin/master' by 1 commit, and can be fast-forwarded.
  (use "git pull" to update your local branch)
hello world
  8. git branch; git pull; cat myFile
      branch1
  branch2
  branch3
* master
Updating 5784923..5ccb801
Fast-forward
 myFile | 1 +
 1 file changed, 1 insertion(+)
Current branch master is up to date.
hello world
I made this commit on branch 'master' after the 'clone'
  9. Looks like there is no Git command to do this automatically for all branches at once. Instead :
    git checkout branch1
    Switched to branch 'branch1'
Your branch is behind 'origin/branch1' by 1 commit, and can be fast-forwarded.	As seen above
  (use "git pull" to update your local branch)						Ok, I'll do this !
    git pull
    Updating b5e84db..6f16d51
Fast-forward
 myFile | 1 +
 1 file changed, 1 insertion(+)
Current branch branch1 is up to date.
    cat myFile
    hello world
hello from 'branch1'
I made this commit on branch 'branch1' after the 'clone'
  10. the generic command is then :
    for branch in $(git branch | tr -d '* '); do git checkout "$branch"; git pull; cat myFile; echo; done
    Already on 'branch1'
Your branch is up to date with 'origin/branch1'.
Already up to date.
Current branch branch1 is up to date.
hello world
hello from 'branch1'
I made this commit on branch 'branch1' after the 'clone'

Switched to branch 'branch2'
Your branch is behind 'origin/branch2' by 1 commit, and can be fast-forwarded.
  (use "git pull" to update your local branch)
Updating 1e4eef9..56d7474
Fast-forward
 myFile | 1 +
 1 file changed, 1 insertion(+)
Current branch branch2 is up to date.
hello world
hello from 'branch2'
I made this commit on branch 'branch2' after the 'clone'

Switched to branch 'branch3'
Your branch is behind 'origin/branch3' by 1 commit, and can be fast-forwarded.
  (use "git pull" to update your local branch)
Updating 545c032..ab21a4f
Fast-forward
 myFile | 1 +
 1 file changed, 1 insertion(+)
Current branch branch3 is up to date.
hello world
hello from 'branch3'
I made this commit on branch 'branch3' after the 'clone'

Switched to branch 'master'
Your branch is up to date with 'origin/master'.
Already up to date.
Current branch master is up to date.
hello world
I made this commit on branch 'master' after the 'clone'
  11. (maybe I'll find further stuff to test / question / investigate / show... ;-)
  12. clean everything before leaving :
    for dirToDelete in "$repoDir" "$cloneDir"; do cd "$baseDir" && [ -d "$dirToDelete" ] && rm -rf "$dirToDelete"; done; cd .. && rmdir "$baseDir"

Replay the full demo with the final git checkout + git pull :

  1. create a repository with several branches
  2. clone it
  3. make some changes on the remote repository
  4. get changes on the cloned repository :
    cd "$cloneDir"; for branch in master branch1 branch2 branch3; do git checkout "$branch"; git pull; cat myFile; done
  5. clean everything before leaving
mail

How to find the root directory of a Git repository ?

mail

Why doesn't git status show whether I'm up-to-date with my remote counterpart ?

Situation

git status 
On branch master
nothing to commit (use -u to show untracked files)
I expected something like : 
On branch master
Your branch is ahead of 'origin/master' by 2 commits.

Details

Try these :
  1. git branch -vv 
    * master 6f3d70e (a very interesting commit message)
    should be : 
    * master 35ca599 [origin/master] (another highly interesting commit message)
  2. git rev-parse --symbolic-full-name "@{u}" 
    fatal: No upstream configured for branch 'master'
    should be : 
    refs/remotes/origin/master

This is because the master local branch has no upstream origin/master remote branch to be compared to.

The presence or absence of an upstream setting mainly affects whether git status can tell you if you are ahead or behind, and whether git merge and git rebase can do their job with no additional parameters. So it's basically just a convenience setting.

Solution

  1. Define a remote-tracking-branch for the current branch
  2. Retry the commands above :
    1. git branch -vv 
      * master 6f3d70e [origin/master: ahead 7] (a very interesting commit message)
    2. git rev-parse --symbolic-full-name "@{u}" 
      refs/remotes/origin/master
  3. And :
    git status
    On branch master
Your branch is ahead of 'origin/master' by 7 commits.
  4. You may later wish to stop tracking the remote branch.
mail

How to untrack a file ... ?

... and delete it from the local filesystem :
  1. git rm fileToUntrack
  2. git commit fileToUntrack
... without deleting it from the local filesystem :
git rm --cached fileToUntrack
If you tracked fileToUntrack by mistake, you may be interested in these articles :

To untrack an entire directory and delete it :

  1. git rm -r path/to/directory/*
  2. then make a simple commit : git commit -m 'cleaning'
mail

How to shrink the .git directory ?

A quick + easy + low-gain solution :

git gc --aggressive --prune
This usually results in a ~10% volume decrease, but works only once. Next time it proves useful will be in months, after working with the repository.
Looks like I'm doing it wrong : read details.

Harder + more effective solution : remove old versions of binary files

  1. list binary files and the storage space they use with this script
  2. from now on, several strategies :
    • keep only the latest version of each binary file
      • sounds nice, but if you versioned it in the 1st place, maybe that's because you wanted to have a few "previous" versions, for any reason ?
      • this leaves you with the latest version of the file with no more history : forget everything about the past, only the present exists ;-)
      Have a look at removeFileFromGitHistory which does exactly this
    • as a workaround to the option above, keep only the n latest versions of a binary file
      • i.e. lose oldest part of the history and keep only the most recent part (which matters most)
      • no idea how to do that (but it sounds possible)
      • n could be :
        • a fixed number
        • determined by date, for instance keep commits of binaryFile that are younger than 3 months :
          git log --oneline --after='3 months' path/to/binaryFile | wc -l
    • squash intermediate commits :
      1. let's consider binaryFile has been committed 20 times throughout the repository history
      2. I keep commit #1
      3. I squash commits 2 to 5 into a single commit
      4. I squash commits 6-10, 11-15, 16-20, so that 20 commits turn into 5 commits
      This actually looks more complex. What if binaryFile was committed with other files ?
mail

How to hide untracked files from git status ?

Once this is defined as an alias in ~/.gitconfig, you may list untracked files with : git status --untracked-files
mail

How may I push to a non-bare repository ?

This is dirty1000 and is actually not recommended.

Situation

Before things are configured : can not push to non-bare repo

git push origin master
Counting objects: 6, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (6/6), done.
Writing objects: 100% (6/6), 697 bytes | 0 bytes/s, done.
Total 6 (delta 4), reused 0 (delta 0)
remote: error: refusing to update checked out branch: refs/heads/master
remote: error: By default, updating the current branch in a non-bare repository				the error
remote: error: is denied, because it will make the index and work tree inconsistent
remote: error: with what you pushed, and will require 'git reset --hard' to match
remote: error: the work tree to HEAD.
remote: error:
remote: error: You can set 'receive.denyCurrentBranch' configuration variable to
remote: error: 'ignore' or 'warn' in the remote repository to allow pushing into			the solution
remote: error: its current branch; however, this is not recommended unless you
remote: error: arranged to update its work tree to match what you pushed in some
remote: error: other way.
remote: error:
remote: error: To squelch this message and still keep the default behavior, set				a workaround
remote: error: 'receive.denyCurrentBranch' configuration variable to 'refuse'.
To gitServer:/path/to/repository
 ! [remote rejected] master -> master (branch is currently checked out)
error: failed to push some refs to 'stuart@gitServer:/path/to/repository'

Solution

Get the initial configuration value

The error message says that, on the gitServer side, we just have to change the value of the receive.denyCurrentBranch directive to ignore or warn. Alright but -just in case- what's the original value of this Git configuration directive ?
git config --get receive.denyCurrentBranch
(nothing but UNIX FAILURE exit code)
This directive is currently unset.

Change the configuration value (source) :

  1. git config receive.denyCurrentBranch ignore
  2. check :
    git config --get receive.denyCurrentBranch
    ignore

With Git 2.3.0+, it is possible to safely push to a non-bare repository with :

git config receive.denyCurrentBranch updateInstead

Let's try to push again :

git push origin master 
Counting objects: 6, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (6/6), done.
Writing objects: 100% (6/6), 697 bytes | 0 bytes/s, done.
Total 6 (delta 4), reused 0 (delta 0)
To gitServer:/path/to/repository
	f5c8ce7..629a87e	master -> master

Hack you have to do after each push to a non-bare repository :

On the gitServer side :
git reset --hard
  • As for why you have to do so, refer to the large Git error message above.
  • On gitServer side, since the index and the working tree are out-of-sync, "addition changes" made by a git push will be reported as "remove changes" by git status (hence the need of the git reset).
  • If there are some uncommitted changes on gitServer, there is a risk of data loss.
mail

How to remove sensitive data (such as passwords, private keys, ...) from the Git history ?

  • Any information that has been committed to a public repository must be considered public, then compromised. So :
    1. clean up your mess
    2. forget about the disclosed passwords or keys, and generate new ones
  • The method below relies on git filter-branch, which has been obsoleted by git filter-repo.
  1. Navigate to the repository's working directory :
    cd root/of/git/repository
  2. Then run a long git filter-branch command :
    git filter-branch --force --index-filter 'git rm --cached --ignore-unmatch path/to/fileContainingSensitiveData' --prune-empty --tag-name-filter cat -- --all
    which :
    • --force : forces Git to process
    • --index-filter '' : apply the specified filter (here : delete a file) to all commits :
      git rm --cached --ignore-unmatch path/to/fileContainingSensitiveData
    • --prune-empty : should an affected commit end up as an empty commit, remove it
    • --tag-name-filter cat : update the tags ( for details)
    • -- --all : affect all branches and tags
    You have to specify path/to/fileContainingSensitiveData —i.e. the path relative to the root of the repository—, not just the file name.
    Example: 
    git filter-branch --force --index-filter 'git rm --cached --ignore-unmatch ./test' --prune-empty --tag-name-filter cat -- --all
    Rewrite de1291f7f10da919f611358539e3d2e0acfd632f (1/9)rm 'test'
Rewrite 023a969f60328055b6cab2e1a82e307760b7a308 (2/9)rm 'test'
Rewrite aaffedfe16c297d02c2fee5eb72b740c91cae942 (3/9)rm 'test'
Rewrite 8a766a9226e360ba0f147ae9c890cec200f958d7 (4/9)rm 'test'
Rewrite 1f9bd643aa3495f46ca9470c021a56f8a0765c62 (5/9)rm 'test'
Rewrite 2e22a397f2283f398038d9a44f451914c0a58f68 (6/9)rm 'test'
Rewrite 15f247b46d558b057ccead7f7980c7f24b58879c (7/9)rm 'test'
Rewrite e54ff82df52d8e7d02749b17e4fac4aa9386122e (8/9)rm 'test'
Rewrite 5bbcaf42adac870f13295b2bbe2673807bf1e059 (9/9)rm 'test'

Ref 'refs/heads/master' was rewritten
Ref 'refs/remotes/origin/master' was rewritten
  3. If fileContainingSensitiveData used to exist with a different path or name (git mv ), repeat the previous step for each previous path or name.
  4. Add fileContainingSensitiveData to .gitignore to prevent committing again by mistake :
    echo fileContainingSensitiveData >> .gitignore; git add .gitignore; git commit -m "I won't commit fileContainingSensitiveData again."
  5. Make sure your history is clean :
    git status
    git log --name-status
    http://stackoverflow.com/questions/1338728/delete-commits-from-a-branch-in-git
git reset --hard commitId	WARNING, this deletes all commits from the current point back to commitId. Use with care !!!
  6. Force-push your local changes to overwrite your public repository, as well as all the branches you've pushed up :
    git push origin --force --all
  7. In order to remove the sensitive file from your tagged releases, you'll also need to force-push against your Git tags :
    git push origin --force --tags
  8. Ask others to rebase instead of merging to avoid sending back the compromised data
  9. After a while, once sure we've not broken anything (and with Git ≥ 1.8.5) : 
    git for-each-ref --format='delete %(refname)' refs/original | git update-ref --stdin
git reflog expire --expire=now --all
git gc --prune=now
mail

How to import an existing Git repository into another ?

Let's consider 2 repositories : To do so :

Before starting this procedure, make sure there are no uncommitted changes on both repositories. commit or stash them accordingly.

  1. cd path/to/bigRepo
  2. git remote add small path/to/smallRepo
  3. git fetch small
  4. git checkout -b small small/master
    At this step, all files are imported into the root of "bigRepo". See the "git ls-files | xargs mv + commit" hack below
  5. git checkout master
  6. git merge small --allow-unrelated-histories
  7. git remote rm small
  8. git branch -d small

pathToBigRepo='/path/to/bigRepo'; pathToSmallRepo='/path/to/smallRepo'; pathToSmallRepoWithinBigRepo="$pathToBigRepo/smallRepo"; mkdir -p "$pathToSmallRepoWithinBigRepo"; cd "$pathToBigRepo"; git remote add remoteRepository "$pathToSmallRepo"; git fetch remoteRepository; git checkout -b newLocalBranchForRemoteData remoteRepository/master; git ls-files | xargs -I {} git mv {} "$pathToSmallRepoWithinBigRepo"; git co -am "Moved into '$pathToSmallRepoWithinBigRepo'"; git checkout master; git merge newLocalBranchForRemoteData --allow-unrelated-histories; git remote rm remoteRepository; git branch -d newLocalBranchForRemoteData
mail

How to get a Git configuration value ?

mail

How to contribute / perform a pull request ?

  1. Fork source project from GitHub UI
  2. Get the URL of your forked version of the project : https://github.com/myGitHubLogin/projectName.git
  3. Get a local copy of your forked repository : git clone https://github.com/myGitHubLogin/projectName.git
  4. Enter your local work directory : cd projectName
  5. Register the remote repo (named upstream by GitHub) : git remote add upstream https://github.com/projectOwner/projectName.git
  6. git fetch upstream
  7. Code, and commit to the local repository. Then push to your GitHub repository : git push origin master
  8. On GitHub UI, open your fork's webpage, then Pull-request
  9. Add title and details to the pull request, then submit.
  10. Done !
Stuff below could be obsolete / wrong

git clone https://github.com/sni/Thruk.git
git remote add --track master upstream git://github.com/Httqm/Thruk.git
git branch newFeature
git checkout newFeature
[do some code modifications, commit]
git push upstream newFeature (?)
mail

How to checkout / clone an existing project ?

checkout is the SVN command to get a local copy of a project from its main repository (when joining a team, for instance).
Git also has a checkout command, but it has a different meaning.
The Git equivalent of SVN checkout is git clone.

clone from a remote host over SSH (I can log in there with ssh stuart@server) :
git clone stuart@server:/path/to/repository /local/path
  • on server, there is /path/to/repository/.git/
  • locally, I'll get /local/path/.git/
clone over SSH and "git checkout" the myDevelopmentBranch branch :
git clone git@host:/path/to/myGreatProject.git -b myDevelopmentBranch /path/to/local/directory
If /path/to/local/directory is not specified, the files will be extracted as ./myGreatProject/*