Git folder relocation - Part one
This post was initially published during June 2019 on both i-logs blog and Medium.
Introduction
Today, we had to move a folder from one git repository to another. We wanted to preserve history of that folder into the destination repository. We also wanted to remove that folder from the source repository.
Because git command line instructions can be really cryptic, and because there is one or two tricks to know, we are sharing our experience to complete the process.
Initial situation
In the beginning we have a repository holding the source code of a web platform and the source code of the related presentational website. As time goes by, we understood that having both projects in the same repository was far from being an ideal situation:
- both website and the platform does not share code
- they have different deployment processes
- they have different lifetime between releases
- issues pertaining to one project have no impact on the other project.
It became obvious to us that the website part should live in its own project. The ascii image below shows the initial situation. We had to move the website folder to its own git repository. The requirements were to keep the website folder history in the destination-repository and completely get rid of this folder in the source-repository.
In this post, we cover the first requirement: move the folder with history from one repository to another.
To avoid a too long article, we will cover the second requirement (removing the folder with history from the source repository) in a future post.
+--------------------------------------+
| source-repository |
+---------|----------------------------+
| +-----------------+
| | Website/ |
+----------> Other/ |
| folers/ |
| and.txt |
| files.txt |
+-----------------+
+--------------------------------------+
| destination-repository |
+---------|----------------------------+
| +------------------+
+---------> (empty) |
+------------------+
Moving the folder
Step 1: prepping the source repository
The first step is to get a clean copy of the repository.
# clone the source repository
git clone https://<server.com>/<path>/source-repository.git
...
# move into that source repository local folder
cd source-repository/
To avoid committing our changes to the source repository, let's cut the link between the local repository and its origin:
git remote rm origin
Next, we clean up the local source repository to only keep the folder(s) we want to move:
git filter-branch --subdirectory-filter <folder> -- -- all
Note: replace <folder> with the name of the folder you want to move.
This is the first trick: we are using the filter-branch git command to
rewrite our repository revision history and apply a custom filter on each
of these revisions. The parameter --subdirectory-filter allows us to
specify the kind of filter we want to use: only keep the history which is
related to the given sub-directory. The result will contain that directory
(and only that) as its project root. Finally, we provide the --all
shortcut to specify we want to inspect all the revision.
It is important to note that when this command completes, the whole contents of the target folder is moved to the root of the repository, as it is shown in the picture below.
Initial directory structure:
+--------+
| / | local repository structure
+-|------+
| +-----------------+
+----> Some/ |
| Other/ |
| Folders/ |
| Website/ |
+----|------------+
| +-----------------+
+----> Folder1/ |
| Folder2/ |
| File1 |
| ... |
+-----------------+
Directory structure after filter-branch:
+--------+
| / | local repository structure
+-|------+
| +-----------------+
+-----> Folder1/ |
| Folder2/ |
| File1 |
| ... |
+-----------------+
It is now time to commit the local changes:
git commit -m"folder website is ready to take off"
We don't want to push our changes to the remote repository (we would loose all other files and folders). As a safety measure, we have remove the link between the local source repository and the remote one in a previous step. If you haven't, please make sure to not push your changes to the remote repository!
Step 2: prepping the target repository:
First we want to create the target repository. As we are using gitlab, this is done in gitlab interface by clicking the new project button and selecting the empty repository option.
Next we clone the new remote target repository in a folder next to the local source repository folder:
# cloning the target repository
cd ..
git clone https://server.com/path/destination-repository.git
...
cd destination-repository/
Step 3: moving the folder:
To actually perform the move of the folder between the source and the destination repositories, we use the second trick of the day: connect the source repository as a remote using a local reference. Let's use git remote to add a new remote to the local destination-repository using a local folder as the target URL!
git remote add source-rep ../source-repository
This is magic. Now we are able:
- to fetch the source from the local source-repository folder and import them in the local destination-repository.
- create a branch with the the source-rep remote contents
- merge the new branch into the master branch of the local target repository
# fetch the source from the remote
git fetch source-rep
...
# branch from master
git branch source-br remotes/source-rep/master
...
# merge the branch locally
git merge source-br
...
Step 4: Cleaning and pushing
We are nearly done. It is now time to clean the local branch, the local remote reference and push the local changes to the destination-repository remote:
# remove the remote reference
git remote rm source-rep
...
# delete the merged branch
git branch -d source-br
...
# push our local changes to the origin
git push origin master
...
Note: What if the target repository is not empty?
In our situation, the target repository was initially empty. We wanted the contents of the website/ folder to become the root of the destination repository.
In some case, you might want to move a folder to a repository with already existing files and folders. In that context, the easiest way is to group all the data to be moved under one single to directory:
- After you have successfully ran the
git filter-branchcommand in the source repository, create a new directory - move the data into the target_folder:
- Commit your changes but don't push them!
# create the new folder
mkdir target_folder
# group data in the new folder
git mv <all the stuff you want to move> target_folder
# commit your modification
git commit -m"grouping files to move one folder"
Please make sure that the new directory name (target_folder in the example above) does not cause a collision with an existing directory in the destination-repository or you might be in for some trouble.
In our next article, we will cover the second requirement which is how to get rid of a folder along with its history from a git repository.
If you are here for copy/paste only:
# clone the source repository
# git clone <your-repository-url>, EG:
git clone https://server.com/path/source-repository.git
# mv to the source repository
cd source-repository/
# cut the link with the source remote repository
git remote rm origin
# delete all entries and history in you repo except the one you need
git filter-branch --subdirectory-filter <directory to move> -- -- all
# commit the changes locally (don't try to push)
git commit -m"folder website ready to take off"
# clone the destination repository
cd ..
git clone https://server.com/path/destination-repository.git
cd destination-repository/
# Add the local source repository as a remote to your destination
# repository
git remote add source-rep ../source-repository
# Fetch the remote source
git fetch source-rep
# create a branch
git branch source-br remotes/source-br/master
# merge the new branch
git merge source-br
# clean up the remote branch
git remote rm source-rep
# clean up branch
git branch -d source-br
# publish our changes
git push origin master