PBS 113 of X: My First Remote (Git)
In the previous instalment we introduced the concepts and jargon around git repositories interacting with each other. Or, to put it into Git jargon, we learned about remotes. We introduced the topic with some simplified example scenarios where you would use Git Remotes (basically all the time), and we introduced the concept of bare repositories.
We also learned that Git Remotes are accessed via URLs, and that there are multiple possible URL schemes for accessing them, but that we would only be using two, local access when the remote is on the same file system as the repository connecting to it, and HTTPS when we’re dealing with cloud-hosted remotes.
In this instalment we’ll get practical by making a start on implementing the first scenario from the previous instalment, the lone developer with a NAS who wishes to keep a copy of their work on their NAS both as a backup, and, to facilitate switching between computers. We’ll focus on just the first of those motivations in this instalment — using a remote repository for backup. We’ll leave facilitating working from multiple computers until next time.
Matching Podcast Episode
Listen along to this instalment on episode 679 of the Chit Chat Across the Pond Podcast.
You can also Download the MP3
Instalment Resources
- The instalment ZIP file — pbs113.zip.
Playing Along
If you’d like to play along with the examples you’ll need to download this instalment’s ZIP file and unzip it. Open a terminal and change to the folder into which you extracted the ZIP . You’ll find a bash script named pbs113-init.sh as well as a bundle file name pbs113a.bundle.
This script automates the steps we’ve been doing manually in the past few instalments to convert bundles into repositories:
- create a folder named
pbs113a - change into that folder
- initialise it as a Git repo
- if needed, change the default branch from
mastertomain - import all branches and tags from the bundle
One you have a terminal open in the folder the ZIP extracted to you’re ready to run the script. We’ll start by making sure the script is executable:
chmod 755 pbs113-init.sh
Now we can execute it with:
./pbs113-init.sh
If you view the contents of the folder (in your file manager or on the Terminal with the ls command) you’ll see that a new folder has been created named pbs113a, this is our repository.
Bonus Extra — Echoing Commands in a Shell Script
You may have noticed that as the shell script executed it showed each command executed pre-fixed with a + symbol. This is not Bash’s default behaviour — it normally only shows the outputs generated by the commands the commands the script executes. It can often be helpful to watch what a script does, so Bash provides an optional -x flag to enable command echoing.
You’ll notice that the so-called shebang line at the very top of the script specifies that the OS should execute the script with bash with the -x flag:
#! /bin/bash -x
Note that Zsh also supports the -x flag.
Creating a Bare Repository
Up until this point we’ve not created any Git repositories we’ve not done coding work directly within. In Git speak, we’ve used the working tree in all our repositories to do our development work. Think of all our repositories so far as working repositories.
As we start to use remote repositories as backups or canonical sources, that assumption falls apart. You’re never going to actively write code in the backup repository on your NAS, or, on the canonical copy of your team’s repository on the server in work. I think if these as storage repositories, and they have no need of a working tree.
Git supports repositories without working trees, and it has a special name for them — bare repositories.
Our new backup repository should be a bare repository, so let’s start by creating one.
In a real-work scenario you’d probably create this bare repository on some kind of NAS or shared drive, but we’ll keep things simple and simply create our in the folder we’re working in (the one with the init script, bundle file, and the pbs113a repo). We’ll name our bare repository pbs113a-backup.git.
To create our bare repository we first create a folder with the desired name, then we initialise it as a bare Git repository using the git init command with the --bare flag:
mkdir pbs113a-backup.git
git init --bare pbs113a-backup.git
Adding a Remote to an Existing Repository
Let’s now enter our primary repository, and add the bare backup repository to it as a remote.
The first step is to change into our repository and check the proverbial lay of the land:
bart-imac2018:pbs113 bart% cd pbs113a
bart-imac2018:pbs113a bart% git status
On branch main
nothing to commit, working tree clean
bart-imac2018:pbs113a bart%
As you can see we’re on the main branch with a clean working tree.
We can also see that this repository has no remotes configured yet by running the git remote command without any sub-commands:
bart-imac2018:pbs113a bart% git remote
bart-imac2018:pbs113a bart%
Let’s use the add sub-command of git remote to define a new remote with the name backup that’s linked to our newly created bare repository.
The git remote add command takes two arguments, a name for the remote, and the URL to access the remote repository. In this case the remote repository is on the local file system, so we’ll use a simple file path as the URL, specifically, we’ll use the relative file path ../pbs113a-backup.git (.. means my parent folder on the terminal).
git remote add backup ../pbs113a-backup.git
We can now see that the remote exists by invoking git remote without any sub-commands or flags:
bart-imac2018:pbs113a bart% git remote
backup
bart-imac2018:pbs113a bart%
Or, more usefully, we can see the details of the defined remotes by adding the -v (for verbose) flag:
bart-imac2018:pbs113a bart% git remote -v
backup ../pbs113a-backup.git (fetch)
backup ../pbs113a-backup.git (push)
bart-imac2018:pbs113a bart%
Fetching & Viewing Remote Branches & Tags
For Git to learn the contents of a remote repository, we have to ask it to retrieve the needed information with the git fetch command. We can pass the command one or more remote names, or the flag --all. Let’s fetch the details from our backup repository:
git fetch backup
We can now list all the remote branches that our repository knows about by passing the -r flag to the git branch command:
git branch -r
We get no output because our only remote contains absolutely nothing.
Note that this command lists all fetched branches on all configured remotes, it does not read the data from the remote itself. This is why it’s important to fetch before trying to work with remote branches.
Viewing remote tags does not work quite as you might expect — since you view local branches with git branch, and remote branches with git branch -r, you might assume that since you view local tags with git tag you could view remote tags with git tag -r, but alas not 🙁
The reason for this is very simple — remote tags are not fetched by the git fetch command, so they can’t be read from the local cache. To view remote branches you need to read the data directly from the remote repository.
The command for reading information directly from a remote repository of git ls-remote. By default it will list both branches and tags, but you can limit it to just tags with the --tags flag.
So, the command to read all branches and tags directly from our backup remote is:
git ls-remote backup
And the command to see just the tags is:
git ls-remote --tags backup
Again, since our remote repository is completely empty, both of these command return nothing at all.
Put a mental pin in these commands for a moment — we’ll use them shortly to verify that our backup has been properly initialised.
Pushing Everything to the Remote
We want our remote to be a full backup of everything in our repository, so we want to send all branches and tags to the remote. We can do this with the --all and --tags flags and the git push command:
First, push all the branches:
git push backup --all
Then push all the tags:
git push backup --tags
Viewing Remote Branches & Tags Revisited
We can verify that we have pushed all the data to our backup using the git ls-remote command:
bart-imac2018:pbs113a bart% git ls-remote backup
d86b2e36f75ddf958c2a9eddd9868e8ce9f1474b refs/heads/main
ff8bc623afaabb9c2bcde5bc1bc492dc685a2f3e refs/tags/v1.0.0
43bca0f7cc7befa88172f217f4d74b3ec58dffc6 refs/tags/v1.1.0
42b42d674716c55d25e4728a0fc8b25823a8e0ed refs/tags/v2.0.0
4f085d37edf75bb8b157793ccf71d48057a9fe27 refs/tags/v2.1.0
069204cbe50a068c512f152ce37dac9e575c76d5 refs/tags/v2.1.1
8f4e695bd09b458c1c09960cfa73fe37fc853282 refs/tags/v2.2.0
4793fd78939e78f3416e6b966e290651589d817b refs/tags/v2.3.0
275f564428b0aa21b50ab40a4fd4a674a444f2b3 refs/tags/v2.4.0
b45a1e487e4f7f09480de3e5fb2ee4f21737ecae refs/tags/v2.4.1
74d6dd0f450097c5c6a792ee8841776cc7f88739 refs/tags/v2.5.0
3e29f1fe0cab9488439bca2c115b012998e45ab3 refs/tags/v2.5.1
b70fb8d6001bbe689786466006f6649242b0c55e refs/tags/v2.5.2
0df6358fbdc40be7037659fcc5c9db6b6a607cc5 refs/tags/v2.6.0
05970fbccd7583bda297487d3276b9034c000e22 refs/tags/v2.6.1
d86b2e36f75ddf958c2a9eddd9868e8ce9f1474b refs/tags/v2.7.0
bart-imac2018:pbs113a bart%
The output of this command is not as clear as it could be. We’ve said before that branches are just references to commits which act as the head of a branch, and tags are just references to commits. When you bear that in mind the output begins to make sense.
The first column in the output lists commit hashes, and the second column the thing referencing those commits. Anything starting with refs/heads/ is a remote branch, and anything starting with refs/tags/ is a remote tag.
Pushing Changes to a Remote Branch
We have now succeeded in creating a backup of all our work up to this point, but that’s not the end of the story, we now need to update this backup as we create new commits.
Let’s make a change and commit it so we have something to work with.
You’ll find an updated version of index.html in the pbs113a-2 folder in the instalment zip, copy that into your working copy. You’ll see this has added a new trivia box about Back to the Future Day.
Because this is a new feature let’s commit it with an appropriate Conventional-Commit-style message:
git commit -am 'feat: added Back to the Future Day trivia box'
Since this is a new feature, let’s also tag it with a new SemVer-style version number. As the git tag command shows us, the current version number is 2.7.0, since this is a non-breaking feature update, we need to bump the middle number, so the new version should be 2.8.0:
git tag v2.8.0
We have made these changes in our local repository, and only in our local repository. Let’s prove that to ourselves by looking at the hash of the commit at the head of our local main branch (using the git log command):
bart-imac2018:pbs113a bart% git log --pretty -1
commit a007430fcb621de92cf877cf865281ef4b89f797 (HEAD -> main, tag: v2.8.0)
Author: Bart Busschots <opensource@bartificer.net>
Date: Fri Apr 2 17:19:54 2021 +0100
feat: added Back to the Future Day trivia box
bart-imac2018:pbs113a bart%
So our local version of main is at a007430fcb621de92cf877cf865281ef4b89f797, and that commit is pointed to by the local tag v2.8.0.
What’s the state of the backup remote?
bart-imac2018:pbs113a bart% git ls-remote backup
d86b2e36f75ddf958c2a9eddd9868e8ce9f1474b refs/heads/main
ff8bc623afaabb9c2bcde5bc1bc492dc685a2f3e refs/tags/v1.0.0
43bca0f7cc7befa88172f217f4d74b3ec58dffc6 refs/tags/v1.1.0
42b42d674716c55d25e4728a0fc8b25823a8e0ed refs/tags/v2.0.0
4f085d37edf75bb8b157793ccf71d48057a9fe27 refs/tags/v2.1.0
069204cbe50a068c512f152ce37dac9e575c76d5 refs/tags/v2.1.1
8f4e695bd09b458c1c09960cfa73fe37fc853282 refs/tags/v2.2.0
4793fd78939e78f3416e6b966e290651589d817b refs/tags/v2.3.0
275f564428b0aa21b50ab40a4fd4a674a444f2b3 refs/tags/v2.4.0
b45a1e487e4f7f09480de3e5fb2ee4f21737ecae refs/tags/v2.4.1
74d6dd0f450097c5c6a792ee8841776cc7f88739 refs/tags/v2.5.0
3e29f1fe0cab9488439bca2c115b012998e45ab3 refs/tags/v2.5.1
b70fb8d6001bbe689786466006f6649242b0c55e refs/tags/v2.5.2
0df6358fbdc40be7037659fcc5c9db6b6a607cc5 refs/tags/v2.6.0
05970fbccd7583bda297487d3276b9034c000e22 refs/tags/v2.6.1
d86b2e36f75ddf958c2a9eddd9868e8ce9f1474b refs/tags/v2.7.0
bart-imac2018:pbs113a bart%
The remote main branch is not at the same commit as our local one, and, there is no remote v2.8.0 tag.
Let’s push these changes to our backup.
First, let’s push the changes on our current branch (main) to the branch with the same name in the remote repository named backup:
bart-imac2018:pbs113a bart% git push backup main
Enumerating objects: 5, done.
Counting objects: 100% (5/5), done.
Delta compression using up to 4 threads
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 673 bytes | 673.00 KiB/s, done.
Total 3 (delta 2), reused 0 (delta 0)
To ../pbs113a-backup.git
d86b2e3..a007430 main -> main
bart-imac2018:pbs113a bart%
Next, let’s push the v2.8.0 tag to the remote repository named backup:
bart-imac2018:pbs113a bart% git push backup v2.8.0
Total 0 (delta 0), reused 0 (delta 0)
To ../pbs113a-backup.git
* [new tag] v2.8.0 -> v2.8.0
bart-imac2018:pbs113a bart%
You can see that first argument to the git push command is the remote we’d like to push to, followed by what we’d like to push. The output shows the changes made in the remote repository.
We can now use the git ls-remote command to verify that we have indeed pushed our changes to our backup repo:
bart-imac2018:pbs113a bart% git ls-remote backup
a007430fcb621de92cf877cf865281ef4b89f797 refs/heads/main
ff8bc623afaabb9c2bcde5bc1bc492dc685a2f3e refs/tags/v1.0.0
43bca0f7cc7befa88172f217f4d74b3ec58dffc6 refs/tags/v1.1.0
42b42d674716c55d25e4728a0fc8b25823a8e0ed refs/tags/v2.0.0
4f085d37edf75bb8b157793ccf71d48057a9fe27 refs/tags/v2.1.0
069204cbe50a068c512f152ce37dac9e575c76d5 refs/tags/v2.1.1
8f4e695bd09b458c1c09960cfa73fe37fc853282 refs/tags/v2.2.0
4793fd78939e78f3416e6b966e290651589d817b refs/tags/v2.3.0
275f564428b0aa21b50ab40a4fd4a674a444f2b3 refs/tags/v2.4.0
b45a1e487e4f7f09480de3e5fb2ee4f21737ecae refs/tags/v2.4.1
74d6dd0f450097c5c6a792ee8841776cc7f88739 refs/tags/v2.5.0
3e29f1fe0cab9488439bca2c115b012998e45ab3 refs/tags/v2.5.1
b70fb8d6001bbe689786466006f6649242b0c55e refs/tags/v2.5.2
0df6358fbdc40be7037659fcc5c9db6b6a607cc5 refs/tags/v2.6.0
05970fbccd7583bda297487d3276b9034c000e22 refs/tags/v2.6.1
d86b2e36f75ddf958c2a9eddd9868e8ce9f1474b refs/tags/v2.7.0
a007430fcb621de92cf877cf865281ef4b89f797 refs/tags/v2.8.0
bart-imac2018:pbs113a bart%
Note that the remote main branch is now also at commit a007430fcb621de92cf877cf865281ef4b89f797, and there’s now a remote tag named v2.8.0 pointing at that same commit.
Final Thoughts
We’ve now seen how we can create bare Git repositories and push commits, branches, and tags to them. This is an important first step in utilising the power of remote repos, but it really is just the first step. At this point we’ve not established any kind of relationship between specific local and remote branches, and as a result, we have to explicitly push our changes at targeted remote branches. We also have to explicitly push our tags. This works, but it’s both error prone and tedious, so of course there must be a better way, and there is 🙂. In the next instalment we’ll discover the power of tracking remote branches, how that simplifies our use of a remote as a backup, and how it enables our use of a remote as a transfer-hub between multiple computers.