Manage dotfiles from anywhere with Git

7 minute read     Updated:

Talha Khalid %
Talha Khalid

Have you ever found yourself in a situation where you had to format your computer and manually reinstall all the programs you were using? Or did you change machines and have to go to the page of each of the software you use and download and run the installer one by one? Or even customize some mouse/keyboard settings?

This is a bit of work for everyone, but it’s even more work for devs. We have a series of environment configuration files, variables, binary configurations, and shells that need to be configured the way we want them to be so we can be as productive as possible!

Let’s understand how dotfiles will solve this problem and how you will be much more productive when you start versioning them using Git.

What Are Dotfiles?

Dotfiles is the name given to the set of hidden files used to store the state configuration or the preferences configuration of a tool.

The term “dotfiles” comes, like most things in computing, from the old Unix kernels that adopted the practice of adding the prefix. in front of the filename to make that file, by default, a hidden file, i.e., a file that is not shown in the listing for ls. For example, `/home/.hushlogin’it is a common file in Linux nowadays to remove the login message when accessing SSH.

As any Unix based system has a very strong relationship with files, since most modules and even devices or network interfaces are shown as files within the file system, they are highly important for software development.

Nowadays, the vast majority of tools use dotfiles to maintain configuration files:

  • Bash:.bashrc
  • Git: .gitconfig,.gitexcludes
  • Vim:.vimrc

The Problem With Dotfiles

Problems

You can save a copy of dotfiles on some online storage service like Dropbox. This is not a problem as long as you only save one file, but what about when you start saving multiple files? How to manage it from several programs, in several folders, and worse, how to restore it all later?

There are several projects to solve the problem of how we can keep our dotfiles synchronized on machines with the same operating system. One solution is to put your dotfiles under version control.

If you use several machines, they may have specific configurations. It is therefore not desirable to synchronize all files. With its branching system, Git can help you overcome this problem. You can create a master branch containing files common to different systems and branches specific to each system. It is then possible to use a logical operator in the Bash configuration file to apply the correct files according to the machine’s name, for example.

Git also allows to use submodules. This way, you can have repositories inside the repository, which gives you more flexibility and modularity to maintain the repository, including other people’s repositories within our repo.

Git, therefore, seems perfectly adapted to our needs since it allows you to save, synchronize, and create specific branches.

Note: It is important to say that if your files contain sensitive data, like passwords, keys, etc., then you may need to manage them in a private repository or use tutorials like this one to make them encrypted.

Managing Your Dotfiles

Managing

The idea of ​​managing configuration files using git and submodules is to make the configuration easier to segment. Being segmented makes it easier to maintain and find the configuration we are looking for. In our case, the structure is:

  • Dotfiles calling repos
  • Repo with app configuration as a submodule
  • Plugins for the app as submodules

Using this structure, we only have files with a few lines that call the file that consolidates the configuration of the entire repo. For example, .bashrc:

##!/bin/bash
## shellcheck disable=1090
source ~/.shell_config/bash/bashrc

The file named above takes care of loading:

  • Alias
  • Command settings
  • Custom commands
  • Environment Variables
  • Prompt configuration

This can be done with all applications. Having a more segmented configuration makes it easier to find the specific configuration we are looking for. This also helps make it harder to break the configuration and easier to debug. Another benefit of this segmentation is the ease of reusing code if we use more than one terminal (bash, zsh, etc.) or want to migrate between them.

And How Do We Do?

How

The process is fairly simple. With git already installed, you can start the repository by executing:

git init --bare $HOME/.cfg

Before you can use it, you must make some configurations:

Let’s avoid loops in versioning echo ".cfg" >> ~/.gitignore

Set up the alias temporarily to make it easier to work on the repo alias gitdot='/usr/bin/git --git-dir=$HOME/.cfg/ --work-tree=$HOME'

Note: the alias can also be persisted in the .bashrc

Hide files that are not being versioned gitdot config --local status.showUntrackedFiles no

Now you just have to push to move configuration files in the repository.

Exporting the Configuration With Dotfiles

Once the repository is created, you can clone it anywhere where you want it. For that, you must:

git clone --bare --depth 1 --recurse-submodules --shallow-submodules url-repo $HOME/.cfg

In this way, you can clone the repository in bare format with the submodules. Fetching only the latest commit from each repository. This allows you to save space and increase the download speed since you do not need the project’s complete history in the local repository. Once you have recreated the repository, you must check it to recreate the files. This process must be done in several parts, considering:

  • Configuration files may already exist in that path; you must back them up and move them
  • If you use submodules, you have to start and update them
  • You must apply the configuration
alias gitdot='/usr/bin/git --git-dir=$HOME/.cfg/ --work-tree=$HOME'
mkdir -p .config-backup && \
gitdot checkout 2>&1 | egrep "\s+\." | awk {'print $1'} | \
xargs -I{} mv {} .config-backup/{}
gitdot checkout
gitdot submodule init
gitdot submodule update
source .bashrc

Note: If the submodules also have submodules, the start and update process must be repeated.

Difference

After moving all the files to a structure that you are happy with in your repository, you can use the hard links functionality of Unix-based systems so that you can create a connection between your original file and the location where the dotfile will exist.

The difference between hard links and soft links is the fact that hard links will be a pointer to the file itself; that is, it is a different name for the same original file that is independent of any other system resource; it is as if you were talking about a file having multiple names.

This creates a great facility when you have to back up these files because hard links are direct pointers to the original file content, so if you change the hard link, let’s change the original file, too, you can perform a test by doing the following:

export temp=$(mktemp -d)
touch $temp/original
ln $temp/original ~/hardlink

Now you just have to edit the file present in ~/hardlink, which is the link itself, and run cat $temp/original to see that the content is also present there.

If you want to delete the original file, you will also need to delete the link, so it’s of no use running only rm -rf $temp; it’s necessary to run rm -rf ~/hardlink too.

Soft links do not allow you to do this, as they are just files that point to other files. So for dotfiles, it is much better to have a hard link that allows you to directly edit a dotfile wherever it is, and those changes are reflected in your Git repository.

Creating

Now that you have the repository with the files, all that’s left is to link them to their original locations.

For this, you will have to use the command ln; the syntax of this command is:

ln <original file> <link location>

It is important to note that the link’s location may need to be an absolute path; that is, it cannot be a path of type ../or ~/ on some systems.

The command does not display any output if everything went well, so the way to identify it is by checking if the file is also present when executing a command ls -la in your destination directory.

In the case of symlinks, when running a listing, the command output shows a path of type file -> original file.

Conclusion

Centralizing Dotfiles online via a version control tool like Github or Gitlabmakes it very easy to synchronize configurations, which helps simplify setup and collaborate, allowing a reference configuration that can be installed on each machine.

As you will have understood, taking an interest in dotfile files is not only reserved for people who want to know the finer cogs of system administration. On the contrary, it can allow you to optimize your development environment(s) in order to increase productivity.

While you’re here:

Earthly is the effortless CI/CD framework.
Develop CI/CD pipelines locally and run them anywhere!

Talha Khalid %
Talha Khalid

Talha Khalid, full stack developer turned writer. If he isn’t spending time with his cat, Petrov you can almost always find him in some random coffee shop.

Updated:

Get notified about new articles!

We won't send you spam. Unsubscribe at any time.