Most Linux administrators today create a plethora of different shell scripts to automate system administration tasks on their servers. However, it can be challenging to coordinate and keep track of all of the different shell scripts on a large number of different systems or locate and copy the right shell scripts to a newly provisioned server. You can use Git to centrally store a master copy of your shell scripts and organize the changes that you and your fellow administrators make across multiple servers.
What Is Git?
Git is a version control system that can be used to keep track of the changes you make to files over time – it allows you to revert files to a previous state or see who made changes to a file at different times if several people are working the those files collaboratively.
Git was originally developed in 2005 by Linus Torvalds (the creator of Linux) to aid in making changes to the Linux kernel, but has since become the most common open-source version control system used today. You can run Git on most platforms, including Windows, Linux and macOS.
Although people primarily use it for version controlling software projects, you can use it to provide version control for any files on your system. In this post, we’ll use it to provide version control and central access for Linux shell scripts.
How to Use Git for Local Version Control
Git essentially takes snapshots (called commits) of the files that you have within a particular folder (called a repository, or repo) on your system.
Each commit contains the changes you’ve made to the files since the last commit, so you can easily rollback those changes (much like a Windows System Restore point).git config
Before you use Git to create commits, you must first tell Git about yourself using the git config command, since that information must be added to each commit that you create:[jason.eckert@csc-studev01 myscripts]$ git config --global user.name "Jason Eckert"
[jason.eckert@csc-studev01 myscripts]$ git config --global user.email "[email protected]"
To turn an existing folder into a Git repo, simply use the git init command.
For example, if you are in the myscripts directory under your home directory on a Linux system, you could run the following commands to turn the myscripts directory into a Git repo (this will also create a hidden .git folder underneath the myscripts directory):
[jason.eckert@csc-studev01 ~]$ pwd
/home/jason.eckert
[jason.eckert@csc-studev01 ~]$ cd myscripts/
[jason.eckert@csc-studev01 myscripts]$ pwd
/home/jason.eckert/myscripts
[jason.eckert@csc-studev01 myscripts]$ ls
chownscript.sh filemaintain.sh newuserscript.sh seccheck.sh sysusage.sh
[jason.eckert@csc-studev01 myscripts]$ git init
Initialized empty Git repository in /home/jason.eckert/myscripts/.git/
[jason.eckert@csc-studev01 myscripts]$ git status
On branch master
Initial commit
Untracked files:
(use "git add <file>..." to include in what will be committed)
chownscript.sh
filemaintain.sh
newuserscript.sh
seccheck.sh
sysusage.sh
nothing added to commit but untracked files present (use "git add" to track)
[jason.eckert@csc-studev01 myscripts]$ _
Notice that the git status command above listed the files in the myscripts directory, but said that they were untracked – this is normal, because Git doesn’t assume you want everything version controlled. After creating a Git repo, you have to tell Git which files you want to version control by staging them with the git add command. Staging simply adds the files to an index that represents the files that Git can take a snapshot/commit of.
git commit
After the files have been staged, you can take snapshots of them using the git commit command. The following commands stage all the files in the myscripts folder using the * wildcard (because I’m lazy), shows that they are ready for committing and then creates a new commit with the description “My first commit”:
[jason.eckert@csc-studev01 myscripts]$ git add *
[jason.eckert@csc-studev01 myscripts]$ git status
On branch master
Initial commit
Changes to be committed:
(use "git rm --cached <file>..." to unstage)
new file: chownscript.sh
new file: filemaintain.sh
new file: newuserscript.sh
new file: seccheck.sh
new file: sysusage.sh
[jason.eckert@csc-studev01 myscripts]$ git commit -m "My first commit"
[master (root-commit) 53f9566] My first commit
5 files changed, 60 insertions(+)
create mode 100755 chownscript.sh
create mode 100644 filemaintain.sh
create mode 100755 newuserscript.sh
create mode 100644 seccheck.sh
create mode 100644 sysusage.sh
[jason.eckert@csc-studev01 myscripts]$ _
Next, let’s modify the filemaintain.sh shell script using the vi editor, see that Git detected the modification, stage the files in our repo again and create a new commit using an appropriate description of the changes that we made (in this example, I added XFS checking to the script):
[jason.eckert@csc-studev01 myscripts]$ vi filemaintain.sh
[jason.eckert@csc-studev01 myscripts]$ git status
On branch master
Changes not staged for commit:
(use "git add <file>..." to update what will be committed)
(use "git checkout -- <file>..." to discard changes in working directory)
modified: filemaintain.sh
no changes added to commit (use "git add" and/or "git commit -a")
[jason.eckert@csc-studev01 myscripts]$ git add *
[jason.eckert@csc-studev01 myscripts]$ git commit -m "Added XFS checking to filemaintain.sh"
[master 08e7f90] Added XFS checking to filemaintain.sh
1 file changed, 1 insertion(+)
[jason.eckert@csc-studev01 myscripts]$ _
git log
To see a list of all commits (and who made them), you can use the git log command, and to rollback to a previous version of the file, you can use the git reset --hard command. Say, for example, that I didn’t like the XFS checking additions I made to filemaintain.sh and wanted to roll it back to the previous version. To see all the changes I made to the files in my repo and rollback the change to “My first commit”, I could use the following commands. (HEAD is simply a reference to the most recent commit):
[jason.eckert@csc-studev01 myscripts]$ git log
commit 08e7f90fd4c1820eab77968ee98c8a7682c43aa8
Author: Jason Eckert <[email protected]>
Date: Mon Aug 13 14:54:20 2018 -0400
Added XFS checking to filemaintain.sh
commit 53f95663c4a9f5b53f5ee8b86b91024fd9e1fc9a
Author: Jason Eckert <[email protected]>
Date: Mon Aug 13 14:53:05 2018 -0400
My first commit
[jason.eckert@csc-studev01 myscripts]$ git reset --hard 53f95663c4a9f5b53f5ee8b86b91024fd9e1fc9a
HEAD is now at 53f9566 My first commit
[jason.eckert@csc-studev01 myscripts]$ git log
commit 53f95663c4a9f5b53f5ee8b86b91024fd9e1fc9a
Author: Jason Eckert <[email protected]>
Date: Mon Aug 13 14:53:05 2018 -0400
My first commit
[jason.eckert@csc-studev01 myscripts]$ _
If I view the filemaintain.sh script now, the XFS stuff I added will no longer be there!
Git Commands to Know
Here is a summary of git commands that you should find useful at this point:- git config = Sets general Git parameters like username and email
- git init = Creates a Git repo within the current directory (also creates .git folder)
- git add <filenames> = Adds the specified filenames to the Git index (called staging)
- git rm <filenames> = Removes the specified filenames from the Git index
- git commit -m <description> = Creates a snapshot/commit with a specified description
- git status = Views the status of a repo
- git log = Views the commit history of a repo
- git reset --hard <commit_ID> = Reverts files within a repo to a previous commit
Using Git to Collaborate
While we just saw how Git can perform version control for your local files and how other users can download (or clone) copies of your Git repos on the same computer or across a network (LAN or Internet). Those users can then create commits periodically after making changes to the files in their cloned repo and push those changes back to your original repo.
Any computer running Git can clone a Git repo from any other computer running Git, regardless of the operating system used. There are many free sites you can use to host Git repos online, including GitHub and GitLab.
Look back at the output of the git status command from earlier, and you will notice it names you as the “On branch master.” A branch is simply a section of your Git repo, much like the different partitions on a hard disk.
Any changes you make to an original or cloned Git repo are part of the master branch by default. But you can create as many other branches as you like to store changes that you may want to experiment with. Once you are satisfied that the changes work as you expected, you can merge the changes you made in your branch with the files in the master branch.
Normally, you maintain an original repo on your computer that other users download (clone). Rather than modifying the master branch on their cloned copy, other users would typically create separate branches on their cloned copy to test their modifications and perform commits as necessary.
Once the modifications are working well, they can upload (push) the branch to the original repo on your computer where you can view the changes and merge them into the master branch. Once there’s an updated master branch on the original repo, others can then download (pull) a fresh copy of the master branch to get the new changes.Root Users
Let’s experiment with this using another user (root) on the same computer – the only requirement is that the other user has read/write access to your repo folder.The following commands run by the root user (the other user) create a cloned copy of the myscripts repo (/home/jason.eckert/myscripts) within the root user’s home directory (/root) using the git clone command:
[root@csc-studev01 myscripts]# pwd
/root
[root@csc-studev01 ~]# git clone /home/jason.eckert/myscripts/
Cloning into 'myscripts'...
done.
[root@csc-studev01 ~]# cd myscripts/
[root@csc-studev01 myscripts]# pwd
/root/myscripts
[root@csc-studev01 myscripts]# ls
chownscript.sh filemaintain.sh newuserscript.sh seccheck.sh sysusage.sh
[root@csc-studev01 myscripts]# _
If you were cloning this from another computer, you’d have to use git clone username@hostname:/path instead. For example, to clone this repo on triosdevelopers.com to your local computer, you’d have to use the command git clone [email protected]:/home/jason.eckert/myscripts/ and supply the root user’s password when prompted.
Now, let’s make a branch called AddSIEM that we can use to test out adding security information and event management (SIEM) functionality to our seccheck.sh script and view our branch when finished:
[root@csc-studev01 myscripts]# git checkout -b AddSIEM
Switched to a new branch 'AddSIEM'
[root@csc-studev01 myscripts]# git branch
* AddSIEM
master
[root@csc-studev01 myscripts]# _
Notice that the git branch command indicates that AddSIEM is our current branch (*). Now, let’s modify the script, view the modification, stage and commit it:
[root@csc-studev01 myscripts]# vi seccheck.sh
[root@csc-studev01 myscripts]# git status
On branch AddSIEM
Changes not staged for commit:
(use "git add <file>..." to update what will be committed)
(use "git checkout -- <file>..." to discard changes in working directory)
modified: seccheck.sh
no changes added to commit (use "git add" and/or "git commit -a")
[root@csc-studev01 myscripts]# git add *
[root@csc-studev01 myscripts]# git commit -m "Added SIEM to seccheck.sh"
[AddSIEM e94d34e] Added SIEM to seccheck.sh
1 file changed, 1 insertion(+), 1 deletion(-)
[root@csc-studev01 myscripts]# _
At this point, you’ve modified the seccheck.sh script in the AddSIEM branch only. If you switched back to the master branch in your cloned repo using the git checkout master command and viewed the seccheck.sh file, you’d see that your changes are not there! That’s because they are only shown in the AddSIEM branch.
Now, let’s push our branch to the original repo. Luckily, you don’t have to remember the location of the original repo, because after you clone a repo, Git remembers the original location and allows you to use the word “origin” to refer to it:
[root@csc-studev01 myscripts]# git push origin AddSIEM
Counting objects: 3, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 293 bytes | 0 bytes/s, done.
Total 3 (delta 2), reused 0 (delta 0)
To /home/jason.eckert/myscripts/
* [new branch] AddSIEM -> AddSIEM
[root@csc-studev01 myscripts]# _
This uploaded the AddSIEM branch from the cloned repo (in /root/myscripts) to the original repo (in /home/jason.eckert/myscripts). Let’s switch back to the jason.eckert user and see if the branch was successfully uploaded to the original repo with the git branch command, and then merge the changes in the AddSIEM branch with our current branch (master) using the git merge command:
[jason.eckert@csc-studev01 myscripts]$ git branch
AddSIEM
* master
[jason.eckert@csc-studev01 myscripts]$ git merge AddSIEM
Updating 53f9566..e94d34e
Fast-forward
seccheck.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
[jason.eckert@csc-studev01 myscripts]$ _
That’s it! Now, other users who have a cloned copy of the original repo can run git pull origin master to download an updated (merged) copy of the master branch from the original location that has the new SIEM feature.
Let’s switch back to the cloned repo in the root user’s home directory and do this:
[root@csc-studev01 myscripts]# git pull origin master
From /home/jason.eckert/myscripts
* branch master -> FETCH_HEAD
53f9566..e94d34e master -> origin/master
Already up-to-date.
[root@csc-studev01 myscripts]# _
For all new modifications made by other users we simply repeat this process.
Now, imagine that you have a repository on a server that contains a central copy of all the shell scripts that can be used on other Linux servers within your organization:
- Linux administrators can clone the repo on the other Linux servers (or pull an updated copy of the master branch), create a new branch to store any modifications to the shell scripts and perform commits in this branch as necessary.
- Once the Linux administrators are happy with the changes, they can push the branch to the original repo, where it can be viewed by others and merged into the master branch.
More Git Commands to Know
Here is a summary of git commands that you should find useful at this point:- git clone /path = Clones a local Git repo to the current directory
- git clone username@hostname:/path = Clones a remote Git repo to the current directory
- git checkout -b <branchname> = Creates a new branch, and switches to it
- git checkout <branchname> = Switches to a different branch
- git branch = Views branches in the repo
- git branch -d <branchname> = Deletes a branch
- git push origin <branchname> = Pushes a branch to the original repo location
- git pull origin <branchname> = Pulls a branch from the original repo location
How to Use Online Git Repositories to Centrally Store Shell Scripts
Well, that’s easier than you think!
- Go to GitHub.com and create a free account
- Create a new public repository called myscripts
Finally, on your local computer, you can run the following commands to push the contents of your repo to GitHub and set GitHub as the original repo (which turns your local computer’s repo into a cloned copy from that point onward).
cd /path/to/repo
git remote add origin https://github.com/accountname/nameofGitHubrepo.git
But you don’t have to remember these commands because when you create a new repo within GitHub, it gives you the commands at the bottom of the screen.
Now, in my example above, I can go to any Linux server and type git clone https://github.com/jasoneckert/myscripts.git to clone the original repo to obtain the central copy of the shell scripts. And I can optionally make branches, do commits, push those commits back to GitHub (you’ll be prompted for your account password) and merge them into the master branch on GitHub!
