gearsix.net/software/setting-up-a-git-server

I recently spent an afternoon setting up a public read-only git server. I'm mostly writing it down for my own future reference, however stuff like this is usually useful to somebody out there.

Here's a repository with the important files used to setup this server: https://notabug.org/gearsix/src.gearsix.net.

Firstly, if you're serious about starting your own git server and aren't going to use something like Gogs or Gitea, then follow the official git documentation. It pretty much covers everything you need.

Setup

Hardware

One of the first things you need to consider for any server is the hardware and device security.

In this case, I was going to be using a Raspberry Pi 1 B+, yeah model 1 from 2014. It was lying around collecting dust and I hate seeing useable hardware like that go to waste, so this project was born.

First thing to consider this hardware isn't very powerful. Consequently I chose to go with a very minimalist approach - hopefully this would also increase the security of the device due to the smaller attack surface.

Software

The OS

The slow hardware meant minimalist, lightweight software. There are loads of OS images for Raspberry Pi devices and lucky for me, somebody else was well ahead of me in the minimalist approach for theirs: dietpi.

DietPi is extremely lightweight at its core, our images start at 400MB in size (3x lighter than 'Raspbian Lite'). With features of low process/memory footprint and DietPi-RAMlog installed by default, DietPi allows you to get the maximum performance from your device.

Git Service

There are also quite a few different git server softwares. For a full-on GitHub-like experience, Gogs and Gitea offer a great experience and do most of the heavy-lifting for you. This was way-overkill for what I wanted though.

The alternative to these is to setup git to respond to server requests manually. You can find details on how to do this here: Git on the Server.

Here's a TL;DR on the types of git services you can setup though:

ssh:// = Quick & easy setup, allows for authenticated r/w. Perfect for small & local servers.
git:// = A git-specific protocol that sits on port 9418. No authentication and read-only access.
http://, https:// = Setup is more complicated, but allows full read/write access with authentication over http/https.

For this use-case, I was only looking to setup read-only access to the repositories. git:// it is.

Web Service

While not totally necessary, it's nice to have webpages for any users to access. For servers like this it's usually to allow browsing the repositories file tree.

There are several services that allow for this, the official one is gitweb, a more popular (and preferred) tool for this is cgit.

An even more minimal solution I found is stagit, stagit is a static webpage generator that generates cgit-like pages for a limited amount of the repository. This was a better fit for my setup since static webpages are more lightweight than CGI scripts, much easier to setup and I was only going to be allowing git access via the git:// protocol anyway.

Guide

Setup DietPi

Flashing to SD

This is the easy part.

Download the correct DietPi image https://www.dietpi.com/
Flash the image to an SD Card https://www.raspberrypi.org/documentation/installation/installing-images/
Plug the SD Card into the Raspberry Pi, boot it up and get access via USB/HDMI or SSH

DietPi boot

Again, this is fairly straight-forward. DietPi has a really nice post-boot setup screen, obviously each setup will have it's own settings configured.

Base Software

Here's the software I installed post-boot:

Git (obviously)
Build Essentials = for building packages
vim.tiny = a tiny vim text editor
fail2ban = login security
certbot = for sorting out SSL certificate
tmux = I can't live without it in a CLI
ed = I prefer using ed for modifying small files

Both tmux and ed had to be installed directly via apt.

Install Stagit

Again, pretty simple.

First clone the stagit repository to a working directory: git pull git://git.codemadness.org/stagit
Next install the dependencies required to build stagit, namely: libgit2-dev
Build & install stagit: "sudo make install"

I ended up making changes to stagit (since it's open-source under MIT/X), you can view these changes here: https://notabug.org/gearsix/stagit.

Install Lighttpd

There are loads of different available webservers, in this case I was limited to the standards: Nginx, Apache 2.4, Lighttpd since I wanted to get an SSL cert with Let's Encrypt.

I chose to go with Lighttpd since it's the most lightweight option of the three, although personally I prefer the other two (they're a bit easier to configure and better documented online). I wasn't doing anything complicated though so this wasn't a problem.

By default Lighttpd serves all files in /var/www/ or sometimes /var/www/html. This is fine.

You don't really need to configure it much for this setup, although I'd recommend looking into the config files and setting them accordingly (/etc/lighttpd/).

The server should be running by default after you install it, you can test this locally by going to http:///. You should be greeted with the default lighttpd page.

Setup git

First we'll setup the git:// protocol. This is actually really easy, just follow the guide. Again, here's a TL;DR:

Create a git service account: "useradd -r git"
Clone your git repositories to wherever you want, the guide recommends "/srv/git"

Create a systemd service that runs the git daemon. The git daemon is what responds to git requests on the git:// port (9481). To do this, just create a file in "/etc/systemd/system/git-daemon.service" with the following contents:

   [Unit]
   Description=Start Git Daemon

   [Service]
   ExecStart=/usr/bin/git daemon --reuseaddr --export-all --base-path=/srv/git/ /srv/git/

   Restart=always
   RestartSec=500ms

   StandardOutput=syslog
   StandardError=syslog
   SyslogIdentifier=git-daemon

   User=git
   Group=git

   [Install]
   WantedBy=multi-user.target

Make sure it works by trying to pull a repo from another machine: "git pull git:///"

Setup stagit

Here's a quick rundown on how stagit works (it's really simple):

stagit <path to git repo> # generates a set of html files based on your repo

stagit-index <path to git repo1> <path to git repo2> # generated an index of stagit pages

So using these you should be able to generate your html pages to put in /var/www/! Done, right?

making it automagic

Obviously, you don't want to log into your device and regenerate the stagit pages everytime you need to update them.

Ontop of that, you don't want to login and git pull the updates to each repo everytime you push a change. So what to do?

write a bash script

Here's a script I wrote that cd's into each git repository directory, runs git pull, then backs up the contents of /var/www/ to /var/www.old/ and regenerates the stagit pages & index.

#!/bin/bash

shopt -s extglob

stagit_dir=/usr/local/src/stagit/
remote=https://notabug.org/gearsix/
repos_dir=/srv/git/
repos=$(ls $repos_dir)

# pull any stagit changes
cd /usr/local/src/stagit
git pull
make install

# git pull all repos in /srv/git
for f in ${repos[*]}; do
	echo "updating $f"
	if [ ! -d $repos_dir$f ]; then git clone $remote$f; fi
	cd $repos_dir$f
	git pull -ff
done

# backup /var/www -> /var/www.old
printf "\nbacking up /var/www/*"
rm /var/www.old/* -rf
cp -r /var/www/* /var/www.old/
rm -r /var/www/!(background.png|logo.png|style.css|favicon.png)

# regenerate /var/www/ git repo pages
echo ""
for f in ${repos[*]}; do
	echo "generating $f stagit pages"
	if [ ! -d /var/www/$f ]; then mkdir /var/www/$f; fi
	cd /var/www/$f
	/usr/local/bin/stagit /srv/git/$f
done

# regenerate /var/www/index.html
printf "\ngenerating stagit index"
/usr/local/bin/stagit-index $(echo $(for f in ${repos[*]}; do echo $repos_dir$f; done)) > /var/www/index.html

certbot

I won't go into detail about how I setup the SSL cert stuff, since it's pretty boring and tedious but I'd recommend using certbot with Let's Encrypt if you're not comfortable doing it all manually (which is even more tedious and boring).

Bonus: DietPi actually does a lot to make setting up an SSL certificate with Let's Encrypt even easier.

Conclusion

So here's a graph of how it all works

gearsix----(git push)---> r/w git repository (external)<---(git pull)----src.gearsix.net<-------
                                      ^                                         |              |
                                      |                                (backup /var/www)  (rebuild /var/www)
                                  (git pull)                                    |              |
                                      |                                         ----------------
                               external mirrors

src.gearsix.net has a cronjob running the bash script daily (doing it any more frequently felt unecessary).

Overall I learnt a lot from doing this, so the project was a success. It's also resulted in me having my own read-only public git server for free which is pretty cool: https://src.gearsix.net/

Learning in-depth the details of setting up a git server was pretty interesting. I've only seen the git:// protocol out in the wild a few times and with its I can understand why it's not used more but it does seem to be faster than http:// and I find it much quicker to use than ssh://. This project gave me a much better understanding and appreciation for git in general as well - it really deserves all the love and use it gets. It's a solid tool.

stagit is pretty cool too. The tool certainly has a hacker-feel to it, being written in C seemed like an odd choice for something generating xml and doing a lot of file i/o (personally I've always found that a PIA with C), but it's my favorite language so I was very interested to learn how the author of this tool managed it in such a tidy manner. The tools minimalist approach makes it incredibly easy to modify, which I ended up doing in some places just so the output HTML would suite my needs. I'm a fan of the developer (Hiltjo Posthuma) so using this was nice. You can see the changes I made here on the gearsix branch.

Writing the bash script is very simple but it represents my bulk of work (outside of setting up other tools) and is a very good summary of how I designed the git server. This part of the project made it feel like my own and not just somebody else's that I setup locally.

The most interesting part was (as always) the design; when running your own web server you always want to worry about security. Of course you can only do so much to mitigate the potential vulnerabilities so creating something very minimalist and read-only seemed like the best approach.

I could have easily setup a git server that allows full r/w, http://, ssh://, etc, but that just seemed unecessary. Not only that but my home internet connection is spotty at-best and I'm not settled into a place permenantly yet, so it would have been very dumb to make this my main git server since it's less reliable and than the loads of perfectly good and free git services.

The creation of this has also changed how I save my git repositories. Now I have a core git repository (https://notabug.org/gearsix) that I do most of my work on and this local mirror repository (which essentially acts as a backup). While I was at it I decided to create a mirror at GitLab as well - triple redundancy!

I've created a repo with the files used to setup the server here: https://notabug.org/gearsix/src.gearsix.net

Let me know if you found this helpful, or if you had any troubles. I'm happy to help out.

If you spot any errors please let me know.

Setting Up A Git Server