• Understand regular version control
• Understand distributed version control

Along with a book, tutorial and cheatsheet, here are the insights that helped git click.

There’s a staging area!

Git has a staging area. Git has a staging area!!!

Yowza, did this ever confuse me. There’s both a repo (”object database”) and a staging area (called “index”). Checkins have two steps:

• git add foo.txt
  • Add foo.txt to the index. It’s not checked in yet!
• git commit -m "message"
  • Put staged files in the repo; they’re now tracked
  • You can “git add --update" to stage all tracked, modified files

Why stage? Git’s flexible: if a, b and c are changed, you can commit them separately or together.

But now there’s two undos:

• git checkout foo.txt
  • Undo local changes (like svn revert)
• git reset HEAD foo.txt
  • Remove from staging area (local copy still modified).

Add and commit, add and commit — Git has a rhythm.

Branching is “Save as…”

Branches are like “Save as…” on a directory. Best of all:

• Easily merge changes with the original (changes tracked and never applied twice)
• No wasted space (common files only stored once)

Why branch? Consider the utility of “Save as…” for regular files: you tinker with multiple possibilities while keeping the original safe. Git enables this for directories, with the power to merge. (In practice, svn is like a single shared drive, where you can only revert to one backup).

Imagine virtual directories

I see branches as “virtual directories” in the .git folder. While inside a physical directory (c:\project or ~/project), you traverse virtual directories with a checkout.

• git checkout master
  • switch to master branch (”cd master”)
• git branch dev
  • create new branch from existing (”cp * dev”)
  • you still need to “cd” with “git checkout dev”
• git merge dev
  • (when in master) pull in changes from dev (”cp dev/* .”)
• git branch
  • list all branches (”ls”)

My inner dialogue is “change to dev directory (checkout)… make changes… save changes (add/commit)… change to master directory… copy in changes from dev (merge)”.

The physical directory is a scratchpad. Virtual directories are affected by git commands:

• rm foo.txt
  • Remove foo.txt from your sandbox (restored if you checkout the branch again)
• git rm foo.txt
  • Remove foo.txt from current virtual directory
  • Gotcha: you need to commit that change!

Know the current branch

Just like seeing your current directory, put the current branch in your prompt!

In my .bash_profile (modified from here):

parse_git_branch() {
    git branch 2> /dev/null | sed -e '/^[^*]/d' -e 's/* \(.*\)/(\1)/'
}

export PS1="\[\033[00m\]\u@\h\[\033[01;34m\] \W \[\033[31m\]\$(parse_git_branch) \[\033[00m\]$\[\033[00m\] "

Visualize your branch structure

Git leaves branch organization to you. Nvie.com has a great branch strategy:

• Have a mainline (master). Mentally it’s on the far right.
• Create branches (master -> dev) and subbranches (dev -> featureX). The further from master, the crazier.
• Only merge with neighbors (master -> dev -> feature X, or featureX -> dev -> master)

Stay sane by choosing a branch layout up front. I have a master tracking a svn project, and dev for my own code. In general, master is clean so I can branch anytime for one-off fixes.

Understand local vs. remote

Git has local and remote commands; seeing both confused me (”When do you checkout vs. pull?”). Work locally, syncing remotely as needed.

Local data

• git init
  
  • create local repo
  • use git add/commit/branch to work locally

Remote data

• git remote add name path-to-repo
  • track a remote repo (usually “origin”) from an existing repo
  • remote branches are “origin/master”, “origin/dev” etc.
    
• git branch -a
  • list all branches (remote and local)

• git clone path-to-repo
  • create a new local git repo copied from a remote one
  • local master tracks remote master
• git pull
  • merge changes from tracked remote branch (if in dev, pull from origin/dev)
• git push
  • send changes to tracked remote branch (if in dev, push to origin/dev)

Why local and remote? Subversion has central checkins, so you avoid committing unfinished work. With git, local commits are frequent and you only push when ready.

GUIDs are GOOD

Git addresses information by a hash (GUID) of its contents. If two branches are the same, they have the same GUID (and vice versa).

Why’s this cool? We can create branches independently, merge them, and have a common GUID. No central numbering needed. Usually, we just compare the first few digits: “Are you on a93?”.

Tips & Tricks

For your .gitconfig:

[alias]
        ci = commit
        st = status
        co = checkout
        oneline = log --pretty=oneline
        br = branch
        la = log --pretty=\"format:%ad %h (%an): %s\" --date=short

There are some GUI tools for git, but I prefer to learn via the command line. Git is opinionated software (which I like), and analogies help me understand its world view.

• Networking is about communication
• Text is the simplest way to communicate
• Protocols are standards for reading and writing text

Beneath the details, networking is an IM conversation. Here's what I wish someone told me when learning how computers communicate.

TCP: The Text Layer

The Transmission Control Protocol (TCP) provides the handy illusion that we can "just" send text between two computers. TCP relies on lower levels and can send binary data, but ignore that for now:

• TCP lets us Instant Message between computers

We IM with Telnet, the 'notepad' of networking: telnet sends and receives plain text using TCP. It's a chat client peacefully free of ads and unsolicited buddy requests.

Let's talk to Google using telnet (or putty, a better utility):

telnet google.com 80
[connecting...]
Hello Mr. Google!

We connect to google.com on port 80 (the default for web requests) and send the message "Hello Mr. Google!". We press Enter a few times and await the reply:

<html>
...
<h1>Bad Request</h1>
Your client has issued a malformed or illegal request
...
</html>

Malformed? Illegal? The mighty Google is not pleased. It didn't understand us and sent HTML telling the same.

But, we had a conversation: text went in, and text came back. In other words: 

Protocols: The Forms To Fill Out

Unstructured chats is too carefree -- how does the server know what we want to do? We need a protocol (standard way of communicating) if we're going to make sense.

We use protocols all the time

• Putting to: and from: addresses in certain places on an envelope
• Filling out bank forms (special place for account number, deposit amount, etc.)
• Saying "Roger" or "10-4" to indicate a radio request was understood

Protocols make communication clear.

Case Study: The HTTP Protocol

We see HTTP in every url: http://google.com/. What does it mean?

• Connect to server google.com (Using TCP, port 80 by default)
• Ask for the resource "/" (the default resource)
• Format the request using the Hypertext Transport Protocol

HTTP is the "form to fill out" when asking for the resource. Using the HTTP format, the above request looks like this:

GET / HTTP/1.0

Remember, it's just text! We're asking for a file, through an IM session, using the format: [Command] [Resource] [Protocol Name/Version].

This command is "IM'd" to the server (your browser adds extra info, a detail for another time). Google's server returns this response:

HTTP/1.0 200 OK
Cache-Control: private, max-age=0
Date: Sun, 15 Mar 2009 03:13:39 GMT
Expires: -1
Content-Type: text/html; charset=ISO-8859-1
Set-Cookie: PREF=ID=5cc6...
Server: gws
Connection: Close

<html>
(Google web page, search box, and cute logo)
</html>

Yowza. The bottom part is HTML for the browser to display. But why the junk up top?

Well, suppose we just got the raw HTML to display. But what about errors: if the server crashed, the file wasn't there, or google just didn't like us?

Some metadata (data about data) is useful. When we order a book from Amazon we expect a packing slip describing the order: the intended recipient, price, return information, etc. You don't want a naked book just thrown on your doorstep.

Protocols are similar: the recipient wants to know if everything was OK. Here we see infamous status codes like 404 (resource not found) or 200 (everything OK). These headers aren't the real data -- they're the packing slip from the server.

Insights From Protocols

Studying existing, popular systems is a great way to understand engineering decisions. Here are a few:

Binary vs Plain Text

Binary data is more efficient than text, but more difficult to debug and generate (how many hex editors do you know to use?). Lower-level protocols, the backbone of the internet, use binary data to maintain performance. Application-level protocols (HTTP and above) use text data for ease of interoperability. You don't have religious wars about endian issues with HTTP.

Stateful vs. Stateless

Some protocols are stateful, which means the server remembers the chat with the client. With SMTP, for example, the client opens a connection and issues commands one at a time (such as adding recipients to an email), and closes the connection. Stateful communication is useful in transactions that have many steps or conditions.

Stateless communication is simpler: you send the entire transaction as one request. Each "instant message" stands on its own and doesn't need the others. HTTP is stateless: you can request a webpage without introducing yourself to the server.

Extensibility

We can't think of everything beforehand. How do we extend old protocols for new users?

HTTP has a simple and effective "header" structure: a metadata preamble that looks like "Header:Value".

If you don't recognize the header sent (new client, old server) just ignore it. If you were expecting a header but don't see it (old client, new server), just use a default. It's like having an "Anything else to tell us?" section in a survey.

Error Correction & Reliability

It's the job of lower-level protocols like TCP to make sure data is transmitted reliably. But higher-level protocols (like HTTP) need to make sure it's the right data. How are errors handled and communicated? Can the client just retry or does the server need to reset state?

HTTP comes with its own set of error codes to handle a variety of situations.

Availability

The neat thing about networking is that works on one computer. Memcached is a great service to cache data. And guess what? It uses plain-old text commands (over TCP) to save and retrieve data.

You don't need complex COM objects or DLLs - you start a Memcached server, send text in, and get text out. It's language-neutral and easy to access because any decent OS supports networking. You can even telnet into Memcached to debug it.

Wireless routers are similar: they have a control panel available through HTTP. There's no "router configuration program" -- you just connect to it with your browser. The router serves up webpages, and when you submit data it makes the necessary configuration changes.

Protocols like HTTP are so popular you can assume the user has a client.

Layering Protocols

Protocols can be layered. We might write a resume, which is part of a larger application, which is stuffed into an envelope. Each segment has its own format, blissfully unaware of the others. Your envelope doesn't care about the resume -- it just wants the to: and from: addresses written correctly.

Many protocols rely on HTTP because it's so widely used (rather than starting from scratch, like Memcached, which needs efficiency). HTTP has well-understood methods to define resources (URLs) and commands (GET and POST), so why not use them?

Web services do just that. The SOAP protocol crams XML inside of HTTP commands. The REST protocol embraces HTTP and uses the existing verbs as much as possible.

Remember: It's All Made Up

Networking involves human conventions. Because plain text is ubiquitous and easy to use, it is the basis for most protocols. And TCP is the simplest, most-supported way to exchange text.

Remembering that everything is a plain text IM conversation helps me wrap my head around the inevitable networking issues. And sometimes you need to jump into HTTP to understand compression and caching.

Don't just memorize the details; see protocols as strategies to solve communication problems. Happy networking.

Traditional version control helps you backup, track and synchronize files. Distributed version control makes it easy to share changes. Done right, you can get the best of both worlds: simple merging and centralized releases.

Distributed? What’s wrong with regular version control?

Nothing — read a visual guide to version control if you want a quick refresher. Sure, some people will deride you for using an “ancient” system. But you’re still OK in my book: using any VCS is a positive step forward for a project.

Centralized VCS emerged from the 1970s, when programmers had thin clients and admired “big iron” mainframes (how can you not like a machine with a then-gluttonous 8 bits to a byte?).

Centralized is simple, and what you’d first invent: a single place everyone can check in and check out. It’s like a library where you get to scribble in the books.

This model works for backup, undo and synchronization but isn’t great for merging and branching changes people make. As projects grow, you want to split features into chunks, developing and testing in isolation and slowly merging changes into the main line. In reality, branching is cumbersome, so new features may come as a giant checkin, making changes difficult to manage and untangle if they go awry.

Sure, merging is always “possible” in a centralized system, but it’s not easy: you often need to track the merge yourself to avoid making the same change twice. Distributed systems make branching and merging painless because they rely on it.

A Few Diagrams, Please

Other tutorials have plenty of nitty-gritty text commands; here’s a visual look. To refresh, developers use a central repo in a typical VCS:

Everyone syncs and checks into the main trunk: Sue adds soup, Joe adds juice, and Eve adds eggs.

Sue’s change must go into main before it can be seen by others. Yes, theoretically Sue could make a new branch for other people to try out her changes, but this is a pain in a regular VCS.

Distributed Version Control Systems (DVCS)

In a distributed model, every developer has their own repo. Sue’s changes live in her local repo, which she can share with Joe or Eve:

But will it be a circus with no ringleader? Nope. If desired, everyone can push changes into a common repo, suspiciously like the centralized model above. This franken-repo contains the changes of Sue, Joe and Eve.

I wish distributed version control had a different name, such as “independent”, “federated” or “peer-to-peer.” The term “distributed” evokes thoughts of distributed computing, where work is split among a grid of machines (like searching for signals with SETI@home or doing protein folding).

A DVCS is not like Seti@home: each node is completely independent and sharing is optional (in Seti you must phone back your results).

Key Concepts In 5 Minutes

Here’s the basics; there’s a book on patch theory if you’re interested.

Core Concepts

• Centralized version control focuses on synchronizing, tracking, and backing up files.
• Distributed version control focuses on sharing changes; every change has a guid or unique id.
• Recording/Downloading and applying a change are separate steps (in a centralized system, they happen together).
• Distributed systems have no forced structure. You can create “centrally administered” locations or keep everyone as peers.

New Terminology

• push: send a change to another repository (may require permission)
• pull: grab a change from a repository

Key Advantages

• Everyone has a local sandbox. You can make changes and roll back, all on your local machine. No more giant checkins; your incremental history is in your repo.
• It works offline. You only need to be online to share changes. Otherwise, you can happily stay on your local machine, checking in and undoing, no matter if the “server” is down or you’re on an airplane.
• It’s fast. Diffs, commits and reverts are all done locally. There’s no shaky network or server to ask for old revisions from a year ago.
• It handles changes well. Distributed version control systems were built around sharing changes. Every change has a guid which makes it easy to track.
• Branching and merging is easy. Because every developer “has their own branch”, every shared change is like reverse integration. But the guids make it easy to automatically combine changes and avoid duplicates.
• Less management. Distributed VCSes are easy to get running; there’s no “always-running” server software to install. Also, DVCSes may not require you to “add” new users; you just pick what URLs to pull from. This can avoid political headaches in large projects.

Key Disadvantages

• You still need a backup. Some claim your “backup” is the other machines that have your changes. I don’t buy it — what if they didn’t accept them all? What if they’re offline and you have new changes? With a DVCS, you still want a machine to push changes to “just in case”. (In Subversion, you usually dedicate a machine to store the main repo; do the same for a DVCS).
• There’s not really a “latest version”. If there’s no central location, you don’t immediately know whether to see Sue, Joe or Eve for the latest version. Again, a central location helps clarify what the latest “stable” release is.
• There aren’t really revision numbers. Every repo has its own revision numbers depending on the changes. Instead, people refer to change numbers: Pardon me, do you have change fa33e7b? (Remember, the id is an ugly guid). Thankfully, you can tag releases with meaningful names.

Mercurial Quickstart

Mercurial is a fast, simple DVCS. The nickname is hg, like the element Mercury.

cd project
hg init                                (create repo here)
hg add list.txt                        (start tracking file)
hg commit -m "Added file"              (check file into local repo)
hg log                                 (see history; notice guid)

changeset:   0:55bbcb7a4c24
user:        Kalid@kazad-laptop
date:        Sun Oct 14 21:36:18 2007 -0400
summary:     Added file

[edit file]
hg revert list.txt                 (revert to previous version)

hg tag v1.0                        (tag this version)
[edit file]
hg update -C v1.0                  ("update" to the older tagged version; -C forces overwrite of local copy)

Once Mercurial has initialized a directory, it looks like this:

You have:

• A working copy. The files you are currently editing.
• A repository. A directory (.hg in Mercurial) containing all patches and metadata (comments, guids, dates, etc.). There’s no central server so the data stays with you.

In our distributed example, Sue, Joe and Eve have their own repos, with independent revision histories.

Understanding Updates and Merging

There’s a few items that confused me when learning about DVCS. First, updates happen in several steps:

• Getting the change into a repo (pushing or pulling)
• Applying the change to the files (update or merge)
• Saving the new version (commit)

Second, depending on the change, you can update or merge:

• Updates happen when there’s no ambiguity. For example, I pull changes to a file that only you’ve been editing. The file just jumps to the latest revision, since there’s no overlapping changes.
• Merges are needed when we have conflicting changes. If we both edit a file, we end up with two “branches” (i.e. alternate universes). One world has my changes, the other world has yours. In this case we (probably) want to merge the changes together into a single universe.

I’m still wrapping my head around how easily branches spring up and collapse in a DVCS:

In this case, a merge is needed because (+Soup) and (+Juice) are changes to a common parent: the list with just “Milk”. After Joe merges the files, Sue can do a regular “pull and update” to get the combined file from Joe. She doesn’t have to merge again on her own.

In Mercurial you can run:

hg incoming ../another-dir  (see pending changes)
hg pull ../another-dir      (download changes)

hg update                   (actually apply changes...)
hg merge                    (... or merge if needed)

hg commit                   (check in merged file; unite branches)

Yep, the “pull-merge-commit” cycle is long. Luckily, Mercurial has shortcuts to combine commands into a single one. Though it seems complex, it’s much easier than handling merges manually in Subversion.

Most merges are automatic. When conflicts come up, they are typically resolved quickly. Mercurial keeps track of the parent/child relationship for every change (our merged list has two parents), as well as the “heads” or latest changes in each branch. Before the merge we have two heads; afterwards, one.

Organizing a Distributed Project

Here’s one way to organize a distributed project:

Sue, Joe and Eve check changes into a common branch. They can trade patches with each other to do simple “buddy builds”: Hey buddy, can you try out these patches? I need to see if it works before I push to the experimental branch.

Later, a maintainer can review and pull changes from the experimental branch into a stable branch, which has the latest release. A distributed VCS helps isolate changes but still provide the “single source” of a centralized system. There are many models of development, from “pull only” (where maintainers decide what to take, and is used when developing Linux) to “shared push” (which acts like a centralized system). A distributed VCS gives you flexibility in how a project is maintained.

Practice And Scathing Ridicule Makes Perfect

I’m a DVCS newbie, but am happy with what I’ve learned so far. I enjoy SVN, but it’s “fun” seeing how easy a merge can be. My suggestion is to start with Subversion, get a grasp for team collaboration, then experiment with a distributed model. With the proper layout a DVCS can do anything a centralized system can, with the added benefit of easy merging.

Online Resources

• Mercurial has an excellent book. On Windows you may need diffing/merging software or TortoiseMerge (if you have TortoiseSVN installed).
• Darcs has a detailed wikibook (has some math theory about changes).
• Git was created by Linus Torvalds. Here’s an interesting lecture on DVCS; prepare to be berated for using a centralized system:

Notable Quotes:

• “How many have done a branch and merged it? How many of you enjoyed it?”
• “When you do a merge, you plan ahead for a week, then set aside a day to do it.”
• “Some people have 5, 10, 15 branches”. One branch is experimental. One branch is maintenance, etc.
• “CVS — you don’t commit. You make changes without committing. You never commit until it passes a giant test suite. People make 1-liner changes, knowing it can’t possibly break.”

So good luck, and watch out for the holy wars. Feel free to share any tips or suggestions below.