Cleanup local git branches - the Gerrit case

If you’ve been working with git for long enough you’ve probably noticed the local branches tend to accumulate. The git branch command provides a handy option to filter branches where the tip commits also belongs to the current branch:

git branch --merged

This is great and works fine in the majority of case and is often found in oneliners to cleanup local branches but this is not enough when working with workflows where the commits are often rewritten a few times before they eventually make it in tree, for example the Gerrit workflow we use in OpenStack. Gerrit uniquely identifies the commit with the Change-Id, a small label it inserts in the commit message and we can make advantage of this to check if a commit has been merged or not and delete the local branch.

First we need to get the Change-Id:

# Get Change-Id of $commit
change_id=$(git log -n1 --pretty=format:%b $commit | awk '/Change-Id:/ {print $0}')

With this Change-Id we can now verify if it was merged or not in the current branch:

# Check that commit was merged into $current_branch
merged_commit=$(git log --pretty=format:%h --grep "$change_id" ${current_branch})
if [ -z "$merged_commit" ]; then
    # This change is missing from $current_branch
fi

Great, so now we can find commits that have not yet been merged in the current branch. But what about the ones that were merged? It is entirely possible someone pushed a new version of the patch in Gerrit, or maybe I made local changes to a patch I haven’t yet submitted for review and the change was merged in the meantime. So if I blindly delete the branch I may lose important local changes. How can I check the two versions of the patch are the same? The interdiff tool compares diff files and we can assume they’re the same when the output is empty:

if [[ $(interdiff <(git show $commit) <(git show $merged_commit) 2>&1) ]]; then
    # The patch that was merged differs from what I have in local branch
fi

Putting it all together:

#!/bin/bash

function prompt_for_missing_commit {
    commit=$1
    branch=$2
    current_branch=$3
    git log --oneline -n1 $commit
    read -p "Commit $commit in $branch is missing from $current_branch. Inspect? [Yn] " answer
    if ! [[ "${answer,,}" =~ ^(n|no)$ ]]; then
        git show $commit
    fi
}

function prompt_for_commit_diff {
    local_commit=$1
    merged_commit=$2
    local_branch=$3
    current_branch=$4
    git log --oneline -n1 $commit
    read -p "Commit $local_commit in $local_branch and $merged_commit in $current_branch differ. Inspect? [Yn] " answer
    if ! [[ "${answer,,}" =~ ^(n|no)$ ]]; then
        interdiff <(git show $local_commit) <(git show $merged_commit) | colordiff
    fi
}

current_branch=$(git symbolic-ref --short HEAD)

for branch in $(git for-each-ref --format='%(refname:short)' refs/heads/); do
    if [ "$branch" == "$current_branch" ]; then
        continue
    fi
    echo
    echo "Checking branch $branch"
    branch_differs=0
    for commit in $(git log --no-merges --pretty=format:"%h" ${current_branch}..${branch}); do
        change_id=$(git log -n1 --pretty=format:%b $commit | awk '/Change-Id:/ {print $0}')
        if [ -z "$change_id" ]; then
            branch_differs=1
            prompt_for_missing_commit $commit $branch $current_branch
            continue
        fi
        merged_commit=$(git log --pretty=format:%h --grep "$change_id" ${current_branch})
        if [ -z "$merged_commit" ]; then
            branch_differs=1
            prompt_for_missing_commit $commit $branch $current_branch
            continue
        else
            # Check that the merged patch is similar to what is in local branch
            # NOTE needs interdiff from patchutils and colordiff
            if [[ $(interdiff <(git show $commit) <(git show $merged_commit) 2>&1) ]]; then
                branch_differs=1
                prompt_for_commit_diff $commit $merged_commit $branch $current_branch
            fi
        fi
    done
    if [ $branch_differs -eq 0 ]; then
        read -p "$branch fully merged. Delete? [yN] " answer
        if [[ "${answer,,}" =~ ^(y|yes)$ ]]; then
            git branch -D $branch
        fi
    else
        read -p "$branch differs from $current_branch. Delete anyway? [yN] " answer
        if [[ "${answer,,}" =~ ^(y|yes)$ ]]; then
            git branch -D $branch
        fi
    fi
done

Challenges in containerizing OpenStack

A presentation I made for the first edition of OpenStack Day France in Paris end of November that covers some of the challenges faced by the Kolla project in its effort to containerize OpenStack:

Setting up Gertty for effective code review

Gertty is a console-based client for the Gerrit code review tool. Once you start doing a lot of code reviews as part of your daily work, gertty proves to be an excellent alternative to the Gerrit web interface. For one, it can work offline so it means it’s possible to review code while on the move. It also means the interface is more reactive since everything is cached locally. Plus the fact that it’s a console tool is really appealing to programmers who spend their life in the terminal.

However, like all great tools, it needs a bit of configuration it in order to deliver its full potential. I’m going to document here a few of the settings I’m using in the context of OpenStack.

Vim key bindings

If like me you like using the vim key bindings whenever possible, then you’re in luck. Gertty ships with a vi keymap and allows you to remap most of the controls. To enable the vi bindings simply set keymap: vi in your .gertty.yaml file. However, don’t stop here or you’ll be heading for serious disappointment. The default vi keymap is rudimentary with only the four direction keys hjkl and you’ll need to redefine most of the combinations to feel at home.

Gertty supports key sequences in the form of [key1, key2] so you can easily emulate a leader key or a command like :q.

Here is the configuration I am using for reference:

keymaps:
  - name: vi
    cursor-down: ['j', 'down']
  - name: vi
    cursor-up: ['k', 'up']
  - name: vi
    cursor-left: ['h', 'left']
  - name: vi
    cursor-right: ['l', 'right']
  - name: vi
    cursor-page-up: ['ctrl u', 'page up']
  - name: vi
    cursor-page-down: ['ctrl d', 'page down']
  - name: vi
    cursor-max-left: ['^', 'home', 'ctrl a']
  - name: vi
    cursor-max-right: ['$', 'end', 'ctrl e']
  - name: vi
    quit: [[':', 'q'], 'ctrl q']
  - name: vi
    interactive-search: ['/']
  - name: vi
    toggle-hidden: [['t', 'h']]
  - name: vi
    toggle-star: [['t', '*']]
  - name: vi
    toggle-reviewed: [['t', 'r']]
  - name: vi
    toggle-list-reviewed: [['t', 'R']]
  - name: vi
    toggle-subscribed: [['t', 's']]
  - name: vi
    toggle-list-subscribed: [['t', 'S']]

Right now, the only thing I’m missing are the G and gg movements to go to the bottom or top of the panel. They are apparently missing from the underlying urwid library.

Dashboards

In the spirit of the web interface, gertty also allows you to define your own dashboards. The syntax is a bit different though, and as a consequence the official OpenStack dashboards can’t be imported directly and need to be adapted slightly. Unfortunately, I couldn’t find documentation for gertty’s query interface, so until someone finds the courage and time to write it down, the source code remains your best resource.

Here is for example what I’m using for the Kolla project:

dashboards:
  - name: "My changes"
    query: "owner:self status:open"
    key: "f2"
  - name: "Incoming reviews"
    query: "is:open is:reviewer"
    key: "f3"
  - name: "Kolla: Needs feedback"
    query: "project:^openstack/kolla.* status:open NOT owner:self label:Workflow>=0 NOT (label:Code-Review<=-1 or label:Code-Review>=1) age:2d"
    key: "f4"
  - name: "Kolla: Needs Approval"
    query: "project:^openstack/kolla.* status:open NOT owner:self label:Workflow>=0 (label:Verified>=1,jenkins or label:Verified>=1,zuul) label:Code-Review=2 NOT label:Code-Review<=-1"
    key: "f5"
  - name: "Kolla: No negative feedback"
    query: "project:^openstack/kolla.* status:open NOT owner:self (label:Verified>=1,jenkins or label:Verified>=1,zuul) NOT (label:Code-Review<=-1 or label:Code-Review>=1)"
    key: "f6"
  - name: "Kolla: Backports"
    query: "project:^openstack/kolla.* status:open NOT owner:self branch:stable/mitaka OR branch:stable/liberty"
    key: "f7"
  - name: "Kolla: Disagreement"
    query: "project:^openstack/kolla.* status:open (label:Verified>=1,jenkins or label:Verified>=1,zuul) label:Code-Review<=-1 label:Code-Review>=1"
    key: "f8"

Keyword replacement

The OpenStack Gerrit instance conveniently replaces the git commit message tags it understands with links to Launchpad issues or blueprints. Gertty offers that feature through keyword replacements.

Here is for instance how to mimic that behavior:

commentlinks:
  # Match external references to bugs on Launchpad
  - match: "(?P<bug_str>(?:[Cc]loses|[Pp]artial|[Rr]elated)-[Bb]ug *: *#?(?P<bug_id>\\d+))"
    replacements:
      - link:
          text: "{bug_str}"
          url: "https://launchpad.net/bugs/{bug_id}"
  # Match external references to blueprints on Launchpad
  - match: "(?P<bp_str>(?:[Bb]lueprint|bp) +(?P<blueprint>[\\w\\-.]+))"
    replacements:
      - link:
          text: "{bp_str}"
          url: "https://blueprints.launchpad.net/openstack/?searchtext={blueprint}"

Filter job results by pipelines

That last one hasn’t yet made it to gertty. This is somewhat specific to OpenStack gerrit instance and how it works with Zuul. The above commit lets you filter the job results by pipeline and displays useful information like the date or the number of rechecks.

See what gertty looks like with the patch applied: