Category Archives: Linux

Extract part of a Subversion repo into a Git repo

A few times I’ve wanted to extract part of a large monolithic Subversion repository out into a seperate Git repo, but maintain the commit history.

Here’s how I do it.

First, I set up a mapping of Subversion user => Username in a file, so that the committer can match up easily via GitHub etc – each committer should have an entry like the below, one per line:


davidp = David Precious

Now, I clone the entire Subversion repo via git svn into a new git repository:


# Clone the Subversion repo into a new Git repo:
# (~/subversion_authors.txt is file mentioned above)
git svn clone file:///shared/svn/scripts --no-metadata -A ~/subversion_authors.txt tmp/scripts-repo-tmp

Some tags get added during this process, I believe; I don’t need/want to preserve them, so I remove any and all tags:


# remove tags - we don't need them
git tag -l | xargs git tag -d

Now, the clever part; using git filter-branch to select the path within the repo I want to preserve, and remove everything else, promoting the desired path to the “root” of the repository:


# remove all except a given path:
git filter-branch --tag-name-filter cat --prune-empty --subdirectory-filter path/to/desired/dir HEAD

In the above, path/to/desired/dir is the path within the repo that I want to move to the root of the repo; everything else will be discarded.

At this point, I can add a GitHub repository via git remove add origin $url, and push the new repository.

I *think*, because I pushed to GitHub, then deleted my temporary repo and cloned back down, that unrelated previous commits were automatically removed. In case that’s not true, though, the following ought to purge unrelated commits from the new Git repo:


git reset --hard
git for-each-ref --format="%(refname)" refs/original/ | xargs -n 1 git update-ref -d
git reflog expire --expire=now --all
git gc --aggressive --prune=now

VLC getting proxy settings via gconf

I had a problem with VLC, using outdated proxy settings rather than connecting directly, even with no proxy configured in VLC’s settings (and even when trying to override it using command-line options).

(It would show that it was trying to use a proxy:

[0x134b4b0] main access error: connection failed: Connection refused
[0x134b4b0] access_http access error: cannot connect to supernova:3140

(supernova:3140 was the proxy setting it was picking up.)

After using strace to follow the execution of VLC when trying to play a network stream, I realised it was getting the settings via gconf; I needed to execute gconf-editor, navigate to system, http_proxy, then untick use_http_proxy – and also edit the value of the host setting to an empty string (without doing this, VLC ignored the use_http_proxy setting being false, and tried to use the proxy anyway!).

I could have done the same using gconftool-2 with:

gconftool-2 -s /system/http_proxy/use_http_proxy --type bool false
gconftool-2 -s /system/http_proxy/host --type string ''

Blogging for my own future reference, and for anyone else who’s using vlc on XFCE and wondering where it was getting the proxy settings. I’ve used Gnome 2 on this machine before, and I think that system proxy setting was set by Chromium (Google Chrome).

SMART monitoring drives on 3ware cards

I’ve been updating my Nagios monitoring to make sure I’m carefully monitoring my server hardware, including temperature, fan speeds and hard drives, and I wanted to use S.M.A.R.T. monitoring to monitor drives on a 3ware RAID controller for signs of imminent failure.

I already monitor the status of the RAID array itself using my nagios_3ware_raid_check Nagios plugin (which I previously blogged about), but I wanted to use SMART monitoring to look for signs of imminent drive trouble, rather than simply finding out when a drive has just failed.

After installing smartmontools, I was able to edit /etc/smartd.conf, disabling the default of scanning for devices, and listing devices explicitly, as follows:


# Monitor the drives on our RAID array; schedule self tests for Sundays.
/dev/twa0 -d 3ware,0 -a -s L/../../7/02
/dev/twa0 -d 3ware,1 -a -s L/../../7/04

The above monitors both drives of a RAID-1 mirrored pair on a 3ware controller card; the -s option schedules a long self-test every Sunday, starting between 2-3am and 4-5am respectively.

I’m still looking for a good way to monitor via Nagios, though; the (poorly-named) check_ide_smart plugin doesn’t support the ability to monitor drives on other interfaces as far as I can see. I found a couple of Perl scripts such as Check-SMART-status-modified, but they had issues.

Increasing loop devices on Xen host

Trying to start another Xen guest domain, and getting an error indicating that there are no loop devices left, like the following? :


Error: Device 51714 (vbd) could not be connected. Failed to find an unused loop device

Increase the number of loop devices that the loop kernel module will create, by editing/creating /etc/modprobe.d/local-loop containing something like:

Continue reading Increasing loop devices on Xen host

Ubuntu 11.04 – X freezing after screensaver active / laptop lid closed

I recently upgraded my laptop (a Dell 17R / N7010) to Ubuntu 11.04 (Natty), and since then, it has been randomly freezing when I come back to it after leaving it for a while.

I see my X session exactly as I left it, and can move the mouse around, but nothing seems to respond to mouse/keyboard events. The system is still alive, I can SSH to it from another box, or switch to a different virtual terminal (e.g. Ctrl + Alt + F1), and I can kill Xorg (sudo pkill X) at which point gdm immediately gets restarted, and I can log back in.

It seems to only strike when the laptop has been left unattended for some time. I can close the lid, wait ten seconds and re-open it, and everything is fine. I can also trigger the screensaver by locking the screen with Ctrl + Alt + L, then unlock, and all is also fine.

I’ve done some Googling, and I’m certainly not the only person experiencing this:

Bug 772925 – Screen freezes after laptop lid closed for more than 1 minute
Bug 781402 – Ubuntu 11.04 / Unity freezing when screen is ‘woken up’
a discussion of bug 781402 on fossplanet.com
Bug 740126 – Disabling an output can cause vblank events to be missed

I’m using ‘Ubuntu Classic’, i.e. using GNOME rather than Unity, but the reports of the problem I’ve seen are split between both.

I’ve found a suggestion of a possible fix – installing compizconfig-settings-manager then disabling Sync to vblankon ubuntuforums.org.

I’ve just tried that out; I’ll update this post when I know whether it has worked or not.

EDIT: I’ve checked logs for anything illuminating and found little, but I did just spot the following in /var/log/Xorg.0.log.old:


[437515.524] (EE) intel(0): [DRI2] DRI2SwapComplete: bad drawable

I’ve no idea if it’s related to the problem or not.

EDIT 2: So far, it would seem that disabling “Sync to vblank” as mentioned above has fixed the problem, for me. (It’s also possible that it was fixed in one of the updates since, but I haven’t spotted any package updates that looked relevant.)

If you’re having the same problem, feel free to comment here on whether the “Sync to vblank” change stopped the freezes for you.

Incremental backups with rdiff-backup

My current backup solution is using rdiff-backup to do incremental backups. I’d previously been using plain rsync, but I wanted snapshots too, so I could retrieve a file as it looked at some point in the past (as Sod’s law guarantees that if a file was mistakenly deleted/clobbered, rsync will run between that happening and you noticing, so the backup will be clobbered too).

My setup involves the backup box running rdiff-backup, connecting to the machine to be backed up via SSH, using a passwordless SSH key for authentication. The entry in ~/.ssh/authorized_keys on the machine to be backed up allows that key to be used only to run rdiff-backup, nothing else, and only from the backup host, to provide as much security as possible.

I thought I’d document my setup here, both for easy future reference for myself when adding other boxes to back up, and for anyone else who may find it useful.

So, the steps I use are as follows:
Continue reading Incremental backups with rdiff-backup

Slow SSH connections – hanging at GSSAPI auth

A particular box at $work had been slow to SSH to for a while, and I finally wanted to spend the time to find out why.

Running ssh -v thatmachine showed that it was hanging whilst attempting to authenticate with GSSAPI, with the slow section looking like:

debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,gssapi-with-mic,password
debug1: Next authentication method: gssapi-with-mic
debug1: Unspecified GSS failure.  Minor code may provide more information
Credentials cache file '/tmp/krb5cc_1000' not found
debug1: Unspecified GSS failure.  Minor code may provide more information
Credentials cache file '/tmp/krb5cc_1000' not found
debug1: Unspecified GSS failure.  Minor code may provide more information
debug1: Next authentication method: publickey
debug1: Offering public key: /home/davidp/.ssh/id_rsa
# authentication by public key then proceeded quickly

SSHing to the machine by IP instead, i.e. ssh -v 10.1.1.192, produced slightly different output:

debug1: Authentications that can continue: publickey,gssapi-with-mic,password
debug1: Next authentication method: gssapi-with-mic
debug1: An invalid name was supplied
Cannot determine realm for numeric host address
debug1: An invalid name was supplied
Cannot determine realm for numeric host address
debug1: An invalid name was supplied
debug1: Next authentication method: publickey
# authentication by public key then succeeded quickly

It’s clear that it’s trying to authenticate using GSS-API (Kerberos), failing, then moving on to public key auth.

The fix is simple – disable attempts to use GSS-API by adding the following to ~/.ssh/config:

GSSAPIAuthentication no

Before adding that:

[davidp@columbia:~]$ time ssh 10.1.1.192 touch /dev/null | grep real
real	0m15.512s

After adding it:

[davidp@columbia:~]$ time ssh 10.1.1.192 touch /dev/null | grep real
real	0m0.611s

Problem solved.

Disabling wifi kill-switch on Inspiron 17R (N7010)

The wireless kill-switch (Fn+F2) on my Dell Inspiron 17R (N7010) laptop is getting on my tits, as I occasionally hi that key combination by accident when trying to use Ctrl + F2.

I can’t seem to see any way to disable this, and Google doesn’t seem to be able to find anything useful. Anyone have any ideas? There must be some way to ignore that key combination.

Running xev whilst pressing that key combination shows me:

KeyPress event, serial 33, synthetic NO, window 0x5c00001,
    root 0xb0, subw 0x0, time 39357273, (121,-345), root:(1426,350),
    state 0x0, keycode 246 (keysym 0x1008ff95, XF86WLAN), same_screen YES,
    XLookupString gives 0 bytes: 
    XmbLookupString gives 0 bytes: 
    XFilterEvent returns: False

KeyRelease event, serial 33, synthetic NO, window 0x5c00001,
    root 0xb0, subw 0x0, time 39357273, (121,-345), root:(1426,350),
    state 0x0, keycode 246 (keysym 0x1008ff95, XF86WLAN), same_screen YES,
    XLookupString gives 0 bytes: 
    XFilterEvent returns: False

So, if it’s being triggered by software, it’s keycode 246 I need to deactivate/map to something else, I believe.

However, I tried with:

xmodmap -e 'keycode 246 = NoSymbol'

… but no effect, it still kills the wireless. I’m hoping that someone might know how to do it, and/or that someone else Googling for how to do this will find this post, and perhaps share any ideas. If I figure it out, I’ll of course update this post with the solution.

UPDATE – I probably should have mentioned that this machine is running Ubuntu Linux. Also, I found that, in System > Preferences > Keyboard Shortcuts, I can assign a shortcut to the Fn+F2 key combination (the shortcut column shows 0xf6), and pressing Fn+F2 does indeed trigger that shortcut (launching a terminal window, for a test), but also still triggers the wifi killswitch. I suspect that, as well as being seen by the OS and software, it’s being directly intercepted by hardware to toggle the card?

UPDATE – to help people Googling for info on how to disable the wireless/wifi kill switch key combination / shortcut for this laptop, it’s a Dell 17R / N7010 with a Broadcom BCM4313 802.11b/g LP-PHY wireless adaptor.