Thursday, June 05, 2008

Automating Repository Setup/Mirroring with svnsync

If you're interested in doing some svn mirroring with svnsync, first read Jeremy Whitlock's article on Mirroring Repositories with svnsync. Then, read the rest of this post, which contains bash scripts that should help you automate the process of creating new repositories for mirroring.

Update: Click on the links noted as (script) to see the Pastie.

Our svn server blew up a few weeks ago, and I had to come back into the office from 12 - 3am to get it running on another box. This got me a little paranoid about our data. While the server was running RAID I, it was all in house. So, after getting a new Dell server, I installed Ubuntu 8.04 Server (64bit) on both the new server and a slice at slicehost (great company!) and wrote the following scripts, which are up on Pastie.

On the servers, I'm assuming the directory structure:

  • /srv
    • /svn
      • /repos
      • /config

/config contains a shared authz.conf (script), passwd.conf (script) and svnserve.conf (script). Click the links to see examples.

On the main svn server, create a bash script called create_repo and paste in the contents of this script: create_repo (script).
You then want to change the ownership and make it executable:
sudo chown user:group create_repo
sudo a+x create_repo

Now, you can execute
sudo create_repo my_new_repo
and the new repo will be created.

NOTE: line 72 of the create_repo (script) script copies the post-commit (script) hook from an existing repo that already works. For the life of me I couldn't get that into the create script and have it work. It worked on my test repo, and that's why I'm copying it INTO my_new_repo. So, you might want to do the first one manually, which you can pretty much follow the steps from the script itself. If you make it work please leave a comment!

UPDATE: SVN doesn't include your environment when executing the hooks, so make sure you're fully qualifying executables in the scrips. /usr/bin/svnsync instead of just svnsync

Ok, now we have the new repo set up on the main host and the post-commit hook script in the right spot. Like the last line says, now it's time to log onto the mirror and execute the next script.

On the mirror create the same structure, and put the same files in the config folder. Now you can create syncsetup by pasting in the following code: syncsetup (script). Notice line 71, which again, copies the pre-revprop-change (script) hook script from a repo that already works. Just like the create_repo script, you might want to run each command manually now too to iron out any issues with your implementation.

Run the following to make the script executable:
sudo a+x syncsetup

Hopefully it'll work, ya know? If you get some strange "svn is locked" or "svnsync pre-revprop-change hook failed with error output" message, try deleting the rev prop on revision 0 of the mirrored repo. Check out this thread too.

If you need help, please post in the comments. I'm hoping the content above makes sense and works for you first try. However, I've been at work since 6am today and I'm fried. And I'll TRY to help, don't get upset if I can't/don't. Good luck!

UPDATE: I couldn't get the post-commit hook to fire properly, so I moved it out into a script and I run it as a cron job. I'm not happy about it, but I think SVN should also give me nice error messages. Go Git?!?

No comments: