Software Development | Bieber Labs

links for 2008-04-25

Lead Architect Blaine Cook Out At Twitter

“I left Twitter just over two weeks ago. It’s an amicable change; Twitter’s architecture is at a good point for them to pursue stability and growth”

(tags: twitter news)
Kigg Starter Kit : The Official Microsoft ASP.NET Site

Digg Clone for .NET

(tags: digg asp.net dotnet vc)

Wordpress 2.5 Upgrade Painless - Really!

Yesterday I read that the 2.5 version of Wordpress was officially released. The cautious part of me wanted to wait to upgrade, but the totally paranoid and non-lazy part wanted to just get it over with. I had read a few mentions of incompatibility, but I decided to just bite the bullet and upgrade.

I hunkered down at my desk, did database and filesystem backups, downloaded the 2.5 release and untarred to my blog directory. Typed in the URL and — poof — there was the blog. Did smoke testing of all the plugins and they worked - every one of them. I was literally done in about 5 minutes.

I don’t think I’ve had an upgrade that wasn’t painful in some way in a long time. This one was nothing. It just worked.

So anyway, long way to say I’m on Wordpress 2.5 now. I was shocked at the ease of the upgrade. Later in the day, I saw all kinds of tweets around other people upgrading and having the same fulfilling sense of surprise when they realized they were done.

Thanks to the Wordpress team for another great (and uneventful) release.

Vacation Learning - PHP and Smarty Templates

I’m on vacation this week and next week. Since I rarely have time to learn anything technical (or blog for that matter anymore), I thought I would take some time during my time off to learn something new around development.

We have a system at work that is essentially a small portal. The core of it was written by me to learn PHP about 8 years ago and has been augmented by ~~me and one other guy at work~~Keith and I over the years. Over that time, as we added new functionality to it, I used it to experiment with other languages as I was learning it. Other pieces were written in Java out of convenience. In total, we have pieces written in PHP, Java, Python, and PERL.

As I usually use this system to learn new things, I figured it would be a good candidate to use to learn how to use the Smarty templating system for PHP. I became interested in this templating system after working with Eventum over the last few weeks and figured that if I am going to do further work with Eventum, it would be helpful to understand the templating framework it uses.

So I’ve started using the system to take our 8 year old PHP code base and separate some of the presentation logic out. Smarty is pretty flexible and easy to use at a high level (I haven’t gotten into any of the really advanced stuff yet).

Here’s an example of how nicely the use of a templating system simplifies your code. Take this example, which enumerated entries from our internal wiki via an RSS feed into a section on the home page:

I’m sure you can appreciate how hard this would be to maintain, and all of the cruft that has accumulated over the years …

Now take the simplified version (sans error checking), written today in about 10 minutes:

function getWikiEntries($url) { $rss = fetch_rss($url); $template = new TemplateEngine(); $firstColumn = array_slice($rss->items, 0, 10); $secondColumn = array_slice($rss->items, 10); $template->assign("firstColumn", $firstColumn); $template->assign("secondColumn", $secondColumn); $template->assign("link", $rss->channel['link']); return($template->renderString("wikiEntries.tpl")); }

… along with its corresponding Smarty template:

<table width="80%" cellpadding="0" cellspacing="0" align="center"> <tr><th class="header-title" colspan="2">Recent Wiki Entries   <a href="{$link}"><img border="0" src="{$applicationURL}/images/rss.png"/></a></th> </tr> <tr><td colspan="2" class="modifications-sectionheader"> </td></tr> {section name="entries" loop="$firstColumn"} <tr> <td class="modifications-data"> <a href="{$firstColumn[entries].link}" title="{$firstColumn[entries].title}">{$firstColumn[entries].title|truncate:28:" ..."}</a>   </td> <td class="modifications-data"> <a href="{$secondColumn[entries].link}" title="{$secondColumn[entries].title}">{$secondColumn[entries].title}</a> </td></tr> {/section} </table>
I don’t know about you, but I think thats quite a difference in maintainability. I’d much rather modify the html in the template than in the original function. Not only that, but the code is actually code, not a bunch of code with a lot of simply horrid markup stuck in the middle of everything.

I’m pretty impressed with how much I’ve been able to use in a short amount of time this week. The libraries are obviously thought out and ramp up time for me was really minimal. I like libraries like that. It also addresses something that has annoyed me for a long time. Embedded HTML is a pain to maintain and I’ve dreaded going into this over the years just because of that.

At some point, I’ll investigate what it takes to write custom plugins, a functionality that the libraries also support.

I think I’ve been able to get a really good start at getting something maintainable. My goal over the next few of weeks is to templatize the whole system, then start taking the non-PHP pieces of the system and rewrite them in PHP. I’ll also add the ability to change configuration in one place, so that we can cut some of the pain that we have in keeping things maintained down - and perhaps be able to install the application in other places.

Should be fun. I’m definitely feeling productive over the past few days. I’ve always liked working in PHP over other languages. I definitely have to do work like this more often.

Flickr Blog This To Draft Plugin

I found this plugin that corrects a pet peeve of mine that I’ve had for a while. The Flickr Blog This To Draft Plugin by Donncha O Caoimh ensures that all of your blogged photos from Flickr come in as drafts, so that you can go in and massage the HTML before publishing.

If you blog photos from Flickr much, and spend time rushing to edit your published picture (to fix HTML, add CSS attributes, etc), grab this plugin to remove the unneeded stress from your life.

LDAP Enabling The Eventum Defect Tracking System

Due to a recent reorg, I have the opportunity to replace our defect tracking system, which has quite a bit of really wasteful process baked into the tool, with a new one. I’ve been looking at defect tracking software for a while, and chose Eventum, an open source project by MySQL AB for a number of reasons, some of them including:

Its open source
Its written in PHP, so I don’t have to worry about messing with fastcgi, mod_perl, or mod_python
It is extensible (you can add custom fields, etc)
It uses MySQL, rather than SQLLite or something like that, so we can integrate it into the rest of our home-grown build software
It supports email integration. While we won’t be using this right away, we’ll be implementing it in a later iteration
Its simple to use, with a very simple interface, once you get use to it. Everything is essentially on one screen.
It has time tracking, along with some basic reporting built in

One thing it doesn’t have built in is LDAP authentication. I wrote a previous article about all of the work we’ve done to integrate both our home grown applications and a few open source applications in with our LDAP store, to minimize the management of multiple passwords across systems, so this was very important to me. I started with many, many Google searches to see if someone else has done this, only to hit one dead end after another. At first I was being lazy and decided to just forget about it. One system not tied to the LDAP tree isn’t that big of a deal, but then my perfectionism set in. Why would I settle for that when LDAP authentication should be really easy to integrate into an Open Source package?

So I decided to spend a few hours to get it working. Since I had no success finding an implementation, I figured I could do my part and post what I have. There are a couple of caveats that I want to throw out before we actually get to the code though:

It isn’t done “right”. This is all extra work for me, so I got enough done so that it would work. The right way to do this would to refactor the auth stuff out into a workflow like hierarchy that could be pluggable (see this post in the eventum mailing list). I’ll get to that someday, but right now this solution hacks the auth module to get authentication working.
LDAP Settings are not configurable through the interface. I don’t have time for that, so a set of defines at the top of the LDAPAuthenticator class contains all of the configuration information for the LDAP server. Bummer, but like I said, I’m on a schedule.
Users still have to be added to the Eventum database - they are not added automagically when they log in. I want control of who is in the system, so I’ve elected to leave this functionality out and just do authentication.

With these three caveats in place though, given my experience looking around for this stuff, at least this code works and will be able to be used by others. Its a starting point - which is more than is out there today. Anyone is free to use this and take the time to do it right. With that said, I’d love to receive updates if someone actually takes this up. For now though, this works for me.

So, now to the code. I wrote a small PHP class called “class.LDAPAuthenticator.php. There are two functions in it. Because Eventum uses email address as the login, we need a way to get the full user DN from the email address. This is what the email_to_dn function does. Given an email address, it returns the full distinguished name of the user. This is called by the main class function, ldap_authenticate. The ldap_authenticate function takes the same arguments as the class.auth.php function isCorrectPassword, which consist of the email address and the password. It binds to the LDAP authentication tree using the full DN of the user and the password supplied. If authentication is successful, it returns TRUE, otherwise it returns FALSE, just like the isCorrectPassword function used to validate the password from the Eventum database.

The code looks like this:

# Change these values to access another LDAP server. define("LDAP_PORT", 636); define("LDAP_HOST", 'ldaps://ldapserver.example.com:' . LDAP_PORT); define("LDAP_BIND_DN", 'PUT THE BIND DN HERE'); define("LDAP_BIND_PASSWORD", 'PUT THE BIND PASSWORD HERE'); define("LDAP_SEARCH_DN", "PATH OF THE TREE TO SEARCH FOR USERS"); class LDAPAuthenticator { # Look up a users full distinguised name from # their email address, since Eventum uses # email address as the login name. function email_to_dn($emailAddress) { $returnDN = ""; $server = ldap_connect(LDAP_HOST); if ($server == FALSE) { return($returnDN); } ldap_set_option($server, LDAP_OPT_PROTOCOL_VERSION, 3) ; $ldapbind = ldap_bind($server, LDAP_BIND_DN, LDAP_BIND_PASSWORD); # verify binding if ($ldapbind) { # find the user based on the entered email address. $result = ldap_search($server, LDAP_SEARCH_DN, "(&(mail=$emailAddress))", array("dn")); $info = ldap_get_entries($server, $result); # if we actually got a value back, return the users DN if ($info["count"] > 0) { $returnDN = $info[0]["dn"]; } ldap_unbind($server); } return($returnDN); } # Authenticate with the LDAP server. Function returns true # if authentication was successful, false otherwise. function ldap_authenticate($email, $password) { $returnValue = FALSE; $userDN = LDAPAuthenticator::email_to_dn($email); if ($userDN == "") { return($returnValue); } $server = ldap_connect(LDAP_HOST); if ($server == FALSE) { return($returnDN); } ldap_set_option($server, LDAP_OPT_PROTOCOL_VERSION, 3) ; $ldapbind = ldap_bind($server, LDAPAuthenticator::email_to_dn($email), $password); if ($ldapbind) { $returnValue = TRUE; ldap_unbind($server); } return($returnValue); } }

Save this file as class.LDAPAuthenticator.php and put it in your Eventum includes directory. Modify the define statements at the top to contain your LDAP server information.

Now, to use it. Go to your Eventum includes directory and add the following line to the top of the class.auth.php file:

require_once(APP_INC_PATH . "class.LDAPAuthenticator.php");

I have this at the end of all of the rest of the require statements.

Now, replace the isCorrectPassword function in class.auth.php with the following function:
/** * Checks whether the provided password match against the email * address provided. * * @access public * @param string $email The email address to check for * @param string $password The password of the user to check for * @return boolean */ function isCorrectPassword($email, $password) { return(LDAPAuthenticator::ldap_authenticate($email, $password)); }

… and VOILA. You can now authenticate off of your LDAP tree.

Now, I know it isn’t pretty, hacking the code directly - but it works, and its more of a starting point than I can find anywhere else. I hope its useful to others. Again, if anyone takes this further and does it “right”, I would be really happy to get a copy of the modifications.

One more thing - don’t forget to require SSL on the URL to your Eventum installation by using the SSLRequireSSL directive in your Apache server. You don’t want these passwords floating around in the clear across the network.

Download the Eventum LDAP hack here and happy authenticating.

Subversion and SSL Troubles

I decided to upgrade my home Subversion repository to version 1.4.3 as soon as it was released. Since then, my ViewVC application has ceased to work, getting a Python exception every time I try to execute it. Creating a small Python program that just imports the library (from svn import fs) gave me the following error:

ImportError: /usr/local/lib/libsvn_ra_dav-1.so.0: undefined symbol: SSL_load_error_strings

Thinking it was an SSL library problem, I upgraded SSL - a few times. I kept mucking with the options, rebuilding Subversion, only to get everything installed and get that same error:

ImportError: /usr/local/lib/libsvn_ra_dav-1.so.0: undefined symbol: SSL_load_error_strings

Over, and over and over again I repeated the process and got the same result. The absolute definition of insanity. This has been going on for a couple of months and I’ve been trying to address it in my spare time, as I’ve been pretty busy lately during the week and gone to the Relaxation Unit the last few weekends.

I googled my ass off to find the error, but to no avail. Finally today I ran across this thread that explained the problem. After going through my distribution directory for 1.4.4 (which I upgraded at the beginning of the month only to receive the same error) I realized that I hadn’t pulled down the Subversion dependencies tarball and rebuilt neon. So, basically I was using an old version of the neon libraries.

I finally settled on the configure statement listed here, after downloading and untarring the deps file:

./configure --with-ssl --with-apxs=/usr/local/apache2/bin/apxs \
            --with-apr=/usr/local/apache2 --with-apr-util=/usr/local/apache2 \
            --enable-shared --with-libs=/usr/local/ssl

This uses the already installed apr libraries that I build with my Apache server, and ensures that the neon shared libraries are built. A quick configure/make/make install/make swig-py/make install-swig-py sequence later and my Python libraries were working fine.

I made it a point this time to document this on the Labs internal wiki, but thought I should throw this out here in public so that others can find it. Hope it helps save the weeks of frustration that I have been suffering for someone out there.

Happy building …

Metrics As A Side Effect

Yesterday morning I found an article by a former employee who has recently introduced Scrum to his organization called Beef Up Your Scrum-Master Toolbox up on the Devx site. Since Doug started his blog and then subsequently left the company, I’ve enjoyed keeping up with his new adventures and I went right to the DevX site to find out what he had been doing lately.

The article is great, outlining the way that his team is accruing metrics for their Scrum teams using an Excel spreadsheet. However, one thing kept nagging at me the whole time I was reading it. The little voice in my head kept saying:

God, thats way too much work!

We recently moved most of our metrics, such as cycle time of items, to an automated process, eliminating about an hour worth of work from one of my managers that was doing it manually. As it stands right now, all of our metrics except one is generated automatically and posted to our internal intranet site. However, the system still isn’t perfect, as it requires development and QA to navigate items through a pretty tedious workflow to get accurate information, something that in the heat of a release is often easy to forget to do - just because its “extra stuff”. I know, because when I’m involved I often forget to update them myself. It seems to me that the real problem getting metrics is how much work is involved in making sure you get the data.

Doug’s article really got me thinking. How can we get the fine grained metrics he’s talking about, in a way where someone isn’t sitting in an Excel spreadsheet for an extended amount of time inputting data, and where all data is updated real time?

As I was thinking about this dilemma, I hit F12 on my Mac to look at the weather and I noticed this little box staring at me:

.. and then I started thinking. What about my experiences with Twitter could help with this problem?

Twitter First Impressions

I signed up for Twitter a few weeks ago after hearing Coté and Jason Calacanis mention it quite a few times on their respective podcasts and blogs. I wasn’t really impressed with it at all, only because I’m not willing to set up my phone to use it (I hate typing on a phone) and I don’t like having a separate web page up all of the time. Then I found the Twidget dashboard plugin for OS X and found that hitting F12 and typing a message was much more conducive to my working style than any of the above methods - it was less of an interruption.

Most of the twitter messages I post now are at home, where I have my F12 key handy and it takes little effort to actually post an answer to the question “What are you doing?”. Its effortless.

Trying To Go Lean

One of the adjustments we’ve made to our development process over the last few months is a logical reduction in the “types” of work we track. Rather than deal with “projects”, “change requests”, and “defects”, we are attempting to reduced these queues into something we call “open items”. For each change we want to implement, one or more “open items” are entered to address them. These items are then tracked for cycle time and completion status. Along with cycle time, we also keep track of the total number of “open items” we have that have not yet been deployed to production. These are reported on using your standard burndown chart that would look something like this:

For larger projects, the overall project is tracked on a Wiki, where we can both keep track of current tasks, and create technical documentation for the work we are doing as we do it. We’ve found that the documentation we create as we are doing the work is much more complete and accurate if we write it during the development effort than it is when we “go back later” to document it. A particular initiative is mapped to several smaller “open items” and scheduled for subsequent, incremental deployments (sprints, increments, whatever). In an ideal world, only the items that are scheduled for the next deployment would be worked on for the next deployment, but we aren’t there yet.

Tracking “open items” as a general unit of work gives us a start in creating a “pull system”, where the development team can work on open items as they come into the queue. The prioritization process, ideally, would be extremely light: pull things from the head of the queue, in order of criticality (Critical, High, Medium, Low), oldest first. This gives us a very simple ruleset to pull from (no prioritization meetings required!), and allows our business users to set a priority that can expedite changes if it is necessary. The defect tracking system, at this point, is a real time reflection of the work we have to do and the priority that the business thinks the work should be done in. This also allows “demand” to come in in a uniform way. The rules are simple and there is one entry point for all work.

On a weekly basis, data is pulled from the defect tracking system and loaded to our intranet database (a small data mart) where we calculate cycle times, and can graph the current rate of work.

Measuring without The Context Switches

The problem is that reporting is only a part of the problem. You can report on the data, but the data will only be accurate if you have an effortless way to update the items you are working on.

Its like Twitter - if I have a separate window that I have to keep going to in order to actually post something I’m less likely to make the effort - because I want to get done with what I’m doing.

Tracking work within the software development process is even worse if, in order to do it, I have to to look up an item over and over again to update it, and there is some complex workflow that it has to go through. Chances are, Ron is going to look like he is behind all the time, no matter how much work he’s getting done.

So what would a “perfect” work tracking system look like for me?

Ron’s Ideal Work Tracking System

I’ve already asserted that I am more likely to Twitter if I can just hotkey to something simple, type something in and press my update button. I think the accuracy of measurements could be increased dramatically if we could have a system that took these types of things into account and reduced the context switches that the team has to go through in order to update status so that they can update real time and it feels like its part of the work that they are doing, rather than something separate. Its an added bonus if this feels like a social, fun activity rather than an extra set of tasks that need to be performed.

The way I see it, work can be entered into the system with the normal interface to the tracking system. This would give full visibility to everything needed to schedule work.

The new developer/QA interface to this system could look a lot like the Twitter widget above. Just a simple “What are you working on” type of interface that could talk to a network based API that, when a message is sent over, could search for a ticket number in the message and add a note to the item. A Subversion post-commit hook could use the same API to update the “code complete” status of the item (pulling the ticket number from the commit message), while a separate QA “widget” could have a pass / fail status pulldown that the QA team could use to signify the testing state.

As an added bonus, the deployment system could also use the API to let the QA team know the deployment status of the given item and whether it is actually there to test. This functionality, intersected with a “Pass” testing designation from the QA team in the production environment, could actually close the item automagically, which would reflect in your measurements.

The key to a system like this for me, is that if the updates feel like more of a social activity, and some of the obvious things are automated out of the process (like having to close an item once it is deployed to production and passed QA testing) the chances are high that you will actually get accurate real time status of what people are working on and the issues they may be running into. This is definitely some of what I am seeing as we use wiki software to track larger development efforts and as a manager the smaller grain information that I would get from a more “socially oriented” system would be extremely valuable to me just to know what is going on at any point in the process.

Now, this is just one view of the problem, from someone who is trying to lighten the large corporate process of getting work done. It might not work in every scenario or every environment - but I think it would be an interesting experiment.

I’d definitely be interested to hear other peoples opinions on this train of thought. Am I nuts or does this actually make some sense?

Video: How Open Source Projects Survive Poisonous People (And You Can Too)

Since getting a 80G iPod about ~~a month ago~~ two weeks ago, I’ve been really getting into watching the Google Tech Talks on Google Video. I recently watched How Open Source Projects Survive Poisonous People (And You Can Too), a lecture given by Brian Fitzpatrick and Ben Collins-Sussman from the Subversion team (now both Google employees) that summarizes a lot of information in Karl Fogels book Producing Open Source Software: How to Run a Successful Free Software Project.

If you haven’t had time to pick up and read Karls book, this video would be a good primer to some of the concepts in it and could very well motivate you to pick it up. Its an excellent book and one that I thoroughly enjoyed reading.

Wordpress 2.1.3 released.

Version 2.1.3 of the Wordpress blogging platform has been released and is available for download. According to the Wordpress blog, this is a security release that “includes fixes for several publicly known minor XSS issues, one major XML-RPC issue, and a proactive full sweep of the WordPress codebase to protect against future problems”.

I’ve upgraded, and so should you. Take a couple of minutes to do this upgrade, as the possible consequences aren’t worth the humiliation.

Wordpress 2.1 Released

The Wordpress team has announced the release of version 2.1 of their Wordpress blogging software on Monday. A list of the changes can be found in the release announcement.

Sure, this is old news (by internet time standards) but I just got around to getting caught up. I’ve upgraded the site and the process was absolutely painless. All of my current plugins worked as far as I can tell.

For those who have a bit of trepidation over upgrades and the compatibility of plugins, I can tell you that my installation, with over 17 plugins active, seems to be working fine after the upgrade this morning after a total upgrade time of around 15 minutes including backups of my installation directory and database. As usual, the upgrade instructions are available on the Wordpress site.

The Wordpress team is planning its next release, Version 2.2, around the April 23 timeframe.

I haven’t had a great deal of time this morning to play around, but it does seem that the site runs just a tad bit faster then it did under the previous version.

Studying Up On Ruby

Photo by rbieber

I’ve been focusing on Ruby a lot over the past 3 days (Rails specifically). What a great environment to work in!

This was taken about a month ago. I’ve been doing quite a bit of reading and experimenting with Rails and I have to say, I love it. While I’m still at the stage of figuring out “how” to do things, once you do it you get why its done that way. That can’t be said for many environments.

Denying Spammers

I’ve been having a real problem with the site being hit hard by spammers lately. Consequently, I have turned comments off on most of the articles on the site at this point.

Due to the implementation of SPAM Karma 2 and Akismet, none of the comment spam actually made it to the blog. I was pretty amazed at how thoroughly these two pieces of software have filtered the incoming comments.

However, the comments not making it to the blog doesn’t mean that the spammers haven’t done any real damage. Twice now I’ve come home to find my comments disabled by my provider, due to basically a Denial of Service attack being executed on the site by these morons.

I found a great page at the Wordpress Codex on Combatting Comment Spam. I would encourage anyone currently dealing with this problem to check out this page. I will be implementing some of these ideas one by one over the coming weeks to see how much they help.

I’ll let you know of the success I have. In the meantime, if you aren’t running Spam Karma, I would encourage you to go take a look at it and its corresponding Akismet plugin. The combination of both has been highly effective in this corner of the web.

Practical Subversion - Second Edition

I received a free copy of Practical Subversion, Second Edition by Daniel Berlin and Garrett Rooney on Friday from their publishers, Apress.

I had reviewed the first edition before it was released and had found it to be an excellent companion to “Version Control with Subversion” (C. Michael Pilato, Ben Collins-Sussman, Brian W. Fitzpatrick), mostly due to its coverage of the Subversion API’s - which I had not seen covered in any real depth in any other book.

I have to say, the authors have outdone themselves with the Second Edition. The book is extremely well written for varying levels of Subversion experience. The beginner will find a very easy to understand introduction to using Subversion in the first two chapters, giving a really great tutorial on how to use the tool along with explanations of many of the concepts embodied in the implementation of the tool, such as locking vs. non-locking systems, properties (from file to revision properties), the basic workflow involved in using version control, and how to use the various commands, from checking out, to using svn blame (or ‘praise’ as I learned from the book is an alias for the command) to find the origin of a change in the system.

Thats just the first two chapters. As the book goes on the reader will learn about repository administration, the differences between the BDB and FSFS file systems, using Apache and Apache modules to squeeze additional functionality into the system, migrating from other version control systems such as CVS and Perforce and third party tools that work with Subversion (such as ViewVC, emacs, etc). The book also covers maintaining vendor branches, and contains a very good chapter on Version Control Best Practices. You then have, from my memory anyway, a greatly expanded chapter on using the Subversion API.

Practical Subversion, Second Edition does a really good job of covering information at many skill levels in an extremely accessible way. This book will definitely be one of those that I would put on the shelf at work as we continue to move people into more advanced roles in the management of our repositories - as there’s really nothing the book doesn’t cover.

I’ve been a user of Subversion for a very long time (I think I started around version 0.19 or so) and as I perused the book last night I walked away with some new distinctions about how we were using the tool and changes I could make to make administration and maintenance easier. That says a lot.

Congratulations to Garrett and Daniel on a fine piece of work. Hopefully the next edition will cover some of the newer features of 1.4, specifically the svnsync tool.

Wordpress 2.0.5 Released

The folks on the Wordpress team have released version 2.0.5 of the Wordpress Blogging Application. This release includes around 50 bug fixes one of which was a missing index on the posts table. I just upgraded and the site performs much better now. I had always thought that the site ran a tad bit slower after the 2.0 upgrade, but for some reason I just figured 2.0 was doing so much more than the 1.x versions. Didn’t even think of looking at table indexes.

Mark Jaquith has also put together his list of change files and corresponding archives containing only the changes from 2.0.4 to 2.0.5, along with a patch file to upgrade your stuff directly. I opted to download from the Wordpress site.

In any event, according to the release announcement, there are some security fixes in this release as well. As I do with every release of Wordpress that contains security release, I am reminding you not to be lazy and get your site upgraded as soon as you can. The performance improvements alone are worth it.

Now … off to see if they fixed that “posting from Flickr mangling CSS thing” …

Building Subversion on The Mac and using Ecto for Blogging

I finally upgraded my Subversion installation on my MacIntosh to the 1.4 version. I was waiting for the “official” packages to come out so that I could just install it, but in looking at the different places recommended by the downloads page, these distributions haven’t been updated since early 1.3 releases.

I’ve had a goal to keep my Mac somewhat pristine. I decided thats not really practical. There are a lot of things that I use that I just like having built from scratch, so that I’m on the most current software and not dependent on someone else’s schedule. Subversion is one of those tools.

One thing I was shocked at was how quickly and seamlessly the build happened on the Mac. These MacBooks are pretty fast machines. I think it took a total of roughly 20 minutes (if that) to build, run tests, and install. The build on the Mac is definitely the fastest configure/check/install cycle I’ve gone through in the many installations of Subversion that I have performed over the years.

I tell you, the more I’m on the Mac, the more I like it. I haven’t run into anything that I’ve found irritating.

Its all good.

This is also the first post I am writing using a trial version of Ecto. I have to say, this application is pretty impressive too. They have both a Mac and a Windows version. I like it much better than blogging in Wordpress directly - and at $17.95 a copy, its practically a no brainer to purchase.

Don’t get me wrong, I’m going to milk the 21 day trial, but it feels like this application is a pretty good fit for me.

Upgrading Subversion to 1.4

This weekend I finally bit the bullet and upgraded our production Subversion server to 1.4. The upgrade was painless, after the usual running of autogen.sh on the Neon libraries that for some reason are necessary when building on the Solaris 9 environment. I also had some weird test failures that wound up being caused by hook scripts being called on the test repositories. For some reason, Python couldn’t find one of the standard libraries, even though I had set my path and library path in my environment. A quick Google search later and all tests passed.

We have been running Subversion with the Berkeley 2.2 back-end since May of 2004, when the Subversion team first released 1.0. We’ve done a few upgrades since then, when I found the time, but wound up getting as far as 1.2.3 and my time to actually do the upgrades never really materialized. Part of that had to do with some pretty large projects that were going on that I didn’t want to interrupt the teams with an upgrade.

So yesterday, finally, I did the upgrade to 1.4. As part of the upgrade, I decided to dump and load the repositories (11 of them) to take advantage of the new SVN Diff format that 1.4 brings to the table. As part of that, I also decided to abandon the Berkeley back-end in favor of the FSFS file system.

The dumps and loads took quite a long time, about sixteen hours. However, the results were worth it. After all was said and done, we saw a reduction in server disk space usage of about 46.8%.

Not bad guys!

As I had mentioned in a previous post, one of the motivators for this upgrade were the working copy improvements, which according to the release notes, should cut down on inode usage on machines running the client. In our environment, we have about 20 people all working on the same machine (in their home directories) and we have had problems with inode max-outs in the past. Again, I’ll have to reserve judgement on these changes until we see the results, but this is one of the things that motivated me to bite the bullet and take a Saturday to do the upgrades.

The other big motivation for me, especially after yesterday was this statement from the FSFS document referenced above:

An FSFS repository can be backed up with standard backup software.
Since old revision files don’t change, incremental backups with
standard backup software are efficient. (See “Note: Backups” for
caveats.)

(BDB repositories can be backed up using “svnadmin hotcopy” and can be
backed up incrementally using “svnadmin dump”. FSFS just makes it
easier.)

Aside from normal system backups done nightly by the admin team, our nightly backups have also consisted of doing nightly dumps of all of the repositories so that, in the event that the repository server goes down, we can just load the repository on another machine (you can never have too many backups - in my opinion). The wake up call from yesterday is that it takes much longer to reload a repository on Solaris 9 than it does on the Linux work-group server I have sitting here that I did all of my original testing on a few years ago. What I thought would be a trivial load (3 hours was the estimate that I had in my head) wound up being 5-6 times that on the 2×2 Solaris box at work. It will be nice to be able to rely on the nightly system backups for these now — and perhaps just a quick tarring of the existing repository structure for quick backups (using hotcopy), rather than the dump / load process we have planned to use up until now. I’ve just never been comfortable for some reason with the Berkeley backend and backups outside of repository dumps.

Finally, I’m really looking forward to playing around with svnsync to set up a shadow fail-over repository in the event of an outage. This was something I threw on a wish list back in 2004, and I’m excited to see a tool to synchronize a “stand by repository” included in this release. There apparently is not much documentation on this new feature yet, but this article should be a good starting point.

Thanks to the Subversion team for another great release of their software. As anyone who reads this blog with any regularity knows, this is one of those pieces of software that I continuously receive value from and cannot say enough about.

Now I’m off to search for more svnsync documentation …

Related posts

Related posts

Related posts

Related posts

Related posts

Related posts

Twitter First Impressions

Trying To Go Lean

Measuring without The Context Switches

Ron’s Ideal Work Tracking System

Related posts

Related posts

Related posts

Related posts

Related posts

Related posts

Related posts

Related posts

Related posts

Related posts

Search Bieberlabs

Most Popular Posts

Categories

Archives

Tags