kate's infrastructure

Finally!

As per my previous two posts on the topic, migration from one web host to another using wordpress eventually worked out great.  I’ll consider periodically deleting spam comments from the DB in the future.

The only things I had to do:

• Upload my customised theme, and re-configure some of the presentation options;
• Manually re-enter the links in the ‘blogroll’ (though there are automated ways around it, this seemed easy enough);
• Upload my plugins, re-enable them and test;

The only problem I noticed is the failed google calendar feed into the wordpress sidebar.  I had been using the gcalendar plugin, which was great.  I now see that due to migration-induced upgrade, the plugin is no longer supported.  I should have actually noticed this on the old site, so this is not a reason to stop the cutover.

Now’s the moment for cutover to the new host.

• Tested through a dedicated IP on the new host seems ok;
• Setup mail services – ok
• Setup all content services -ok

Next step is to cutover to the new nameservers at my registrar of choice, Directnic.

Aside from the bugs I’ve now got in the site, my next follow-up is with extra services like feedburner and google analytics to ensure they work.

A few days away from the problem hasn’t really given me any new perspective.  I’ve cleaned up my old host and tried to get rid of anything not absolutely mandatory.  It’s really frustrating to run the same export repeatedly and get different results each time.  Kind of makes it hard to pinpoint the problem.

Interestingly, I did notice that the webalizer software installed on my old host shows completely different stats from google analytics.  Probably worthy fodder for a future investigation. 

Anyway I had decided to give up on WordPress exports and simply deal with this as a mySQL issue, here’s the score:

• I created a db export using PHPmyAdmin on my old host.  The script generated the full create data for the DB.  Somewhat less useful than it seems.
• On recommendation from TechDuke, I tried using the bigdump script to import my db from the db, though this is generally targeted at people with specific import issues.  Laziness was my specific issue.  It didn’t work on the first run, but I think I’ll bookmark it to review again.

Reviewing the previous wordpress export attempts, the most frustrating thing was how many comments I’d previously marked as spam were clearly still in the DB.   Overall, the function wasn’t working for the posts I wanted, but it treated spam nicely.  I decided if this process was going to succeed, by any method, I had to fix the spam.

• A quick search and I stumbled upon a stroke of genius – just use phpMyAdmin to run a delete script on the spam comments.
• Query run in a few seconds.
• Re-ran export and I at least appear to have my latest posts.

Success seems closer.  Right now I’m deleting the wordpress install on my new host, I’ll re-install then try re-importing in a few minutes and we’ll see how it goes.

Storage illustrated through russian dolls: http://www.artlebedev.com/everything/matryoshkus/

Excellent, except for ending with Terabyte.  Must future proof gimmicks.

A reminder of what’s already in place, and maybe the soon to be common:

• Kilobyte (KB) - 1024 Bytes
• Megabyte (MB) - 1024 KB
• Gigabyte (GB) - 1024 MB
• Terabyte (TB) - 1024 GB
• Petabyte (PT) – 1024 TB
• Exabyte (EB) – 1024 PT
• Zettabyte (ZB) – 1024 EB
• Yottabyte (YB) – 1024 ZB

I remember having under a GB storage on my home computer, and in 2007 a TB is now in reach for home users.  Even Mac users. Crazy times.

I’ve not used this, but if I don’t record the link someplace I’m bound to need it.  This looks good: http://home.eunet.no/pnordahl/ntpasswd/

Though it says NTPassword, it words right through to Vista.

I’m about to move web hosts from one that’s not so great, to one that seems pretty great.  I’ve had a pretty active wordpress blog on another host for over a year, so I’d prefer a smooth change.

I’ve been damn impressed by wordpress as a really simple and extensible web content management system.  I like it, it’s easy to use, I can dabble in PHP code or MySQL if I feel the need, but it’s not a requirement.  WordPress provide pretty regular code updates and plug-ins that would satify my every whim.  Right now, my only dream is a smooth migration.  I want to retain the look & feel, content, posts and users from the previous system.

But how?

WordPress have their act together as usual and have included an import/export function within their v2.2 codebase.  In a simple click of a button it throws my year long rambles into a massive XML file.  Three seconds into the exercise and I seem to be living the dream.  Naturally, not everything quite works as it should and I seem to be missing a few months worth of the most recent posts.  As this is when I feel I was at my wittiest, I must retrieve them.

Here’s my approach so far (will update the list as I learn more):

FACT #1: Opening the XML file generated in Internet Explorer suggests some problems in the XML validation. 

THEORY #1: Internet explorer often sucks at tasks like this.

INVESTIGATE #1:

• [LowTech - NOTEPAD / IE] Look for possible XML-unfriendly characters in the text, tracking to the char that IE identified.  I opened the file in notepad (ouch), and did a search and replace where I found bad characters.  Periodically reopened in IE.  Repeat.  It’s tedious and really didn’t look to be a quick fix, particularly as I was struggling to find some of the bad characters.
• [FreeTech - UltraEdit] Oh how I used to love UltraEdit back in my ASP dev days.  I think I might purchase it, but certainly not for this purpose.  Made finding the duds easier, but essentially it was a pretty notepad for this exercise.
• [ShareTech - XML] I downloaded an XML validator, hoping for a ‘fix stuff’ button.  No joy, and the validation was pretty inconsistent.  Uninstall. 

CONCLUSION #1: Trial import into wordpress actually accepts the file.  Assume that this is wasted effort for now.  Valid XML is not the problem (for now).

[ -------]

FACT #2: The file is missing the content and isn’t closed properly. 

THEORY #2: WordPress or the web server are misbehaving.

INVESTIGATE #2:

• [LowTech - Repeat] Sounds stupid, but I thought I’d try it again.  I actually got a slightly larger file, but a few posts closer to success still isn’t success.  Problem remains.
• [FreeTech - Cheat] Checked the various WordPress support forums.  Others have similar problems, but no one seems to have described mine.  Post my own question.  Fingers crossed. 
• [ShareTech - Upgrade] Newer = better, right?  I upgraded both blogs to v2.2.  Same result. 
• [HighTech - Web server logs] Simple.  Grab the IIS logs from my old server, find the line which suggests someone was running export.php, check the status code.  In this case it was 200, which is fine.  There’s a chance security or max file sizes on the server forced the truncation, but nothing I can locate so far.  This one’s not officially ruled out yet.

CONCLUSION #2: Investigation on web server settings are ongoing.  Good times.

[ -------]

FACT #3: …

THEORY #3: The export function doesn’t work.

INVESTIGATE #3:

• [HighTech - Master mySQL] This is boat rocking at it’s finest.  Basically its all about mucking around with the DB behind the scenes.  Normally I’d be ok with this, but I love that wordpress doesn’t usually require this.  I’m going to hold out as long as I can, but the steps I’ll need to do are:

I’ve installed an ‘empty’ wordpress DB on my new host, using the Dreamhost one-click install;

I’ve generated an SQL script from my old host which would populate a WordPress using phpMyAdmin;

I’ve ftp’ed the file to my server ready to run;

I’ve completely lost the ability to access mySQL via SSH on my new host.  Problem is likely me, though this will stall my progress a tad in the meantime.

 CONCLUSION #3: Searching for alternatives, but mySQL play looks likely.

 More updates to come.

When it comes to reading server logs, wikipedia loves you: http://en.wikipedia.org/wiki/List_of_HTTP_status_codes

Rule of thumb = 200/300 series are lovely. 

So the domain name may be lame, but the intentions are grand.  Once upon a time, in the days before blogs were fashionable, I used to read a site about the many challenges we face day to day in the industry.  This wasn’t a big industry blog or vendor backed site, it was just a guy who was sharing his ideas, lessons and often some pretty innovative solutions.  If only I could find that site again…

I’d never make such grandiose claims, but it would be nice to keep this site as a bit of a reference for stuff I learn along my infrastructure adventure.  If it helps anyone else, that’d be lovely too.

My world is infrastructure and IT Operations.  Not always the most glamorous of topics, but it’s a truly exciting time to be in this industry.  Hopefully I can share some of the experiences and they’ll be useful to others too.