The Code Cave

Well I can’t call it the 5 second upgrade script any more… Since adding full file backups, and compressed database backups, from Windows desktop, through manual log in and launch of the update script, it took me ~15 seconds to update an active blog with a couple dozen posts and log all of the results to a html log. I’m fairly certain I could type my password faster and shave off a few seconds. The script itself, which now shows start and stop times, only took 2 seconds to do its work. The rest was connect and login time. Sometimes it took a whole 30 seconds for the process to complete, web and server usage being what it is, but either way, wow. I should say that I used this last night with all of the options turned on, creating file backups AND gzipped backups AND database backups AND an HTML log file AND adding extra verbosity AND updating my 6 WordPress blogs and it took a full 8.5 minutes. I had to actually minimize the window to get it out of my way…. Between that and typing in the script name at the shell prompt, I was exhausted!

When I think of how long it used to take me to update just my wife’s blog, I just have to shake my head. Each release was a many-night, if not many-week process till I had the spare time to concentrate on doing the whole thing right. And I had to look up the instructions on the database backup every time… I’m just so glad this script is done.

The basic functionality is now complete and I am calling this a RC 1 release. It still needs further testing (especially the MySQLDump stuff. Does everyone HAVE MySQLDump? Should I disable this feature by default?), but it is fairly stable now.

Here’s the basic functionality

#  You can use this program in several ways:  
#    * In the default mode to download the latest and greatest update,
#      make an uncompressed copy of your files, makes a compressed backup
#      of your database(s), distributes the file to any number of 
#      directories, and performs the web steps
#    * Configure it to make a online copy of your files you use for easy 
#      recovery AND a compressed copy that you can download.
#    * Add custom directories and backup MORE than just WP. 
#    * Configure it update from a local file each night and start with
#      a clean blog every morning.  
#    * Use it as a nightly backup script by disabling all other steps

I’ve made about a gazzilion improvements and took the advice of a dozen or more reviewers out there. I think the new script is much improved. I’m really pleased with how well the database backup stuff works. I change into each blog directory, read all of the connection information from the wp-config.php file and use that to connect to the database. (I’m fairly certain this will work well for most servers, but I’m a little worried that *nix gurus will not be using TCP to connect to their databases and I have that hard coded. You gurus should let me know if this is an issue in the comments for this post!) I also then query the tables names from the specified database using the prefix specified for the blog. This means that this process will work for ALL versions of WP and will not grab non-WP stuff like vBulletin tables. It also means that it works just as well if you have 1 blog per DB or ALL blogs in 1 DB. It doesn’t matter. The DB backup for Blog1 has ONLY the data for Blog1. That is better for security, size, time and opens a neat avenue for testers who want to restore their blog to a different database/databasename and then test a major upgrade running their full blog out of a different directory. I’ve structured the tarball backups to make this easy too. *SORRY* There I go into tech speak again, but it is neat stuff, that is normally totally hidden from view.

You can peruse the text version, EasyWPUpdate.txt, here: link

Like the new name? I think it is better. I put TCC in front of all of my plugins, but really there is no need here. And yes, the sample version has grown to 851 lines. That’s not ALL code of course. It is heavily documented and includes some HTML that will give you a nice webpage log for you to peruse after the process is done. You can see a sample log here: link.

Now, I had deliberately made that last post very intimidating. I wanted people to be wary of the script. Now, I have much more confidence in its ability and quality. I’ve learned a lot in the last week. From using procedures, to the trap function, to sed and MySQLDump, to basic shell coding practices. It was all fun and you get the benifit. Especially because there are three versions of WP in the pipeline: 2.0.7 (Very, Very Soon), 2.0.8 (in the works), 2.1 (Very Soon).

So, I’ve made this post easier to read and the script easier to configure. I’ll do a full document later, but here are the basic steps to install this script:
1. Use Telnet or Putty to connect to your website and log into the shell
2. Type the following line:
wget http://www.thecodecave.com/downloads/EasyWPUpdate
3. Type the following line:
chmod +x EasyWPUpdate
4. Use an editor to change the values in Step 1 and save it again.
5. Run the script by typing:
EasyWPUpdate

That’s it. You will have just made backups of the files and database and updated all of your blogs. When 2.0.8 comes out, the process will be:
1. Log in
2. Type
EasyWPUpdate

And you are done.

Now, step 1 looks like this:

# ##################################################################
# Step 1. Tell the script where to find the blogs
# ##################################################################
# List all of your WordPress directories and urls here.
#
# Each Blog should have a BlogDir and a BlugURL.
# Each Blog should have its own number [1],[2],[3] etc
# Delete the ones you don’t need.
#
BlogDir[1]=’site1dir’
BlogURL[1]=’www.example.com’

BlogDir[2]=’site2/news’
BlogURL[2]=’www.site2.com/news’

BlogDir[3]=’wordpress’
BlogURL[3]=’blog.site3.com’

That isn’t that hard to change is it? Even in VI.

Some quick tips on editing the script
1. type
vi EasyWPUpdate
2. Hit i
3. Make your changes
4. Hit ESCAPE COLON W to save your changes (or skip this step to lose changes)
5. Hit ESCAPE COLON !Q to immediately quit

Also, if your root directory is accessible from the web, you might want to change the name of the script
mv EasyWPUpdate SomeSneakyName
to prevent unauthorized access.

If you ran the alpha 3 version of the script, you can copy and paste configuration over BUT!!!! you have to make the following changes:
The BlogDirs[] array has been renamed to BlogDir[]. Drop the “s” from all of those variables.

You should not need to copy the Common*Prefix variables over, but if you do, make sure to remove the trailing slash from the CommonRootPrefix variable.

I think that’s all you need to be aware of.

If you are a guru, please read through all 6 setup steps (and the rest too) there may be things you want to change.

I’ve also updated my Did That Help page and added a forum specficly for this script.

That might make discussions a little bit easier.

Well that’s about it. Let me know how it works. I’d like to get some good testing in before 2.0.7 comes out. I’ll also do some testing with updating to 2.1 so I am certain that will work well. I also need write instructions for the database recovery steps. The script has built in instructions if it blows up in the middle of updating the files. So, that is handled.

I’ll leave you with the change history and credits section from the script. Enjoy!

# History:
#    01/AUG/2006 - BL - Created
#    21/DEC/2006 - BL - Added multiple blog arrays 
#                       Added options at the top of the script
#    04/JAN/2007 - BL - Added File Backup routines
#                       Added web update
#                       Added tmp directory usage
#                       Added local source ”freshen” option
#    11/JAN/2007 - BL - Added ”steps” and further comments
#                       Added quotes around many vars to protect against spaces
#                       Changed TMPDIR-/tmp to TMPDIR:-/tmp
#                       Changed `pwd` != ”$tmp” to `$pwd` != ”$tmp”
#                       Added further error trapping around cd and cp routines
#                       Fixed file backup procedure, was adding extra layers
#                       Added ability to backup to tarball
#                       Added SQL backup procedure
#                       Fixed local file backup procedure
#                       Removed “Verbose” from cp to make messages clearer
#                       Added log to webpage for Joe.
#                       Fixed inconsistent use of trailing / in path variables
#                       Added status messages throughout
#                       Added recovery instructions in case of failure mid backup
#                       Added a list of directories to backup
#                       Added Credits section
#
# Credits - I want to thank all of the readers of TheCodeCave.com, for
#   their testing of this script. I especially appreciated Michael, Maciek, 
#   Aaron and Joe for all of the helpful suggestions.  
#   A very special thanks goes out to goldfish on the FreeNode #sed channel                     
#   who will be PayPaled a Cafe Voltaire tomorrow.  I would have spent days
#   figuring out the RegEx for the SED commands.  Prec, also from #sed gave 
#   provided me with a working cr/lf stripper.  For bash, lhunath, jp-_ and the                     
#   whole crew at #bash on FreeNode gave great line by line suggestions.
#   They basically gave it a full code review!  None of this would have been 
#   possible without Advanced Bash-Scripting Guide. 20 days ago I didn’t know
#   what bash was.  Now I’ve written a powerful script with features I’ve not
#   seen anywhere else.  If you have any questions about the code in this 
#   script, you’ll find the answers here:&nbsphttp://www.tldp.org/LDP/abs/html/
#

The 35 Second upgrade processhas been replaced with the “5 Second Upgrade Script” (link). Follow the link for more information.

This post describes a process to upgrade your post to the latest version of WordPress in 35 seconds. Each additional site would add only 5 to 10 seconds, depending upon how fast you type. I’ve measured it myself. The first two runs were 40 seconds and the rest were 35. And just so you know, that is real time and includes manually logging in by typing my password, doing the work, logging out, clicking start run and typing the full URL to the upgrade.php file. If I bookmarked the site and stored my password. I bet I could upgrade my site in 20 seconds. Anyway, I thought you might find this technique useful, so I thought I would share.

The only automation I used is to setup a shell script that will go out and get the latest version of WordPress whenever I ask it to. And since that is a generic process that will work for all versions, I don’t count it as cheating to reach that 35 second mark. I really wanted to hit 30 seconds, but hey this is better than my old methot which involved 15-20 minutes of upload time to take the files from my Windows machine over to the my 1and1.com site.

A Short Warning

Some things you should be scared by:

• DO NOT USE THIS PROCESS IF YOU HAVE INSTALLED WORDPRESS INTO A DIRECTORY NAMED WORDPRESS
• Some plugins make/require changes to your php code. Older (WP1.5) plugins often did this. It is safer to disable the plugins before upgrading. Know your site. I don’t need to disable my plugins for this to work. There’s a chance you don’t either.
• This script uses a directory named wordpress for its work and deletes it when it is done. Anything in the directory off the root name wordpress will be deleted by this process. ~/wordpress is emptied and removed. Gone shall its contents be. No more. Erased and cannot be retrieved. And so shall they be deleted. *blip*
• This process is meant to update any number of directories hanging off your root directory. It will work if you run out of the root too. In fact if you have a typo in your cd commands, it will update the root and you will have a lot of files to cleanup.
• This process works to upgrade any site that is running WordPress 1.5 or newer.
• This process will NOT delete any obsolete files. So, if you use this process to upgrade to WordPress 2.1 when it is released, because it includes a major rearranging of the files, you will end up with some legacy files cluttering up your webspace. This is not horrible unless a security hole is known to exist in one of the existing files. You could, on your own, modify this script to delete existing WordPress files before the install.
• This script will replace all standard WordPress files with the latest and greatest. If you are using an older/custom plugin that requires you to actually edit files distributed through wordpress, you’ll have to re-make those changes.
• REMEMBER TO BACKUP YOUR SITE. If you are thinking about updating your site, you should have a ready backup handy incase frogs and locusts start swarming out of your CPU fan. If such things seem unlikely to occur, or your one of those that loves the smell of fried chips in the morning, you may opt to proceed without a backup.

Pre-Requisites

Other things you should know:

• This instructions are optimized for a 1and1 Linux host account. If you have a linux account from another provider, the instruction should be close enough for you to follow.
• I’m gonna assume you have an account on a *nix server.
• I’m gonna assume you have the access and a way to Telnet/SSH into your account. Basicaly, to the untrained eye, telnet/ssh allows you to access your account from a DOS prompt. (No hate mail please! I said to the untrained eye!)
• I’m gonna assume you have the vi editor on your server. vi is a text editor that you will propably rarely use and that is optimized to make you very quickly forget the basics of how to use it. After stumbling around in it this time, I think I came up with a quick and easy set of instructions that will work for this post.

How to get Telnet/SSH access

So your first task is to get putty and that’s this PuTTY:

Not this putty:

(Old Joke. I’m sorry, but it was a good one and heck, cut and paste is easier than looking up the urls and retyping all of this.)

Just download it from here: http://www.chiark.greenend.org.uk/~sgtatham/putty/ and install.

Now, configure PuTTY to log into your site. It is straight forward, if you have difficulties. Please take a break and read the help docs on it and decided if you really want to take on updating via scripts without further reading and studying. Manual updates take longer but they are safer.

Once you have putty installed, login using your user number and main password (u35555555 or something like that for a 1and1.com user)

Once you are logged you’ll be at a prompt that looks like this:
USERNAME:~>

You can now use unix commands to wander around in your account. Be careful, you CAN break stuff. *nix is a strange world if you are not familiar with it.

The Script

As mentioned before, this upgrade process requires you to create a script. A BASH script is like a DOS Batch file. It is a series of commands that are executed sequentially. You create the script once and can run it MANY times. In fact you should be able to use it for all future WP releases. However, if you are performing more than a maintenence upgrade (i.e. if you are switching from 1.5 to 2.04 or from 2.0.4 to 2.1) you might want to follow the manual instructions that come with WordPress as it will result in a cleaner install. It’s up to you. The point is, you can always use this script to update to the latest release over and over again.

The process the script follows is simple.
1. Remove the files and directories used by this process to ensure a clean run.
2. Grab the latest gz file from wordpress.org
3. Uncompress the gz file - this creates a tar file
4. Uncomress the tar file - this creates a wordpress directory tree
5. Change to the directory housing the active wp site
6. Copy all of the files and directories from wordpress over to the current directory.
7. Remove the wordpress directory
8. Remove the tar

In terms of BASH commands, the script file will contain something like this:

rm latest.tar.gz
rm latest.tar
wget http://wordpress.org/latest.tar.gz
gunzip latest.tar.gz
tar -xf latest.tar
cd ~/wpdir4site1
cp -R -v –remove-destination ~/wordpress/* .
rm ~/wordpress/ -R
rm latest.tar

“wpdir4site1″ represents the directory in which your site has WP installed. You’ll need to change that text to use the right name. It is the directory in which WordPress’s Index.PHP resides and WP-Admin hangs off of it. If you are unsure of what to put here, STOP. Script updates are not for you. It is better to take a couple minute to update your site than to put something wrong in here and have the script erase everything on your site.

The next line is the one that does all of the work. So, you need to copy the cd and the cp lines if you want to change this script to update more sites. Then your script would contain something like this:

rm latest.tar.gz
rm latest.tar
rm ~/wordpress/ -R
wget http://wordpress.org/latest.tar.gz
gunzip latest.tar.gz
tar -xf latest.tar
cd ~/wpdir4site1
cp -R -v –remove-destination ~/wordpress/* .
cd ~/wpdir4site2
cp -R -v –remove-destination ~/wordpress/* .
cd ~/wpdir4site3
cp -R -v –remove-destination ~/wordpress/* .
cd ~/wpdir4site4
cp -R -v –remove-destination ~/wordpress/* .
rm ~/wordpress/ -R
rm latest.tar

Create The Script

So, here’s where you use PuTTY to log into your account (see above). Once you are at the bash prompt, you will use the vi program to create your script in your root directory.

So just type in:
vi updatewp.sh

Then hit the letter i
That will put vi in insert mode.
Then just type in the script as I described in the previous section. If you want you can use Notepad to customize the script to fit your directory structure and then copy it into the Windows clipboard, and paste it into PuTTY by right clicking the window.
Once, you have the script file looking as you want it to appear or if you want to give up and start over, hit Escape. That will take you out of Insert mode.
If you are satisfied with what you see, hit colon (:), hit w and then hit enter. That will save your work. Then hit :q and press enter to quit.
If you just wish to quit without saving, hit colon (:) and then type q! and press enter.
In review here are all of the vi commands you need to know

• i = Insert mode
• escape = Exit Insert Mode
• :w = Write file
• :q = Quit softly
• :q! = Quit forcibly

Oh, and don’t paste the file in without updating the directory names, because if you run it like that, say when you are proof reading a blog post, you will update your root directory with the current version of WP and that means loads of files to clean up. Luckily you can identify them via their modified date. And no, I will not tell you how I know this!

Let’s get busy

Now you’ve done all of the hard work. You’re up to the 35 second challenge.

Each time you upgrade, do this process:
1. Telnet/SSH/PuTTY into your account.
2. Type “sh updatewp.sh” and then press enter
3. Close your Telnet/SSH/PuTTY program.
4. Visit your upgrade url and press the “Step 1″ link (I do this by hitting Start->Run and typing “http://www.MySiteUrl.com/wp-admin/upgrade.php”)

This process is also great for quickly restoring a corrupt/hacked site back to operation.

I can do all that in 35 seconds, how fast can you do it?