Load reduction

#4 (go back to main post)

To minimise the server loads and avoid crashing, the whole replace in the database will be made one table at a time that too with php sleep function and in batches of few hundered entries.
I am looking for the phpclasses code which keeps track of the processes running at the server and cpu usage and if I find that it can be used without troubling the end user, I will use that to keep a check on the processes. Also the checklist model of file transfer would be great as it will help in allowing breaks wherever wanted and also not keep running and cause the server to get slow.
Another very important aspect is error handling. As this process involves a lot of data manipulation and handling, it needs to have all ways of handling various unexpected situations. During the course of the project, I will aim at always having a fallback plan-B for everything that can go wrong.

2010
04/08
CATEGORY
Ideas
WPmove
TAGS





Write comment

File move

#3 (go back to main post)

A checklist of files on Source Server(say Server A) will be made first.
XML-RPC can be used to maintain the list between the 2 installations
As the files get transferred, they can be striked off from the list.
FTP is a great option but there may be cases where the user permissions arent available and also where the user himself isnt sure about the FTP credentials. CURL and SST can be a good options too here…
After a little bit of research (I am making a plugin right now for a comics site to be able to allow download and embed for their daily comics update image episodes), I figured out that the existing HTTP_API can be of great use here to either download each file from a list generated at the Source Server ( Server A) to the Destination Server (Server B).
Possible issues here can be the directory permissions of teh destination server causing the copy impossible. To check if a file was copied correctly or not, we can use the md5 checksum method used by search engines for pages when they revisit them.( we will have a md5 sum on Server A before sending data and one on Server B after the data is recieved, if they match, the file can be striked off a checklist and regarded as downloaded) As this is being done from a list, this can be put up as a cron job or like a pauseable task through the browser of the user in cases of huge number of files.

2010
04/08
CATEGORY
Ideas
WPmove
TAGS





Write comment

Handle blog occurences

#2 (go back to main post)

There may be a lot of cases when the domain replace in the blog posts may result in an error. For example:- Say I run a blog and I want all my subscribers and visitors to know that the domain will be “changed from abc.tld to xyz.tld” and I write a post about it. After our simple replace will cause the post to show “changed from xyz.tld to xyz.tld” which is undesirable. Also people will want to change texts like “At ABC we believe in serving our customers” to “At XYZ we believe in serving our customers”(Note that the names ABC and XYZ arent domain names as they dont have the .tld extensions ) but not in cases where they write that “ABC was established in the year 2000″. We can skip the ABC name to XYZ name because thats just text to wordpress and we arent changing it so we arent answerable to the customer(though it wil be a good bonus if we made that replace for them). But in cases where wordpress had inserted a link or where a change is occuring because of the domain replace concept, there we do need the user’s permission atleast.
One potential solution that came up during chats with mentors is to change the urls of files directly to the new ones as there wont be any case where the user will need the old links unless he is using the old server and domain to host his files(we can have him decide that during the move) and to log the changes for the user to review and revert or may be better log the changes first and let him have a checkbox type interface where he can select where and where not to make the changes… There is just one small issue here that all the places where potential changes may occur are to be made must be tagged first so that we know the difference between “changed from abc.tld to xyz.tld” and “please visit us back at abc.tld for more info” from within the same post…

2010
04/08
CATEGORY
Ideas
WPmove
TAGS





Write comment

Backup of db

#1 (go back to main post)

The main aim here is to search and replace the old domain name with the new one…

It seems that the simple Like %example.com% search and a replace but its lot more than that. One case as mentioned in the main idea text is the presence of serialized array in the tables. For that we can have a simple solution of
looking for serialized data first in the tables > unserialize them > making the replace in the resulting array> serialize  > update
But this can be done easily only on tables… In sql files the solution will become a lot more complex as just finding serialized data will need matching. So here we can make another set of backup tables from the code and start the above process on them,  say of the format [dbprefix]_wpmove_[originaltablename] but that will mean a live running site where data can be changed and it will vary from the backup table we have

So final idea sums up to
take backup> post a site down for maintainance message> look-up all occurrences of the domain in the db(except the posts table, will come to that in next point)> check if the data is serialized array>if yes follow the code section mentioned above>replace n update.

One very cool reply I got for the [wpurl] mail on wp-hackers was of Otto

function maybe_unserialize( $original ) {
if ( is_serialized( $original ) ) // don’t attempt to unserialize
{
$fixed = preg_replace_callback(
‘!(?<=^|;)s:(\d+)(?=:”(.*?)”;( ?:}|a:|s:|b:|i:|o:|N;))!s’,
‘serialize_fix_callback’,
$original );
return @unserialize( $fixed );
}
return $original;
}

this creates an array output even if we have a  search and replace on the domain directly and the new domain has different characters.

try changing the count before the element data in serialized text and pass it thru this function

2010
04/07
CATEGORY
Ideas
WPmove
TAGS





Write comment

WPmove

The Problem

Move WordPress

Currently, if you want to move your WordPress install to a new host and keep the same domain, you only need to move all the files (WordPress Core, Themes, and Plugins), and then export/import the database. It’s pretty straight forward, but still more complex than many users are capable of.

However, if you want to change the domain (even without moving to a new host) you have to export the database, do some search and replace on the .sql file, and then re-import the updated database. Even then, if the new domain has a different number of characters in it than the old one did, you can run against problems if any of the places you replaced the name happened to be inside a serialized array (almost all the options, including things like widget settings, text widget text, etc).

The goal of this project would be to make a these kinds of transitions simple and smooth. If only the domain is changing, then when the user updates the WordPress URL we would update internal links in their posts, settings, image urls, etc. If they are moving to a new host, allowing them to install WordPress at their new host and enter their credentials for their old host (WordPress admin credentials as well as FTP credentials) and WordPress would simply import everything (posts, settings, uploads, theme, plugins, etc).

The Proposal

I talked to John James Jacoby ( jjj ) and John Godley (john__g)  on IRC about this and came up with the following tentative points..

I will keep posting about each of them as a blog post and may be change the list too as I go deeper into the matter

  1. Backup of db n a way to search n replace the domain name in the backed up sql file or replicated [dbprefix]_wpmove_[table] table.
  2. Handle where or where not to make the replace within blog posts
  3. Set-up ftp move / zip-unzip package move
  4. Keep a check on the server load and error-handling
  5. Make progress bars and other UI elements to help the novice through the process
  6. Roundup n check for single user blogs if all the above points are met n make all bug fixes with #1 to #5
  7. Introduce multi user single server support (Extra point for now as I already have found a lot of issues in the first 5 points)
  8. Profiling the server types and best possible way out of all the functions and options we have
  9. [wpurl] tag replace

I will surely keep blogging about other issues and future ideas I get on this project for the community as a resource and for next years or may be this yrs GSOC participants (If I dont get selected)

Read more

2010
04/06
CATEGORY
Ideas
WPmove
TAGS



Write comment