Tuesday, October 30, 2012

AppEngine : Migrating to High Replication Datastore and retaining old application id

Let me begin by saying that, if you have a memorable application id on appengine's *.appspot.com, but the application uses the now deprecated Master Slave Datastore and you would like to move to the new High Replication Datastore, but would like to retain the old application id, then you have no reason to worry. Because, the default migration tool, provided by Google, does exactly that. It migrates your datastore indexes and makes the old application id an alias for the new application id.

If that is enough to allay your fears, then move on to the Migration docs provided by Google. For a step by step, read on.

Old application id: appname

New application id with High Replication Datastore: appname-hrd

Creating a new application with High Replication Datastore:

This can be done in two ways.

1. Goto http://appengine.google.com and click on create aplication
Click on Create Application

Fill in the new name, in this case, appname-hrd. Fill in a title, select other options.
Give a new name to the application

Make sure you select High Replication Datastore as the datastore.
Select High Replication and click on Create Application

Then click on Create Application.


2. Go to http://appengine.google.com and click on the name of your old application, in this case appname.
Click on application name

Go to Application Settings
Click on Application Settings

Under Duplicate Application Settings, type in the new name, i.e appname-hrd and see if it is available.
Type in a new application name and click on Duplicate Application

If available, click Duplicate Application. This will create an application named appname-hrd with the same title as your old application but with a High replication Datastore.

 If you followed any of the above two methods, you will now see the new application name on the http://appengine.google.com page.
Your new application has been created

Pre-requisites before migrating.

You are trying to migrate your app from the old Master-Slave Datastore to the new High Replication Datastore. You have created a new empty application. For the migration to be successfull, you should upload your application code from the old application to the new application id, i.e from appname to  appname-hrd

If you do not have the application code on your computer, you can download it from appengine as follows.

Open command prompt(Windows) or terminal(Linux)
cd to the appropriate directory(location of google_appengine_sdk) and run the following command

appcfg.py -A appname download_app /path/to/dir

appname is the name of the old application from where you are downloading the code. /path/to/dir/ is the path to the folder where you want the code to be saved on your computer.

If you have the application code on your computer, uploading the code to the new application id is as follows.

(Before proceeding, backup your application code)

Open the app.yaml file in your code folder and edit the application name to the new name


application: appname-hrd



Save the changes, and then upload the code using the following

appcfg.py update /path/to/dir

Where /path/to/dir is the path where your new code is saved.

On completion, your new application-id should now be serving the same app that exists on your old-appication id, although without the datastore. To migrate the datastore, read on.

Migrate old application datastore to the new application.

Go to http://appengine.google.com. Click on the old application-id i.e appname.

Then click on Application Settings
Click on Application Settings

Under Migrate Application, click on View Migration Tool.
Click on View Migration Tool

On the HRD Migration Page, select the new application-id from the drop down menu as the Destination Application
Select appname-hrd(new application id) as the Destination Application

And click on Start Migration
Starting Migration

The Migration should start and you should see the process status. 

Possible errors:

Error: "Please deploy your application code and index configuration on your target application before beginning the migration."

Reason: You haven't uploaded your code to the new application-id. If you did, check the application name in the app.yaml file

Error: Warning: The application you selected to migrate to already contains data.
You can delete this data using the Datastore Admin tool. Furthermore, you can proceed anyway by checking the box below.
Reason: Your default app may have created a few entries in the datastore. As long as there is no user generated data that you need to backup, you need not worry. Select Confirm migration to application with existing data and click Start Migration.

There are some steps in the migration, in which you will have to intervene.

Activate Read Only

Activate read only.
You can select Activate Read Only directly or select Launch Incremental Copy first and then Activate Read Only.
According to Google, the Launch Incremental Copy option is beneficial if you have a large amount of data and doing so reduces the amount of time the app has to be in a read only mode.
If you want to cut short your read only time, click Launch Incremental Copy.

After Launch Incremental Copy, you will see Sync as Running.

When sync is complete, you can click on Activate Read-only.

After Catch up and Sync is complete, you will be prompted for alias assignment
Click on Finish Migration

Click on Finish Migration to assign the alias and complete your migration.

Now your old application will automatically point to your new application-id, without doing a 301 Redirect.


  1. Great post and walkthrough- thanks!

    since the old application hangs around as an 'alias', it looks like a migration of this sort ties up two applications out of the allotted ten though... your thoughts?

  2. Welcome!

    Late late reply, but, I have no idea. Google could keep your old app only for redirection purposes(if you send them a request) and increase your quota by that much.

  3. Really good tutorial. I really appreciate your effort making it.