AppEngine : Migrating to High Replication Datastore and retaining old application id
Let me begin by saying that, if you have a memorable application id on appengine's *.appspot.com, but the application uses the now deprecated Master Slave Datastore and you would like to move to the new High Replication Datastore, but would like to retain the old application id, then you have no reason to worry. Because, the default migration tool, provided by Google, does exactly that. It migrates your datastore indexes and makes the old application id an alias for the new application id.
If that is enough to allay your fears, then move on to the Migration docs provided by Google. For a step by step, read on.
Old application id: appname
New application id with High Replication Datastore: appname-hrd
Creating a new application with High Replication Datastore:
This can be done in two ways.
1. Goto http://appengine.google.com and click on create aplication
Fill in the new name, in this case, appname-hrd. Fill in a title, select other options.
Make sure you select High Replication Datastore as the datastore.
Then click on Create Application.
OR
2. Go to http://appengine.google.com and click on the name of your old application, in this case appname.
Go to Application Settings
Under Duplicate Application Settings, type in the new name, i.e appname-hrd and see if it is available.
If available, click Duplicate Application. This will create an application named appname-hrd with the same title as your old application but with a High replication Datastore.
If you followed any of the above two methods, you will now see the new application name on the http://appengine.google.com page.
Pre-requisites before migrating.
You are trying to migrate your app from the old Master-Slave Datastore to the new High Replication Datastore. You have created a new empty application. For the migration to be successfull, you should upload your application code from the old application to the new application id, i.e from appname to appname-hrd
If you do not have the application code on your computer, you can download it from appengine as follows.
Open command prompt(Windows) or terminal(Linux)
cd to the appropriate directory(location of google_appengine_sdk) and run the following command
appname is the name of the old application from where you are downloading the code. /path/to/dir/ is the path to the folder where you want the code to be saved on your computer.
If you have the application code on your computer, uploading the code to the new application id is as follows.
(Before proceeding, backup your application code)
Open the app.yaml file in your code folder and edit the application name to the new name
Save the changes, and then upload the code using the following
Where /path/to/dir is the path where your new code is saved.
On completion, your new application-id should now be serving the same app that exists on your old-appication id, although without the datastore. To migrate the datastore, read on.
Migrate old application datastore to the new application.
Go to http://appengine.google.com. Click on the old application-id i.e appname.
Then click on Application Settings
Under Migrate Application, click on View Migration Tool.
On the HRD Migration Page, select the new application-id from the drop down menu as the Destination Application
And click on Start Migration
The Migration should start and you should see the process status.
Possible errors:
Error: "Please deploy your application code and index configuration on your target application before beginning the migration."
Reason: You haven't uploaded your code to the new application-id. If you did, check the application name in the app.yaml file
Error: Warning: The application you selected to migrate to already contains data.
You can delete this data using the Datastore Admin tool. Furthermore, you can proceed anyway by checking the box below.
Reason: Your default app may have created a few entries in the datastore. As long as there is no user generated data that you need to backup, you need not worry. Select Confirm migration to application with existing data and click Start Migration.
There are some steps in the migration, in which you will have to intervene.
Activate read only.
You can select Activate Read Only directly or select Launch Incremental Copy first and then Activate Read Only.
According to Google, the Launch Incremental Copy option is beneficial if you have a large amount of data and doing so reduces the amount of time the app has to be in a read only mode.
If you want to cut short your read only time, click Launch Incremental Copy.
After Launch Incremental Copy, you will see Sync as Running.
When sync is complete, you can click on Activate Read-only.
After Catch up and Sync is complete, you will be prompted for alias assignment
Click on Finish Migration to assign the alias and complete your migration.
Now your old application will automatically point to your new application-id, without doing a 301 Redirect.
If that is enough to allay your fears, then move on to the Migration docs provided by Google. For a step by step, read on.
Old application id: appname
New application id with High Replication Datastore: appname-hrd
Creating a new application with High Replication Datastore:
This can be done in two ways.
1. Goto http://appengine.google.com and click on create aplication
![]() |
Click on Create Application |
![]() |
Give a new name to the application |
Make sure you select High Replication Datastore as the datastore.
![]() |
Select High Replication and click on Create Application |
Then click on Create Application.
OR
2. Go to http://appengine.google.com and click on the name of your old application, in this case appname.
![]() |
Click on application name |
Go to Application Settings
![]() |
Click on Application Settings |
Under Duplicate Application Settings, type in the new name, i.e appname-hrd and see if it is available.
![]() |
Type in a new application name and click on Duplicate Application |
If available, click Duplicate Application. This will create an application named appname-hrd with the same title as your old application but with a High replication Datastore.
If you followed any of the above two methods, you will now see the new application name on the http://appengine.google.com page.
![]() |
Your new application has been created |
Pre-requisites before migrating.
You are trying to migrate your app from the old Master-Slave Datastore to the new High Replication Datastore. You have created a new empty application. For the migration to be successfull, you should upload your application code from the old application to the new application id, i.e from appname to appname-hrd
If you do not have the application code on your computer, you can download it from appengine as follows.
Open command prompt(Windows) or terminal(Linux)
cd to the appropriate directory(location of google_appengine_sdk) and run the following command
appcfg.py -A appname download_app /path/to/dir
appname is the name of the old application from where you are downloading the code. /path/to/dir/ is the path to the folder where you want the code to be saved on your computer.
If you have the application code on your computer, uploading the code to the new application id is as follows.
(Before proceeding, backup your application code)
Open the app.yaml file in your code folder and edit the application name to the new name
app.yaml application: appname-hrd ... ...
Save the changes, and then upload the code using the following
appcfg.py update /path/to/dir
Where /path/to/dir is the path where your new code is saved.
On completion, your new application-id should now be serving the same app that exists on your old-appication id, although without the datastore. To migrate the datastore, read on.
Migrate old application datastore to the new application.
Go to http://appengine.google.com. Click on the old application-id i.e appname.
Then click on Application Settings
![]() |
Click on Application Settings |
Under Migrate Application, click on View Migration Tool.
![]() |
Click on View Migration Tool |
On the HRD Migration Page, select the new application-id from the drop down menu as the Destination Application
![]() |
Select appname-hrd(new application id) as the Destination Application |
And click on Start Migration
![]() |
Starting Migration |
The Migration should start and you should see the process status.
Possible errors:
Error: "Please deploy your application code and index configuration on your target application before beginning the migration."
Reason: You haven't uploaded your code to the new application-id. If you did, check the application name in the app.yaml file
Error: Warning: The application you selected to migrate to already contains data.
You can delete this data using the Datastore Admin tool. Furthermore, you can proceed anyway by checking the box below.
Reason: Your default app may have created a few entries in the datastore. As long as there is no user generated data that you need to backup, you need not worry. Select Confirm migration to application with existing data and click Start Migration.
There are some steps in the migration, in which you will have to intervene.
![]() |
Activate Read Only |
Activate read only.
You can select Activate Read Only directly or select Launch Incremental Copy first and then Activate Read Only.
According to Google, the Launch Incremental Copy option is beneficial if you have a large amount of data and doing so reduces the amount of time the app has to be in a read only mode.
If you want to cut short your read only time, click Launch Incremental Copy.
After Launch Incremental Copy, you will see Sync as Running.
When sync is complete, you can click on Activate Read-only.
After Catch up and Sync is complete, you will be prompted for alias assignment
![]() |
Click on Finish Migration |
Click on Finish Migration to assign the alias and complete your migration.
Now your old application will automatically point to your new application-id, without doing a 301 Redirect.
Great post and walkthrough- thanks!
ReplyDeletesince the old application hangs around as an 'alias', it looks like a migration of this sort ties up two applications out of the allotted ten though... your thoughts?
Welcome!
ReplyDeleteLate late reply, but, I have no idea. Google could keep your old app only for redirection purposes(if you send them a request) and increase your quota by that much.
Really good tutorial. I really appreciate your effort making it.
ReplyDelete