Migration of Standalone Application and Database from VM (Google Compute Engine) to a modern architecture, using Google Kubernetes Engine (APP) + Cloud SQL (DB) with HA
In another amazing project based on a real-world scenario, I acted as a Cloud Specialist to migrate the Compute Engine (VM) application and database to the Australia region.
Once the region migration was completed, the task was to modernize the application. I had to migrate resources running on Google Compute Engine (VMs) to a modern and containerized architecture using Google Kubernetes Engine (GKE) managed services, and Cloud SQL for the database layer, including high availability (HA).
Overview of Implementation: This hands-on endeavor prioritizes the modernization of a legacy application through the utilization of GCP PaaS services, specifically Google Kubernetes Engine (GKE) and Google Cloud SQL, both renowned for their high availability. Below delineates the steps to be undertaken:
Step 1. Create a Cloud SQL instance and configuration.
Step 2. Create image of application, pushing the image in Google container registry and then using it on GKE.
Step 3. Create a GKE cluster and deploying application.
Step 4. Migrate the Database to Cloud SQL instance.
Step 5. Testing Failover DB
Prerequisite: This project is the continuation of legacy application and database server to Australia region. The prerequisite is to have application and database servers running in Australia region.
Step 1. Cloud SQL creation and configuration.
A- Creating Cloud SQL instance key configurations.
- Creating Cloud SQL instance basic configuration (Australia region)
- Enable HA: Multiple Zones. Multi zone is required for HA.
- Primary zone: australia-southest1-a
- Secondary zone: australia-southest1-b
Connectivity: SQL instance requires private connection with Application server. It does not require public IP.
- Uncheck Public IP, and Select Private IP.
Click on Enable API to enable API Service Networking.
Allocate an IP range -> Select Use an automatically allocated IP range. (In production, ensure to select your VPC and specific IP range).
Click on Create Instance.
B- Creating Database.
Click on Databases. Then, click on Create “clinic” Database. We will need this database to migrate the data from ausdb-01 instance.
C- Creating a user in the Cloud SQL instance. (this user is used to login the SQL instance).
Click on Users, Add user account:
- Username: app
- Password: welcome1
- Host name | Allow any host
Step 2. Containerizing (docker) of application source code.
(For this project, I already prepared the Dockerfile. However, if you don’t have one, you can write your docker file like below. Dockerfile helps create the image of your application using Docker.)
# Dockerfile
FROM node:14
WORKDIR /usr/src/app
COPY package*.json ./
RUN npm install
COPY . .
EXPOSE 3000
CMD [ “node”, “index.js” ]Creating a new image:
the middlewares in the application file index.js carry environmental variables pointing to the parameters. these environmental variables are set apart from the source code to so that the container can use it dynamically. (in legacy application, these parameters are hardcoded that is not a best practice.)
In the contollers files, we can set a user friendly message to pop when application is not accessible (during failover). res.json(‘Whoops, failed to load … Try again in 2 minutes.’)
Use below commands to build image, tag the image and push it to gcp container registry.
docker build -t tcb-clinic-app .
docker tag tcb-clinic-app asia.gcr.io/$DEVSHELL_PROJECT_ID/tcb-clinic-app:latest
gcloud docker -- push asia.gcr.io/$DEVSHELL_PROJECT_ID/tcb-clinic-app:latest
Step 3. Create GKE cluster and deploy the application.
Create GKE Autopilot cluster with below configurations:
- Name: autopilot-cluster-1
- Region: australia-southeast1
In Cloud Shell, download deployment file tcb-clinic.yaml
Edit the file tcb-clinic.yaml using the Cloud Editor, change the Private IP (DBHOST) of the Cloud SQL:
When cluster is up and running, click on connect.
Connecting to the Cluster (Console -> GKE -> the Cluster created -> Connect)
Deploying the application using deployment yaml file.
kubectl apply -f tcb-clinic.yaml
Step 4. Migrate the Database to Cloud SQL instance.
Accessing the Compute Engine service and turn off the GCE aus-app01
Exporting the DB of the MySQL GCE
Access the aus-db01 VM via ssh
- Export the data to clinic.sql
mysqldump --add-drop-table -u root -p clinic > clinic.sql2. Importing data into the created DB (Cloud SQL)
3. Connecting to Cloud SQL DB and checking the imported data:
use clinic;
select * from patient;
Testing the access to the application with the External IP of the Load Balancing.
Step 5. Testing Failover DB
1. Access the Cloud SQL
2. Click on Failover
3. Then, click on Trigger Failover.
During failover, Zones will be changed.
In browser, the message Whoops, failed to load…Try again in 2 minutes will appear.
Refresh until the connection is re-established after failover.
Conclusion and Key Insights:
Throughout this hands-on experience, we embarked on a journey starting with the creation of an application and database hosted in the US on GCP GCEs. Subsequently, we executed a seamless migration of these VMs to the Australia region utilizing the disk snapshot method, aligning with client specifications. Responding to evolving requirements, we then transitioned the application and database to GCP PaaS services, leveraging GKE autopilot for application hosting and Cloud SQL for database management. Notably, the client’s mandate for high availability of databases was diligently addressed. This hands-on exercise mirrors real-world scenarios encountered in the migration of a node.js application across different geographical regions. The insights gained from this practical endeavor are invaluable, equipping me with the expertise to architect similar projects effectively.
References:
