Sunday, September 25, 2016

Net702 - Creating a Distributed, Highly Available Web System in the Cloud

DinoStore

Here I am creating a highly available web system called DinoStore, working in Amazon Web Services (AWS). Initially, I have created the web application (DinoStore) on my local computer with the help of Visual Studio and MySql database. Then I have transferred it to AWS and made it highly available. This exercise is part of my study program NET-702. Through this  exercise, I am mainly focusing on covering the main features of AWS and familiarizing cloud environment.   

Below are the links to the steps to the DinoStore creation.


Lab:1 http://ramvijo.blogspot.com/2016/09/net702-lab-1-provisioning-durable.html

Lab:2 http://ramvijo.blogspot.com/2016/09/net-702-lab-2-using-rds-with-aspnet.html

Lab:3 http://ramvijo.blogspot.com/2016/09/net702-lab-3-using-dynamodb-as-session.html

Lab:4 http://ramvijo.blogspot.com/2016/09/net702-lab-4-configuring-system-to-use.html

Lab:5 http://ramvijo.blogspot.com/2016/09/net702-lab-5-adding-ec2-virtual.html


Lab:6 http://ramvijo.blogspot.com/2016/09/net702-lab-6-creating-and-using-amis.html

Lab:7 http://ramvijo.blogspot.com/2016/09/net702-lab-7-using-elastic-ips.html

Lab:8 http://ramvijo.blogspot.com/2016/09/net702-lab-8-using-elb-to-scale.html


Lab:9 http://ramvijo.blogspot.com/2016/09/net702-lab-9-enabling-auto-scale-to.html

Lab:10 http://ramvijo.blogspot.com/2016/09/net702-lab-10-configuring-dns-with.html



Billing and usage details 

Currently, I have used 23$ from my 100$ credit by the end of the project, but AWS charges monthly for the services. I have checked the current status of the usage and found out that, I have consumed 44.77$. This is used by RDS, Elastic compute cloud and Rout 53 services. Among them, RDS costs are really high because my region is Sidney and its region charges are high. 





Net 702 - Lab 2: Using RDS with ASP.NET Applications

Using RDS with ASP.NET Applications 

We have already shifted the Images and other files of DinoStore to the Cloud. Now in this lab, we are going to shift the database from local MySQL to the AWS RDS.

Amazon Relational Database Service (RDS)


RDS is the relational database service from Amazone. Amazon RDS provides you six familiar database engines to choose from, including Amazon Aurora, Oracle, Microsoft SQL Server, PostgreSQL, MySQL, and MariaDB.
I have launched an RDS instance in my AWS account. 

We have to select the region we need as per our requirement, I have selected Sydney as my region because it is the nearest one for me. I am using MySQL on my local computer so that I have selected MySQL as my RDS Engine.


Instance class I have selected a microDB storage I selected SSD, 5GB. Then I have given a username and password for the instance. I have given the database name as dinostoredb and enabled automatic backups by selecting a period of days to 0. 


In the security group option select to create a new security group and in the inbound rules I have added my IP to be allowed. I have copied the endpoint connection string from the dinostoredb instance detail.


In my local SQL, I have made a new connection and given a connection mane than pasted the copied
endpoint string in the hostname. Then given the username and password which was used for the RDS instance and saved it. I have tested the connection and it was successful then I clicked ok to connect to the RDS.
 

I have connected to the DB in the cloud and used a script to create tables in the cloud database.


Then I have log back tho the local database and exported the product list from there as CSV format and imported it into the cloud database.

I can view the imported data in my cloud table.


Next, I am going to create a read replica of the RDS Instance in a different availability zone. This will act as a read-only copy of the current database,  this will give our database a better redundancy and load balancing option. Here I am creating a replica and redirecting the website to use the images from there to make a load balancing.

I have created a replica by selecting the dinostoredb and right click to get an action menu to select the replica option. Then I have followed the same steps as for the previous database, but I have selected a different availability zone for the new database.


Now we can find two databases in the RDS.


I have made a new connection for the replica database as well.


I have received the same copy of the main database here in the replica.


Then I have redirected Visual Studio program to use new databases by editing the details in Web config file.


Then I have redirected the storage link to replica and for all other activities use the main database, in the Visual Studio Default.aspx  file.



Then I have built the project and tested the website. Initially, I received an error then I have foud out an error in the database details in web config. I changed the database details then built again and this time, it was successful.




Now we have to test the settings, currently there are no  tables in the database. Here I am creating a new user on the website.


Now I am logged in as the new user. Let us check the database now.


Here in the users table, I can see the newly created user.

Now we can check the replica by restarting the main database.

I have rebooted the database while rebooting I have refreshed the website and found that the website was available all the time. 

Net702: Lab 4: Configuring the System to use Simple Queue Service

Configuring the System to use Simple Queue Service

In this lab, we are creating a queue for the orders to control the traffic of orders. The orders will first move to the queue then it will be processed to the database. 

Amazon Simple Queue Service (SQS)

SQS is a fast, reliable, salable, fully managed message queuing service provided by AWS. SQS can manage to transmit any amount of data without data loss. We are using SQS to queue the orders in DinoStroe project. 

I have created a new queue in the SQS console and named it as dinoorders.
.



Then I have tested the queue by sending a message in ‘Queue Actions’



It has successfully come to the queue.

  I started polling and I could see my message. Now I have confirmed that my SQS is properly working, then I have deleted the test message.



Now I have to configure the Visual Studio program to move the orders to the queue. I have placed some coding in Checkout.aspx.cs to point it to the queue.



Then right click on the DinoStore – References object, and Manage NuGet Packages. Search for and install the Json.NET package. This allows the shopping cart to be serialised as a JSON object and added to the queue.


Next, I had to add credentials and location of the SQS in web config file. I have added following comments with the region, access key, and secret key.

<add key="AWSRegion" value="YOUR REGION HERE"/>
<add key="AWSAccessKey" value="YOUR ACCESS KEY HERE"/>

<add key="AWSSecretKey" value="YOUR SECRET KEY HERE"/>



Now the time to check the order queue, I have built the project and logged in to the DinoStore with the user and made a purchase.


Then I have checked the SQS queue and found the purchase in the queue.


In the next step, we need to pull the order from SQS to RDS.We now need to add a new application to the project that can pull the orders from the queue. I have already prepared the application called 'NET702.DinoStore.OrderProcessor'. I have added the order processor by right-clicking on the Solution Dinostore\Add\Add Existing project. Then inserted the dinoorders queue URL into the NET702.DinoStore.OrderProcessor ‘Program.cs’ code.



Then edited the app.config file by giving the access keys and region details. Changed the StoreSqlDb server connection string to the dinostoreinstance URL, and add DB username and password to this line.


Then rRight click on NET702.DinoStore.OrderProcessor project and select ‘Set as Startup Project’ this will make the program to run from this application.  


Right click the new project, choose Properties-Signing and untick the ‘Sign the ClickOnce manifests’



Then I built the project to fetch the order from the queue but I received several errors. Then I have updated NuGets in the order processor references.As well as, I have fixed some errors in the coding and then it was successfully built.



Now it has received and deleted a message from the queue and it started polling the queue.


To verify, I have checked the SQS and found the queue has been cleared.


Then I have checked the database and found the order has been created in the orders table.





Net702 - Lab 3: Using DynamoDB as a Session State Provider

Using DynamoDB as a Session State Provider 

Amazon DynamoDB is a fully managed non-relational database service, here we are using it to pre-create a session table for DinoStore. Here we will be storing all the session data of the web activities (Big Data) which we can use later on. 

I have created a DinamoDB table (ASP.NET_SessionState) with the hash key named SessionId.




After creating the table we need access key and private key from AWS. It is in the AWScaccount tab security creational option. I have downloaded it and saved it on my local computer, these keys are necessary to access the AWS while connecting through some programs.



 Now we have to prepare Visual Studio application by installing some NuGets. I have installed AWSSDK NuGet.


Then I have edited web config for the sessionstate.


I have given the details of the DynamoDB table with the Access key and Secret Key. I have made necessary changes in Shopingcart.cs also.


 Then I was getting an error when I tried to build the project, then I have installed the AWS sessonprovider and Extensions NuGets. Then I have found out that I was using a different region in the coding, I have changed it also.



I was successfully built the project then login to the website and made an item into the shopping cart.


Then went back to the DynamoDB and found that the item has came to the table.