Part:10 Failover Test Operation Procedures in Zerto
In previous post I discussed basic DR procedures. In this post allow me to discuss Failover Test Operation (sand box testing). Primary reason to do this test is to ensure VMs are properly replicated, they can start in particularly defined order without any resource constraints at DR site especially compute. No end user testing is allowed. It can be done but best be avoid. Choose the reason for sandbox testing and proceed accordingly.
Failover Test is completely non-disruptive as this test doesn’t touch any production work load. It creates VMDK based on scratch volumes (using Journal History) in thin format. Using scratch volumes eases the clean up process. In this test VPGs continue to replicate, create new checkpoints (automatically or manually).
Failover Test Operation has following basic steps
It is recommended to test all VPGs of a cluster, this will ensure all VMs belonging to the cluster are successfully recovered. It also assure that there is no resource constraint and there is sufficient capacity to restart VMs.
During my experience I have seen VMs provisioning request continue to grow. VMs which need DR service also continue to grow, IT manager remains ignored to see if the DR site has sufficient capacity to power on the VMs. DR tests of all VPG in a cluster will give you clear picture and heads up as to when you will need more capacity addition at DR site.
Actual Procedure to start DR Test
Failover button you see at the bottom right hand corner is common for both Test DR or Live DR. You need to toggle button as shown below
The moment you toggle it Live, it turns Failover to red color (not shown in this figure).
In SRM you get pop-up warning. I would expect similar mechanism to be adopted by Zerto. It helps a lot.
ok. Now press toggle switch for Test and Press Failover Button. In next screen you should select all the VPG you wish to test. Press Next >
In above screen you have major decision point. You must select checkpoint. I have separate blog post on Checkpoint here. Please go through the posts for detailed explanation. Let me explain checkpoint in short for you- you get to select as to which point you wish to restore VM at DR site to test. You get to choose both automatically created checkpoint and manually created checkpoint.
After checkpoint is selected press Next, select Failover button and DR Test starts.
Highlighted in yellow is DR Test progress status
Above figure provides task list which gets executed at vCenter of DR site.
In above figure you can’t see VMDK at all. It explains VM is created out of scratch volume which is not visible in the original place.
Please do note that VMs names are appended with “testing recovery”
Since we are in DR test, the only option left is to do Live Failover. You can do a move operation as well. It is worth noting you can do Failover even if DR test is going on. Unsure if any such scenario is really going to happen. But it is DR product it address all unanticipated challenges and protect Business reputation.
Roll back Procedure
Roll back procedure will simply delete these test VM. Procedure is simple in the same screen i.e. VPG tab we can use stop button “Stop Failover Test” to stop DR test. Select the VPG and press “Stop Selected”
At DR site you can see list of task executed in below figure after rolling back DR test
When DR Test roll back is completed Checkpoint by the name “StartDateAndTimeOfTest (Using checkpoint) Below similar checkpoint is created. We tested on 1st Jan, 2014 using last checkpoint which was taken on Checkpoint of 31st Dec, 10:42 PM
Most important part of DR product is about reporting. Be it actual DR or DR test. In order to get reports, go to reports tab > from drop down select Recovery reports
Select the date, VPG and other options and click generate PDF Report or PDF Reports (Zipped). I’m pleased with report and its description. Well format definitely helps to prove both IT manager and Business manager. I have uploaded sample recovery report here as it was bit difficult to share it via blog post.
If you wish to follow entire series of Zerto go to the Landing Page