At VELO, keeping our services reliable is not only about technology — it is also about preparation. Earlier this year, our team conducted a Business Continuity Plan (BCP) drill to make sure we are ready to respond quickly and effectively during unexpected situations.
The drill was designed to simulate real incidents that could potentially affect data center operations. Instead of treating it like a formal test, the entire activity became a valuable learning experience for everyone involved.
Simulating a Power Failure
The first scenario focused on a complete power outage in our data center environment. The goal was simple: ensure that all critical systems could continue running without interruption even if the main power source failed.
During the simulation, the team intentionally disconnected the main power source to observe how the backup systems responded. Our UPS systems immediately switched to battery mode while the backup generator prepared to take over the load. Within seconds, the generator became active and restored stable power to the data center.
The engineering and monitoring teams carefully watched every transition — from PLN power to battery mode, from battery mode to generator power, and finally back to normal conditions. Throughout the process, all monitoring dashboards and customer services continued operating normally without noticeable disruption.
We also tested redundancy by shutting down one UPS unit while another backup UPS handled the workload automatically. The transition happened smoothly, showing that the backup design worked as expected.
Of course, not everything was perfect. During the first attempt, the generator did not immediately activate because of a configuration issue in the ATS panel. The team quickly identified the root cause, corrected the configuration, and repeated the test successfully. This became an important reminder that even small details matter in disaster recovery preparation.
Simulating a Fire Emergency
The second scenario was even more challenging. This drill simulated a fire incident inside one of the data center racks. The objective was not only technical recovery, but also emergency response coordination and disaster recovery execution.
The simulation began when smoke detectors triggered an alarm in the control room. The emergency alert system activated immediately, and our operations team worked together with security personnel to evacuate everyone from the affected area. At the same time, the NOVEC1230 fire suppression system was activated to contain the simulated fire before it could spread further.
After the evacuation was completed, the BCP coordinators officially declared a disaster condition and activated the Disaster Recovery Plan (DRP). Teams responsible for cloud services and managed services then prepared to move operations to the disaster recovery site.
The recovery teams traveled to the backup data center location, verified server conditions, checked backups, and ensured that applications and databases could run properly from the recovery environment. Once everything was ready, VPN connections were redirected so users could reconnect to services through the DR site.
One of the most satisfying moments during the drill was when users successfully accessed the applications again from the backup location. This proved that the recovery procedures were working and that service continuity could be maintained even during a serious incident.
More Than Just a Drill
What made this exercise meaningful was not only the technology, but also the teamwork behind it. Every department had a role — from data center engineers and NOC operators to security personnel, cloud teams, and communication coordinators.
The drill also helped us identify areas for improvement. Faster coordination, clearer communication, and more structured DR procedures are some of the lessons we will continue improving in future simulations.
At VELO, we believe preparedness is part of delivering trusted services. A disaster recovery plan should never exist only on paper — it must be tested, practiced, and continuously improved.
Because when unexpected situations happen, preparation makes all the difference.