Business Needs
Facilitate and support the continual growth and demand for data for city services, while optimizing performance and redundancy.
Solution
The Nutanix Xtreme Computing Platform: Nutanix Acropolis distributed storagefabric; Nutanix Prism management solution.
Benefits
- Measured a 20x performance increase for all virtual servers and production SQL databases.
- Enabled replication of the entire datacenter to a remote location without taking systems offline, ensuring business continuity in the event of a disaster.
- Obtained a 2/3 reduction in operational expenses.
Lenoir City Utilities Board Background
The Lenoir City Utilities Board (LCUB) provides water, electrical, wastewater, and gas services to all residents in the rapidly growing Lenoir, Tennessee region. As the sixth largest municipal utility in the state, LCUB has invested millions of dollars into its infrastructure to continue to provide reliable and affordable utility services to all of its customers.
IT Environment and Challenges
LCUB’s IT department was getting ready for an infrastructure refresh to support the increasing demands for data, while optimizing performance and redundancy in the city datacenter. The legacy IT environment consisted of three VMware ESXi Hosts. Each host was a Dell PowerEdge R710 with 8 Intel Xeon E5520 CPUs (2.26 GHz each) and 128GB of RAM, and an iSCSI SAN with 12TB of space. “Our IT environment was running very close to full capacity,” noted William Jason Tuttle, enterprise data management administrator for the Lenoir City Utilities Board. “Our high resource utilization made it very difficult to maneuver in the event of peak demand or a failure, or even to make changes without experiencing downtime.”
LCUB made the decision to purchase two Nutanix clusters in 2015 in order to increase storage capacity. Each Nutanix production cluster consists of nine VMware ESXi hosts. One Nutanix NX-3050 is being used to host the servers, and a NX-7110 series will be used to host LCUB’s virtual desktop infrastructure. Each NX-3050 has 16 Intel Xeon E5-2650 v2 CPUs and 256GB of RAM, for a combined 36 TB of usable space that is accessible by all nodes in the cluster.
Deploying Nutanix
To achieve the level of performance needed for its production SQL database, LCUB decided to use the SSD tier on the NX-7110 series node. “The NX-7110 nodes are reserved for our virtual desktop environment; however, we have not yet implemented VDI, so we can use the SSD tier on the NX-7110 temporarily,” noted Tuttle. “This solution is fine in the short-term and gives us the opportunity to get a feel for the kind of performance we can expect with dedicated SSD resources on the Nutanix platform.
We have already consumed all but 47.19 GB of the SSD tier on the NX-7110 host with our production database. In order to give us room for growth we will require a host with a larger SSD tier.” Tuttle then described what the environment would be like without the Nutanix solution. “If we were using conventional methods to improve performance, we would need to decide on a way to split our database so that workloads could be shared among several controllers and disks. We would need to place indexes on file-groups that were on separate disks (in a RAID configuration) with their own controllers. And to further speed things up, we would have to build a non-clustered index on yet another disk.
The Nutanix platform takes care of all that optimization for us, without having to reconfigure our production database. The same results could be accomplished by having multiple controllers and multiple drives configured in a striped RAID.”
“It really wasn’t an option for us to reconfigure our database using one of these methods, since it would have taken 6 to 12 months,” continued Tuttle. “The Nutanix option was much faster. I moved 10TB of servers over to Nutanix in just one day, and that was with doing it all in baby steps. You can deploy Nutanix a lot faster than I did, but since it was my first experience with their systems, I was very cautious about moving servers over and testing them. The next time it will only take hours instead of a day.”
20x Faster Performance
LCUB created a performance baseline before and after moving the production database from the existing EMC SAN to the 7000-series Nutanix platform. LCUB is now able to easily capture comparative performance metrics at the hardware, virtual machine, and hypervisor level using Nutanix’s Prism interface. “The interface for the old equipment was simply horrible,” Tuttle shared. “Anytime we wanted to manage the SAN, we had to have Windows XP with an old version of Java or it wouldn’t work. It was a major headache. Whereas the Nutanix Prism interface is a beautiful masterpiece, and it just works.”
After migrating its primary database to the Nutanix environment, LCUB saw an increase of 652 million disk bytes/sec – which equates to a 20x improvement in system performance. “Several of our power users immediately told me that they noticed a significant performance improvement in their daily tasks of running reports and navigating large data-sets,” Tuttle shared. “There have been zero complaints of performance issues from any other staff after the migration. The combination of results from the performance baseline and direct feedback from users confirms that we have significantly improved the daily working environment, based on Nutanix’s performance alone.”
2/3 Reduction in OpEx
LCUB received a huge reduction in operational costs by moving to the Nutanix hyper-converged platform. “With traditional 3-tier systems, you’re juggling a lot more pieces and parts,” noted Tuttle. “You also have to work with several different admins and teams. I can cover the server side, but I would need at least two more people for deploying and managing the network, storage and the SAN. Nutanix eliminates the need for ‘drawing lines in the sand’, between infrastructure teams. This is a tremendous benefit of Nutanix’s hyperconverged approach—it eliminates the ‘political minefield’ that you have to navigate when interworking between departments. As a result, we have been able to significantly streamline our IT management duties and obtain a 2/3 reduction in operational expenses.”
Monitoring Nutanix on a Smart Watch
Tuttle recently used the Restful API within the Prism Interface to create an innovative smart watch monitoring application. “The app pulls data from our production cluster and displays a lot of different statistics, including remaining free space, memory utilization, CPU utilization,” Tuttle explained. “It extracts all of that information and populates our Pebble smart watches, which we now use for monitoring performance. It updates the data in real time, giving us a very good overview of overall system health of the cluster. If everything is working well, it displays a smiling face. If not, it shows a frustrated face and with a letter that indicates the location that might be failing or down. It also monitors how many people we have out of power at any given moment in our district and the weather forecast. All of these factors come into play when you are providing public utility services that need to be available around the clock.
Excellent Nutanix Support
LCUB enlisted the help of Nutanix support for the initial implementation phase. “A Nutanix engineer came out and advised us how to install the equipment in the rack,” Tuttle said. “He had a lot of good advice on the placement and the way to run the wires so we wouldn’t have to bring down a cluster just to do a replacement if there was a problem moving forward. That was invaluable advice. You don’t want to have to shut everything down to replace a mechanical part, like a power supply, that is going to eventually run out.” The Nutanix engineers also helped LCUB configure the storage backplane and switches. “There are some ‘real pros’ working in Nutanix Support – they put me at ease,” Tuttle acknowledged. “The entire implementation went very smoothly. And post deployment, every interaction with Nutanix Support has been excellent.”
Non-Disruptive Upgrades
“We are a utility company, so any downtime is simply not an option–our systems have to be up and running 24×7,” noted Tuttle. “We have almost 100 servers, so it would take several hours just to shut everything down properly. And then there are a lot of processes that are involved with restoring all of those servers. Without Nutanix, it would take a week or two to do an upgrade, assuming everything goes smoothly. With Nutanix, we can perform upgrades on–the-fly without taking our systems offline.”
Implementing Disaster Recovery
One disaster scenario that LCUB must plan for involves a direct hit on its datacenter by a tornado. Tennessee has experienced 1,131 tornados since 1950, several of which have been near LCUB’s datacenter. “If we are to remain operational in the event of a major catastrophe, we must have at least one secondary site that can be turned up quickly to take over production workloads,” Tuttle stated. “As our workflows become more dependent on technology, we must plan for the worst so that we are able to restore power and other lifesaving services in a timely manner after any catastrophic event that impacts our primary datacenter.”
The Nutanix environment offers LCUB the ability to setup disaster recovery by easily replicating all data to a second or third cluster. Data can be deduplicated, reducing bandwidth requirements by 70%, for efficiency and performance. The Lenoir City Utilities Board is now undergoing a fiber optics project to connect all of their offices and substations. This will allow LCUB to place a disaster recovery cluster at an accessible and safe location on its existing network. LCUB is now building a new datacenter for DR. “Without Nutanix, we would have to take our systems off-line until our systems were physically moved to the new data center,” noted Tuttle. “Nutanix will enable us to stay up and running through-out the move. Now that we have replication in place, we have the ability to run off replicated data and rebuild it again. We will be able to relocate to our new data center and never miss a beat. That’s a huge benefit of the Nutanix approach.”
We have been able to achieve blisteringly fast speeds at the data level utilizing the clustered Nutanix architecture. Nutanix allows us to take advantage of our existing infrastructure and investments—to scale out horizontally and realize the benefits of SSD.
William Jason Tuttle, Enterprise Data Management and Administration, Lenoir City Utilities Board
Future Plans
The Lenoir City Utilities Board is now considering the purchase of a Nutanix NX-8150 series node to accommodate the entirety of its production database server and software in the SSD tier. This will provide one node for its production cluster and one node for the disaster recovery cluster. At that point, LCUB can free up the resources that are dedicated for the upcoming VDI environment and provide enough room to onboard additional databases as needed.
“We now have blisteringly fast speeds at the data level by utilizing the clustered Nutanix SSD architecture,” noted Tuttle. “Nutanix allows us to take advantage of our existing infrastructure and investments, and scale out horizontally and realize the benefits of SSD. As a utility company, we are responsible for keeping the lights on. It’s critical infrastructure for us, and Nutanix is ‘knocking it out of the park’ with their hyperconverged arrays.”