Salem Hospital works with NetXperts, Inc. to provide five nines of availability for the new Patient Care Tower
Customer Name: Salem Hospital
Location: Salem, OR
Number of Employees: Over 4,000
- 24×7 Critical Care Facility requires five nines of uptime
- Network required to support Critical Applications such as Epic EMR, Voice, PACS Imaging, Video Surveillance and Video Conferencing
- Ability to upgrade switch IOS with minimal impact to end users
- Build a network that meet unpredictable and expanding needs 10 years into the future Solution
- High Availability Campus Network Design with Routed Access Layer using OSPF
- 10Gbps Ethernet uplinks
- Enhanced and Standards based POE at the access layer Results
- May 2009 go live with 0 network issues reported.
- Failure of 10Gbps X2 in July had 0 impact to end users, X2 replacement had 0 impact to end users.
- In service upgrades of switches with 0 impact to end users is now possible because of redundant supervisors.
Salem Hospital Regional Health Services is one of the largest of Oregon’s 57 acute care hospitals and operates the busiest emergency department in Oregon. It is a not-for-profit hospital, licensed for 454 acute-care beds and employs over 4,000 people whose mission is to improve the health and well-being of the people and communities they serve. The new Patient Care Tower is a 24×7 Critical Care facility that includes state-of-the-art interventional suites for surgery, 54 patient rooms in the ER, space for treatment of critical care and trauma patients and a rooftop helipad with an elevator dedicated
to the ER.
As the healthcare industry moves towards Electronic Medical Records, the network becomes more and more critical. Network outages directly affect the ability of doctors and nurses to provide excellent patient care. Add to that, the requirement to support Voice over Internet Protocol (VOIP), IP Video surveillance cameras, PACS medical
imaging applications and Wireless service the need for a high availability campus network foundation is critical. Another problem the hospital faced in the past was outdated and inconsistent versions on network infrastructure. It simply was not feasible to upgrade the equipment because of the downtime involved, the hospital needed a solution to maintain consistency of version across the enterprise and patch security holes with minimal impact to end users.
For help designing and deploying the new Patient Care Tower network, Salem Hospital needed a partner with experience working in healthcare and directly with hospitals, they needed a partner who understands the critical nature of hospital networks and understands the business rules when working at hospitals. The partner Salem Hospital chose was NetXperts, Inc., a Cisco Silver Partner headquartered in San Ramon, CA. NetXperts met with Salem Hospital IS and different business units within the hospital to design a campus LAN that met all of the hospitals needs. “A highly available network that has no single points of failure is a key part of providing 99.999% reliability for these and future critical applications.” Said Nate Helfrey, Sr. Network Engineer/Project Manager assigned to the Salem Hospital PCT Project.
The decision to go with a layer 3 routing protocol such as OSPF at the access layer enables the hospital to load balance across the redundant uplink ports and most importantly, eliminate Spanning Tree Protocol as the loop prevention technology. Spanning Tree Protocol had been the root cause of network outages at the hospital in the past.
A three tier design with Core, Distribution, and Access Layers was chosen. 10 Gigabit per second Ethernet connections served as uplink ports connecting the switches. This solved the problem of future scalability, enabling Salem Hospital to feel confident in the performance of the network now and its ability to scale for future bandwidth intensive traffic such as video conferencing, increased medical imaging and future unknown requirements.
The Cisco Catalyst 6509-E with redundant Supervisor Engine 720, redundant 6000 WAC Power Supplies and eight port 10Gigabit Ethernet modules were chosen for the distribution layer and network core. The distribution layer supports 18 closets with redundant and load balanced 10Gbps connections utilizing layer 3 connections with OSPF to18 Cisco Catalyst 4510R-Es. The redundant supervisors allow for 0 impact upgrade to IOS. Redundant 40Gbps Ether Channel links provided the connectivity to the network core. Redundant power supplies insure that a single power supply or circuit failure will not take down the switch.
Cisco 4510R-E Catalyst Switches with redundant Supervisor 6-E configured for SSO failover, redundant 4200 WAC Power Supplies were chosen for the closet Access Layer switches. Line cards chosen included a combination of 48 port 10/100/1000 RJ45 blades with 802.3af standards based Power over Ethernet and 48 port 10/100/1000 RJ45 blades with support for Cisco Enhanced Power over Ethernet capable of supporting up to 30 Watts of power per port. The choice of the Cisco Catalyst 4510R-E allowed for scalability of up to 384 Ethernet ports per IDF. The choice of the Supervisor 6-E allowed for redundant 10Gbps connections to the Cisco Catalyst 6509E switches at the distribution layer, support for OSPF routing protocol and for upgrades to the supervisor engines IOS with 0 impact to end users. The enhanced POE blades allowed for Salem Hospital to deploy IP Video Surveillance cameras with pan/tilt/zoom throughout the PCT eliminating the need for a separate video network. Standards based POE blades provided power to IP Phones and 802.11n Wireless Access Points.
- Cisco Catalyst 650
- Switches with Sup720
- Cisco Catalyst 4510
- Switches with Sup6e
- 10Gbps line cards
- 802.3af POE line cards
- Cisco Enhanced POE line cards
- 10Gbps X2
“NetXperts worked with the customer to develop a test matrix for the Quality Assurance and Stress tests. QA testing is a key part of NetXperts project methodology. It’s important that we uncover potential issues and address them prior to cutover and go live” Jason Blick, VP of Engineering stated.
Prior to the go live, NetXperts conducted Quality Assurance and Stress Tests. The tests proved to Salem Hospital that the network met the requirements, failure of a single interface, single blade, single power supply, had 0 impact to end users. Software upgrade of IOS was also performed with 0 impact.
Eddy Louis, NetXperts Field Engineer was onsite as part of the go live support team. “From a network perspective we were pretty bored. We had no calls or tickets to work on, so we pitched in on other issues where we could be helpful.”
In May of 2009 the new Patient Care Tower opening was a smashing success. NetXperts Engineers were onsite for the go live and 0 network issues were reported. About 2 months later in July, the first failure was experienced, an X2 interface, the monitoring application reported the outage, but end users didn’t notice a thing. The X2 was replacedand the link put back into service with out issue.