Az-fcc re-Certification Application

Appendix D: Disaster Recovery

Sprint offers emergency options and uninterruptible power that exceeds the State’s minimum requirements by offering an end-to-end approach that is unmatched in the relay industry. Sprint has emergency operations and uninterruptible power systems (UPS) supporting relay call centers, the TRS switches (located at wireline switch sites). Sprint knows a large-scale loss of commercial power is one of the most critical factors impacting access to communication. We have proven programs to keep that from impacting relay services. Both TRS and CapTel offer uninterruptible power supplies and generators to ensure relay users will continue to have access to the service in the event of power outages.

Call Center Power Solutions

Sprint provides a cost effective solution with a UPS using a combination of standard battery backup and an auxiliary generator to provide uninterrupted power for an unlimited duration for key components.

The switch peripherals

Switch room environment, including:

Air conditioning, if required to maintain service

Fire suppression systems

Emergency lights and system alarms

CA consoles/ terminals

CA work site emergency lights

Call Detail Recording (CDR)

Sprint ensures the UPS system capacity is sufficient to operate the call center during busy season and busy hour load. Sprint has installed power-generating equipment capable of operating call centers for extended periods. In the event of a power outage, the UPS and back-up power generator ensure seamless power transition until normal power is restored. UPS is used only long enough for the backup power generators to come on line – a matter of minutes. Backup power generators are supplied with sufficient fuel to maintain operations for at least 24 hours. Generators can stay in service for longer periods of time as long as fuel is supplied. As a safety precaution (in case of a fire during a power failure), the fire suppression system is not electrically powered. Once the back-up generator is on line, stable power is established and maintained to all TRS system equipment and facility environmental controls until commercial power is restored.

Emergency Procedures Training

All Sprint Relay employees are trained on emergency procedures to minimize or prevent disruption to relay users. Sprint instructs its staff on the procedures to be followed in the event of an emergency or service impacting issue. Sprint provides annual training to ensure familiarity with systems and processes. Ad-hoc training is conducted for new procedures or team members.

Sprint’s response organizations use exercises to evaluate plans, educate personnel, test functions, and operational capability. Information related to these exercises is propriety to Sprint. Additionally, as part of the nation’s critical infrastructure, Sprint participates in coordinated situation drills with Federal Emergency Management Agency (FEMA), the Department of Homeland Security (DHS), and state emergency management agencies to ensure coordinated preparedness and response during a disaster.

Tabletop Exercises: In a round-table setting, members of the response team meet to discuss responsibilities and describe how to react as a team in an emergency.

Walk-Through Drills: Both the response team and management perform their emergency functions within the emergency response location.

Functional Drills: Tests designed to target specific functional processes within the recovery plan such as notification, response, communications, documentation, and team cohesiveness. Often, these functions are tested separately to help identify improvement areas and to eliminate confusion.

Full-scale Exercises: Exercises simulated to be as close as possible to a real-life disaster. They may involve a combination of response teams, management, field operations, and outside agencies.

After Action Reviews (AARs): Following an incident or an exercise, an AAR is conducted to ask participants to identify areas of success and improvement. These are documented as Lessons Learned and tracked to satisfactory completion.

Maturity: Sprint uses an internally developed Maturity Model for benchmarking the Business Continuity Program success and progress. The model is based on the Capability Maturity Model as developed by Carnegie Mellon University.

Business Continuity

Industry accepted principles are the basis for Sprint’s BC program. Sprint has adopted key principles from standards set by organizations such as the Disaster Recovery Institute International (DRII), ASIS Organizational Resilience Standard, FEMA, Business Continuity Institute (BCI), American National Standards Institute (ANSI), NFPA 1600, International Organization for Standardization (ISO) 27001 and ISO 22301, and several Military Specifications (Mil-Spec) standards. Sprint’s Business Continuity Program Overview is reviewed and approved on an annual basis.

Sprint Relay network has a Business Continuity (BC) plan to deal with all types of natural and man-made problems which may prevent calls from reaching the relay center or impact the operation of the TRS platform. The plan identifies how Sprint minimizes impact to relay users and restores relay services. Sprint brings more value when it comes to maintaining operations during natural and man-made events. Sprint’s BC methodology and implementation standards are consistent with industry-wide best practices and trusted by experts in the field. The Sprint dedicated BC Teams (BCTs) participate in government-provided and private sector training, and maintain certifications from:

DRII

International Association of Emergency Managers (IAEM)

DHS

Business Continuity Institute (BCI)

Sprint understands the BC challenges faced by government organizations and has designed state relay services accordingly. Sprint has experience in serving more than 160 federal entities and more than 150 military bases worldwide including the Department of Defense (DOD), State/Local Governments, Law Enforcement, and DHS.

Sprint's Business Continuity Management Team works as a customer advocate when large network outages occur. The team works closely with network recovery teams to establish customer prioritization once the backbone, Telecommunications Service Priority (TSP) and Critical Life Circuits are re-established.

All departments within Sprint, including the Sprint Relay program, follow these well-established programs to ensure top-notch support for our customers.

Call Center Evacuation Events

Sprint has plans in place to deal with call center events such as fires. Each call center has a designated Safety Marshal and clear chain of command. As a first step, the situation is identified and the threat is assessed. If evacuation is necessary, the local authorities (e.g., 911) are immediately alerted along with the Call Center Service Assurance Center (CCSA) and the Traffic Management Control Center (TMCC). Call center management and Sprint Corporate Security are also alerted.

Traffic will be re-routed immediately to other call centers not impacted and work with those call centers to increase staffing, as needed. Once the issue is resolved, all communication assistants (CAs)/operators return to the center and the incident is fully documented.

Proactive Measures

Over the past 26 years, Sprint Relay users have rarely experienced any type of inability to place calls. Sprint’s backup capabilities are unmatched in the TRS industry with 6 call centers (including the location at Sprint headquarters in Overland Park, KS) capable of handling TRS calls and multiple switch locations supporting the TRS platform.

Sprint’s switches and call centers are staffed with spare positions and platform components to deal with all types of technical issues. The TRS platform offers automated alarming to notify personnel of issues.

Redundancy is built into our infrastructure to deliver outstanding performance for all of our TRS customers. These attributes will ensure functional equivalency for state relay service callers during disasters. The benefits of our leading-edge platform and flexible configuration include:

Switches, call controllers, and databases are housed in geographically-dispersed locations that conform to “critical” grade physical security requirements. Sprint’s switches and peripherals are located at switch sites in telecom bunkers.

Redundant connections between switch sites, 800 network, and call centers

If the problem is within Sprint's TRS center, maintenance can usually be performed from Sprint's centralized center, the CCSA.

Sprint retains hardware spares at each center to allow for the most common type of repair required without the ordering of additional equipment (except for complete loss of a building).

Centralized routing and reporting systems enables Sprint to treat the entire call center complex as a single virtual call center rather than standalone call centers

All TRS positions are capable of handling calls for any State customer.

All training seats are configured and immediately ready to take production traffic.

Sprint has pre-established plans for all types of outages.

Sprint automatic routes calls away from a center undergoing a service recovery event. For example, if a fire drill forces CAs/operators to evacuate, the call router automatically sends calls to other relay centers.

Sprint has historically been the best at dealing with natural and man-made disasters that have caused outages. With each incident Sprint has managed to be prepared, respond and ensure ongoing service delivery. Sprint’s processes as detailed here take into consideration every aspect of an outage and/or natural disaster that includes a higher call volume likelihood due to the natural disaster. Some examples of disasters that affected Sprint facilities in the past are:

Wind burst that blew off a portion of the roof of our Syracuse, NY call center

Farmer cuts Fiber Optic cable servicing Lubbock, TX when burying a cow

Hurricanes that impacted call centers in Miami and Jacksonville

Tornado warnings impacting upper Midwest call centers. One evening, 37 Tornadoes were within range of our call center. Our center had to be evacuated. Sprint continued to provide service without interruption.

These list just a few of the natural and man-made disasters we faced, and with each one we were able to maintain our service levels with the processes we have in place. Our employees are the best at ensuring we maintain these service levels.

TRS Data Center Disaster Planning

Sprint has implemented a distributed architecture for interconnection redundancy utilizing dual fiber facilities at all of our switch locations. These main switch locations currently have battery backup as well as permanent generators. In addition, site recovery plans have been developed for all major switch locations, prioritizing available options for relocation, and ensuring agility when faced with disaster recovery issues. Most switches also have tap boxes to readily connect the output of a portable generator in the event of primary generator issues.

TRS Winter Preparedness Plan

Sprint has processes in place if a known weather event is encountered. These known contingency plans are designed to mitigate our customers’ degradation of service and are maintained by the TMCC. Each service has back-up locations to ensure redundancy.

Known Event

Four days prior - TMCC and Ron Peay (Operations Manager) will make a determination as to the severity and number of centers which might be affected.

Three days prior - TMCC and Ron will verify previous day’s potential impact and begin calling to non-effected centers to post overtime (OT). All centers will be advised to put a list together of employees who will work overnight and weekends. TMCC will notify John Moore (Manager - Customer Relations) and CCSA of our “game plan”

Two days prior - TMCC will meet with Ron to update impacts and plan. All non-impacted centers will be called to update OT requirements and overnight requests.

One day prior - TMCC will meet with Ron to update impacts and plan.

Day of Event - TMCC will invoke emergency call routing as required. TMCC will be the point of contact for all notifications. Affected centers will update TMCC every four hours. TMCC will update Ron who will update Business Continuity Manager through executive level. Management is also responsible for notifying the Business Continuity Team.

Unknown Event

The Activation Criteria Plan will be used when either weather or other events cause potential significant (excess of 25 percent) increase in call volumes or one or more TRS call centers is off-line for more than two hours, using the following procedure:

Automated alarming and/or TRS call center notifies TMCC

TMCC contact CCSA

CCSA sends notification to a pre-established distribution list

CCSA establishes a conference call to work on resolving the issue with impacted groups

After fix agencies are unable to re-establish center operations – the Business Continuity Plan (BCP) is invoked and Management will notify the Business Continuity Management Team.

CapTel-Specific Disaster Recovery Information

CapTel, Inc. (CTI) and Sprint have worked together to develop a complete plan for dealing with all types of natural and man-made problems including but not limited to terrorism and phone line cut accidents. Performance at the CapTel call center is monitored continuously by CTI technicians 24/7. Sprint will be notified by the CapTel Service Center Manager immediately upon determination of any type of natural or man-made problem that causes disruption either:

CapTel has established contingency plans in the event of a complete and extended loss of a CapTel call center. The plan includes a number of steps based on the estimated duration of the outage and takes advantage of the relative short travel time between the Wisconsin CapTel call centers. The first phase is organized to initiate the recovery process within hours and can be fully completed within days. This involves expanding service into available space in the operating call center locations and other CapTel facilities

All training seats are configured and immediately ready to take production traffic.

Additional production seats are established in unused and available space within the existing facilities.

Regular shuttle services are established to transport qualified CapTel CAs/operators and staff from the outage area to and from the expanded facilities.

The recovery plan includes a second phase for extended outages. To support this longer duration, CapTel has identified additional disaster recovery locations with appropriate facilities in the metropolitan area of each of the call centers.

The addition of the Orlando, FL and Sprint’s TRS/CapTel call centers has alleviated many of the inclement weather challenges presented by the winter season. However, if inclement weather affects the CapTel staffs’ ability to arrive to work, in most cases, with minor adjustments, CTI can still meet the call volume demand with enough staff coverage in a wide range of snow fall amounts. However, if necessary, Sprint and CTI will institute proven tactics, as necessary, to motivate, encourage, and enable CapTel CAs/operators to be present or to pick up additional hours so CTI can meet its service level requirements during inclement weather

Customer Notification Procedures

Sprint will inform the state contract manager of any major interruptions to the TRS/CapTel service that exceeds five minutes in duration or isolates part of the state. To provide the contract manager with the most complete and timely information on problems affecting relay service, Sprint’s trouble reporting procedure for TRS and CapTel includes multiple levels of response:

Immediate notification of events that last 5 minutes or isolate part of the State

Notification when the issue is resolved and/or status updates (every 24 hours)

Comprehensive final report within 3 days

Within 24 hours of the Relay service disruption, an intermediate report provides problem status and more detail of what action is necessary. In most cases, the 24-hour report reveals the problem has been corrected and full relay service has been restored. The state contract manger (or designate) will receive this notification from your Sprint Customer Relationship Manager (CRM). He/she and/or a member of the management team will provide the final report and follow up on steps Sprint will take to ensure we can minimize the likelihood of this event occurring again.

Final reports include a comprehensive look at the event, including the following:

How the problem occurred

When the problem occurred

The number of impacted customers (if known)

What was required to correct the problem

Time and date the relay service resumed full operation

Avoidance plan for future (if applicable)

Temporary Delay Message

If approved by the state, Sprint can also provide a temporary delay message for TRS users that is turned on only when long hold times may occur as a result of weather or other event impacting service. For example, if there were a terrorist attack or natural disaster that significantly increased the number of calls to the relay center, Sprint can add a temporary recording that alerts voice and TTY users, such as: “THE RELAY CENTER IS EXPERIENCING LONGER THAN NORMAL HOLD TIMES. PLEASE HOLD FOR THE NEXT AVAILABLE CA OR TRY YOUR CALL AGAIN LATER.”

Telecommunications Service Priority (TSP)

All of Sprint’s circuits supporting TRS and CapTel services have qualified for priority restoration under the TSP program. Sprint’s participation in the TSP Program strengthens our robust reliability. If a national or regional emergency causes service to be disrupted and the call center cannot receive or place calls, Sprint’s participation in the TSP program means Local Exchange Carriers (LECs) would be required to restore service as rapidly as possible consistent with the priority status assigned. Sprint’s reliable network and TSP participation ensures Sprint’s disaster recovery ability is unmatched by any Relay provider in the world.

Appendix E – Arizona Relay Service Newsletters

Appendix F – Arizona Relay Brochures & Advertisements




Arizona Relay Service Website screenshots




Appendix G – Complaint Logs 2014-2017

Complaint Tracking for AZ (06/01/2014-05/31/2015). Total Customer Contacts: 0

Tally

Date of Complaint

Nature of Complaint

Date of Resolution

Explanation of Resolution

1

04/16/15

TTY user had asked the agent to repeat the last message that was voiced by the outbound party because the message received had garbled, the agent responded with "CA can't be involved in your call". The customer felt this response was rude and uncalled for. Supervisor documenting the concern apologized for the inconvenience. Follow up requested to be sent via email.

04/16/15

The agent was confused on what the caller was asking. Supervisor coached the agent on when it is okay to repeat information when requested. Follow up sent via email as per request.