Internet2 IP Network Operations Report
12/23/2013 - 12/29/2013

Internet2 IP Network Operations Center
at Indiana University

New Connections

None


Upgrades

None


New Services

None


Unscheduled Outages Summary

Ticket Number Customer Impact Network Impact Title Outage Type Start Time (UTC) End Time (UTC)

220792-High 2-High Brief Outage - I2 IP Backbone ATLA-HOUS Undetermined 12/23/2013 7:44 AM 12/23/2013 7:45 AM

Unscheduled Outages Detail


Ticket No.:22079:45
Subject:Brief Outage - I2 IP Backbone ATLA-HOUS
Affected:Backbone ATLA-HOUS
Start Time:Monday, December 23, 2013, 7:44 AM (0744) UTC
End Time:Monday, December 23, 2013, 7:45 AM (0745) UTC
Description:Backbone Circuit ATLA-HOUS was briefly operating at a reduced capacity. Level(3) Communications is investigating the cause; however, the reason for the outage is pending. Traffic continued to pass over the backbone, at a reduced capacity. The Internet2 IP Network (R&E/Commodity) Backbone ATLA-HOUS remained available.

Scheduled Maintenances Summary

Ticket Number Customer Impact Network Impact Title Maintenance Type Start Time (UTC) End Time (UTC)

22059 3-Elevated 1-Critical Maintenance Completed - I2 IP Core Node CHIC Software 12/23/2013 10:02 AM 12/23/2013 10:46 AM
22060 3-Elevated 1-Critical Maintenance Completed - I2 IP Core Node KANS Software 12/23/2013 10:17 AM 12/23/2013 11:08 AM
22061 3-Elevated 1-Critical Maintenance Completed - I2 IP Core Node HOUS Software 12/23/2013 10:27 AM 12/23/2013 10:58 AM
22062 3-Elevated 1-Critical Maintenance Completed - I2 IP Core Node SALT Software 12/23/2013 10:42 AM 12/23/2013 11:08 AM
22063 3-Elevated 1-Critical Maintenance Completed - I2 IP Core Node LOSA Software 12/23/2013 10:53 AM 12/23/2013 11:08 AM
22064 3-Elevated 1-Critical Maintenance Completed - I2 IP Core Node SEAT Software 12/23/2013 11:04 AM 12/23/2013 11:08 AM
22075 3-Elevated 1-Critical Maintenance Completed - I2 IP Core Node (CLEV) Software 12/27/2013 5:59 AM 12/27/2013 6:04 AM
22076 3-Elevated 1-Critical Maintenance Completed - I2 IP Core Node (NEWY32AoA) Software 12/27/2013 5:45 AM 12/27/2013 5:50 AM
22077 3-Elevated 1-Critical Maintenance Completed - I2 IP Core Node (WASH) Software 12/27/2013 5:27 AM 12/27/2013 5:32 AM
22078 3-Elevated 1-Critical Maintenance Completed - I2 IP Core Node (ATLA) Software 12/27/2013 5:04 AM 12/27/2013 5:09 AM

Scheduled Maintenances Detail

Ticket No.:22059:45
Subject:Maintenance Completed - I2 IP Core Node CHIC
Affected:Core Node (CHIC) and its Connector(s) and Participants
Start Time:Monday, December 23, 2013, 10:02 AM (1002) UTC
End Time:Monday, December 23, 2013, 10:46 AM (1046) UTC
Description: Core Node (CHIC) was unavailable while Internet2 Engineers performed maintenance to swap the Routing engines. All connections to the Core Node were affected. Maintenance has been completed.

Ticket No.:22060:45
Subject:Maintenance Completed - I2 IP Core Node KANS
Affected:Core Node KANS
Start Time:Monday, December 23, 2013, 10:17 AM (1017) UTC
End Time:Monday, December 23, 2013, 11:08 AM (1108) UTC
Description: Core Node KANS was unavailable while Internet2 Engineers performed maintenance to swap the Routing engines. All connections to the Core Node were affected. Maintenance has been completed.

Ticket No.:22061:45
Subject:Maintenance Completed - I2 IP Core Node HOUS
Affected:Core Node HOUS
Start Time:Monday, December 23, 2013, 10:27 AM (1027) UTC
End Time:Monday, December 23, 2013, 10:58 AM (1058) UTC
Description: Core Node (HOUS) was unavailable while Internet2 Engineers performed maintenance to swap the Routing engines. All connections to the Core Node were be affected. Maintenance has been completed.

Ticket No.:22062:45
Subject:Maintenance Completed - I2 IP Core Node SALT
Affected:Core Node SALT
Start Time:Monday, December 23, 2013, 10:42 AM (1042) UTC
End Time:Monday, December 23, 2013, 11:08 AM (1108) UTC
Description: Core Node (SALT) was unavailable while Internet2 Engineers performed maintenance to swap the Routing engines. All connections to the Core Node were affected. Maintenance has been completed.

Ticket No.:22063:45
Subject:Maintenance Completed - I2 IP Core Node LOSA
Affected:Core Node LOSA and its Connector(s) and Participants
Start Time:Monday, December 23, 2013, 10:53 AM (1053) UTC
End Time:Monday, December 23, 2013, 11:08 AM (1108) UTC
Description: Core Node LOSA was unavailable while Internet2 Engineers performed maintenance to swap the Routing engines. All connections to the Core Node were affected. Maintenance has been completed.

Ticket No.:22064:45
Subject:Maintenance Completed - I2 IP Core Node SEAT
Affected:Core Node SEAT and its Connector(s) and Participants
Start Time:Monday, December 23, 2013, 11:04 AM (1104) UTC
End Time:Monday, December 23, 2013, 11:08 AM (1108) UTC
Description: Core Node SEAT was unavailable while Internet2 Engineers performed maintenance to swap the Routing engines. All connections to the Core Node were affected. Maintenance has been completed.

Ticket No.:22075:45
Subject:Maintenance Completed - I2 IP Core Node (CLEV)
Affected:Core Node (CLEV) and its Connectors and Participants
Start Time:Friday, December 27, 2013, 5:59 AM (0559) UTC
End Time:Friday, December 27, 2013, 6:04 AM (0604) UTC
Description: Core Node (CLEV) was unavailable while Internet2 Engineers performed maintenance to swap the Routing engines. Maintenance has been completed. All connections to the Core Node were affected.

Ticket No.:22076:45
Subject:Maintenance Completed - I2 IP Core Node (NEWY32AoA)
Affected:Core Node (NEWY32AoA) and its Connectors and Participants
Start Time:Friday, December 27, 2013, 5:45 AM (0545) UTC
End Time:Friday, December 27, 2013, 5:50 AM (0550) UTC
Description: Core Node (NEWY32AoA) was unavailable while Internet2 Engineers performed maintenance to swap the Routing engines. Maintenance has been completed. All connections to the Core Node were affected.

Ticket No.:22077:45
Subject:Maintenance Completed - I2 IP Core Node (WASH)
Affected:Core Node (WASH) and its Connectors and Participants
Start Time:Friday, December 27, 2013, 5:27 AM (0527) UTC
End Time:Friday, December 27, 2013, 5:32 AM (0532) UTC
Description: Core Node (WASH) was unavailable while Internet2 Engineers performed maintenance to swap the Routing engines. Maintenance has been completed. All connections to the Core Node were affected.

Ticket No.:22078:45
Subject:Maintenance Completed - I2 IP Core Node (ATLA)
Affected:Core Node (ATLA) and its Connectors and Participants
Start Time:Friday, December 27, 2013, 5:04 AM (0504) UTC
End Time:Friday, December 27, 2013, 5:09 AM (0509) UTC
Description: Core Node (ATLA) was unavailable while Internet2 Engineers performed maintenance to swap the Routing engines. Maintenance has been completed. All connections to the Core Node were affected.


Downtime and Availability



Internet2 IP Core Nodes Down TimeReporting Period Availability52 Week Availability
ATLA MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.98501% 99.99981%
CHIC MX960 0 hr 44 min0 hr 0 min 99.56349%100.00000% 99.97382% 99.99905%
CLEV MX960 0 hr 5 min0 hr 0 min 99.95040%100.00000% 99.99753% 99.99924%
HOUS MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.96983% 100.00000%
KANS MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99886% 100.00000%
LOSA MX960 0 hr 15 min0 hr 0 min 99.85119%100.00000% 99.96774% 100.00000%
NEWY MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.98729% 99.99867%
SALT MX960 0 hr 26 min0 hr 0 min 99.74206%100.00000% 99.97382% 100.00000%
SEAT MX960 0 hr 4 min0 hr 0 min 99.96032%100.00000% 99.96528% 100.00000%
WASH MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.95086% 99.99962%

Aggregate Internet2 IP Core Nodes1 hr 34 min0 hr 34 min99.90675%100.00000%99.97700%99.99964%

** Less than 52 weeks of data available

NOTE: Numbers in italics represent downtime with scheduled maintenance excluded. Internet2's definition of scheduled maintenance is any anticipated outage announced at least 72 hours in advance of the outage start time.

Internet2 IP Backbone CircuitsDown Time Reporting Period Availability52 Week Availability
R&E 100G Backbone: I2-ATLA-CHIC-VLAN-09181 0 hr 49 min0 hr 0 min 99.51389%100.00000% 99.98558%99.99943%
R&E 100G Backbone: I2-ATLA-HOUS-VLAN-09182 0 hr 31 min0 hr 0 min 99.69246%100.00000% 99.98956%100.00000%
R&E 100G Backbone: I2-ATLA-WASH-VLAN-09179 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.94877%99.99848%
R&E 100G Backbone: I2-CHIC-CLEV-VLAN-09180 0 hr 49 min0 hr 0 min 99.51389%100.00000% 99.82221%99.87591%
R&E 100G Backbone: I2-CHIC-KANS-VLAN-09183 1 hr 35 min0 hr 0 min 99.05754%100.00000% 99.96642%100.00000%
R&E 100G Backbone: I2-CLEV-NEWY32AOA-VLAN-09177 0 hr 15 min0 hr 0 min 99.85119%100.00000% 99.99070%99.99943%
R&E 100G Backbone: I2-CLEV-WASH-VLAN-09178 0 hr 5 min0 hr 0 min 99.95040%100.00000% 99.99146%100.00000%
R&E 100G Backbone: I2-HOUS-KANS-VLAN-09184 1 hr 22 min0 hr 0 min 99.18651%100.00000% 99.98387%100.00000%
R&E 100G Backbone: I2-HOUS-LOSA-VLAN-09189 0 hr 46 min0 hr 0 min 99.54365%100.00000% 99.75751%99.81785%
R&E 100G Backbone: I2-KANS-SALT-VLAN-09185 1 hr 17 min0 hr 0 min 99.23611%100.00000% 99.98482%100.00000%
R&E 100G Backbone: I2-LOSA-SALT-VLAN-09188 0 hr 41 min0 hr 0 min 99.59325%100.00000% 99.99165%100.00000%
R&E 100G Backbone: I2-LOSA-SEAT-VLAN-09187 0 hr 19 min0 hr 0 min 99.81151%100.00000% 99.97116%99.99924%
R&E 100G Backbone: I2-NEWY32AOA-WASH-VLAN-09176 0 hr 10 min0 hr 0 min 99.90079%100.00000% 99.95351%99.95636%
R&E 100G Backbone: I2-SALT-SEAT-VLAN-09186 0 hr 30 min0 hr 0 min 99.70238%100.00000% 99.99298%99.99924%
R&E 100G Backbone: I2-WASH-ATLA-VLAN-09179 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
Legacy 10G Backbone: I2-ATLA-CHIC-10GE-05419 0 hr 49 min0 hr 0 min 99.51389%100.00000% 99.74423%99.81538%
Legacy 10G Backbone: I2-ATLA-CHIC-10GE-05638 0 hr 49 min0 hr 0 min 99.51389%100.00000% 99.74290%99.81538%
Legacy 10G Backbone: I2-ATLA-HOUS-10GE-05423 0 hr 31 min0 hr 0 min 99.69246%100.00000% 99.90172%99.96983%
Legacy 10G Backbone: I2-ATLA-HOUS-10GE-05562 0 hr 31 min0 hr 0 min 99.69246%100.00000% 99.90058%99.97002%
Legacy 10G Backbone: I2-ATLA-WASH-10GE-05133 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.76947%99.86092%
Legacy 10G Backbone: I2-ATLA-WASH-10GE-05251 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.77080%99.86092%
Legacy 10G Backbone: I2-ATLA-WASH-10GE-09142 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99981%99.99981%
Legacy 10G Backbone: I2-ATLA-WASH-10GE-6366 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
Legacy 10G Backbone: I2-CHIC-CLEV-10GE-08891 0 hr 49 min0 hr 0 min 99.51389%100.00000% 99.88179%99.89754%
Legacy 10G Backbone: I2-CHIC-CLEV-10GE-08892 0 hr 49 min0 hr 0 min 99.51389%100.00000% 99.88179%99.89754%
Legacy 10G Backbone: I2-CHIC-KANS-10GE-05781 1 hr 35 min0 hr 0 min 99.05754%100.00000% 99.93909%100.00000%
Legacy 10G Backbone: I2-CHIC-KANS-10GE-05976 1 hr 35 min0 hr 0 min 99.05754%100.00000% 99.92999%100.00000%
Legacy 10G Backbone: I2-CLEV-NEWY32AOA-10GE-08893 0 hr 15 min0 hr 0 min 99.85119%100.00000% 99.90893%99.95731%
Legacy 10G Backbone: I2-CLEV-NEWY32AOA-10GE-08894 0 hr 15 min0 hr 0 min 99.85119%100.00000% 99.91519%99.95731%
Legacy 10G Backbone: I2-CLEV-WASH-10GE-08889 0 hr 5 min0 hr 0 min 99.95040%100.00000% 99.94327%99.99696%
Legacy 10G Backbone: I2-CLEV-WASH-10GE-08890 0 hr 5 min0 hr 0 min 99.95040%100.00000% 99.93739%99.99696%
Legacy 10G Backbone: I2-HOUS-KANS-10GE-05560 1 hr 22 min0 hr 0 min 99.18651%100.00000% 99.88521%99.90570%
Legacy 10G Backbone: I2-HOUS-KANS-10GE-05561 1 hr 22 min0 hr 0 min 99.18651%100.00000% 99.88521%99.90570%
Legacy 10G Backbone: I2-HOUS-KANS-LAG-08934 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
Legacy 10G Backbone: I2-HOUS-LOSA-10GE-05559 0 hr 46 min0 hr 0 min 99.54365%100.00000% 99.43211%99.59396%
Legacy 10G Backbone: I2-HOUS-LOSA-10GE-05581 0 hr 46 min0 hr 0 min 99.54365%100.00000% 99.43173%99.59358%
Legacy 10G Backbone: I2-KANS-CHIC-10GE-05781 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
Legacy 10G Backbone: I2-KANS-SALT-10GE-05138 1 hr 17 min0 hr 0 min 99.23611%100.00000% 99.78085%99.90930%
Legacy 10G Backbone: I2-KANS-SALT-10GE-05566 1 hr 17 min0 hr 0 min 99.23611%100.00000% 99.78104%99.90949%
Legacy 10G Backbone: I2-KANS-SALT-LAG-08935 1 hr 17 min0 hr 0 min 99.23611%100.00000% 99.82942%99.90930%
Legacy 10G Backbone: I2-LOSA-SALT-10GE-05563 0 hr 41 min0 hr 0 min 99.59325%100.00000% 99.74917%99.80666%
Legacy 10G Backbone: I2-LOSA-SALT-10GE-05571 0 hr 41 min0 hr 0 min 99.59325%100.00000% 99.75239%99.80666%
Legacy 10G Backbone: I2-LOSA-SEAT-10GE-05564 0 hr 19 min0 hr 0 min 99.81151%100.00000% 99.73418%99.82297%
Legacy 10G Backbone: I2-LOSA-SEAT-10GE-05572 0 hr 19 min0 hr 0 min 99.81151%100.00000% 99.73418%99.82620%
Legacy 10G Backbone: I2-NEWY32AOA-WASH-10GE-07612 0 hr 10 min0 hr 0 min 99.90079%100.00000% 99.84233%99.90228%
Legacy 10G Backbone: I2-NEWY32AOA-WASH-10GE-07613 0 hr 10 min0 hr 0 min 99.90079%100.00000% 99.76985%99.82980%
Legacy 10G Backbone: I2-SALT-SEAT-10GE-05565 0 hr 30 min0 hr 0 min 99.70238%100.00000% 99.96034%99.98596%
Legacy 10G Backbone: I2-SALT-SEAT-10GE-05573 0 hr 30 min0 hr 0 min 99.70238%100.00000% 99.97382%99.99943%

Aggregate All Internet2 IP Backbone Circuits 28 hr 44 min28 hr 44 min 99.64368%100.00000% 99.87581%99.92393%


NOTE: Numbers in italics represent downtime with scheduled maintenance excluded. Internet2's definition of scheduled maintenance is any anticipated outage announced at least 72 hours in advance of the outage start time.

* Internet2 IP 100G Backbone Circuits Ride over AL2S for connectivity.



Upcoming Scheduled Maintenances

Ticket No.:22091:45
Subject:Maintenance - I2 IP Backbone ATLA-CHIC
Affected:Backbone ATLA-CHIC
Start Time:Friday, January 3, 2014, 10:00 AM (1000) UTC
End Time:Friday, January 3, 2014, 10:30 AM (1030) UTC
Description: IP Backbone ATLA-CHIC is expected to remain available while Internet2 Engineers adjust routing metrics, increasing the routing preference for traffic between Internet2 IP core nodes to use AL2S. No outage is expected, however, the entire maintenance window is reserved.

Ticket No.:22089:45
Subject:Maintenance - I2 IP Backbone LOSA-SALT
Affected:Backbone LOSA-SALT
Start Time:Friday, January 3, 2014, 10:30 AM (1030) UTC
End Time:Friday, January 3, 2014, 11:00 AM (1100) UTC
Description: IP Backbone LOSA-SALT is expected to remain available while Internet2 Engineers adjust routing metrics, increasing the routing preference for traffic between Internet2 IP core nodes to use AL2S. No outage is expected, however, the entire maintenance window is reserved.

Ticket No.:22090:45
Subject:Maintenance - I2 IP Backbone CLEV-NEWY
Affected:Backbone CLEV-NEWY
Start Time:Monday, January 6, 2014, 10:00 AM (1000) UTC
End Time:Monday, January 6, 2014, 10:30 AM (1030) UTC
Description: IP Backbone CLEV-NEWY is expected to remain available while Internet2 Engineers adjust routing metrics, increasing the routing preference for traffic between Internet2 IP core nodes to use AL2S. No outage is expected, however, the entire maintenance window is reserved.

Ticket No.:22092:45
Subject:Maintenance - I2 IP Backbone KANS-SALT
Affected:Backbone KANS-SALT
Start Time:Monday, January 6, 2014, 10:30 AM (1030) UTC
End Time:Monday, January 6, 2014, 11:00 AM (1100) UTC
Description: IP Backbone KANS-SALT is expected to remain available while Internet2 Engineers adjust routing metrics, increasing the routing preference for traffic between Internet2 IP core nodes to use AL2S. No outage is expected, however, the entire maintenance window is reserved.

Ticket No.:22047:45
Subject:Maintenance - I2 IP Backbones NEWY-WASH and ATLA-WASH
Affected:Backbones NEWY-WASH and ATLA-WASH
Connector OARnet (WASH)
Connector 3ROX (WASH)
Connector MAGPI (WASH)
Start Time:Tuesday, January 7, 2014, 4:00 AM (0400) UTC
End Time:Tuesday, January 7, 2014, 8:00 AM (0800) UTC
Description: The above listed will be unavailable while Level(3) Communications performs maintenance between Mclean, VA and Millersville, MD. Outages are expected throughout the maintenance window. IP traffic will re-route during this maintenance.

Ticket No.:22031:45
Subject:Maintenance 1 of 3 - I2 IP Backbone ATLA-HOUS
Affected:Backbone ATLA-HOUS
Start Time:Wednesday, January 8, 2014, 5:00 AM (0500) UTC
End Time:Wednesday, January 8, 2014, 11:00 AM (1100) UTC
Description: Backbone ATLA-HOUS will be unavailable while vendor Engineers perform circuit maintenance in Jacksonville, Florida. Outages are expected, and the entire window is reserved. IP traffic will re-route during this maintenance. This is the first of three maintenance windows.

Ticket No.:22032:45
Subject:Maintenance 2 of 3 - I2 IP Backbone ATLA-HOUS
Affected:Backbone ATLA-HOUS
Start Time:Thursday, January 9, 2014, 5:00 AM (0500) UTC
End Time:Thursday, January 9, 2014, 11:00 AM (1100) UTC
Description: Backbone ATLA-HOUS will be unavailable while vendor Engineers perform circuit maintenance in Jacksonville, Florida. Outages are expected, and the entire window is reserved. IP traffic will re-route during this maintenance. This is the second of three maintenance windows.

Ticket No.:22093:45
Subject:Maintenance - I2 IP Backbone NEWY-WASH
Affected:Backbone NEWY-WASH
Start Time:Thursday, January 9, 2014, 10:00 AM (1000) UTC
End Time:Thursday, January 9, 2014, 10:30 AM (1030) UTC
Description: IP Backbone NEWY-WASH is expected to remain available while Internet2 Engineers adjust routing metrics, increasing the routing preference for traffic between Internet2 IP core nodes to use AL2S. No outage is expected, however, the entire maintenance window is reserved.

Ticket No.:22094:45
Subject:Maintenance - I2 IP Backbone ATLA-HOUS
Affected:Backbone ATLA-HOUS
Start Time:Thursday, January 9, 2014, 10:30 AM (1030) UTC
End Time:Thursday, January 9, 2014, 11:00 AM (1100) UTC
Description: IP Backbone ATLA-HOUS is expected to remain available while Internet2 Engineers adjust routing metrics, increasing the routing preference for traffic between Internet2 IP core nodes to use AL2S. No outage is expected, however, the entire maintenance window is reserved.

Ticket No.:21972:45
Subject:Maintenance - I2 IP Backbone HOUS-KANS
Affected:Backbone HOUS-KANS
Net+ Peer Dell (KANS)
Connector LEARN (KANS)*
Start Time:Friday, January 10, 2014, 6:01 AM (0601) UTC
End Time:Friday, January 10, 2014, 12:00 PM (1200) UTC
Description: Backbone HOUS-KANS will be unavailable while Level(3) Communications implement scheduled maintenance to splice backbone cable as mandated by the Oklahoma Department of Transportation due road widening project. An outage is expected, and the entire window is reserved. * indicates connector is multihomed and therefore should remain available. Backbone traffic will re-route during this maintenance.

Ticket No.:22033:45
Subject:Maintenance 3 of 3 - I2 IP Backbone ATLA-HOUS
Affected:Backbone ATLA-HOUS
Start Time:Saturday, January 11, 2014, 5:00 AM (0500) UTC
End Time:Saturday, January 11, 2014, 11:00 AM (1100) UTC
Description: Backbone ATLA-HOUS will be unavailable while vendor Engineers perform circuit maintenance in Jacksonville, Florida. Outages are expected, and the entire window is reserved. IP traffic will re-route during this maintenance. This is the third of three maintenance windows.

Ticket No.:22085:45
Subject:Maintenance - I2 IP Node oob.hous.net.internet2.edu
Affected:oob.hous.net.internet2.edu
Start Time:Tuesday, January 21, 2014, 6:00 AM (0600) UTC
End Time:Tuesday, January 21, 2014, 12:00 PM (1200) UTC
Description: The OOB Circuit for the POP in Houston, TX will be unavailable while Level(3) Engineers perform maintenance. Outages are expected throughout the window and the entire window is reserved.