Unified - Internet2 IP Network Operations Report
05/18/2015 - 05/24/2015

Unified - Internet2 IP Network Operations Center
at Indiana University

New Connections

None


Upgrades

None


New Services

None


Unscheduled Outages Summary

Ticket Number Customer Impact Network Impact Title Outage Type Source Of Impact Start Time (UTC) End Time (UTC)

38684-Normal Outage Resolved - I2 AL2S Backbone ASHB-WASH Unannounced MaintenanceVendor 05/18/2015 10:05 PM 05/18/2015 10:06 PM
05/18/2015 10:28 PM 05/18/2015 10:31 PM
38934-Normal 1-Critical Brief Outage Resolved - I2 Optical DWDM System EUGE-SCAR Unannounced MaintenanceVendor 05/19/2015 5:28 PM 05/19/2015 5:29 PM
39244-Normal 1-Critical Brief Outage Resolved- I2 Optical DWDM System RALE-WASH Human ErrorVendor 05/21/2015 1:56 AM 05/21/2015 1:58 AM

Unscheduled Outages Detail


Ticket No.:3868:144
Subject:Outage Resolved - I2 AL2S Backbone ASHB-WASH
Source Of Impact:Vendor
Affected:---AL2S---
Backbone ASHB-WASH
---IP---
Backbone CLEV-WASH
Backbone NEWY32AOA-WASH
Backbone CHIC-WASH
Backbone ATLA-CLEV
Start Time:Monday, May 18, 2015, 10:05 PM (2205) UTC
End Time:Monday, May 18, 2015, 10:06 PM (2206) UTC
Start Time:Monday, May 18, 2015, 10:28 PM (2228) UTC
End Time:Monday, May 18, 2015, 10:31 PM (2231) UTC
Description:The above listed assets were briefly unavailable. The reason for the outage was unannounced maintenance. No further outages are expected.

Ticket No.:3893:144
Subject:Brief Outage Resolved - I2 Optical DWDM System EUGE-SCAR
Source Of Impact:Vendor
Affected:DWDM System EUGE-SACR
AL2S Backbone PORT-SUNN
IP Backbone LOSA-SEAT
Optical Member CENIC
Start Time:Tuesday, May 19, 2015, 5:28 PM (1728) UTC
End Time:Tuesday, May 19, 2015, 5:29 PM (1729) UTC
Description:DWDM System EUGE-SCAR was briefly unavailable to the community. Level (3) Communications Engineers were contacted and determined the reason for the outage was unannounced maintenance. Workgroup administrators received notifications from OESS regarding circuits re-routed on the backbone. IP traffic re-routed during the event.

Ticket No.:3924:144
Subject:Brief Outage Resolved- I2 Optical DWDM System RALE-WASH
Source Of Impact:Vendor
Affected:DWDM System RALE-WASH
AL2S Backbone RALE-WASH, ATLA-WASH
IP Backbones ATLA-CLEV, ATLA-NEWY32AOA, ATLA-WASH
Start Time:Thursday, May 21, 2015, 1:56 AM (0156) UTC
End Time:Thursday, May 21, 2015, 1:58 AM (0158) UTC
Description:DWDM System RALE-WASH is available after a brief outage to the community. Level (3) Communications Engineers have investigated and determined this was caused by a fiber bump.

Scheduled Maintenances Summary

Ticket Number Customer Impact Network Impact Title Maintenance Type Source Of Impact Start Time (UTC) End Time (UTC)

3813 3-Elevated Maintenance Completed - I2 AL2S Core Node SEAT Hardware Internal05/21/2015 10:23 AM 05/21/2015 10:46 AM
3831 3-Elevated Maintenance Completed - I2 AL2S Core Node EQCH HardwareInternal 05/22/2015 5:00 AM 05/22/2015 5:09 AM
3974 3-Elevated Emergency Maintenance Completed - I2 AL2S Core Node CLEV Circuit Internal05/22/2015 7:00 PM 05/22/2015 7:06 PM

Scheduled Maintenances Detail

Ticket No.:3813:144
Subject:Maintenance Completed - I2 AL2S Core Node SEAT
Source Of Impact:Internal
Affected:AL2S Core Node SEAT
AL2S Backbones MISS2-SEAT and PORT-SEAT
IP Backbones LOSA-SEAT and SALT-SEAT
Start Time:Thursday, May 21, 2015, 10:23 AM (1023) UTC
End Time:Thursday, May 21, 2015, 10:46 AM (1046) UTC
Description: The above listed assets were unavailable while Internet2 Engineers performed maintenance to replace the routing engines. All connections to the Core Node were affected. The maintenance has been completed. No further outages are expected.

Ticket No.:3831:144
Subject:Maintenance Completed - I2 AL2S Core Node EQCH
Source Of Impact:Internal
Affected:AL2S Core Node EQCH
AL2S Backbone EQCH-MINN
AL2S Backbone EQCH-STAR
AL2S Backbone CLEV-EQCH
IP Backbone CHIC-CLEV
Net+ Provider Code42 (CHIC)
Start Time:Friday, May 22, 2015, 5:00 AM (0500) UTC
End Time:Friday, May 22, 2015, 5:09 AM (0509) UTC
Description: The above listed assets are now available after Internet2 Engineers completed hardware maintenance. AL2S connections to the node will be affected. AL2S workgroup administrators received notifications from OESS regarding circuits re-routed on the backbone. IP Backbones re-routed during the event.

Ticket No.:3974:144
Subject:Emergency Maintenance Completed - I2 AL2S Core Node CLEV
Source Of Impact:Internal
Affected:AL2S Core Node CLEV
IP Backbone ATLA-CLEV, CHIC-CLEV, CLEV-NEWY32AOA, & CLEV-WASH
Start Time:Friday, May 22, 2015, 7:00 PM (1900) UTC
End Time:Friday, May 22, 2015, 7:06 PM (1906) UTC
Description: The above listed core node was unavailable while engineers performed an emergency maintenance. Engineers had discovered alarms for what indicated a packet loss for traffic through this node. To resolve this potential service degradation, engineers swapped a card at the site. This maintenance has been completed.


Downtime and Availability



Unified - Internet2 IP Core NodesDown TimeReporting Period Availability52 Week Availability
ATLA MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000% 100.00000%
CHIC MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000% 100.00000%
CLEV MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000% 100.00000%
HOUS MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000% 100.00000%
KANS MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000% 100.00000%
LOSA MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000% 100.00000%
NEWY MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000% 100.00000%
SALT MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000% 100.00000%
SEAT MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000% 100.00000%
WASH MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000% 100.00000%
rtsw0.sing1.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000% 100.00000%
rtsw.sing2.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000% 100.00000%

Aggregate Unified - Internet2 IP Core Nodes0 hr 0 min0 hr 0 min100.00000%100.00000%100.00000%100.00000%

NOTE: Numbers in italics represent downtime with scheduled maintenance excluded. Internet2's definition of scheduled maintenance is any anticipated outage announced at least 72 hours in advance of the outage start time.

Unified - Internet2 IP Backbone CircuitsDown Time Reporting Period Availability52 Week Availability
I2-ATLA-CLEV-VLAN-14832 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
I2-ATLA-CLEV-VLAN-14837 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
I2-ATLA-NEWY32AOA-VLAN-14833 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
I2-ATLA-NEWY32AOA-VLAN-14838 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
I2-CHIC-NEWY32AOA-VLAN-14831 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
I2-CHIC-NEWY32AOA-VLAN-14834 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
R&E 100G Backbone: I2-ATLA-CHIC-VLAN-09181 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.53134%99.86908%
R&E 100G Backbone: I2-ATLA-HOUS-VLAN-09182 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.60876%99.99602%
R&E 100G Backbone: I2-ATLA-WASH-VLAN-09179 0 hr 2 min0 hr 2 min 99.98016%99.98016% 99.88654%99.96774%
R&E 100G Backbone: I2-CHIC-CLEV-VLAN-09180 0 hr 13 min0 hr 0 min 99.87103%100.00000% 99.91993%99.99924%
R&E 100G Backbone: I2-CHIC-KANS-VLAN-09183 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.94896%99.99848%
R&E 100G Backbone: I2-CLEV-NEWY32AOA-VLAN-09177 0 hr 5 min0 hr 0 min 99.95040%100.00000% 99.94061%99.99469%
R&E 100G Backbone: I2-CLEV-WASH-VLAN-09178 0 hr 9 min0 hr 4 min 99.91071%99.96032% 99.91291%99.94706%
R&E 100G Backbone: I2-HOUS-KANS-VLAN-09184 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.84954%99.98672%
R&E 100G Backbone: I2-HOUS-LOSA-VLAN-09189 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.62583%99.75106%
R&E 100G Backbone: I2-KANS-SALT-VLAN-09185 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.89375%99.98975%
R&E 10G Backbone: I2-LOSA-PAIX-VLAN-23105 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
Commodity 10G Backbone: I2-LOSA-PAIX-VLAN-23106 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
R&E 100G Backbone: I2-LOSA-SALT-VLAN-09188 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.98330%99.99905%
R&E 100G Backbone: I2-LOSA-SEAT-VLAN-09187 0 hr 24 min0 hr 1 min 99.76190%99.99008% 99.97192%99.99905%
R&E 100G Backbone: I2-LOSA-SEAT-VLAN-09328 0 hr 24 min0 hr 1 min 99.76190%99.99008% 99.97192%99.99905%
R&E 100G Backbone: I2-LOSA-WILC-VLAN-23103 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
Commodity 100G Backbone: I2-LOSA-WILC-VLAN-23104 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
R&E 100G Backbone: I2-NEWY32AOA-WASH-VLAN-09176 0 hr 4 min0 hr 4 min 99.96032%99.96032% 99.94137%99.95921%
R&E 100G Backbone: I2-SALT-SEAT-VLAN-09186 0 hr 23 min0 hr 0 min 99.77183%100.00000% 99.99222%99.99962%
R&E 100G Backbone: I2-SALT-SEAT-VLAN-09327 0 hr 23 min0 hr 0 min 99.77183%100.00000% 99.99222%99.99962%
R&E 100G Backbone: I2-WASH-ATLA-VLAN-09179 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
R&E 100G Backbone: I2-ATLA-WASH-VLAN-09318 0 hr 2 min0 hr 2 min 99.98016%99.98016% 99.88654%99.96774%
Legacy 10G Backbone: I2-ATLA-CHIC-10GE-05419 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.71634%99.97287%
Legacy 10G Backbone: I2-ATLA-HOUS-10GE-05423 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.20993%99.56208%
Legacy 10G Backbone: I2-ATLA-WASH-10GE-05251 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.93871%100.00000%
Legacy 10G Backbone: I2-ATLA-WASH-10GE-09142 0 hr 2 min0 hr 2 min 99.98016%99.98016% 99.93701%99.99829%
Legacy 10G Backbone: I2-CHIC-CLEV-10GE-08891 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.94023%99.99981%
Legacy 10G Backbone: I2-CHIC-KANS-10GE-05781 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
Legacy 10G Backbone: I2-CHIC-KANS-10GE-05976 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.97989%99.99962%
Legacy 10G Backbone: I2-CLEV-NEWY32AOA-10GE-08894 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.96376%99.99962%
Legacy 10G Backbone: I2-CLEV-WASH-10GE-08889 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99867%99.99867%
Legacy 10G Backbone: I2-HOUS-KANS-10GE-05560 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.89982%99.99753%
Legacy 10G Backbone: I2-HOUS-KANS-LAG-08934 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
Legacy 10G Backbone: I2-HOUS-LOSA-10GE-05559 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.75789%99.82772%
Legacy 10G Backbone: I2-KANS-SALT-10GE-05566 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.92505%99.99924%
Legacy 10G Backbone: I2-KANS-SALT-LAG-08935 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.92505%99.99924%
Legacy 10G Backbone: I2-LOSA-PAIX-10GE-05853 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99621%100.00000%
Legacy 10G Backbone: I2-LOSA-SALT-10GE-05563 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99013%100.00000%
Legacy 10G Backbone: I2-LOSA-SEAT-10GE-05564 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.97818%99.99905%
Legacy 10G Backbone: I2-LOSA-SEAT-10GE-05572 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99412%99.99981%
Legacy 100G Backbone: I2-LOSA-WILC-100GE-23893 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
Legacy 10G Backbone: I2-NEWY32AOA-WASH-10GE-07612 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99696%99.99810%
Legacy 10G Backbone: I2-SALT-SEAT-10GE-05565 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99981%99.99981%

Aggregate All Unified - Internet2 IP Backbone Circuits 2 hr 11 min1 hr 55 min 99.97348%99.99676% 99.91848%99.97499%


NOTE: Numbers in italics represent downtime with scheduled maintenance excluded. Internet2's definition of scheduled maintenance is any anticipated outage announced at least 72 hours in advance of the outage start time.



Upcoming Scheduled Maintenances

Ticket No.:3975:144
Subject:Maintenance 1 of 3 - I2 Various Optical Circuits (Northern Tier)
Affected:AL2S Backbone MINN-MISS2
CA*Net/CANARIE
Net+ Provider Code42
Start Time:Tuesday, June 2, 2015, 5:00 AM (0500) UTC
End Time:Tuesday, June 2, 2015, 10:00 AM (1000) UTC
Description: The above listed assets may be unavailable while Zayo Engineers perform power maintenance. An outage is not expected, however the entire maintenance window has been reserved.

Ticket No.:3976:144
Subject:Maintenance 2 of 3 - I2 Various Optical Circuits (Northern Tier)
Affected:AL2S Backbone MINN-MISS2
CA*Net/CANARIE
Net+ Provider Code42
Start Time:Wednesday, June 3, 2015, 5:00 AM (0500) UTC
End Time:Wednesday, June 3, 2015, 10:00 AM (1000) UTC
Description: The above listed assets may be unavailable while Zayo Engineers perform power maintenance. An outage is not expected, however the entire maintenance window has been reserved.

Ticket No.:4007:144
Subject:Maintenance 1 of 2 - I2 AL2S Backbone SALT-SUNN
Affected:AL2S Backbone SALT-SUNN
Start Time:Wednesday, June 3, 2015, 7:00 AM (0700) UTC
End Time:Wednesday, June 3, 2015, 9:00 AM (0900) UTC
Description: The above listed asset will be unavailable while Internet2 Engineers perform maintenance. An outage of 45 minutes is expected. The entire window is reserved. Backbone traffic will re-route during this maintenance. Workgroup administrators will receive notifications from OESS regarding circuits re-routed on the backbone.

Ticket No.:3977:144
Subject:Maintenance 3 of 3 - I2 Various Optical Circuits (Northern Tier)
Affected:AL2S Backbone MINN-MISS2
CA*Net/CANARIE
Net+ Provider Code42
Start Time:Thursday, June 4, 2015, 5:00 AM (0500) UTC
End Time:Thursday, June 4, 2015, 10:00 AM (1000) UTC
Description: The above listed assets may be unavailable while Zayo Engineers perform power maintenance. An outage is not expected, however the entire maintenance window has been reserved.

Ticket No.:4012:144
Subject:Maintenance 2 of 2 - I2 AL2S Backbone SALT-SUNN
Affected:AL2S Backbone SALT-SUNN
Start Time:Thursday, June 4, 2015, 7:00 AM (0700) UTC
End Time:Thursday, June 4, 2015, 9:00 AM (0900) UTC
Description: The above listed asset will be unavailable while Internet2 Engineers perform maintenance. An outage of 45 minutes is expected. The entire window is reserved. Backbone traffic will re-route during this maintenance. Workgroup administrators will receive notifications from OESS regarding circuits re-routed on the backbone.

Ticket No.:4058:144
Subject:Maintenance - I2 AL2S Participant FLR
Affected:AL2S Participant FLR
Start Time:Thursday, June 4, 2015, 9:00 AM (0900) UTC
End Time:Thursday, June 4, 2015, 10:00 AM (1000) UTC
Description: The above listed assset will be unavailable while FLR Engineers perform hardware maintenance. An outage of 15 minutes is expected. The entire window is reserved.

Ticket No.:4067:144
Subject:Maintenance - I2 AL2S Backbone LOSA-SALT
Affected:AL2S Backbone LOSA-SALT
IP Backbone LOSA-SALT
Start Time:Thursday, June 4, 2015, 10:00 AM (1000) UTC
End Time:Thursday, June 4, 2015, 12:00 PM (1200) UTC
Description: The above listed asset will be unavailable while Internet2 Engineers perform maintenance. An outage of up to 2 hours is expected. The entire window is reserved. Backbone traffic will re-route during this maintenance. Workgroup administrators will receive notifications from OESS regarding circuits re-routed on the backbone.

Ticket No.:4068:144
Subject:Maintenance Correction 1 of 2 - I2 AL2S Core Node Insertion in Las Vegas, NV
Affected:AL2S Core Node LASV
AL2S Backbone LOSA-SALT
IP Backbone LOSA-SALT
Start Time:Tuesday, June 9, 2015, 7:00 AM (0700) UTC
End Time:Tuesday, June 9, 2015, 9:00 AM (0900) UTC
Description: *Please note that the maintenance window has been modified* The above listed assets will be unavailable while Internet2 Engineers perform a core node insertion in Las Vegas. An outage of 45 minutes is expected; however the entire window is reserved. Backbone traffic will re-route during this maintenance. Workgroup administrators will receive notifications from OESS regarding circuits re-routed on the backbone. *This is maintenance window 1 of 2*

Ticket No.:4073:144
Subject:Maintenance 1 of 2 - I2 Optical DWDM System CINC-LOUI
Affected:DWDM System CINC-LOUI
AL2S Backbone(s) ATLA-CHIC
IP Backbone(s) ATLA-CHIC
IP Connector University of Memphis and its Participants
IP Connector Indiana Gigapop and its Participants
Start Time:Wednesday, June 10, 2015, 4:00 AM (0400) UTC
End Time:Wednesday, June 10, 2015, 10:00 AM (1000) UTC
Description: DWDM System CINC-LOUI will be unavailable while Level(3) Communications performs cable relocation in Rexville, IN and Cincinnati, OH. An outage of 30 minutes is expected; however the entire window is reserved.

Ticket No.:4071:144
Subject:Maintenance 2 of 2 - I2 AL2S Core Node Insertion in Las Vegas, NV
Affected:AL2S Core Node LASV
AL2S Backbone LOSA-SALT
IP Backbone LOSA-SALT
Start Time:Wednesday, June 10, 2015, 7:00 AM (0700) UTC
End Time:Wednesday, June 10, 2015, 9:00 AM (0900) UTC
Description: The above listed assets will be unavailable while Internet2 Engineers perform a core node insertion in Las Vegas. An outage of 45 minutes is expected; however the entire window is reserved. Backbone traffic will re-route during this maintenance. Workgroup administrators will receive notifications from OESS regarding circuits re-routed on the backbone. *This is maintenance window 2 of 2*

Ticket No.:4074:144
Subject:Maintenance 2 of 2 - I2 Optical DWDM System CINC-LOUI
Affected:DWDM System CINC-LOUI
AL2S Backbone(s) ATLA-CHIC
IP Backbone(s) ATLA-CHIC
IP Connector University of Memphis and its Participants
IP Connector Indiana Gigapop and its Participants
Start Time:Thursday, June 11, 2015, 4:00 AM (0400) UTC
End Time:Thursday, June 11, 2015, 10:00 AM (1000) UTC
Description: DWDM System CINC-LOUI will be unavailable while Level(3) Communications performs cable relocation in Rexville, IN and Cincinnati, OH. An outage of 30 minutes is expected; however the entire window is reserved. This is an alternate maintenance window, to be used if necessary.

Ticket No.:3989:144
Subject:Maintenance - I2 Various Optical Circuits (Northern Tier)
Affected:AL2S Backbone MINN-MISS2
Net+ Provider Code42 (SEAT)
Start Time:Saturday, June 20, 2015, 1:00 PM (1300) UTC
End Time:Saturday, June 20, 2015, 11:00 PM (2300) UTC
Description: The above listed assets may be unavailable while Zayo Engineers perform circuit maintenance. An outage of up to 3 hours is expected, however the entire maintenance window has been reserved.