Generated Report for Unified

Unified - Internet2 IP Network Operations Report
05/04/2015 - 05/10/2015

Unified - Internet2 IP Network Operations Center
at Indiana University

New Connections

None

Upgrades

None

New Services

None

Unscheduled Outages Summary

Ticket Number	Customer Impact	Network Impact	*Title*	Outage Type	Source Of Impact	*Start Time (UTC)*	*End Time (UTC)*

3707	4-Normal		Outage Resolved - I2 AL2S Backbone ASHB-WASH	Hardware	Internal	05/04/2015 1:03 PM	05/04/2015 3:34 PM
3717	4-Normal		Stability - I2 AL2S Backbone ATLA-JCSN	Investigating	Vendor	05/05/2015 8:07 AM	05/05/2015 9:28 AM
3736	4-Normal		Brief Outage - I2 AL2S Backbone PHIL-WASH	Investigating	Undetermined	05/06/2015 12:48 AM	05/06/2015 12:49 AM
3763	2-High	1-Critical	Outage Resolved - I2 AL2S Backbone PHIL-WASH	Software	Vendor	05/08/2015 9:23 AM	05/08/2015 12:39 PM
3776	4-Normal	1-Critical	Brief Outage Resolved - I2 Optical DWDM System KANS-SAIN	Hardware	Vendor	05/09/2015 6:09 PM	05/09/2015 6:10 PM

Unscheduled Outages Detail

Ticket No.:	3707:144
Subject:	Outage Resolved - I2 AL2S Backbone ASHB-WASH
Source Of Impact:	Internal
Affected:	AL2S Backbone ASHB-WASH IP Backbones CLEV-WASH and ATLA-WASH
Start Time:	Monday, May 4, 2015, 1:03 PM (1303) UTC
End Time:	Monday, May 4, 2015, 3:34 PM (1534) UTC
Description:	The above listed asset is now available. Internet2 Engineers have confirmed that the outage was due to a line card failure and worked with Level 3 Technicians to swap to a spare card. No further outages are expected. Workgroup administrators receive notifications from OESS regarding circuits re-routed on the backbone.

Ticket No.:	3717:144
Subject:	Stability - I2 AL2S Backbone ATLA-JCSN
Source Of Impact:	Vendor
Affected:	Optical Member ESNet AL2S Backbone ATLA-JCSN IP Backbone ATLA-HOUH
Start Time:	Tuesday, May 5, 2015, 8:07 AM (0807) UTC
End Time:	Tuesday, May 5, 2015, 9:28 AM (0928) UTC
Description:	The above listed assets are now stable. Level(3) Engineers are investigating a possible fiber issue between Memphis, TN and Tuscon, MS; however, the reason for the instability is pending. Workgroup administrators receive notifications from OESS regarding circuits re-routed on the backbone.

Ticket No.:	3736:144
Subject:	Brief Outage - I2 AL2S Backbone PHIL-WASH
Source Of Impact:	Undetermined
Affected:	AL2S Backbone PHIL-WASH IP Backbone ATLA-NEWY32AOA IP Backbone NEWY32AOA-WASH
Start Time:	Wednesday, May 6, 2015, 12:48 AM (0048) UTC
End Time:	Wednesday, May 6, 2015, 12:49 AM (0049) UTC
Description:	The above listed asset is available after a brief outage. Internet2 Engineers are investigating the cause. The reason for the outage is pending. Workgroup administrators receive notifications from OESS regarding circuits re-routed on the backbone.

Ticket No.:	3763:144
Subject:	Outage Resolved - I2 AL2S Backbone PHIL-WASH
Source Of Impact:	Vendor
Affected:	AL2S Backbone PHIL-WASH IP Backbone ATLA-NEWY32AOA IP Backbone NEWY32AOA-WASH
Start Time:	Friday, May 8, 2015, 9:23 AM (0923) UTC
End Time:	Friday, May 8, 2015, 12:39 PM (1239) UTC
Description:	The above listed assets are available. Internet2 Engineers addressed a firmware issue on the Al2S switch in Washington, DC. Workgroup administrators received notifications from OESS regarding circuits re-routed on the backbone. IP Backbones re-routed during the outage. No further outages are expected.

Ticket No.:	3776:144
Subject:	Brief Outage Resolved - I2 Optical DWDM System KANS-SAIN
Source Of Impact:	Vendor
Affected:	DWDM System KANS-SAIN AL2S Backbone(s) CHIC-KANS IP Backbone(s) CHIC-KANS Optical Member ESnet Optical Member NOAA N-Wave
Start Time:	Saturday, May 9, 2015, 6:09 PM (1809) UTC
End Time:	Saturday, May 9, 2015, 6:10 PM (1810) UTC
Description:	DWDM System KANS-SAIN was briefly unavailable to the community. Level (3) Communications Engineers have determined the outage was caused by a line card failure. A replacement card was installed. No further outages are expected.

Scheduled Maintenances Summary

Ticket Number	Customer Impact	Network Impact	*Title*	Maintenance Type	Source Of Impact	*Start Time (UTC)*	*End Time (UTC)*

3692	3-Elevated		Maintenance Completed - I2 AL2S Backbone RALE-WASH	Hardware	Internal	05/06/2015 5:22 AM	05/06/2015 5:24 AM
3777	3-Elevated	1-Critical	Emergency Maintenance Completed - I2 Optical DWDM System KANS-SAIN	Circuit	Vendor	05/10/2015 3:04 AM	05/10/2015 3:15 AM

Scheduled Maintenances Detail

Ticket No.:	3692:144
Subject:	Maintenance Completed - I2 AL2S Backbone RALE-WASH
Source Of Impact:	Internal
Affected:	AL2S Backbone RALE-WASH IP Backbones ATLA-CLEV, ATLA-NEWY32AOA, and ATLA-WASH
Start Time:	Wednesday, May 6, 2015, 5:22 AM (0522) UTC
End Time:	Wednesday, May 6, 2015, 5:24 AM (0524) UTC
Description:	The above listed assets were unavailable while Level(3) Engineers performed hardware maintenance. A brief outage was observed. Backbone traffic re-routed during this maintenance. Workgroup administrators received notifications from OESS regarding circuits re-routed on the backbone. Maintenance was not completed successfully and will be rescheduled at a later date.

Ticket No.:	3777:144
Subject:	Emergency Maintenance Completed - I2 Optical DWDM System KANS-SAIN
Source Of Impact:	Vendor
Affected:	DWDM System KANS-SAIN AL2S Backbone(s) CHIC-KANS IP Backbone(s) CHIC-KANS Optical Member ESnet Optical Member NOAA N-Wave
Start Time:	Sunday, May 10, 2015, 3:04 AM (0304) UTC
End Time:	Sunday, May 10, 2015, 3:15 AM (0315) UTC
Description:	DWDM System KANS-SAIN was unavailable while Level(3) Communications performed emergency maintenance in Elm, MO. A brief outage was observed. Maintenance has been completed. No further outages are expected.

Downtime and Availability

Unified - Internet2 IP Core Nodes	Down Time		Reporting Period Availability		52 Week Availability
ATLA MX960	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%
CHIC MX960	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%
CLEV MX960	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%
HOUS MX960	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%
KANS MX960	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%
LOSA MX960	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%
NEWY MX960	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%
SALT MX960	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%
SEAT MX960	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%
WASH MX960	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%
rtsw0.sing1.net.internet2.edu	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%
rtsw.sing2.net.internet2.edu	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%

Aggregate Unified - Internet2 IP Core Nodes	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%

NOTE: Numbers in italics represent downtime with scheduled maintenance excluded. Internet2's definition of scheduled maintenance is any anticipated outage announced at least 72 hours in advance of the outage start time.

Unified - Internet2 IP Backbone Circuits	Down Time		Reporting Period Availability		52 Week Availability
I2-ATLA-CLEV-VLAN-14832	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%
I2-ATLA-CLEV-VLAN-14837	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%
I2-ATLA-NEWY32AOA-VLAN-14833	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%
I2-ATLA-NEWY32AOA-VLAN-14838	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%
I2-CHIC-NEWY32AOA-VLAN-14831	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%
I2-CHIC-NEWY32AOA-VLAN-14834	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%
R&E 100G Backbone: I2-ATLA-CHIC-VLAN-09181	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.53134%	99.86908%
R&E 100G Backbone: I2-ATLA-HOUS-VLAN-09182	1 hr 21 min	1 hr 21 min	99.19643%	99.19643%	99.59339%	99.98065%
R&E 100G Backbone: I2-ATLA-WASH-VLAN-09179	2 hr 33 min	2 hr 31 min	98.48214%	98.50198%	99.89488%	99.96812%
R&E 100G Backbone: I2-CHIC-CLEV-VLAN-09180	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.92240%	99.99924%
R&E 100G Backbone: I2-CHIC-KANS-VLAN-09183	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.94934%	99.99848%
R&E 100G Backbone: I2-CLEV-NEWY32AOA-VLAN-09177	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.94213%	99.99469%
R&E 100G Backbone: I2-CLEV-WASH-VLAN-09178	2 hr 31 min	2 hr 31 min	98.50198%	98.50198%	99.91462%	99.94782%
R&E 100G Backbone: I2-HOUS-KANS-VLAN-09184	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.89337%	99.98691%
R&E 100G Backbone: I2-HOUS-LOSA-VLAN-09189	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.62583%	99.75106%
R&E 100G Backbone: I2-KANS-SALT-VLAN-09185	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.89375%	99.98975%
R&E 10G Backbone: I2-LOSA-PAIX-VLAN-23105	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%
Commodity 10G Backbone: I2-LOSA-PAIX-VLAN-23106	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%
R&E 100G Backbone: I2-LOSA-SALT-VLAN-09188	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.98330%	99.99905%
R&E 100G Backbone: I2-LOSA-SEAT-VLAN-09187	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.97647%	99.99924%
R&E 100G Backbone: I2-LOSA-SEAT-VLAN-09328	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.97647%	99.99924%
R&E 100G Backbone: I2-LOSA-WILC-VLAN-23103	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%
Commodity 100G Backbone: I2-LOSA-WILC-VLAN-23104	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%
R&E 100G Backbone: I2-NEWY32AOA-WASH-VLAN-09176	3 hr 17 min	3 hr 17 min	98.04563%	98.04563%	99.94213%	99.95997%
R&E 100G Backbone: I2-SALT-SEAT-VLAN-09186	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.99658%	99.99962%
R&E 100G Backbone: I2-SALT-SEAT-VLAN-09327	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.99658%	99.99962%
R&E 100G Backbone: I2-WASH-ATLA-VLAN-09179	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%
R&E 100G Backbone: I2-ATLA-WASH-VLAN-09318	2 hr 33 min	2 hr 31 min	98.48214%	98.50198%	99.89488%	99.96812%
Legacy 10G Backbone: I2-ATLA-CHIC-10GE-05419	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.71634%	99.97287%
Legacy 10G Backbone: I2-ATLA-HOUS-10GE-05423	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.20993%	99.56208%
Legacy 10G Backbone: I2-ATLA-WASH-10GE-05251	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.93871%	100.00000%
Legacy 10G Backbone: I2-ATLA-WASH-10GE-09142	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.93739%	99.99867%
Legacy 10G Backbone: I2-CHIC-CLEV-10GE-08891	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.94023%	99.99981%
Legacy 10G Backbone: I2-CHIC-KANS-10GE-05781	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%
Legacy 10G Backbone: I2-CHIC-KANS-10GE-05976	0 hr 12 min	0 hr 1 min	99.88095%	99.99008%	99.97989%	99.99962%
Legacy 10G Backbone: I2-CLEV-NEWY32AOA-10GE-08894	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.96376%	99.99962%
Legacy 10G Backbone: I2-CLEV-WASH-10GE-08889	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.99867%	99.99867%
Legacy 10G Backbone: I2-HOUS-KANS-10GE-05560	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.94346%	99.99753%
Legacy 10G Backbone: I2-HOUS-KANS-LAG-08934	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%
Legacy 10G Backbone: I2-HOUS-LOSA-10GE-05559	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.75789%	99.82772%
Legacy 10G Backbone: I2-KANS-SALT-10GE-05566	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.92505%	99.99924%
Legacy 10G Backbone: I2-KANS-SALT-LAG-08935	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.92505%	99.99924%
Legacy 10G Backbone: I2-LOSA-PAIX-10GE-05853	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.99621%	100.00000%
Legacy 10G Backbone: I2-LOSA-SALT-10GE-05563	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.99013%	100.00000%
Legacy 10G Backbone: I2-LOSA-SEAT-10GE-05564	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.97818%	99.99905%
Legacy 10G Backbone: I2-LOSA-SEAT-10GE-05572	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.99412%	99.99981%
Legacy 100G Backbone: I2-LOSA-WILC-100GE-23893	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%
Legacy 10G Backbone: I2-NEWY32AOA-WASH-10GE-07612	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.99696%	99.99810%
Legacy 10G Backbone: I2-SALT-SEAT-10GE-05565	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.99981%	99.99981%

Aggregate All Unified - Internet2 IP Backbone Circuits	12 hr 27 min	0 hr 15 min	99.84876%	99.85180%	99.92080%	99.97474%

Upcoming Scheduled Maintenances

Ticket No.:	3797:144
Subject:	Maintenance - I2 Optical DWDM System ALBA-BUFF
Affected:	DWDM System ALBA-BUFF AL2S Backbone(s) ALBA-CLEV IP Backbone(s) CLEV-NEWY32AOA
Start Time:	Friday, May 15, 2015, 4:00 AM (0400) UTC
End Time:	Friday, May 15, 2015, 6:00 AM (0600) UTC
Description:	DWDM System ALBA-BUFF will be unavailable while Level(3) Communications replaces a line interface module in Utica, NY. An outage of 2 hours is expected. The entire window is reserved.

Ticket No.:	3686:144
Subject:	Maintenance 2 of 2 - I2 Optical DWDM System DALL-HOUS
Affected:	DWDM System DALL-HOUS AL2S Backbone(s) DALL-HOUS IP Backbone(s) HOUS-KANS Connector(s)/Peer(s)
Start Time:	Friday, May 15, 2015, 5:00 AM (0500) UTC
End Time:	Friday, May 15, 2015, 11:00 AM (1100) UTC
Description:	DWDM System DALL-HOUS will be unavailable while Level(3) Communications performs cable relocation in Montgomery, TX. An outage of 2 hours is expected. The entire window is reserved. This is an alternate window, to be used if necessary

Ticket No.:	3792:144
Subject:	Maintenance - I2 AL2S Backbone RALE-WASH
Affected:	AL2S Backbone RALE-WASH IP Backbone ATLA-CLEV IP Backbone ATLA-NEWY32AOA IP Backbone ATLA-WASH
Start Time:	Friday, May 15, 2015, 6:00 AM (0600) UTC
End Time:	Friday, May 15, 2015, 8:00 AM (0800) UTC
Description:	The above listed assets will be unavailable while Level(3) Engineers perform hardware maintenance. An outage is expected. The entire window is reserved. Backbone traffic will re-route during this maintenance. Workgroup administrators will receive notifications from OESS regarding circuits re-routed on the backbone.

Ticket No.:	3813:144
Subject:	Maintenance - I2 AL2S Core Node SEAT
Affected:	AL2S Core Node SEAT AL2S Backbones MISS2-SEAT and PORT-SEAT IP Backbones LOSA-SEAT and SALT-SEAT
Start Time:	Thursday, May 21, 2015, 10:00 AM (1000) UTC
End Time:	Thursday, May 21, 2015, 12:00 PM (1200) UTC
Description:	The above listed assets will be unavailable while Internet2 Engineers perform maintenance to replace the routing engines. An outage is expected and the entire window is reserved. All connections to the Core Node will be affected.

Ticket No.:	3773:144
Subject:	Maintenance - I2 Optical Member ESnet
Affected:	I2 Optical Member ESnet
Start Time:	Friday, May 22, 2015, 4:00 AM (0400) UTC
End Time:	Friday, May 22, 2015, 9:00 AM (0900) UTC
Description:	Member ESnet's Optical circuit ESNET-CHAT-ORNL-100GE-06347 may be unavailable while ESnet Engineers perform maintenance. An outage of up to 4 hours and 30 minutes is expected; however the entire maintenance window has been reserved.

Ticket No.:	3827:144
Subject:	Maintenance - I2 Various Optical Circuits (Northern Tier)
Affected:	Optical Member CA*Net/CANARIE AL2S Backbone MINN-MISS2 Net+ Provider Code42 (CHIC)
Start Time:	Sunday, May 31, 2015, 1:00 PM (1300) UTC
End Time:	Sunday, May 31, 2015, 7:00 PM (1900) UTC
Description:	The above listed optical circuits will be unavailable while Zayo Engineers perform maintenance. An outage of 3+ hours is expected, however the entire maintenance window has been reserved.