Generated Report for Unified

Unified - Internet2 IP Network Operations Report
03/23/2015 - 03/29/2015

Unified - Internet2 IP Network Operations Center
at Indiana University

New Connections

None

Upgrades

None

New Services

None

Unscheduled Outages Summary

Ticket Number	Customer Impact	Network Impact	*Title*	Outage Type	Source Of Impact	*Start Time (UTC)*	*End Time (UTC)*

3199	4-Normal	1-Critical	Brief Outage Resolved - I2 Optical DWDM System CHIC-EQCH	Human Error	Vendor	03/24/2015 5:59 PM	03/24/2015 6:00 PM
3223	4-Normal		Brief Outage Resolved - I2 Optical Various Circuits	Undetermined	Vendor	03/26/2015 5:38 PM	03/26/2015 5:39 PM
3227	4-Normal	1-Critical	Brief Outage - I2 Optical DWDM System EUGE-PORT	Investigating	Undetermined	03/26/2015 9:52 PM	03/26/2015 9:53 PM
3248	4-Normal		Outage Resolved - I2 AL2S Core Node (ASHB)	Hardware	Vendor	03/29/2015 1:02 PM	03/29/2015 2:30 PM

Unscheduled Outages Detail

Ticket No.:	3199:144
Subject:	Brief Outage Resolved - I2 Optical DWDM System CHIC-EQCH
Source Of Impact:	Vendor
Affected:	DWDM System CHIC-EQCH AL2S Backbone ASHB-CHIC IP Backbone CHIC-CLEV
Start Time:	Tuesday, March 24, 2015, 5:59 PM (1759) UTC
End Time:	Tuesday, March 24, 2015, 6:00 PM (1800) UTC
Description:	DWDM System CHIC-EQCH was briefly unavailable to the community. Level (3) Communications Engineers determined that the outage was caused due to a fiber bump.

Ticket No.:	3223:144
Subject:	Brief Outage Resolved - I2 Optical Various Circuits
Source Of Impact:	Vendor
Affected:	AL2S Backbone LOSE-PHOE IP Backbone HOUS-LOSA Optical Member ESnet
Start Time:	Thursday, March 26, 2015, 5:38 PM (1738) UTC
End Time:	Thursday, March 26, 2015, 5:39 PM (1739) UTC
Description:	The above listed circuits were briefly unavailable to the community. Level(3) Engineers reviewed the issue and were not able to confirm an RFO.

Ticket No.:	3227:144
Subject:	Brief Outage - I2 Optical DWDM System EUGE-PORT
Source Of Impact:	Undetermined
Affected:	Optical DWDM EUGE-PORT Optical Member ESNet AL2S Backbone PORT-SUNN IP Backbone LOSA-SEAT
Start Time:	Thursday, March 26, 2015, 9:52 PM (2152) UTC
End Time:	Thursday, March 26, 2015, 9:53 PM (2153) UTC
Description:	DWDM System EUGE-PORT was briefly unavailable to the community. Level (3) Communications Engineers are currently investigating the reason for the outage.

Ticket No.:	3248:144
Subject:	Outage Resolved - I2 AL2S Core Node (ASHB)
Source Of Impact:	Vendor
Affected:	AL2S Backbone ASHB-CHIC AL2S Backbone ASHB-PITT AL2S Backbone ASHB-WASH IP Backbone CLEV-WASH AL2S Participant MARIA AL2S Participant NIH AL2S Participant Smithsonian AL2S Participant Merit AL2S Net+ Provider Amazon AL2S Net+ Provider Blackboard IP Connector MARIA (WASH)* IP Participant NIH (WASH)* IP Connector Smithsonian (WASH)* IP Participant Merit (WASH)* IP Net+ Provider Amazon IP Net+ Provider Blackboard
Start Time:	Sunday, March 29, 2015, 1:02 PM (1302) UTC
End Time:	Sunday, March 29, 2015, 2:30 PM (1430) UTC
Description:	The above listed assets are available after an outage. Internet2 Engineers have investigated the cause and determined that it was caused by a routing engine failure. All connections to the Core Node were affected. This outage has been resolved and no further outages are expected.

Scheduled Maintenances Summary

Ticket Number	Customer Impact	Network Impact	*Title*	Maintenance Type	Source Of Impact	*Start Time (UTC)*	*End Time (UTC)*

3197	3-Elevated		Maintenance Completed - I2 IP Backbone CHIC-KANS	Circuit	Internal	03/27/2015 7:27 AM	03/27/2015 7:29 AM

Scheduled Maintenances Detail

Ticket No.:	3197:144
Subject:	Maintenance Completed - I2 IP Backbone CHIC-KANS
Source Of Impact:	Internal
Affected:	Backbone CHIC-KANS
Start Time:	Friday, March 27, 2015, 7:27 AM (0727) UTC
End Time:	Friday, March 27, 2015, 7:29 AM (0729) UTC
Description:	The above assets were briefly unavailable while Internet2 Engineers, with the assistance of Level3 Technicians, performed maintenance to swap fiber on the AL2S to IP interconnect in Kansas City. IP traffic will re-routed during this maintenance.

Downtime and Availability

Unified - Internet2 IP Core Nodes	Down Time		Reporting Period Availability		52 Week Availability
ATLA MX960	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%
CHIC MX960	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%
CLEV MX960	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%
HOUS MX960	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%
KANS MX960	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%
LOSA MX960	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%
NEWY MX960	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%
SALT MX960	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%
SEAT MX960	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%
WASH MX960	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%
rtsw0.sing1.net.internet2.edu	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%
rtsw.sing2.net.internet2.edu	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%

Aggregate Unified - Internet2 IP Core Nodes	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%

NOTE: Numbers in italics represent downtime with scheduled maintenance excluded. Internet2's definition of scheduled maintenance is any anticipated outage announced at least 72 hours in advance of the outage start time.

Unified - Internet2 IP Backbone Circuits	Down Time		Reporting Period Availability		52 Week Availability
I2-ATLA-CLEV-VLAN-14832	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%
I2-ATLA-CLEV-VLAN-14837	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%
I2-ATLA-NEWY32AOA-VLAN-14833	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%
I2-ATLA-NEWY32AOA-VLAN-14838	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%
I2-CHIC-NEWY32AOA-VLAN-14831	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%
I2-CHIC-NEWY32AOA-VLAN-14834	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%
R&E 100G Backbone: I2-ATLA-CHIC-VLAN-09181	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.63798%	99.93682%
R&E 100G Backbone: I2-ATLA-HOUS-VLAN-09182	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.60876%	99.99602%
R&E 100G Backbone: I2-ATLA-WASH-VLAN-09179	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.92676%	99.99696%
R&E 100G Backbone: I2-CHIC-CLEV-VLAN-09180	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.94004%	99.99924%
R&E 100G Backbone: I2-CHIC-KANS-VLAN-09183	0 hr 2 min	0 hr 0 min	99.98016%	100.00000%	99.96850%	99.99867%
R&E 100G Backbone: I2-CLEV-NEWY32AOA-VLAN-09177	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.94213%	99.99469%
R&E 100G Backbone: I2-CLEV-WASH-VLAN-09178	1 hr 28 min	1 hr 28 min	99.12698%	99.12698%	99.94649%	99.97666%
R&E 100G Backbone: I2-HOUS-KANS-VLAN-09184	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.90627%	99.99677%
R&E 100G Backbone: I2-HOUS-LOSA-VLAN-09189	0 hr 1 min	0 hr 1 min	99.99008%	99.99008%	99.62602%	99.75125%
R&E 100G Backbone: I2-KANS-SALT-VLAN-09185	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.89507%	99.98975%
R&E 10G Backbone: I2-LOSA-PAIX-VLAN-23105	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%
Commodity 10G Backbone: I2-LOSA-PAIX-VLAN-23106	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%
R&E 100G Backbone: I2-LOSA-SALT-VLAN-09188	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.98330%	99.99905%
R&E 100G Backbone: I2-LOSA-SEAT-VLAN-09187	0 hr 1 min	0 hr 1 min	99.99008%	99.99008%	99.97647%	99.99924%
R&E 100G Backbone: I2-LOSA-SEAT-VLAN-09328	0 hr 1 min	0 hr 1 min	99.99008%	99.99008%	99.97647%	99.99924%
R&E 100G Backbone: I2-LOSA-WILC-VLAN-23103	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%
Commodity 100G Backbone: I2-LOSA-WILC-VLAN-23104	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%
R&E 100G Backbone: I2-NEWY32AOA-WASH-VLAN-09176	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.98216%	99.99734%
R&E 100G Backbone: I2-SALT-SEAT-VLAN-09186	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.99677%	99.99981%
R&E 100G Backbone: I2-SALT-SEAT-VLAN-09327	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.99677%	99.99981%
R&E 100G Backbone: I2-WASH-ATLA-VLAN-09179	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%
R&E 100G Backbone: I2-ATLA-WASH-VLAN-09318	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.92676%	99.99696%
Legacy 10G Backbone: I2-ATLA-CHIC-10GE-05419	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.73399%	99.97287%
Legacy 10G Backbone: I2-ATLA-HOUS-10GE-05423	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.20993%	99.56208%
Legacy 10G Backbone: I2-ATLA-WASH-10GE-05251	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.93871%	100.00000%
Legacy 10G Backbone: I2-ATLA-WASH-10GE-09142	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.93739%	99.99867%
Legacy 10G Backbone: I2-CHIC-CLEV-10GE-08891	0 hr 1 min	0 hr 1 min	99.99008%	99.99008%	99.95788%	99.99981%
Legacy 10G Backbone: I2-CHIC-KANS-10GE-05781	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%
Legacy 10G Backbone: I2-CHIC-KANS-10GE-05976	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.99981%	99.99981%
Legacy 10G Backbone: I2-CLEV-NEWY32AOA-10GE-08894	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.96376%	99.99962%
Legacy 10G Backbone: I2-CLEV-WASH-10GE-08889	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.99867%	99.99867%
Legacy 10G Backbone: I2-HOUS-KANS-10GE-05560	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.94346%	99.99753%
Legacy 10G Backbone: I2-HOUS-KANS-LAG-08934	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%
Legacy 10G Backbone: I2-HOUS-LOSA-10GE-05559	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.75789%	99.82772%
Legacy 10G Backbone: I2-KANS-SALT-10GE-05566	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.92505%	99.99924%
Legacy 10G Backbone: I2-KANS-SALT-LAG-08935	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.92505%	99.99924%
Legacy 10G Backbone: I2-LOSA-PAIX-10GE-05853	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.99621%	100.00000%
Legacy 10G Backbone: I2-LOSA-SALT-10GE-05563	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.99013%	100.00000%
Legacy 10G Backbone: I2-LOSA-SEAT-10GE-05564	0 hr 1 min	0 hr 1 min	99.99008%	99.99008%	99.97818%	99.99905%
Legacy 10G Backbone: I2-LOSA-SEAT-10GE-05572	0 hr 1 min	0 hr 1 min	99.99008%	99.99008%	99.99412%	99.99981%
Legacy 100G Backbone: I2-LOSA-WILC-100GE-23893	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	100.00000%	100.00000%
Legacy 10G Backbone: I2-NEWY32AOA-WASH-10GE-07612	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.99696%	99.99810%
Legacy 10G Backbone: I2-SALT-SEAT-10GE-05565	0 hr 0 min	0 hr 0 min	100.00000%	100.00000%	99.99981%	99.99981%

Aggregate All Unified - Internet2 IP Backbone Circuits	1 hr 36 min	0 hr 2 min	99.98056%	99.98097%	99.92824%	99.97919%

Upcoming Scheduled Maintenances

Ticket No.:	2943:144
Subject:	Maintenance 3 of 4 - I2 Optical OneControl Upgrade
Affected:	oc-denv.net.internet2.edu
Start Time:	Monday, March 30, 2015, 2:00 PM (1400) UTC
End Time:	Wednesday, April 1, 2015, 9:00 PM (2100) UTC
Description:	During the above time frame, system engineers will be upgrading oc-denv.net.internet2.edu to OneControl 4.2. Users should switch to using oc-indi.net.internet2.edu as the primary OneControl server during this time frame. A second notification will be sent when oc-denv has been upgrade to 4.2 and is ready for use.

Ticket No.:	3295:144
Subject:	Emergency Maintenance - I2 AL2S Backbones CHIC-KANS and DENV-KANS
Affected:	AL2S Backbones CHIC-KANS and DENV-KANS IP Backbones CHIC-KANS and KANS-SALT
Start Time:	Thursday, April 2, 2015, 5:00 AM (0500) UTC
End Time:	Thursday, April 2, 2015, 6:00 AM (0600) UTC
Description:	The above listed assets will be unavailable while Internet2 Engineers, with the assistance of Level3 Technicians, perform emergency maintenance. An outage is expected and the entire window is reserved. Backbone traffic will re-route during this maintenance. Workgroup administrators will receive notifications from OESS regarding circuits re-routed on the backbone.

Ticket No.:	3232:144
Subject:	Maintenance - I2 Optical Member ESNET (CHIC-NASH)
Affected:	ESNET-CHIC-NASH-100GE-07807
Start Time:	Friday, April 3, 2015, 3:00 AM (0300) UTC
End Time:	Friday, April 3, 2015, 5:00 AM (0500) UTC
Description:	Member ESnet's Optical circuit ESNET-CHIC-NASH-100GE-07807 will be unavailable while Level3 Engineers perform maintenance. A brief outage is expected. The entire maintenance window has been reserved.

Ticket No.:	3245:144
Subject:	Maintenance - I2 Optical Member ESnet
Affected:	Optical Member ESnet
Start Time:	Friday, April 3, 2015, 3:00 AM (0300) UTC
End Time:	Friday, April 3, 2015, 5:00 AM (0500) UTC
Description:	Member ESnet's Optical circuit ESNET-CHIC-NASH-100GE-07807 will be unavailable while Level(3) Engineers perform maintenance to troubleshoot a "Pre-FEC Signal Degrade" on an OCLD. . A brief outage is expected, however the entire maintenance window has been reserved.

Ticket No.:	3297:144
Subject:	Maintenance Tracking - I2 Optical Node opt.sacr
Affected:	Optical Node opt.sacr
Start Time:	Saturday, April 4, 2015, 7:00 AM (0700) UTC
End Time:	Saturday, April 4, 2015, 11:00 AM (1100) UTC
Description:	*Tracking Ticket The above listed assets should remain available while Internet2 Engineers perform maintenance. An outage is not expected; however, the entire window is reserved.

Ticket No.:	2978:144
Subject:	Maintenance - I2 AL2S Connector CAAREN
Affected:	Connector CAAREN and its Participants
Start Time:	Tuesday, April 7, 2015, 4:00 AM (0400) UTC
End Time:	Tuesday, April 7, 2015, 5:00 AM (0500) UTC
Description:	The above listed will be unavailable while Internet2 and CAAREN Engineers migrate VLANs carrying CAAREN peerings. A brief outage is expected; however, the entire maintenance window is reserved.

Ticket No.:	3219:144
Subject:	Maintenance 1 of 2- I2 Optical Member ESnet
Affected:	I2 Optical Member ESnet
Start Time:	Tuesday, April 7, 2015, 4:00 AM (0400) UTC
End Time:	Tuesday, April 7, 2015, 9:00 AM (0900) UTC
Description:	Please note that this maintenance has been rescheduled Member ESnet's Optical circuit ESNET-CHAT-ORNL-100GE-06347 will be unavailable while Level (3) Engineers perform maintenance. A brief outage is expected, however the entire maintenance window has been reserved. This is maintenance window 1 of 2.

Ticket No.:	3221:144
Subject:	Maintenance 2 of 2- I2 Optical Member ESnet
Affected:	I2 Optical Member ESnet
Start Time:	Wednesday, April 8, 2015, 4:00 AM (0400) UTC
End Time:	Wednesday, April 8, 2015, 9:00 AM (0900) UTC
Description:	Please note that this maintenance has been rescheduled Member ESnet's Optical circuit ESNET-CHAT-ORNL-100GE-06347 will be unavailable while Level (3) Engineers perform maintenance. A brief outage is expected, however the entire maintenance window has been reserved. This is maintenance window 2 of 2.