Generated Report for Internet2 IP

Internet2 IP Network Operations Report
09/19/2011 - 09/25/2011

Internet2 IP Network Operations Center
at Indiana University

New Connections

None

Upgrades

None

New Services

None

Unscheduled Outages Summary

Ticket Number	Customer Impact	Network Impact	*Title*	Outage Type	*Start Time (UTC)*	*End Time (UTC)*

17916	2-High	2-High	Internet2 IP Network Backbone Circuit ATLA-CHIC Brief Outage Resolved	Circuit - Damaged Fiber	09/22/2011 2:00 PM	09/22/2011 2:02 PM
17931	3-Elevated	3-Elevated	Internet2 IP Network Backbone Circuit CHIC-NEWY Brief Outage	Unannounced Maintenance	09/25/2011 8:38 PM	09/25/2011 8:50 PM

Unscheduled Outages Detail

Ticket No.:	17916:45
Subject:	Internet2 IP Network Backbone Circuit ATLA-CHIC Brief Outage Resolved
Affected:	Backbone Circuit ATLA-CHIC Connector Indiana GigaPOP(ATLA)* Connector KyRON
Start Time:	Thursday, September 22, 2011, 2:00 PM (1400) UTC
End Time:	Thursday, September 22, 2011, 2:02 PM (1402) UTC
Description:	The above listed was briefly unavailable. Level(3) Communications investigated and determined the reason for outage was fiber damage outside of Birmingham, AL. Fiber splicing of the entire conduit will not be required. All services have been restored and no further outages are expected. (*denotes multihomed status and, therefore, remained reachable)

Ticket No.:	17931:45
Subject:	Internet2 IP Network Backbone Circuit CHIC-NEWY Brief Outage
Affected:	Backbone Circuit CHIC-NEWY
Start Time:	Sunday, September 25, 2011, 8:38 PM (2038) UTC
End Time:	Sunday, September 25, 2011, 8:50 PM (2050) UTC
Description:	Backbone circuit CHIC-NEWY was briefly unavailable. Level(3) Communications technicians report they were performing testing to troubleshoot a degraded light level. GRNOC Engineers are now investigating the cause of the degraded light levels.

Scheduled Maintenances Summary

Ticket Number	Customer Impact	Network Impact	*Title*	Maintenance Type	*Start Time (UTC)*	*End Time (UTC)*

17711	3-Elevated	2-High	Internet2 IP Network Backbone Circuits LOSA-SALT and LOSA-SEAT Maintenance Completed	Circuit	09/20/2011 11:35 AM	09/20/2011 12:06 PM
17892	3-Elevated	1-Critical	Internet2 IP Network Various Core Node(s) Emergency Maintenance Completed	Software	09/22/2011 6:02 AM	09/22/2011 6:34 AM
17887	3-Elevated	2-High	Internet2 IP Network Backbone Circuit ATLA-WASH Maintenance Completed	Hardware	09/23/2011 6:09 AM	09/23/2011 6:27 AM

Scheduled Maintenances Detail

Ticket No.:	17711:45
Subject:	Internet2 IP Network Backbone Circuits LOSA-SALT and LOSA-SEAT Maintenance Completed
Affected:	Backbone Circuit LOSA-SALT Backbone Circuit LOSA-SEAT IXP PacWave Connector Oregon Gigapop (LOSA) and its Participants
Start Time:	Tuesday, September 20, 2011, 11:35 AM (1135) UTC
End Time:	Tuesday, September 20, 2011, 12:06 PM (1206) UTC
Description:	The above listed were unavailable while Level(3) Communications performed maintenance to relocate fiber due to UPRR bridge project at Hollister and UPRR, Goleta, CA. This maintenance has been completed. Alternate night is reserved for 9/21/2011 for the same time, if required.

Ticket No.:	17892:45
Subject:	Internet2 IP Network Various Core Node(s) Emergency Maintenance Completed
Affected:	Core Nodes ATLA, CHIC, SEAT, SALT, NEWY, LOSA, KANS, HOUS, WASH
Start Time:	Thursday, September 22, 2011, 6:02 AM (0602) UTC
End Time:	Thursday, September 22, 2011, 6:34 AM (0634) UTC
Description:	Routing in the above listed was unstable as network engineers restarted iBGP on the routers across the backbone routers. No external or customer peerings were affected, and traffic should rerouted as the internal peering sessions restarted. Maintenance has been completed.

Ticket No.:	17887:45
Subject:	Internet2 IP Network Backbone Circuit ATLA-WASH Maintenance Completed
Affected:	Backbone Circuit ATLA-WASH Connector MCNC and its Participants (WASH)
Start Time:	Friday, September 23, 2011, 6:09 AM (0609) UTC
End Time:	Friday, September 23, 2011, 6:27 AM (0627) UTC
Description:	Backbone Circuit ATLA-WASH and connector MCNC's connection to WASH were unavailable while Level(3) Communications replaced failing hardware in King George, VA. Maintenance has been completed.

Downtime and Availability

Internet2 IP Core Nodes	Down Time	Reporting Period Availability	52 Week Availability
CHIC T1600	0 hr 32 min	99.68254%	99.89261%
NEWY T1600	0 hr 32 min	99.68254%	99.97021%
SEAT MX960	0 hr 32 min	99.68254%	99.99336%
WASH T1600	0 hr 32 min	99.68254%	99.94403%
HOUS MX960	0 hr 32 min	99.68254%	99.99279%
LOSA T1600	0 hr 32 min	99.68254%	99.96888%
ATLA T1600	0 hr 32 min	99.68254%	99.98008%
SALT T1600	0 hr 32 min	99.68254%	99.96793%
KANS T1600	0 hr 32 min	99.68254%	99.95351%

Aggregate Internet2 IP Core Nodes	4 hr 48 min	99.68254%	99.96260%

** Less than 52 weeks of data available

Internet2 IP Backbone Circuits	Down Time	Reporting Period Availability	52 Week Availability
I2-HOUS-LOSA-10GE-05559	0 hr 0 min	100.00000%	99.95959%
I2-ATLA-CHIC-10GE-05419	0 hr 2 min	99.98016%	99.87876%
** I2-HOUS-KANS-10GE-05560	0 hr 0 min	100.00000%	99.81500%
I2-CHIC-NEWY32AOA-10GE-05239	0 hr 0 min	100.00000%	99.84498%
** I2-CHIC-NEWY32AOA-10GE-05667	0 hr 12 min	99.88095%	99.84252%
I2-NEWY32AOA-WASH-10GE-05242	0 hr 0 min	100.00000%	99.91063%
I2-CHIC-WASH-10GE-05250	0 hr 0 min	100.00000%	99.91765%
** I2-HOUS-KANS-10GE-05561	0 hr 0 min	100.00000%	99.81557%
** I2-NEWY32AOA-WASH-10GE-04643	0 hr 0 min	100.00000%	99.91101%
I2-ATLA-WASH-10GE-05251	0 hr 18 min	99.82143%	99.85124%
I2-ATLA-WASH-10GE-05133	0 hr 18 min	99.82143%	99.85371%
I2-ATLA-HOUS-10GE-05423	0 hr 0 min	100.00000%	99.91120%
** I2-ATLA-HOUS-10GE-05562	0 hr 0 min	100.00000%	99.91120%
** I2-KANS-SALT-10GE-05566	0 hr 0 min	100.00000%	99.95845%
I2-KANS-SALT-10GE-05138	0 hr 0 min	100.00000%	99.95845%
** I2-SALT-SEAT-10GE-05565	0 hr 0 min	100.00000%	99.95712%
** I2-SALT-SEAT-10GE-05573	0 hr 0 min	100.00000%	99.95712%
** I2-LOSA-SALT-10GE-05571	0 hr 31 min	99.69246%	99.93815%
** I2-LOSA-SALT-10GE-05563	0 hr 31 min	99.69246%	99.93815%
I2-ATLA-CHIC-10GE-05638	0 hr 2 min	99.98016%	99.88009%
I2-CHIC-KANS-10GE-05567	0 hr 0 min	100.00000%	99.82278%
I2-CHIC-KANS-10GE-05568	0 hr 0 min	100.00000%	99.82032%
I2-CHIC-KANS-10GE-05976	0 hr 0 min	100.00000%	99.89223%
I2-CHIC-WASH-10GE-05637	0 hr 0 min	100.00000%	99.97439%
I2-LOSA-SEAT-10GE-05564	0 hr 31 min	99.69246%	99.98653%
I2-LOSA-SEAT-10GE-05572	0 hr 31 min	99.69246%	99.98653%
I2-HOUS-LOSA-10GE-05581	0 hr 0 min	100.00000%	99.95959%

Aggregate All Internet2 IP Backbone Circuits	2 hr 56 min	99.93533%	99.90567%

Upcoming Scheduled Maintenances

Ticket No.:	17913:45
Subject:	Internet2 IP Network Backbone Circuit CHIC-WASH Maintenance
Affected:	Backbone Circuit CHIC-WASH
Start Time:	Thursday, September 29, 2011, 4:00 AM (0400) UTC
End Time:	Thursday, September 29, 2011, 10:00 AM (1000) UTC
Description:	Backbone Circuit CHIC-WASH will be unavailable while Level(3) Engineers perform fiber maintenance. An outage of 2 hours is expected; however, the entire window is reserved. IP traffic will re-route during this maintenance. An alternate window has been reserved for 09/30/2011 from 0400 to 1000 UTC, should it be required.

Ticket No.:	17914:45
Subject:	Internet2 IP Network Backbone Circuit CHIC-WASH Maintenance Alternate Window
Affected:	Backbone Circuit CHIC-WASH
Start Time:	Friday, September 30, 2011, 4:00 AM (0400) UTC
End Time:	Friday, September 30, 2011, 10:00 AM (1000) UTC
Description:	Backbone Circuit CHIC-WASH will be unavailable while Level(3) Engineers perform fiber maintenance. An outage of 2 hours is expected; however, the entire window is reserved. IP traffic will re-route during this maintenance. This is an alternate window for ticket 17913:45, should it be required.