Internet2 IP Network Operations Report
06/23/2014 - 06/29/2014

Internet2 IP Network Operations Center
at Indiana University

New Connections

None


Upgrades

None

New Services

None



Unscheduled Outages Summary







Ticket Number
Customer Impact
Network Impact
Title
Outage Type
Source Of Impact
Start Time (UTC)

End Time (UTC)





233712-High
2-High

Brief Outage Resolved - I2 AL2S Backbone ATLA-HOUS
UndeterminedUndetermined
06/26/2014 11:08 AM
06/26/2014 11:09 AM


233452-High
2-High

Outage Resolved - I2 IP Backbone CHIC-CLEV
SoftwareVendor
06/26/2014 12:00 PM
06/26/2014 7:02 PM


233532-High
2-High

Brief Outage - I2 IP Backbone HOUS-KANS
UndeterminedUndetermined
06/27/2014 6:48 AM
06/27/2014 6:49 AM






Unscheduled Outages Detail






Ticket No.:23371:45



Subject:Brief Outage Resolved - I2 AL2S Backbone ATLA-HOUS



Source Of Impact:Undetermined



Affected:Backbone HOUS-KANS



Start Time:Thursday, June 26, 2014, 11:08 AM (1108) UTC



End Time:Thursday, June 26, 2014, 11:09 AM (1109) UTC


Description:Backbone HOUS-KANS is now available following a brief outage. Level(3)
Communications and Internet2 Engineering has investigated, but the reason
for the outage is indeterminate. Workgroup administrators receive
notifications from OESS regarding circuits re-routed on the backbone.







Ticket No.:23345:45



Subject:Outage Resolved - I2 IP Backbone CHIC-CLEV



Source Of Impact:Vendor



Affected:Backbone CHIC-CLEV
Participant FRGP (CHIC)*
Connector CENIC (CHIC)*
NET+ Provider Code42 (CHIC)*



Start Time:Thursday, June 26, 2014, 12:00 PM (1200) UTC



End Time:Thursday, June 26, 2014, 7:02 PM (1902) UTC


Description:Backbone CHIC-CLEV is now available. Level(3) Communications 
experienced trouble after a software upgrade maintenance. Hardware has 
been replaced and no further outages are expected. IP traffic will re-route.
* indicates multihomed status and therefore the entity remained available.







Ticket No.:23353:45



Subject:Brief Outage - I2 IP Backbone HOUS-KANS



Source Of Impact:Undetermined



Affected:Backbone HOUS-KANS



Start Time:Friday, June 27, 2014, 6:48 AM (0648) UTC



End Time:Friday, June 27, 2014, 6:49 AM (0649) UTC


Description:Backbone HOUS-KANS was brielfy unavailable. Internet2
Engineers are investigating the cause; however, the
reason for the outage is pending.  IP traffic will
re-route.







Scheduled Maintenances Summary





Ticket Number
Customer Impact
Network Impact
Title

Maintenance Type
Source Of Impact
Start Time (UTC)
End Time (UTC)





23219
3-Elevated
2-High

Maintenance Completed - I2 IP Various Backbones
Circuit
Vendor06/26/2014 4:05 AM
06/26/2014 12:00 PM


23221
3-Elevated
2-High

Maintenance Completed - I2 IP Various Backbones
CircuitVendor
06/27/2014 7:02 AM
06/27/2014 7:27 AM




Scheduled Maintenances Detail





Ticket No.:23219:45



Subject:Maintenance Completed - I2 IP Various Backbones



Source Of Impact:Vendor



Affected:Backbone ATLA-CHIC
Backbone CHIC-CLEV
Backbone HOUS-KANS
Backbone CLEV-WASH
ION Backbone CLEV-WASH
ION Backbone HOUS-KANS
ION Backbone ATLA-CHIC
ION Backbone CHIC-CLEV



Start Time:Thursday, June 26, 2014, 4:05 AM (0405) UTC



End Time:Thursday, June 26, 2014, 12:00 PM (1200) UTC


Description:   The backbones listed above were unavailable while
Internet2 fiber provider Engineers performed
maintenance.  IP traffic re-routed during this maintenance. This maintenance has been completed.



  







Ticket No.:23221:45



Subject:Maintenance Completed - I2 IP Various Backbones



Source Of Impact:Vendor



Affected:Backbone CLEV-NEWY32AOA
Backbone LOSA-SEAT
Backbone NEWY32AOA-WASH
ION Backbone CLEV-NEWY32AOA
ION Backbone NEWY32AOA-WASH



Start Time:Friday, June 27, 2014, 7:02 AM (0702) UTC



End Time:Friday, June 27, 2014, 7:27 AM (0727) UTC


Description:   The backbones listed above were unavailable while Level(3)
Communications Engineers performed maintenance. Maintenance has
been completed. No further outages are expected. Backbone traffic re-
routed during this maintenance.



  










Downtime and Availability







Internet2 IP Core Nodes
Down TimeReporting Period Availability52 Week Availability


ATLA MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99355%  99.99867%
CHIC MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.98501%  99.99905%
CLEV MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99089%  99.99924%
HOUS MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.97571%  100.00000%
KANS MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99639%  100.00000%
LOSA MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99184%  100.00000%
NEWY MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99943%  100.00000%
SALT MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99317%  99.99981%
SEAT MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99639%  100.00000%
WASH MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.98843%  100.00000%
rtsw0.sing1.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%  100.00000%
rtsw.sing2.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%  100.00000%
      
Aggregate Internet2 IP Core Nodes0 hr 0 min0 hr 0 min100.00000%100.00000%99.99257%99.99973%

** Less than 52 weeks of data available
 NOTE: Numbers in italics represent downtime with scheduled  maintenance excluded. Internet2's definition of scheduled
  maintenance is any anticipated outage announced at least 72
  hours in advance of the outage start time.  
	    

	    
Internet2 IP Backbone CircuitsDown Time
Reporting Period Availability52 Week Availability


R&E 100G Backbone: I2-ATLA-CHIC-VLAN-09181
               7 hr 55 min0 hr 0 min
               95.28770%100.00000%
               99.76700%99.95579%
R&E 100G Backbone: I2-ATLA-HOUS-VLAN-09182
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.72260%99.89526%
R&E 100G Backbone: I2-ATLA-WASH-VLAN-09179
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.78749%99.84138%
R&E 100G Backbone: I2-CHIC-CLEV-VLAN-09180
               14 hr 56 min7 hr 1 min
               91.11111%95.82341%
               99.58978%99.74860%
R&E 100G Backbone: I2-CHIC-KANS-VLAN-09183
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.92126%99.97723%
R&E 100G Backbone: I2-CLEV-NEWY32AOA-VLAN-09177
               0 hr 25 min0 hr 0 min
               99.75198%100.00000%
               99.97742%99.99924%
R&E 100G Backbone: I2-CLEV-WASH-VLAN-09178
               7 hr 55 min0 hr 0 min
               95.28770%100.00000%
               99.86092%99.98179%
R&E 100G Backbone: I2-HOUS-KANS-VLAN-09184
               0 hr 2 min0 hr 2 min
               99.98016%99.98016%
               99.72810%99.77288%
R&E 100G Backbone: I2-HOUS-LOSA-VLAN-09189
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.58428%99.68333%
R&E 100G Backbone: I2-KANS-SALT-VLAN-09185
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.93568%99.97002%
R&E 100G Backbone: I2-LOSA-SALT-VLAN-09188
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.94592%100.00000%
R&E 100G Backbone: I2-LOSA-SEAT-VLAN-09187
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.96205%99.99905%
R&E 100G Backbone: I2-LOSA-SEAT-VLAN-09328
               0 hr 25 min0 hr 0 min
               99.75198%100.00000%
               99.95731%99.99905%
R&E 100G Backbone: I2-NEWY32AOA-WASH-VLAN-09176
               0 hr 25 min0 hr 0 min
               99.75198%100.00000%
               99.94479%99.95598%
R&E 100G Backbone: I2-SALT-SEAT-VLAN-09186
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.98710%99.99924%
R&E 100G Backbone: I2-SALT-SEAT-VLAN-09327
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.98729%99.99943%
R&E 100G Backbone: I2-WASH-ATLA-VLAN-09179
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               100.00000%100.00000%
Legacy 10G Backbone: I2-ATLA-CHIC-10GE-05419
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.89564%99.96774%
Legacy 10G Backbone: I2-ATLA-HOUS-10GE-05423
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.38202%99.73399%
Legacy 10G Backbone: I2-ATLA-WASH-10GE-05251
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.84707%99.85030%
Legacy 10G Backbone: I2-ATLA-WASH-10GE-09142
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.84707%99.85030%
Legacy 10G Backbone: I2-CHIC-CLEV-10GE-08891
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.96604%99.98786%
Legacy 10G Backbone: I2-CHIC-KANS-10GE-05781
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.95048%99.98710%
Legacy 10G Backbone: I2-CHIC-KANS-10GE-05976
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.95048%99.98710%
Legacy 10G Backbone: I2-CLEV-NEWY32AOA-10GE-08894
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.96053%99.99981%
Legacy 10G Backbone: I2-CLEV-WASH-10GE-08889
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.82886%99.83891%
Legacy 10G Backbone: I2-HOUS-KANS-10GE-05560
               0 hr 1 min0 hr 1 min
               99.99008%99.99008%
               99.67839%99.75011%
Legacy 10G Backbone: I2-HOUS-KANS-LAG-08934
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               100.00000%100.00000%
Legacy 10G Backbone: I2-HOUS-LOSA-10GE-05559
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.50269%99.60667%
Legacy 10G Backbone: I2-KANS-SALT-10GE-05566
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.81500%99.90930%
Legacy 10G Backbone: I2-KANS-SALT-LAG-08935
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.81481%99.90912%
Legacy 10G Backbone: I2-LOSA-SALT-10GE-05563
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.74992%99.80666%
Legacy 10G Backbone: I2-LOSA-SEAT-10GE-05564
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.96243%99.99772%
Legacy 10G Backbone: I2-LOSA-SEAT-10GE-05572
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.95921%99.99772%
Legacy 10G Backbone: I2-NEWY32AOA-WASH-10GE-07612
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.78825%99.91177%
Legacy 10G Backbone: I2-SALT-SEAT-10GE-05565
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.98956%99.99943%
      
Aggregate All Internet2 IP Backbone Circuits
                   32 hr 4 min25 hr 0 min
                   99.46980%99.88316%
                   99.84854%99.91305%


 NOTE: Numbers in italics represent downtime with scheduled  maintenance excluded. Internet2's definition of scheduled
  maintenance is any anticipated outage announced at least 72
  hours in advance of the outage start time.  
*  Internet2 IP 100G Backbone Circuits Ride over AL2S for connectivity.

Upcoming Scheduled Maintenances





Ticket No.:23222:45



Subject:Maintenance - I2 IP Various Backbones



Affected:ION Backbone ATLA-CHIC
Backbone ATLA-CHIC
ION Backbone ATLA-WASH
Backbone ATLA-WASH
ION Backbone CHIC-KANS
Backbone CHIC-KANS
ION Backbone HOUS-KANS
Backbone HOUS-KANS



Start Time:Thursday, July 3, 2014, 4:00 AM (0400) UTC



End Time:Thursday, July 3, 2014, 11:00 AM (1100) UTC



Description:   The above listed backbones will be unavailable while Vendor Engineers
perform maintenance.  Outages are anticipated; however, the entire
window is reserved.  IP traffic will re-route during this maintenance.



  







Ticket No.:23358:45



Subject:Maintenance - I2 IP Backbones KANS-SALT and HOUS-KANS



Affected:Backbone KANS-SALT
Backbone HOUS-KANS



Start Time:Friday, July 4, 2014, 4:00 AM (0400) UTC



End Time:Friday, July 4, 2014, 11:00 AM (1100) UTC



Description:   The above listed backbones will be unavailable while Level 3 Engineers
perform maintenance. Outages are expected, and the entire maintenance
window has been reserved.  IP traffic will re-route during this maintenance.



  







Ticket No.:23283:45



Subject:Maintenance - I2 IP Backbone CHIC-KANS



Affected:Backbone CHIC-KANS



Start Time:Sunday, July 6, 2014, 5:00 AM (0500) UTC



End Time:Sunday, July 6, 2014, 7:00 AM (0700) UTC



Description:   Backbone CHIC-KANS will be unavailable while Level(3) Communications
performs maintenance to replace a card in Plainfield, IN. An outage is
expected, and the entire window is reserved.  IP traffic will re-route during
this maintenance.



  







Ticket No.:23341:45



Subject:Maintenance 1 of 2 - I2 IP Backbones HOUS-KANS



Affected:Backbone HOUS-KANS
ION Backbone HOUS-KANS



Start Time:Thursday, July 10, 2014, 5:00 AM (0500) UTC



End Time:Thursday, July 10, 2014, 11:00 AM (1100) UTC



Description:   Backbone HOUS-KANS will be unavailable while Level(3) Communications 
performs maintenance to repair damaged fiber in Bardwell, TX.  An outage 
is expected, and the entire window is reserved.  IP traffic will re-route 
during this maintenance.
* This is the 1st of 2 maintenance windows *



  







Ticket No.:23342:45



Subject:Maintenance 2 of 2 - I2 IP Backbones HOUS-KANS



Affected:Backbone HOUS-KANS
ION Backbone HOUS-KANS



Start Time:Friday, July 11, 2014, 5:00 AM (0500) UTC



End Time:Friday, July 11, 2014, 11:00 AM (1100) UTC



Description:   Backbone HOUS-KANS will be unavailable while Level(3) Communications
performs maintenance to repair damaged fiber in Bardwell, TX.  An outage
is expected, and the entire window is reserved.  IP traffic will re-route
during this maintenance.
* This is the 2nd of 2 maintenance windows *