Unified - Internet2 IP Network Operations Report
08/17/2015 - 08/23/2015

Unified - Internet2 IP Network Operations Center
at Indiana University

New Connections

None


Upgrades

None

New Services

None



Unscheduled Outages Summary







Ticket Number
Customer Impact
Network Impact
Title
Outage Type
Source Of Impact
Start Time (UTC)

End Time (UTC)





52274-Normal


Brief Outage Resolved - I2 Optical Various Circuits
Human ErrorVendor
08/20/2015 4:43 PM
08/20/2015 4:45 PM


52414-Normal
1-Critical

Brief Outage Resolved - I2 Optical DWDM System ASHB-PITT
Cut or Damaged FiberVendor
08/21/2015 1:11 PM
08/21/2015 1:15 PM


52494-Normal
1-Critical

Brief Outage - I2 Optical DWDM System SACR-SUNN
InvestigatingVendor
08/21/2015 7:05 PM
08/21/2015 7:06 PM


52574-Normal
1-Critical

Outage Resolved - I2 Optical DWDM System ASHB-PITT
Unannounced MaintenanceVendor
08/22/2015 5:37 AM
08/22/2015 8:15 AM


08/22/2015 10:03 AM
08/22/2015 10:18 AM






Unscheduled Outages Detail






Ticket No.:5227:144



Subject:Brief Outage Resolved - I2 Optical Various Circuits



Source Of Impact:Vendor



Affected:AL2S Backbone RENO-SUNN
IP Backbone LOSA-SEAT
Optical Member ESNet
Optical Member CENIC
Optical Member University of Oregon



Start Time:Thursday, August 20, 2015, 4:43 PM (1643) UTC



End Time:Thursday, August 20, 2015, 4:45 PM (1645) UTC


Description:The above listed optical circuits was briefly unavailable. Internet2
Engineers have advised that this issue was related to a fiber bump in
the Sunnyvale, CA area.  No further outages are expected.







Ticket No.:5241:144



Subject:Brief Outage Resolved - I2 Optical DWDM System ASHB-PITT



Source Of Impact:Vendor



Affected:DWDM System ASHB-PITT
AL2S Backbones ASHB-PITT and ASHB-CHIC
IP Backbones ATLA-CLEV and CLEV-WASH
Connector 3ROX (ASHB)



Start Time:Friday, August 21, 2015, 1:11 PM (1311) UTC



End Time:Friday, August 21, 2015, 1:15 PM (1315) UTC


Description:The above listed assets are currently available. The Internet2 Fiber Vendor has
indicated that this issue was caused by partial fiber damage between Cumberland, MD 
and Uniontown, PA. Level(3) engineers migrated affected services to spare fiber and no 
further outages are expected.







Ticket No.:5249:144



Subject:Brief Outage - I2 Optical DWDM System SACR-SUNN



Source Of Impact:Vendor



Affected:DWDM System SACR-SUNN
AL2S Backbone(s) PORT-SUNN and RENO-SUNN
IP Backbone(s) LOSA-SEAT
Optical Member CENIC
Optical Member University of Oregon



Start Time:Friday, August 21, 2015, 7:05 PM (1905) UTC



End Time:Friday, August 21, 2015, 7:06 PM (1906) UTC


Description:The above listed assets were briefly unavailable to the community.
Level (3) Communications Engineers are currently investigating the
reason for the outage.  No ETR is available at this time.  Updates
will be provided as they are received.







Ticket No.:5257:144



Subject:Outage Resolved - I2 Optical DWDM System ASHB-PITT



Source Of Impact:Vendor



Affected:DWDM System ASHB-PITT
AL2S Backbones ASHB-PITT and ASHB-CHIC
IP Backbones ATLA-CLEV and CLEV-WASH
Connector 3ROX (ASHB)



Start Time:Saturday, August 22, 2015, 5:37 AM (0537) UTC



End Time:Saturday, August 22, 2015, 8:15 AM (0815) UTC


Start Time:Saturday, August 22, 2015, 10:03 AM (1003) UTC



End Time:Saturday, August 22, 2015, 10:18 AM (1018) UTC


Description:The above listed assets are currently available to the community. Level (3)
Communications Engineers have completed repairs and services have stabilized. No
further outages are expected.







Scheduled Maintenances Summary





Ticket Number
Customer Impact
Network Impact
Title

Maintenance Type
Source Of Impact
Start Time (UTC)
End Time (UTC)





5119
3-Elevated


Maintenance Completed - I2 AL2S Interconnect to I2 IP (KANS)
Circuit
Internal08/18/2015 5:13 AM
08/18/2015 5:51 AM




Scheduled Maintenances Detail





Ticket No.:5119:144



Subject:Maintenance Completed - I2 AL2S Interconnect to I2 IP (KANS)



Source Of Impact:Internal



Affected:AL2S-IP Interconnect (KANS)
IP Backbone CHIC-KANS



Start Time:Tuesday, August 18, 2015, 5:13 AM (0513) UTC



End Time:Tuesday, August 18, 2015, 5:51 AM (0551) UTC


Description:   The above listed asset was unavailable while Internet2 Engineers, with the assistance
of Level3 Technicians, performed maintenance. Backbone traffic was re-routed during this maintenance. Workgroup
administrators received notifications from OESS regarding circuits re-routed on the
backbone. Maintenance has been completed, no further outages are expected.



  










Downtime and Availability







Unified - Internet2 IP Core NodesDown TimeReporting Period Availability52 Week Availability


ATLA MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%  100.00000%
CHIC MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%  100.00000%
CLEV MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%  100.00000%
HOUS MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%  100.00000%
KANS MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%  100.00000%
LOSA MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%  100.00000%
NEWY MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%  100.00000%
SALT MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%  100.00000%
SEAT MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%  100.00000%
WASH MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%  100.00000%
rtsw0.sing1.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%  100.00000%
rtsw.sing2.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%  100.00000%
      
Aggregate Unified - Internet2 IP Core Nodes0 hr 0 min0 hr 0 min100.00000%100.00000%100.00000%100.00000%

 NOTE: Numbers in italics represent downtime with scheduled  maintenance excluded. Internet2's definition of scheduled
  maintenance is any anticipated outage announced at least 72
  hours in advance of the outage start time.  
	    

	    
Unified - Internet2 IP Backbone CircuitsDown Time
Reporting Period Availability52 Week Availability


I2-ATLA-CLEV-VLAN-14832
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               100.00000%100.00000%
I2-ATLA-CLEV-VLAN-14837
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               100.00000%100.00000%
I2-ATLA-NEWY32AOA-VLAN-14833
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               100.00000%100.00000%
I2-ATLA-NEWY32AOA-VLAN-14838
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               100.00000%100.00000%
I2-CHIC-NEWY32AOA-VLAN-14831
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               100.00000%100.00000%
I2-CHIC-NEWY32AOA-VLAN-14834
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               100.00000%100.00000%
R&E 100G Backbone: I2-ATLA-CHIC-VLAN-09181
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.40536%99.79812%
R&E 100G Backbone: I2-ATLA-HOUS-VLAN-09182
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.53533%99.97932%
R&E 100G Backbone: I2-ATLA-WASH-VLAN-09179
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.72583%99.84271%
R&E 100G Backbone: I2-CHIC-CLEV-VLAN-09180
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.87079%99.99867%
R&E 100G Backbone: I2-CHIC-KANS-VLAN-09183
               0 hr 38 min0 hr 0 min
               99.62302%100.00000%
               99.90285%99.99526%
R&E 100G Backbone: I2-CLEV-NEWY32AOA-VLAN-09177
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.93663%99.99469%
R&E 100G Backbone: I2-CLEV-WASH-VLAN-09178
               2 hr 57 min2 hr 57 min
               98.24405%98.24405%
               99.87572%99.91329%
R&E 100G Backbone: I2-HOUS-KANS-VLAN-09184
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.61578%99.81235%
R&E 100G Backbone: I2-HOUS-LOSA-VLAN-09189
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.33288%99.46740%
R&E 100G Backbone: I2-KANS-SALT-VLAN-09185
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.84973%99.97002%
R&E 10G Backbone: I2-LOSA-PAIX-VLAN-23105
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.99184%100.00000%
Commodity 10G Backbone: I2-LOSA-PAIX-VLAN-23106
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.99184%100.00000%
R&E 100G Backbone: I2-LOSA-SALT-VLAN-09188
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.95959%99.99905%
R&E 100G Backbone: I2-LOSA-SEAT-VLAN-09187
               0 hr 1 min0 hr 1 min
               99.99008%99.99008%
               99.80077%99.88521%
R&E 100G Backbone: I2-LOSA-SEAT-VLAN-09328
               0 hr 1 min0 hr 1 min
               99.99008%99.99008%
               99.80077%99.88521%
R&E 100G Backbone: I2-LOSA-WILC-VLAN-23103
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.98918%99.99602%
Commodity 100G Backbone: I2-LOSA-WILC-VLAN-23104
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.98918%99.99602%
R&E 100G Backbone: I2-NEWY32AOA-WASH-VLAN-09176
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.89754%99.95959%
R&E 100G Backbone: I2-SALT-SEAT-VLAN-09186
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.96471%99.99962%
R&E 100G Backbone: I2-SALT-SEAT-VLAN-09327
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.96471%99.99962%
R&E 100G Backbone: I2-WASH-ATLA-VLAN-09179
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               100.00000%100.00000%
R&E 100G Backbone: I2-ATLA-WASH-VLAN-09318
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.72583%99.84271%
Legacy 10G Backbone: I2-ATLA-CHIC-10GE-05419
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.61388%99.90912%
Legacy 10G Backbone: I2-ATLA-HOUS-10GE-05423
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.36779%99.72355%
Legacy 10G Backbone: I2-ATLA-WASH-10GE-05251
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.93871%100.00000%
Legacy 10G Backbone: I2-ATLA-WASH-10GE-09142
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.93340%99.99829%
Legacy 10G Backbone: I2-CHIC-CLEV-10GE-08891
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.93663%99.99981%
Legacy 10G Backbone: I2-CHIC-KANS-10GE-05781
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               100.00000%100.00000%
Legacy 10G Backbone: I2-CHIC-KANS-10GE-05976
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.97647%99.99962%
Legacy 10G Backbone: I2-CLEV-NEWY32AOA-10GE-08894
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.96376%99.99962%
Legacy 10G Backbone: I2-CLEV-WASH-10GE-08889
               2 hr 57 min2 hr 57 min
               98.24405%98.24405%
               99.96148%99.96490%
Legacy 10G Backbone: I2-HOUS-KANS-10GE-05560
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.77212%99.87306%
Legacy 10G Backbone: I2-HOUS-KANS-LAG-08934
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               100.00000%100.00000%
Legacy 10G Backbone: I2-HOUS-LOSA-10GE-05559
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.74822%99.82734%
Legacy 10G Backbone: I2-KANS-SALT-10GE-05566
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.93245%99.99905%
Legacy 10G Backbone: I2-KANS-SALT-LAG-08935
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.93720%99.99924%
Legacy 10G Backbone: I2-LOSA-PAIX-10GE-05853
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.98805%100.00000%
Legacy 10G Backbone: I2-LOSA-SALT-10GE-05563
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.98103%100.00000%
Legacy 10G Backbone: I2-LOSA-SEAT-10GE-05564
               0 hr 3 min0 hr 3 min
               99.97024%99.97024%
               99.83493%99.85732%
Legacy 10G Backbone: I2-LOSA-SEAT-10GE-05572
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.99981%99.99981%
Legacy 100G Backbone: I2-LOSA-WILC-100GE-23893
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               100.00000%100.00000%
Legacy 10G Backbone: I2-NEWY32AOA-WASH-10GE-07612
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.99412%99.99810%
Legacy 10G Backbone: I2-SALT-SEAT-10GE-05565
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.99658%99.99981%
      
Aggregate All Unified - Internet2 IP Backbone Circuits
                   6 hr 37 min0 hr 38 min
                   99.91962%99.92732%
                   99.87762%99.94864%


 NOTE: Numbers in italics represent downtime with scheduled  maintenance excluded. Internet2's definition of scheduled
  maintenance is any anticipated outage announced at least 72
  hours in advance of the outage start time.  

Upcoming Scheduled Maintenances





Ticket No.:5218:144



Subject:Maintenance *Tracking* 2 of 2 - I2 Various Circuits



Affected:CA*Net/CANARIE
Net+ Provider Code42



Start Time:Thursday, August 27, 2015, 6:00 AM (0600) UTC



End Time:Thursday, August 27, 2015, 11:00 AM (1100) UTC



Description:   Internet2's Optical circuits should remain available
while Zayo Engineers perform power maintenance. An outage is not expected;
however, the entire maintenance window has been reserved. This is a backup 
window, to be used if necessary.



  







Ticket No.:5135:144



Subject:Maintenance - I2 AL2S Backbone Various Circuits



Affected:AL2S Backbone DENV-KANS
AL2S Backbone PORT-SUNN
AL2S Backbone DENV-ELPA
IP Backbone KANS-SALT
IP Backbone LOSA-SEAT



Start Time:Friday, August 28, 2015, 4:00 AM (0400) UTC



End Time:Friday, August 28, 2015, 10:00 AM (1000) UTC



Description:   The above listed asset will be unavailable while Level 3 Engineers perform 
maintenance.  An outage of up to 6 hours is
expected. The entire window is reserved.  Backbone traffic will
re-route during this maintenance. Workgroup administrators will
receive notifications from OESS regarding circuits re-routed on the
backbone.



  







Ticket No.:5274:144



Subject:Maintenance - I2 Optical Circuits I2-COLU4-KANS-100GE-11579 and I2-COLU4-SAIN-100GE-11580



Affected:AL2S Backbone COLU4-KANS and COLU4-SAIN



Start Time:Friday, August 28, 2015, 5:00 AM (0500) UTC



End Time:Friday, August 28, 2015, 10:00 AM (1000) UTC



Description:   The above listed assets may be unavailable
while MOREnet Engineers perform maintenance. A brief
outage is expected, however the entire maintenance window has been
reserved.



  







Ticket No.:4955:144



Subject:Maintenance - I2 Various Optical Circuits (Northern Tier)



Affected:Backbone MISS2-SEAT



Start Time:Saturday, August 29, 2015, 2:00 PM (1400) UTC



End Time:Sunday, August 30, 2015, 12:00 AM (0000) UTC



Description:   The above listed assets will be unavailable while Zayo Group's third party vendor 
relocates fiber. An outage up of up to an hour is expected; however the entire 
window has been reserved. Backbone traffic will
re-route during this maintenance. Workgroup administrators will
receive notifications from OESS regarding circuits re-routed on the
backbone.



  







Ticket No.:4436:144



Subject:Maintenance - I2 Optical Various Circuits (MINN)



Affected:AL2S Backbone MINN-MISS2
Optical Member Canarie/CA*net
Net+ Provider Code42



Start Time:Sunday, August 30, 2015, 1:00 PM (1300) UTC



End Time:Sunday, August 30, 2015, 10:00 PM (2200) UTC



Description:   The above listed asset will be unavailable while fiber providor performs
maintenance.  An
outage of 3 hours is expected. The entire window is reserved.  Backbone
traffic will
re-route during this maintenance. Workgroup administrators will receive
notifications from
OESS regarding circuits re-routed on the backbone.



  







Ticket No.:5302:144



Subject:Maintenance - I2 Optical Member ESNet



Affected:



Start Time:Monday, August 31, 2015, 12:00 PM (1200) UTC



End Time:Friday, September 4, 2015, 10:00 PM (2200) UTC



Description:   Member ESNet's above listed optical circuits will be unavailable
while CenturyLink performs maintenance to relocate fiber. Multiple 10 minute 
outages are expected during the window. The entire maintenance window has 
been reserved.



  







Ticket No.:5112:144



Subject:Maintenance - I2 Optical Member NOAA N-Wave



Affected:



Start Time:Tuesday, September 1, 2015, 4:00 AM (0400) UTC



End Time:Tuesday, September 1, 2015, 9:00 AM (0900) UTC



Description:   Member NOAA N-Wave's Optical circuit I2-PITT-WASH-10GE-07664 will be
unavailable
while N-Wave Engineers perform maintenance. A brief outage is expected,
however the
entire maintenance window has been reserved.



  







Ticket No.:5266:144



Subject:Maintenance Update - I2 Optical Various Circuits



Affected:CA*Net/CANARIE
Net+ Provider Code42



Start Time:Tuesday, September 1, 2015, 6:00 AM (0600) UTC



End Time:Tuesday, September 1, 2015, 11:00 AM (1100) UTC



Description:   Zayo's above optical circuits may be unavailable 
while Zayo Engineers performs power maintenance. No
outage is expected, however the entire maintenance window has been
reserved.



  







Ticket No.:5113:144



Subject:Maintenance - I2 Optical Member NOAA N-Wave



Affected:Optical Member NOAA



Start Time:Wednesday, September 2, 2015, 4:00 AM (0400) UTC



End Time:Wednesday, September 2, 2015, 9:00 AM (0900) UTC



Description:   Member NOAA N-Wave's Optical circuit I2-PITT-WASH-10GE-07664 will be
unavailable while N-Wave Engineers perform maintenance. A brief outage is
expected, however the entire maintenance window has been reserved.