Unified - Internet2 IP Network Operations Report
09/08/2014 - 09/14/2014

Unified - Internet2 IP Network Operations Center
at Indiana University

New Connections

None


Upgrades

None

New Services

None



Unscheduled Outages Summary







Ticket Number
Customer Impact
Network Impact
Title
Outage Type
Source Of Impact
Start Time (UTC)

End Time (UTC)





3521-Critical
1-Critical

Brief Outage Resolved - I2 Optical Various Circuits (HOUS)
UndeterminedUndetermined
09/08/2014 4:59 PM
09/08/2014 5:00 PM


3701-Critical
1-Critical

Outage Resolved - I2 Optical DWDM System HOUS-JCSN
Cut or Damaged FiberVendor
09/09/2014 2:43 PM
09/10/2014 12:34 AM


3751-Critical
1-Critical

Brief Outage - I2 Optical DWDM System DENV-SALT
UndeterminedVendor
09/09/2014 7:48 PM
09/09/2014 7:52 PM






Unscheduled Outages Detail






Ticket No.:352:144



Subject:Brief Outage Resolved - I2 Optical Various Circuits (HOUS)



Source Of Impact:Undetermined



Affected:Optical Various Circuits
AL2S Backbone ELPA-HOUH
IP Backbone HOUS-LOSA



Start Time:Monday, September 8, 2014, 4:59 PM (1659) UTC



End Time:Monday, September 8, 2014, 5:00 PM (1700) UTC


Description:The above listed were briefly unavailable to the community.  Level (3)
Communications technicians report the reason for outage is undetermined.







Ticket No.:370:144



Subject:Outage Resolved - I2 Optical DWDM System HOUS-JCSN



Source Of Impact:Vendor



Affected:DWDM System HOUS-JCSN
AL2S Backbone HOUH-JCSN
IP Connector Mission
NDDI Backbone LOSA-RALE



Start Time:Tuesday, September 9, 2014, 2:43 PM (1443) UTC



End Time:Wednesday, September 10, 2014, 12:34 AM (0034) UTC


Description:DWDM System HOUS-JCSN is now available to the community.  Level (3)
Communications has confirmed that the outage was due to a fiber cut
caused by
construction. Splicing has been completed and services have restored.
No further outages are expected.  IP and AL2S Backbones re-routed.
Workgroup administrators received notifications from OESS regarding
circuits re-routed on the backbone.







Ticket No.:375:144



Subject:Brief Outage - I2 Optical DWDM System DENV-SALT



Source Of Impact:Vendor



Affected:DWDM System DENV-SALT
AL2S Backbone DENV-SALT
IP Backbone KANS-SALT
NDDI Backbone CHIC-SEAT
IP Connector FRGP (SALT)*



Start Time:Tuesday, September 9, 2014, 7:48 PM (1948) UTC



End Time:Tuesday, September 9, 2014, 7:52 PM (1952) UTC


Description:DWDM System DENV-SALT was briefly unavailable to the community.  Level (3) 
Communications technicians are currently investigating the reason for the outage.  Layer 
2 and 3 backbone traffic rerouted during the outage.  Workgroup administrators receive 
notifications from OESS regarding circuits re-routed on the backbone. *Denotes 
multihomed and entity remained available.







Scheduled Maintenances Summary





Ticket Number
Customer Impact
Network Impact
Title

Maintenance Type
Source Of Impact
Start Time (UTC)
End Time (UTC)





268
3-Elevated
1-Critical

Maintenance Completed - I2 Optical DWDM System BOST-NEWY
Hardware
Internal09/09/2014 4:08 AM
09/09/2014 5:19 AM


269
3-Elevated


Maintenance Completed - I2 Optical DWDM System BOST-NEWY
CircuitInternal
09/10/2014 4:31 AM
09/10/2014 6:29 AM




Scheduled Maintenances Detail





Ticket No.:268:144



Subject:Maintenance Completed - I2 Optical DWDM System BOST-NEWY



Source Of Impact:Internal



Affected:DWDM System BOST-NEWY
AL2S Backbone BOST-NEWY
AL2S Connector NOX
NDDI Backbone CHIC-NEWY
IP Connector OARnet (CLEV)*
IP Connector NOX (WASH)*



Start Time:Tuesday, September 9, 2014, 4:08 AM (0408) UTC



End Time:Tuesday, September 9, 2014, 5:19 AM (0519) UTC


Description:   DWDM System BOST-NEWY is now available after a brief outage while
Internet2
Engineers with the assistance of Level(3) Communications
replaced a fuse in  Hartford, CT.   The maintenance is completed. No
further outages are expected related to this event.
* Denotes that the Entity is multihomed and therefore should
have remained reachable.



  







Ticket No.:269:144



Subject:Maintenance Completed - I2 Optical DWDM System BOST-NEWY



Source Of Impact:Internal



Affected:DWDM System BOST-NEWY
AL2S Backbone BOST-NEWY
AL2S Connector NOX
NDDI Backbone CHIC-NEWY
IP Connector OARnet (CLEV)*
IP Connector NOX (WASH)*



Start Time:Wednesday, September 10, 2014, 4:31 AM (0431) UTC



End Time:Wednesday, September 10, 2014, 6:29 AM (0629) UTC


Description:   DWDM System BOST-NEWY was unavailable while Internet2 Engineers with 
the assistance of Level(3) Communications converted a site from an 
amplifier to an add/drop in Hartford, CT.   Maintenance has been 
successfully completed.
* Denotes that the Entity is multihomed and therefore should remain 
reachable.



  










Downtime and Availability







Unified - Internet2 IP Core NodesDown TimeReporting Period Availability52 Week Availability


ATLA MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%  100.00000%
CHIC MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%  100.00000%
CLEV MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%  100.00000%
HOUS MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%  100.00000%
KANS MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%  100.00000%
LOSA MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%  100.00000%
NEWY MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%  100.00000%
SALT MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%  100.00000%
SEAT MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%  100.00000%
WASH MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%  100.00000%
rtsw0.sing1.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%  100.00000%
rtsw.sing2.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%  100.00000%
      
Aggregate Unified - Internet2 IP Core Nodes0 hr 0 min0 hr 0 min100.00000%100.00000%100.00000%100.00000%

 NOTE: Numbers in italics represent downtime with scheduled  maintenance excluded. Internet2's definition of scheduled
  maintenance is any anticipated outage announced at least 72
  hours in advance of the outage start time.  
	    

	    
Unified - Internet2 IP Backbone CircuitsDown Time
Reporting Period Availability52 Week Availability


R&E 100G Backbone: I2-ATLA-CHIC-VLAN-09181
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.94061%99.99962%
R&E 100G Backbone: I2-ATLA-HOUS-VLAN-09182
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.99677%99.99943%
R&E 100G Backbone: I2-ATLA-WASH-VLAN-09179
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.99753%100.00000%
R&E 100G Backbone: I2-CHIC-CLEV-VLAN-09180
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.98235%99.99943%
R&E 100G Backbone: I2-CHIC-KANS-VLAN-09183
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.99488%99.99962%
R&E 100G Backbone: I2-CLEV-NEWY32AOA-VLAN-09177
               3 hr 9 min0 hr 0 min
               98.12500%100.00000%
               99.96395%99.99981%
R&E 100G Backbone: I2-CLEV-WASH-VLAN-09178
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.99279%100.00000%
R&E 100G Backbone: I2-HOUS-KANS-VLAN-09184
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.99279%99.99981%
R&E 100G Backbone: I2-HOUS-LOSA-VLAN-09189
               0 hr 1 min0 hr 1 min
               99.99008%99.99008%
               99.99981%99.99981%
R&E 100G Backbone: I2-KANS-SALT-VLAN-09185
               0 hr 4 min0 hr 4 min
               99.96032%99.96032%
               99.97040%99.99924%
R&E 100G Backbone: I2-LOSA-SALT-VLAN-09188
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               100.00000%100.00000%
R&E 100G Backbone: I2-LOSA-SEAT-VLAN-09187
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.99222%100.00000%
R&E 100G Backbone: I2-LOSA-SEAT-VLAN-09328
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.99222%100.00000%
R&E 100G Backbone: I2-NEWY32AOA-WASH-VLAN-09176
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.99962%99.99962%
R&E 100G Backbone: I2-SALT-SEAT-VLAN-09186
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               100.00000%100.00000%
R&E 100G Backbone: I2-SALT-SEAT-VLAN-09327
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               100.00000%100.00000%
R&E 100G Backbone: I2-WASH-ATLA-VLAN-09179
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               100.00000%100.00000%
R&E 100G Backbone: I2-ATLA-WASH-VLAN-09318
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.99753%100.00000%
Legacy 10G Backbone: I2-ATLA-CHIC-10GE-05419
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.94099%100.00000%
Legacy 10G Backbone: I2-ATLA-HOUS-10GE-05423
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.83777%99.83834%
Legacy 10G Backbone: I2-ATLA-WASH-10GE-05251
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               100.00000%100.00000%
Legacy 10G Backbone: I2-ATLA-WASH-10GE-09142
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               100.00000%100.00000%
Legacy 10G Backbone: I2-CHIC-CLEV-10GE-08891
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.98558%100.00000%
Legacy 10G Backbone: I2-CHIC-KANS-10GE-05781
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               100.00000%100.00000%
Legacy 10G Backbone: I2-CHIC-KANS-10GE-05976
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               100.00000%100.00000%
Legacy 10G Backbone: I2-CLEV-NEWY32AOA-10GE-08894
               3 hr 9 min0 hr 0 min
               98.12500%100.00000%
               99.96376%99.99962%
Legacy 10G Backbone: I2-CLEV-WASH-10GE-08889
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               100.00000%100.00000%
Legacy 10G Backbone: I2-HOUS-KANS-10GE-05560
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               100.00000%100.00000%
Legacy 10G Backbone: I2-HOUS-KANS-LAG-08934
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               100.00000%100.00000%
Legacy 10G Backbone: I2-HOUS-LOSA-10GE-05559
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               100.00000%100.00000%
Legacy 10G Backbone: I2-KANS-SALT-10GE-05566
               0 hr 4 min0 hr 4 min
               99.96032%99.96032%
               99.97382%99.99924%
Legacy 10G Backbone: I2-KANS-SALT-LAG-08935
               0 hr 4 min0 hr 4 min
               99.96032%99.96032%
               99.97382%99.99924%
Legacy 10G Backbone: I2-LOSA-SALT-10GE-05563
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               100.00000%100.00000%
Legacy 10G Backbone: I2-LOSA-SEAT-10GE-05564
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.99222%100.00000%
Legacy 10G Backbone: I2-LOSA-SEAT-10GE-05572
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.99431%100.00000%
Legacy 10G Backbone: I2-NEWY32AOA-WASH-10GE-07612
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               100.00000%100.00000%
Legacy 10G Backbone: I2-SALT-SEAT-10GE-05565
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               100.00000%100.00000%
      
Aggregate All Unified - Internet2 IP Backbone Circuits
                   6 hr 31 min6 hr 18 min
                   99.89516%99.99651%
                   99.98583%99.99548%


 NOTE: Numbers in italics represent downtime with scheduled  maintenance excluded. Internet2's definition of scheduled
  maintenance is any anticipated outage announced at least 72
  hours in advance of the outage start time.  

Upcoming Scheduled Maintenances





Ticket No.:183:144



Subject:Maintenance - I2 Optical DWDM System PHIL-WASH



Affected:NDDI Backbone NEWY-WASH
AL2S Backbone PHIL-WASH
IP Backbone NEWY32AoA-WASH
Connector OARnet (WASH)*
Connector MAX Gigapop (WASH)*
Connector MAGPI
Connector Drexel



Start Time:Thursday, September 18, 2014, 4:00 AM (0400) UTC



End Time:Thursday, September 18, 2014, 6:00 AM (0600) UTC



Description:   DWDM System PHIL-WASH will be unavailable while Level(3) Communications performs 
circuit maintenance in Perryville and Baltimore, MD.  An outage is expected, and the 
entire window is reserved.

*Denotes that the Entity is multihomed and therefore will remain reachable.*



  







Ticket No.:185:144



Subject:Maintenance - I2 Optical DWDM System HOUS-SANA



Affected:DWDM System HOUS-SANA
AL2S Backbone ELPA-HOUH
IP Backbone HOUS-LOSA
NDDI Backbone LOSA-WASH2



Start Time:Thursday, September 18, 2014, 5:00 AM (0500) UTC



End Time:Thursday, September 18, 2014, 11:00 AM (1100) UTC



Description:   DWDM System HOUS-SANA will be unavailable while Level(3)
Communications performs maintenance to repair damage from natural
causes in Giddings, TX.  An outage is expected, and the entire window is
reserved.



  







Ticket No.:409:144



Subject:Maintenance - I2 Optical DWDM System DENV-KANS



Affected:DWDM System DENV-KANS
AL2S Backbone DENV-KANS
NDDI Backbone CHIC-SEAT
IP Backbone KANS-SALT



Start Time:Friday, September 19, 2014, 6:00 AM (0600) UTC



End Time:Friday, September 19, 2014, 10:00 AM (1000) UTC



Description:   DWDM System DENV-KANS will be unavailable while Level(3)
Communications perform hardware maintenance in Bethune, CO.
Outages are expected; the entire window has been reserved.



  







Ticket No.:456:144



Subject:Maintenance  - I2 AL2S Backbone BOST-NEWY32AOA



Affected:AL2S Backbone BOST-NEWY32AOA
IP Backbone CLEV-NEWY32AOA



Start Time:Monday, September 22, 2014, 8:00 AM (0800) UTC



End Time:Monday, September 22, 2014, 10:00 AM (1000) UTC



Description:   The above listed will be unavailable while Internet2 Engineers perform
maintenance to insert a new AL2S Node in Hartford, CT.  An outage is
expected, and
the entire window is reserved.  Backbone traffic will re-route during
this maintenance.
Workgroup administrators will receive notifications from OESS
regarding circuits
re-routed on the backbone.



  







Ticket No.:73:144



Subject:Maintenance Rescheduled 1 of 2 - I2 Optical Various Circuits (Northern Tier)



Affected:Northern Tier Optical Circuits
Optical Member Canarie



Start Time:Tuesday, September 23, 2014, 7:00 AM (0700) UTC



End Time:Tuesday, September 23, 2014, 12:00 PM (1200) UTC



Description:   *Please Note the Updated Maintenance Window*
The above listed Optical circuits will be unavailable while Zayo Engineers
perform maintenance. A outage of up to 3 hours is expected, however the
entire maintenance window has been reserved.

* This is the 1st of 2 maintenance windows *



  







Ticket No.:74:144



Subject:Maintenance Rescheduled 2 of 2 - I2 Optical Various Circuits (Northern Tier)



Affected:Northern Tier Optical Circuits
Optical Member Canarie



Start Time:Wednesday, September 24, 2014, 7:00 AM (0700) UTC



End Time:Wednesday, September 24, 2014, 12:00 PM (1200) UTC



Description:   *Please Note the Updated Maintenance Window*
The above listed Optical circuits will be unavailable while Zayo
Engineers perform maintenance. A outage of up to 3 hours is expected,
however the entire maintenance window has been reserved.

* This is the 2nd of 2 maintenance windows *



  







Ticket No.:454:144



Subject:Maintenance 2 of 2 - I2 Optical Member CENIC



Affected:



Start Time:Wednesday, September 24, 2014, 7:00 AM (0700) UTC



End Time:Wednesday, September 24, 2014, 1:00 PM (1300) UTC



Description:   Member CENIC's above-listed Optical circuits will be unavailable
while CENIC Engineers perform maintenance. A brief
outage is expected, however the entire maintenance window has been
reserved.
*This is maintenance is 2 of 2.*



  







Ticket No.:414:144



Subject:Maintenance Rescheduled - I2 Optical Circuit I2-COLU4-KANS-100GE-11579 and I2-COLU4-SAIN-100GE-11580



Affected:AL2S Backbone CHIC-COLU4
AL2S Backbone COLU4-KANS



Start Time:Friday, September 26, 2014, 5:00 AM (0500) UTC



End Time:Friday, September 26, 2014, 10:00 AM (1000) UTC



Description:   *This maintenance has been rescheduled*
The above listed circuits will be unavailable while MOREnet Fiber Vendor Engineers relocate fiber in Stanton, MO.  An outage of up to 3 hours is expected, however the 
entire maintenance window has been reserved.



  







Ticket No.:266:144



Subject:Maintenance *Tracking* - I2 Optical Various Circuits (Northern Tier)



Affected:Northern Tier Optical Circuits



Start Time:Monday, September 29, 2014, 5:00 AM (0500) UTC



End Time:Friday, October 3, 2014, 10:00 AM (1000) UTC



Description:   The above listed Optical circuits will remain available while Zayo Engineers perform 
maintenance.  No outages are expected, however the entire maintenance window has 
been reserved.



  







Ticket No.:373:144



Subject:Maintenance Window 1 of 4 - I2 Optical Circuits I2-FARG-SEAT-10GE-09204 & I2-EQCH-FARG-10GE-09203



Affected:Optical Circuits EQCH-FARG and FARG-SEAT



Start Time:Tuesday, October 7, 2014, 6:00 AM (0600) UTC



End Time:Tuesday, October 7, 2014, 11:00 AM (1100) UTC



Description:   The above listed will become unavailable while vendor engineers conduct 
maintenance to insert a new Infinera Node in Helena, Montana. An outage 
of 3 hours is anticipated; however the entire window has been reserved. 
This is maintenance window 1 of 4.



  







Ticket No.:387:144



Subject:Maintenance Window 2 of 4 - I2 Optical Circuits I2-FARG-SEAT-10GE-09204 & I2-EQCH-FARG-10GE-09203



Affected:Optical Circuits EQCH-FARG and FARG-SEAT



Start Time:Wednesday, October 8, 2014, 6:00 AM (0600) UTC



End Time:Wednesday, October 8, 2014, 11:00 AM (1100) UTC



Description:   The above listed will become unavailable while vendor engineers conduct
maintenance to insert a new Infinera Node in Helena, Montana. An outage
of 3 hours is anticipated; however the entire window has been reserved.
This is maintenance window 2 of 4.



  







Ticket No.:388:144



Subject:Maintenance Window 3 of 4 - I2 Optical Circuits I2-FARG-SEAT-10GE-09204 & I2-EQCH-FARG-10GE-09203



Affected:Optical Circuits EQCH-FARG and FARG-SEAT



Start Time:Thursday, October 9, 2014, 6:00 AM (0600) UTC



End Time:Thursday, October 9, 2014, 11:00 AM (1100) UTC



Description:   The above listed will become unavailable while vendor engineers conduct
maintenance to insert a new Infinera Node in Helena, Montana. An outage
of 3 hours is anticipated; however the entire window has been reserved.
This is maintenance window 3 of 4.



  







Ticket No.:389:144



Subject:Maintenance Window 4 of 4 - I2 Optical Circuits I2-FARG-SEAT-10GE-09204 & I2-EQCH-FARG-10GE-092034



Affected:Optical Circuits EQCH-FARG and FARG-SEAT



Start Time:Friday, October 10, 2014, 6:00 AM (0600) UTC



End Time:Friday, October 10, 2014, 11:00 AM (1100) UTC



Description:   The above listed will become unavailable while vendor engineers conduct
maintenance to insert a new Infinera Node in Helena, Montana. An outage
of 3 hours is anticipated; however the entire window has been reserved.
This is maintenance window 4 of 4.