I-Light Network Operations Report
2021-07-23 - 2021-08-05
I-Light Network Operations Center
| Incident Number | Customer Impact | Network Impact | Title | Outage Type | Source of Impact | Current State | Start Time (UTC) | End Time (UTC) | Duration |
|---|---|---|---|---|---|---|---|---|---|
| INC0088008 | 3 - Moderate | 3 - Moderate | Availability - I-Light Backbone CLMB-NEWA | Undetermined | Undetermined | Closed | 2021-07-23 06:55:27
2021-07-23 07:17:06 |
2021-07-23 06:55:37
2021-07-23 07:17:17 |
0 days 0 hr 0 min
0 days 0 hr 0 min |
| INC0088253 | 3 - Moderate | 3 - Moderate | Brief Outage - I-Light Various Backbones | Undetermined | Undetermined | Closed | 2021-07-27 03:17:39 | 2021-07-27 03:18:53 | 0 days 0 hr 1 min |
| INC0088494 | 3 - Moderate | 3 - Moderate | Brief Routing Instability - I-Light Provider Indiana GigaPOP | Investigating | Undetermined | Resolved | 2021-07-29 17:03:00 | 2021-07-29 17:18:00 | 0 days 0 hr 15 min |
| INC0088615 | 3 - Moderate | 3 - Moderate | Brief Routing Instability - I-Light Provider Indiana GigaPOP | Investigating | Undetermined | Resolved | 2021-07-31 00:50:00 | 2021-07-31 01:42:00 | 0 days 0 hr 52 min |
| INC0088798 | 3 - Moderate | 3 - Moderate | Brief Outage - I-Light Backbone BLTN-IPHE | Undetermined | Undetermined | Resolved | 2021-08-03 16:49:20 | 2021-08-03 16:50:43 | 0 days 0 hr 1 min |
| INC0088840 | 3 - Moderate | 2 - High | Brief Outage - I-Light Backbone IPHE-SBND | Undetermined | Undetermined | Resolved | 2021-08-04 05:22:44 | 2021-08-04 05:23:01 | 0 days 0 hr 0 min |
| Ticket No.: | INC0088008 |
| Subject: | Availability - I-Light Backbone CLMB-NEWA |
| Source Of Impact: | Undetermined |
| Affected: | ILN-CLMB-NEWA-100GE-24295
Backbone CLMB-NEWA Member IU School of Medicine Health Science Member Indiana State University Member Indiana State University Federal Hall Member Indiana University Southeast Member Indiana University Southeast Water Tower Square Member Ivy Tech Evansville Member Ivy Tech Sellersburg Member Oakland City University Member Purdue Polytechnic New Albany Member Rose Hulman Institute of Technology Member St. Mary of the Woods College Member University of Evansville Member University of Southern Indiana Member Vincennes University Fort Branch Member Vincennes University Jasper Member Vincennes University Plainfield Member Vincennes University Aviation Member Vincennes University Lebanon Member Wabash College |
| Current State: | Closed |
| Start Time: | 2021-07-23 06:55:27 |
| End Time: | 2021-07-23 06:55:37 |
| Outgoing Notification Text | Backbone CLMB-NEWA is now available following two (2) brief outages. Vendor Zayo Engineers have been engaged for further investigation into the root cause for the brief outages. The reason for brief outages is undetermined at this time. Traffic rerouted via redundant connections. |
| Ticket No.: | INC0088253 |
| Subject: | Brief Outage - I-Light Various Backbones |
| Source Of Impact: | Undetermined |
| Affected: | ILN-LFYT-MTAY-100GE-24336
ILN-IPIU-SBND-100GE-24338 Backbone IPIU-SBND Backbone LFYT-MTAY Backbone GARY-LFYT Backbone IPHE-IPIU Member Goshen College Member IU Foundation Member Indiana University Northwest Member Ivy Tech East Chicago Member Ivy Tech Gary Member Ivy Tech Lafayette Member Ivy Tech Michigan City Member Ivy Tech Northwest Member Ivy Tech Valparaiso Member Ivy Tech Warsaw Member Manchester University Ft. Wayne Campus Member Manchester University North Manchester Campus Member Michiana Hematology Member Purdue NorthWest Hammond Campus Member Purdue NorthWest Westville Campus Member Purdue Polytechnic Anderson Member Purdue Polytechnic South Bend Member Purdue University Member Purdue University Ft. Wayne Member Trine University Member Valparaiso University Member Wabash College Member University of Notre Dame Member WNIT Michiana Public Broadcasting Corporation Member University of St. Francis Member Huntington University Member Indiana Wesleyan University Member Ivy Tech Fort Wayne Member Indiana University South Bend Member Ivy Tech South Bend Member Ivy Tech Richmond Member Indiana University East Member Ball State University Member Ivy Tech Muncie Member Ivy Tech Marion Member Ivy Tech Lawrenceburg Member Ivy Tech Elkhart Member Indiana University Fort Wayne Member Indiana Institute of Technology Ft. Wayne Member Huntington University Member Holy Cross College Member Earlham College Member Ivy Tech Connersville Member Bethel University Member Anderson University |
| Current State: | Closed |
| Start Time: | 2021-07-27 03:17:39 |
| End Time: | 2021-07-27 03:18:53 |
| Outgoing Notification Text | The listed was briefly unavailable. Vendor Zayo Engineers have been engaged to investigate the root cause for the brief outage observed. The reason for brief outage is undetermined at this time. Further updates to be provided as they are made available. Traffic rerouted via redundant connections. |
| Ticket No.: | INC0088494 |
| Subject: | Brief Routing Instability - I-Light Provider Indiana GigaPOP |
| Source Of Impact: | Undetermined |
| Affected: | ILN-ICTC-IPIU-100GE-25087
Backbone ICTC-IPIU Member Wabash College |
| Current State: | Resolved |
| Start Time: | 2021-07-29 17:03:00 |
| End Time: | 2021-07-29 17:18:00 |
| Outgoing Notification Text | Provider Indiana GigaPOP Engineers installed a new MPC in rtr.ictc.indiana.gigapop.net at approximately 1:03 PM ET. At completion of the MPC booting, alarms were raised against the chassis fabric on Core Node ICTC. Initial observation indicated only the new MPC was impacted; however, alarms for connectivity were raised shortly after. The new MPC was offlined at 1:18 PM ET, which cleared the alarms for connectivity. Provider Indiana GigaPOP Engineers are currently investigating with Vendor Juniper to determine the cause of the fabric impact. Re-creating may not be possible and attempts to install the new MPC will be done via scheduled maintenance. |
| Ticket No.: | INC0088615 |
| Subject: | Brief Routing Instability - I-Light Provider Indiana GigaPOP |
| Source Of Impact: | Undetermined |
| Affected: | ILN-ICTC-IPIU-100GE-25087
Backbone ICTC-IPIU Member Wabash College |
| Current State: | Resolved |
| Start Time: | 2021-07-31 00:50:00 |
| End Time: | 2021-07-31 01:42:00 |
| Outgoing Notification Text | Provider Indiana GigaPOP's previously troublesome and "offlined" MPC in rtr.ictc.indiana.gigapop.net suddenly came back online this evening around 8:50 PM ET. At completion of the MPC booting, alarms were raised against the chassis fabric on Core Node ICTC. Initial observation indicated only the new MPC was impacted; however, alarms for connectivity were raised shortly after. The new MPC was again offlined at 9:42 PM ET, which cleared the alarms for connectivity. Shortly after this shutdown, this card was physically removed from the chassis. Provider Indiana GigaPOP Engineers are continuing the investigation with Vendor Juniper. A fresh set of logs and update has been sent to Juniper for further investigation. |
| Ticket No.: | INC0088798 |
| Subject: | Brief Outage - I-Light Backbone BLTN-IPHE |
| Source Of Impact: | Undetermined |
| Affected: | ILN-BLTN-IPIU-100GE-04017
Backbone BLTN-IPHE Member Vincennes University Plainfield Member Vincennes University Lebanon Member Vincennes University Jasper Member Vincennes University Aviation Member WYIN Lakeshore Public TV Member WNIN Tri State Public Media Evansville Member Ivy Tech Columbus Member Indiana University Southeast Water Tower Square Member IU Foundation Member St. Francis Crown Point Member Martin University Member Ivy Tech Bloomington Member IU Research Network Member Indiana University Purdue University Columbus Learning Center Member Franklin College Member Delaware County CES Member Indiana University South Bend Member Indiana University East Member Indiana University Fort Wayne Member Indiana University Kokomo Member IU School of Medicine Health Science Member Indiana Office of Technology Member Indiana University Northwest Peer Indiana GigaPOP |
| Current State: | Resolved |
| Start Time: | 2021-08-03 16:49:20 |
| End Time: | 2021-08-03 16:50:43 |
| Outgoing Notification Text | Backbone BLTN-IPHE was briefly unavailable. I-Light Engineering is engaged and investigating the root cause for the outage. The reason for outage is undetermined at this time. Further updates to be provided as they are made available. Traffic rerouted via redundant connections. |
| Ticket No.: | INC0088840 |
| Subject: | Brief Outage - I-Light Backbone IPHE-SBND |
| Source Of Impact: | Undetermined |
| Affected: | ILN-IPHE-SBND-100GE-24342
Backbone IPHE-SBND Member Bethel University Member Holy Cross College Member IU Foundation Member Indiana University Northwest Member Indiana University South Bend Member Ivy Tech East Chicago Member Ivy Tech Elkhart Member Ivy Tech Gary Member Ivy Tech Lafayette Member Ivy Tech Michigan City Member Ivy Tech Northwest Member Ivy Tech South Bend Member Ivy Tech Valparaiso Member Michiana Hematology Member Purdue NorthWest Hammond Campus Member Purdue NorthWest Westville Campus Member Valparaiso University Member WNIT Michiana Public Broadcasting Corporation Member Wabash College Member University of Notre Dame |
| Current State: | Resolved |
| Start Time: | 2021-08-04 05:22:44 |
| End Time: | 2021-08-04 05:23:01 |
| Outgoing Notification Text | Backbone IPHE-SBND was briefly unavailable. Vendor Zayo Engineers have been engaged to investigate the root cause for the brief outage observed. The reason for brief outage is undetermined at this time. Further updates to be provided as they are made available. Traffic rerouted via redundant connections. |
| Ticket No.: | CHG0053374 |
| Subject: | Maintenance - I-Light Member Vincennes University - Fort Branch |
| Source Of Impact: | Vendor |
| Affected: | Member Vincennes University - Fort Branch |
| Start Time: | 2021-08-10 05:00:00 |
| End Time: | 2021-08-10 09:00:00 |
| Outgoing Notification Text | Member Vincennes University - Fort Branch will be unavailable while Vendor WOW Business Engineers performs maintenance to conduct a software update. An outage of twenty (20) minutes is expected. The entire window is reserved. |
| Ticket No.: | CHG0053548 |
| Subject: | Maintenance Completed - I-Light Member Vincennes University |
| Source Of Impact: | Internal |
| Affected: | Member Vincennes University |
| Start Time: | 2021-08-06 01:00:00 |
| End Time: | 2021-08-06 04:00:00 |
| Outgoing Notification Text |