SOX Network Operations Report
2018-11-01 - 2018-11-30
SOX Network Operations Center
Connector | Type | Date | Sponsor |
---|
None
None
None
Incident Number | Customer Impact | Network Impact | Title | Outage Type | Source of Impact | Current State | Start Time (UTC) | End Time (UTC) |
---|---|---|---|---|---|---|---|---|
INC0022594 | 3 - Moderate | 3 - Moderate | Availability - SoX Participant University of Alabama in Huntsville | Undetermined | Undetermined | Resolved | 2018-11-03 12:32:00 | 2018-11-03 21:26:06 |
INC0024247 | 3 - Moderate | 3 - Moderate | Availability - SoX Participant Emory | Undetermined | Undetermined | Closed | 2018-11-26 00:14:14 | 2018-11-26 00:52:00 |
INC0024318 | 3 - Moderate | 3 - Moderate | Availability - SoX Node 365DC (Management) | Undetermined | Undetermined | Resolved | 2018-11-26 17:59:40 | 2018-11-26 20:02:17 |
INC0024212 | 3 - Moderate | 3 - Moderate | Availability - SoX Participant Emory University | Undetermined | Undetermined | Resolved | 2018-11-25 12:50:58 | 2018-11-25 21:54:46 |
INC0024267 | 3 - Moderate | 3 - Moderate | Availability - SoX Peer Google | Undetermined | Undetermined | Resolved | 2018-11-26 07:08:47 | 2018-11-26 07:19:20 |
INC0024507 | 3 - Moderate | 3 - Moderate | Brief Outage - SoX Participant CDC | Undetermined | Undetermined | Closed | 2018-11-28 15:07:36 | 2018-11-28 15:28:38 |
INC0024270 | 3 - Moderate | 3 - Moderate | Availability - SoX Node 365DC (Management) | Undetermined | Undetermined | Resolved | 2018-11-26 08:14:29 | 2018-11-26 10:28:13 |
INC0024361 | 3 - Moderate | 3 - Moderate | Availability - SoX Core Node 56MAR OOB | Undetermined | Undetermined | Resolved | 2018-11-27 05:43:00 | 2018-11-27 06:12:52 |
INC0023226 | 3 - Moderate | 3 - Moderate | Availability - SoX Core Node 56MAR OOB | Undetermined | Undetermined | Resolved | 2018-11-12 06:20:37 | 2018-11-12 06:38:50 |
INC0024590 | 3 - Moderate | 3 - Moderate | Availability - SoX Core Node 56MAR OOB | Other | Internal | Resolved | 2018-11-29 07:33:56 | 2018-11-29 08:03:33 |
INC0023458 | 3 - Moderate | 3 - Moderate | Availability - SoX Node 365DC (Management) | Undetermined | Undetermined | Resolved | 2018-11-14 09:51:27 | 2018-11-14 10:39:00 |
INC0022750 | 3 - Moderate | 3 - Moderate | Outage Resolved - SoX Participant CLIGHT | Unannounced Maintenance | Undetermined | Resolved | 2018-11-06 05:26:42 | 2018-11-06 06:41:51 |
INC0024571 | 3 - Moderate | 3 - Moderate | Availability - SoX Participant Auburn | Undetermined | Undetermined | Resolved | 2018-11-29 01:17:00 | 2018-11-29 06:39:00 |
INC0023844 | 3 - Moderate | 3 - Moderate | Availability - SoX Participant Oak Ridge National Labs | Undetermined | Undetermined | Resolved | 2018-11-19 11:49:04 | 2018-11-19 17:53:00 |
Ticket No.: | INC0022594 |
Subject: | Availability - SoX Participant University of Alabama in Huntsville |
Source Of Impact: | Undetermined |
Affected: | SOX-56MAR-56MAR-10GE-01658
Participant University of Alabama in Huntsville |
Current State: | Resolved |
Start Time: | 2018-11-03 12:32:00 |
End Time: | 2018-11-03 21:26:06 |
Description | Participant University of Alabama in Huntsville is now available. University of Alabama in Huntsville Engineers have been contacted; however, the reason for the outage is pending. |
Ticket No.: | INC0024247 |
Subject: | Availability - SoX Participant Emory |
Source Of Impact: | Undetermined |
Affected: | SoX-56MAR-56MAR-GIGE-01789
Participant Emory |
Current State: | Closed |
Start Time: | 2018-11-26 00:14:14 |
End Time: | 2018-11-26 00:52:00 |
Description | SoX Participant Emory is now available. Emory Engineers have been contacted; however the reason for outage is pending. |
Ticket No.: | INC0024318 |
Subject: | Availability - SoX Node 365DC (Management) |
Source Of Impact: | Undetermined |
Affected: | mgmtsw.365dc.sox.net
oob.365dc.sox.net Node 365DC (Management) |
Current State: | Resolved |
Start Time: | 2018-11-26 17:59:40 |
End Time: | 2018-11-26 20:02:17 |
Description | The 365 Data Center OOB is now available. 365 Data Center Engineers have been contacted; however the reason for outage is pending. |
Ticket No.: | INC0024212 |
Subject: | Availability - SoX Participant Emory University |
Source Of Impact: | Undetermined |
Affected: | SoX-56MAR-56MAR-GIGE-01789
Participant Emory University |
Current State: | Resolved |
Start Time: | 2018-11-25 12:50:58 |
End Time: | 2018-11-25 21:54:46 |
Description | Participant Emory University is Available. Emory University Engineers have been contacted however; the reason for the outage is pending. |
Ticket No.: | INC0024267 |
Subject: | Availability - SoX Peer Google |
Source Of Impact: | Undetermined |
Affected: | SOX-56MAR-56MAR-LAG-01922
Peer Google |
Current State: | Resolved |
Start Time: | 2018-11-26 07:08:47 |
End Time: | 2018-11-26 07:19:20 |
Description | SoX Peer Google is now available. Google Engineers have been contacted; however the reason for outage is pending. |
Ticket No.: | INC0024507 |
Subject: | Brief Outage - SoX Participant CDC |
Source Of Impact: | Undetermined |
Affected: | SOX-56MAR-56MAR-10GE-01715
Participant CDC |
Current State: | Closed |
Start Time: | 2018-11-28 15:07:36 |
End Time: | 2018-11-28 15:28:38 |
Description | Participant CDC was briefly unavailable. Center for Disease Control Engineers have been engaged; however, the reason for the outage is pending. |
Ticket No.: | INC0024270 |
Subject: | Availability - SoX Node 365DC (Management) |
Source Of Impact: | Undetermined |
Affected: | oob.365dc.sox.net
mgmtsw.365dc.sox.net Node 365DC |
Current State: | Resolved |
Start Time: | 2018-11-26 08:14:29 |
End Time: | 2018-11-26 10:28:13 |
Description | The 365 Data Center Management Nodes are now available. 365DC and SoX Engineers have been contacted; however the reason for outage is pending. |
Ticket No.: | INC0024361 |
Subject: | Availability - SoX Core Node 56MAR OOB |
Source Of Impact: | Undetermined |
Affected: | oob.56mar.sox.net
Core Node 56MAR OOB |
Current State: | Resolved |
Start Time: | 2018-11-27 05:43:00 |
End Time: | 2018-11-27 06:12:52 |
Description | The OOB for Core Node 56 Marietta St is Available. SoX Engineers have been contacted to investigate; however the reason for outage is pending. |
Ticket No.: | INC0023226 |
Subject: | Availability - SoX Core Node 56MAR OOB |
Source Of Impact: | Undetermined |
Affected: | oob.56mar.sox.net
Core Node 56MAR OOB |
Current State: | Resolved |
Start Time: | 2018-11-12 06:20:37 |
End Time: | 2018-11-12 06:38:50 |
Description | The OOB device at 56 Marietta St is now available. SoX Engineers have been contacted to investiage; however the reason for outage is pending. |
Ticket No.: | INC0024590 |
Subject: | Availability - SoX Core Node 56MAR OOB |
Source Of Impact: | Internal |
Affected: | oob.56mar.sox.net
Core Node 56MAR OOB |
Current State: | Resolved |
Start Time: | 2018-11-29 07:33:56 |
End Time: | 2018-11-29 08:03:33 |
Description | The 56 Marietta OOB is now available. SoX Engineers investigated and determined the reason for outage was a temporary drop in LTE connectivity. The cause of the LTE drop is undetermined. No further outages are expected. |
Ticket No.: | INC0023458 |
Subject: | Availability - SoX Node 365DC (Management) |
Source Of Impact: | Undetermined |
Affected: | mgmtsw.365dc.sox.net
oob.365dc.sox.net Node 365DC (Management) |
Current State: | Resolved |
Start Time: | 2018-11-14 09:51:27 |
End Time: | 2018-11-14 10:39:00 |
Description | Various management nodes at the 365 Data Center POP are now available. 365 Data Center and SoX Engineers have been contacted; however the reason for outage is pending. All alarms have cleared. |
Ticket No.: | INC0022750 |
Subject: | Outage Resolved - SoX Participant CLIGHT |
Source Of Impact: | Undetermined |
Affected: | SOX-56MAR-56MAR-10GE-01608
Participant CLIGHT |
Current State: | Resolved |
Start Time: | 2018-11-06 05:26:42 |
End Time: | 2018-11-06 06:41:51 |
Description | SoX Participant CLIGHT is Available. Pinged Internet2 and found that CLIGHT is doing internal maintenance affecting SoX connection. |
Ticket No.: | INC0024571 |
Subject: | Availability - SoX Participant Auburn |
Source Of Impact: | Undetermined |
Affected: | SOX-56MAR-56MAR-10GE-01847
Participant Auburn |
Current State: | Resolved |
Start Time: | 2018-11-29 01:17:00 |
End Time: | 2018-11-29 06:39:00 |
Description | Participant Auburn is now available. Auburn Engineers have been contacted; however the reason for the outage is pending. |
Ticket No.: | INC0023844 |
Subject: | Availability - SoX Participant Oak Ridge National Labs |
Source Of Impact: | Undetermined |
Affected: | SOX-56MAR-56MAR-10GE-01708
Participant Oak Ridge National Labs |
Current State: | Resolved |
Start Time: | 2018-11-19 11:49:04 |
End Time: | 2018-11-19 17:53:00 |
Description | Participant Oak Ridge National Labs is now unavailable. Oak Ridge National Labs Engineers have engaged their fiber vendor, TVA. TVA has dispatched a field tech with an estimated time of arrival of 14:15 UTC. The reason for the outage is pending. |
Ticket Number | Customer Impact | Network Impact | Title | Maintenance Type | Source Of Impact | Current State | Start Time (UTC) | End Time (UTC) |
---|---|---|---|---|---|---|---|---|
CHG0033604 | 3 - Moderate | 3 - Moderate | Maintenance Completed - SoX Peer Internet2 (Backup) | Circuit | Internal | Closed | 2018-11-02 03:15:59 | 2018-11-02 04:00:00 |
CHG0033801 | 3 - Moderate | 3 - Moderate | Decommission Completed - SoX POP IMART | Other | Internal | Closed | 2018-11-13 19:47:11 | 2018-11-14 18:00:00 |
Ticket No.: | CHG0033604 |
Subject: | Maintenance Completed - SoX Peer Internet2 (Backup) |
Source Of Impact: | Internal |
Affected: | SOX-MPLEX-SIDCO-VLAN-01513
Peer Internet2 (Backup) |
Current State: | Closed |
Start Time: | 2018-11-02 03:15:59 |
End Time: | 2018-11-02 04:00:00 |
Description | Peer Internet2's backup connection was unavailable while SoX Engineers moved the peering from rtr.mplex to rtr.sidco in preparation for decommissioning rtr.mplex. No service impact was observed, as traffic continued to route out of the the primary 100G interface to the peer. Maintenance has been completed. |
Ticket No.: | CHG0033801 |
Subject: | Decommission Completed - SoX POP IMART |
Source Of Impact: | Internal |
Affected: | rtr.imart.sox.net
POP IMART |
Current State: | Closed |
Start Time: | 2018-11-13 19:47:11 |
End Time: | 2018-11-14 18:00:00 |
Description | SoX Engineers will be decommissioning the InfoMart Data Center co-location in Dallas, TX during the listed window. During the window, Engineers will remove all the equipment at the site. Decommission work is expected to be completed by 12:00AM CST 11/14; however, the entire window is reserved. |
SOX Nodes | Down Time | Reporting Period Availability | 52 Week Availability |
rtr.365dc.sox.net | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 99.69314% | 100.00000% |
dcp.365dc.sox.net | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
rtr.56mar.sox.net | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
mgmtsw.court.sox.net | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
sw2.gtrch.sox.net | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
sw1.55mar.sox.net | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 99.97310% | 100.00000% |
dcp.sidco.sox.net | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
sw1.gtrch.sox.net | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
sw1.56mar.sox.net | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
rtr.sidco.sox.net | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
dcp.court.sox.net | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
bbsw.56mar.sox.net | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
mgmtsw.365dc.sox.net | 5 hr 3 min | 5 hr 3 min | 99.29653% | 99.29653% | 99.94234% | 100.00000% |
sw2.56mar.sox.net | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 99.77091% | 100.00000% |
mgmtsw.sidco.sox.net | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
Aggregate SOX Core Nodes | 0 days 5 hr 3 min | 0 days 5 hr 3 min | 99.95310% | 99.95310% | 99.95863% | 100.00000% |
NOTE: Numbers in italics represent downtime with scheduled maintenance excluded.
SOX Backbone Circuits | Down Time | Reporting Period Availability | 52 Week Availability |
SOX-56MAR-56MAR-FE-01935 | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
SOX-56MAR-56MAR-GIGE-01828 | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
SOX-56MAR-56MAR-FE-01938 | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
SOX-56MAR-56MAR-GIGE-01946 | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
SOX-56MAR-56MAR-FE-01941 | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
SOX-56MAR-56MAR-VLAN-01665 | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
SOX-COURT-COURT-GIGE-01950 | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
SOX-55MAR-56MAR-LAG-01892 | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
SOX-56MAR-56MAR-GIGE-01934 | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
SOX-56MAR-56MAR-10GE-01718 | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
SOX-365DC-365DC-FE-01954 | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
SOX-56MAR-56MAR-FE-01944 | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
SOX-55MAR-55MAR-LAG-01894 | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
SOX-SIDCO-SIDCO-FE-01959 | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
SOX-COURT-COURT-GIGE-01952 | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
SOX-56MAR-56MAR-GIGE-01948 | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
SOX-56MAR-56MAR-10GE-01775 | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
SOX-365DC-365DC-FE-01955 | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
SOX-56MAR-56MAR-GIGE-01947 | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
SOX-56MAR-56MAR-10GE-01719 | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
SOX-56MAR-56MAR-FE-01937 | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
SOX-365DC-365DC-FE-01956 | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
SOX-56MAR-56MAR-LAG-01862 | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
SOX-365DC-365DC-FE-01953 | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
SOX-56MAR-COURT-GIGE-01930 | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
SOX-56MAR-COURT-GIGE-01929 | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
SOX-365DC-56MAR-VLAN-01506 | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
SOX-SIDCO-SIDCO-FE-01962 | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
SOX-56MAR-56MAR-GIGE-01939 | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
SOX-56MAR-56MAR-GIGE-01940 | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
SOX-SIDCO-SIDCO-FE-01961 | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
SOX-SIDCO-SIDCO-FE-01958 | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
SOX-56MAR-56MAR-GIGE-01945 | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
SOX-56MAR-COURT-GIGE-01927 | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
SOX-56MAR-56MAR-10GE-01743 | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
SOX-56MAR-56MAR-100GE-01666 | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 99.99571% | 100.00000% |
SOX-56MAR-56MAR-10GE-01936 | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
SOX-56MAR-56MAR-GIGE-01933 | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
SOX-56MAR-GTRCH-GIGE-01931 | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
SOX-56MAR-56MAR-FE-01943 | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
SOX-SIDCO-SIDCO-FE-01960 | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
SOX-COURT-COURT-GIGE-01951 | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
SOX-55MAR-55MAR-LAG-01893 | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
SOX-56MAR-55MAR-10GE-01730 | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
SOX-56MAR-COURT-GIGE-01932 | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
SOX-56MAR-56MAR-GIGE-01949 | 00 hr 00 min | 00 hr 00 min | 100.00000% | 100.00000% | 100.00000% | 100.00000% |
Aggregate All SOX Backbone Circuits | 0 days 0 hr 0 min |0 days 0 hr 0 min | 100.00000% | 99.95310% | 99.99991% | 100.00000% |
NOTE: Numbers in italics represent downtime with scheduled maintenance excluded.
Task Number | Task Title | Reason not in Report |
CHG0033572 | Maintenance Completed - SoX Peer Netflix (OCA) | Not in report because source of impact was set to customer. |
INC0024245 | Availability - SoX Peer Great Access | No circuits, nodes, or other elements were listed as impacted in the report. |
INC0023849 | Availability - SoX Peer Amazon | No circuits, nodes, or other elements were listed as impacted in the report. |
INC0022004 | Outage Resolved - SoX Peer NationalNet | No circuits, nodes, or other elements were listed as impacted in the report. |
INC0022579 | Brief Outage - SoX Participant CLIGHT | Not in report because source of impact was set to customer. No circuits, nodes, or other elements were listed as impacted in the report. |
INC0022845 | Outage Resolved - SoX Peer OneNet | Not in report because source of impact was set to customer. No circuits, nodes, or other elements were listed as impacted in the report. |
INC0023797 | Outage Resolved - SoX Peer ATLNX (Atlantic Nexus) | Not in report because source of impact was set to customer. |
INC0022423 | Availability - SoX Peer Yahoo | No circuits, nodes, or other elements were listed as impacted in the report. |
INC0023459 | Availability - SoX Peer Georgia Public Web | No circuits, nodes, or other elements were listed as impacted in the report. |
INC0022747 | Availability - SoX Peer Yahoo | No circuits, nodes, or other elements were listed as impacted in the report. |