SOX Network Operations Report
2018-07-01 - 2018-07-31

SOX Network Operations Center

New Connections

ConnectorTypeDateSponsor


Upgrades

None


New Services


None


Other


None


Unscheduled Outages Summary

Incident Number Customer Impact Network Impact Title Outage Type Source of Impact Current State Start Time (UTC) End Time (UTC)

INC0015133 3 - Moderate2 - High Instability Resolved - SoX Backbone 56MAR-55MARHardware Internal Closed 2018-07-12 16:38:00 2018-07-18 18:56:00
INC0015362 3 - Moderate3 - Moderate Outage Resolved - SoX Core Node 55MARHardware Undetermined Closed 2018-07-18 17:08:03 2018-07-18 18:56:00

Unscheduled Outages Detail


Ticket No.:INC0015133
Subject:Instability Resolved - SoX Backbone 56MAR-55MAR
Source Of Impact: Internal
Affected:SOX-56MAR-55MAR-10GE-01884
Backbone 56MAR-55MAR
Current State:Closed
Start Time:2018-07-12 16:38:00
End Time:2018-07-18 18:56:00
DescriptionThe Backbone Circuit between 56 Marietta Street and 55 Marietta Street is now stable. SoX Engineering have determined that the reason for the outage was a bad optic. The optic has been replaced, the node has been rebooted, and the fiber has been cleaned. No further outages are expected.


Ticket No.:INC0015362
Subject:Outage Resolved - SoX Core Node 55MAR
Source Of Impact: Undetermined
Affected:sw1.55mar.sox.net
Core Node 55MAR
Current State:Closed
Start Time:2018-07-18 17:08:03
End Time:2018-07-18 18:56:00
DescriptionSoX Core Node 55MAR is now available. SoX Engineering performed a reboot of the switch to restore the Netflix Cache.


Scheduled Maintenances Summary

Ticket Number Customer Impact Network Impact Title Maintenance Type Source Of Impact Current State Start Time (UTC) End Time (UTC)

CHG00323743 - Moderate3 - Moderate Maintenance Completed - SoX Backbone SIDCO-SIDCO via Internet2 Circuit Vendor Closed 2018-07-27 04:14:542018-07-27 06:33:16

Scheduled Maintenances Detail


Ticket No.:CHG0032374
Subject:Maintenance Completed - SoX Backbone SIDCO-SIDCO via Internet2
Source Of Impact: Vendor
Affected:SOX-SIDCO-SIDCO-10GE-01502
Backbone SIDCO-SIDCO
Current State:Closed
Start Time:2018-07-27 04:14:54
End Time:2018-07-27 06:33:16
DescriptionBackbone SIDCO-SIDCO was unavailable while Internet2 Engineers performed circuit maintenance. An outage was observed during the maintenance. Maintenance has been completed.


Downtime and Availability



SOX NodesDown TimeReporting Period Availability52 Week Availability
rtr.akard.sox.net00 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
rtr.56mar.sox.net00 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 99.87036% | 100.00000%
sw2.gtrch.sox.net00 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
sw1.55mar.sox.net1 hr 47 min | 1 hr 47 min 99.75818% | 99.75818% 99.97310% | 100.00000%
dcp.sidco.sox.net00 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
sw1.56mar.sox.net00 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
rtr.imart.sox.net00 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
dcp.court.sox.net00 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
bbsw.56mar.sox.net00 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
sw1.365dc.sox.net00 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
rtr.mplex.sox.net00 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
dcp.mplex.sox.net00 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
mgmtsw.court.sox.net00 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
sw1.gtrch.sox.net00 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
rtr.sidco.sox.net00 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 99.99843% | 100.00000%
mgmtsw.mplex.sox.net00 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
sw2.56mar.sox.net00 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 99.77091% | 100.00000%
mgmtsw.sidco.sox.net00 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%

Aggregate SOX Core Nodes0 days 1 hr 47 min | 0 days 1 hr 47 min 99.98657% | 99.98657% 99.97849% | 100.00000%



NOTE: Numbers in italics represent downtime with scheduled maintenance excluded.



SOX Backbone CircuitsDown TimeReporting Period Availability52 Week Availability
SOX-56MAR-56MAR-FE-0193500 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
SOX-56MAR-56MAR-GIGE-0182800 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 99.95731% | 100.00000%
SOX-56MAR-56MAR-FE-0193800 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
SOX-MPLEX-MPLEX-FE-0195300 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
SOX-56MAR-56MAR-GIGE-0194600 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
SOX-56MAR-56MAR-FE-0194100 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
SOX-56MAR-SIDCO-VLAN-0150600 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 99.95121% | 100.00000%
SOX-56MAR-56MAR-VLAN-0166500 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 99.95731% | 100.00000%
SOX-56MAR-56MAR-10GE-0189500 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
SOX-56MAR-MPLEX-VLAN-0173100 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 99.91462% | 100.00000%
SOX-COURT-COURT-GIGE-0195000 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
SOX-55MAR-56MAR-LAG-0189200 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 99.91462% | 100.00000%
SOX-56MAR-56MAR-GIGE-0193400 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 99.95731% | 100.00000%
SOX-IMART-56MAR-VLAN-0156200 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 99.91255% | 100.00000%
SOX-56MAR-56MAR-10GE-0171800 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 99.95731% | 100.00000%
SOX-56MAR-56MAR-FE-0194400 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
SOX-55MAR-55MAR-LAG-0189400 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
SOX-SIDCO-SIDCO-FE-0195900 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
SOX-COURT-COURT-GIGE-0195200 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
SOX-56MAR-56MAR-GIGE-0194800 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
SOX-MPLEX-SIDCO-VLAN-0153100 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
SOX-56MAR-56MAR-10GE-0177500 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 99.95731% | 100.00000%
SOX-MPLEX-MPLEX-FE-0195400 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
SOX-56MAR-56MAR-GIGE-0194700 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
SOX-56MAR-56MAR-10GE-0171900 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 99.95731% | 100.00000%
SOX-AKARD-AKARD-10GE-0156300 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
SOX-56MAR-56MAR-FE-0193700 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
SOX-56MAR-56MAR-LAG-0186200 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 99.95731% | 100.00000%
SOX-56MAR-55MAR-10GE-0188500 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 99.95731% | 100.00000%
SOX-56MAR-COURT-GIGE-0193000 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
SOX-56MAR-COURT-GIGE-0192900 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
SOX-AKARD-IMART-VLAN-0154900 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
SOX-SIDCO-SIDCO-FE-0196200 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
SOX-MPLEX-SIDCO-VLAN-0150900 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
SOX-56MAR-COURT-GIGE-0192800 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
SOX-56MAR-56MAR-GIGE-0193900 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
SOX-MPLEX-MPLEX-FE-0195500 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
SOX-56MAR-56MAR-GIGE-0194000 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
SOX-SIDCO-SIDCO-FE-0196100 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
SOX-MPLEX-MPLEX-FE-0195600 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
SOX-55MAR-56MAR-10GE-0196300 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
SOX-SIDCO-SIDCO-FE-0195800 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
SOX-56MAR-56MAR-GIGE-0183900 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
SOX-56MAR-56MAR-GIGE-0194500 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
SOX-56MAR-COURT-GIGE-0192700 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
SOX-56MAR-56MAR-10GE-0174300 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 99.95731% | 100.00000%
SOX-56MAR-56MAR-100GE-0166600 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 99.91255% | 100.00000%
SOX-56MAR-56MAR-10GE-0193600 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
SOX-AKARD-56MAR-VLAN-0159000 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
SOX-56MAR-56MAR-GIGE-0193300 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
SOX-56MAR-GTRCH-GIGE-0193100 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
SOX-56MAR-56MAR-FE-0194300 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
SOX-SIDCO-SIDCO-FE-0196000 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
SOX-MPLEX-MPLEX-FE-0195700 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
SOX-COURT-COURT-GIGE-0195100 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
SOX-AKARD-MPLEX-VLAN-0154100 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
SOX-MPLEX-56MAR-VLAN-0152200 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 99.95731% | 100.00000%
SOX-55MAR-55MAR-LAG-0189300 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
SOX-56MAR-55MAR-10GE-0173000 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 99.95731% | 100.00000%
SOX-56MAR-COURT-GIGE-0193200 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%
SOX-56MAR-56MAR-GIGE-0194900 hr 00 min | 00 hr 00 min 100.00000% | 100.00000% 100.00000% | 100.00000%

Aggregate All SOX Backbone Circuits0 days 0 hr 0 min |0 days 0 hr 0 min 100.00000% | 99.98657%99.98584% | 100.00000%



NOTE: Numbers in italics represent downtime with scheduled maintenance excluded.





Upcoming Scheduled Maintenances

Ticket No.:CHG0032445
Subject:Emergency Maintenance Completed - SoX Peer Internet2
Source Of Impact: Vendor
Affected:Peer Internet2
Start Time:2018-08-01 09:00:00
End Time:2018-08-01 13:00:00
DescriptionPeer Internet2 remained available while fiber vendor engineers performed maintenance in Houston, TX. Maintenance has been completed.


Ticket No.:CHG0032136
Subject:Maintenance 4 of 4 - SoX Core Node 56MAR
Source Of Impact: Vendor
Affected:Core Node 56MAR
Start Time:2018-08-02 11:00:00
End Time:2018-08-03 21:00:00
DescriptionThe Core Node at 56 Marietta St. will remain available while Digital Realty Engineers perform power maintenance. No outages are expected during the maintenance window. The entire window is reserved. This is the 4th of 4 maintenance windows.


Ticket No.:CHG0032127
Subject:Maintenance - SoX Core Node 56MAR
Source Of Impact: Customer
Affected:Core Node 56MAR
Start Time:2018-08-03 11:00:00
End Time:2018-08-03 21:00:00
DescriptionThe Core Node at 56 Marietta St. will be available while Digital Realty Engineers perform power maintenance. An outage is not expected; however, the entire window is reserved.





Task Records not Included in this report

Task NumberTask TitleReason not in Report
CHG0032207Maintenance 1 of 5 Completed - SoX Core Node 56MARDid not have Actual Start Time set, so there was no actual downtime for the scheduled maintenance and was not included in the report.
CHG0032179Maintenance 2 of 2 Completed - SoX Peer NetflixNot in report because source of impact was set to customer.
No circuits, nodes, or other elements were listed as impacted in the report.
CHG0032214Maintenance 4 of 5 Completed - SoX Core Node 56MARDid not have Actual Start Time set, so there was no actual downtime for the scheduled maintenance and was not included in the report.
CHG0032152Maintenance Completed - Sox Peer Internet2Not in report because source of impact was set to customer.
Did not have Actual Start Time set, so there was no actual downtime for the scheduled maintenance and was not included in the report.
CHG0032239Maintenance Completed - SoX Peer Internet2 (DWDM System CINC-INDI)Not in report because source of impact was set to customer.
Did not have Actual Start Time set, so there was no actual downtime for the scheduled maintenance and was not included in the report.
No circuits, nodes, or other elements were listed as impacted in the report.
CHG0032228Maintenance Completed - SoX Participant Georgia TechNot in report because source of impact was set to customer.
No circuits, nodes, or other elements were listed as impacted in the report.
CHG0032414Emergency Maintenance Completed - SoX Peer GoogleNot in report because source of impact was set to customer.
Did not have Actual Start Time set, so there was no actual downtime for the scheduled maintenance and was not included in the report.
No circuits, nodes, or other elements were listed as impacted in the report.
CHG0032155Maintenance Completed- SoX Peer Internet2Not in report because source of impact was set to customer.
CHG0032175Maintenance 1 of 2 Completed - SoX Peer NetflixNot in report because source of impact was set to customer.
No circuits, nodes, or other elements were listed as impacted in the report.
CHG0032108Maintenance Completed - SoX Core Node 56MARDid not have Actual Start Time set, so there was no actual downtime for the scheduled maintenance and was not included in the report.
No circuits, nodes, or other elements were listed as impacted in the report.