A daily NOC routine and reading CDR patterns

The earlier lessons taught individual skills: KPIs, testing, fraud signs, troubleshooting. A NOC shift is where they all meet, under time pressure. This lesson gives you a repeatable daily routine and shows how to read the patterns hidden in call detail records, so you spot problems from the data before a customer emails you. Times, names and numbers are examples; adapt the checklist to your own tools.

Why a fixed routine helps

On a quiet day a routine feels unnecessary. On a busy day it is what stops you missing the one route that silently went bad while you handled three tickets. A routine also makes shifts comparable: if every engineer checks the same things in the same order, a problem found at 2 pm can be compared with the morning baseline. In the example below the NOC is staffed 6:00 AM – 6:00 PM (Macau time, GMT+8); outside those hours, automatic alerts and limits do the watching.

The shift at a glance

Start of shift (first 30–60 minutes)

1. Read the handover note from the previous shift and the overnight alert log. Open tickets come first.
2. Platform health: switch services running, CPU and memory normal, disk space comfortable (a full disk stops CDR writing), database replication and backups completed.
3. Traffic is flowing: current concurrent calls compared with the same time last week. A sudden zero on one customer or vendor is a red flag.
4. Overnight review: attempts, minutes and spend per customer while the NOC was unstaffed. Look for any customer whose night traffic is far above normal, or new destinations they never call.
5. Balances and credit: prepaid customers close to zero, and vendor prepayments that will run out today.
6. Rate and routing changes taking effect today: confirm they were loaded with the right effective dates.

The hourly sweep (about 10 minutes)

Once an hour, look at your top destinations and top vendors, sorted by traffic. For each, compare the last hour with that route’s own normal values (see Lesson 6):

• ASR and NER, for answer and network problems.
• ACD, for audio problems and early hang-ups.
• PDD, for slow or looping routes.
• The mix of final SIP codes, especially a rising 503 or 480 share (see Lesson 11).
• Concurrent calls against each vendor’s channel limit.

Write a one-line note even when everything is fine: “10:00 sweep – all normal”. A record of normal makes abnormal easier to prove later.

Reading CDR patterns

KPIs are averages, and averages hide things. Once a day — and whenever a KPI looks odd — open the raw CDRs for a busy route and look at the shape of the data. These are the patterns worth recognising:

A quick way to see duration shape: group the answered calls into duration bands and count them. Example with invented numbers for one route over a day:

Duration band     Healthy route    Suspicious route
0–5 s                   4%                 31%
6–30 s                 18%                 22%
31–120 s               41%                 27%
121 s +                 37%                 20%

Both routes might show a similar ASR, but the second has a large spike of very short answered calls. That is exactly the kind of difference an average ACD hides and a duration band shows.

Handling an alert during the shift

1. Confirm it is real: is the drop visible in the CDRs, or is it a reporting delay?
2. Scope it: one vendor, one destination, one customer, or everything?
3. Protect traffic first: fail over or reroute if a good alternative exists.
4. Collect evidence: three to five example calls with time, Call-ID and final code.
5. Open a ticket with the partner, and note the ticket number and time.
6. Follow up and test before moving traffic back.

End of shift

Before you leave, make the next few hours safe and leave a clear trail:

• Check that customer limits and spend alerts are in place for the unstaffed hours, especially for new and prepaid customers.
• Confirm any routes moved away during the day are either restored or deliberately left off, with a reason.
• Write the handover note.

HANDOVER – 31 Oct, 17:30 (GMT+8)
Open tickets:
  #1042 Vendor X – 503 on Country Z mobile since 13:10. Traffic moved to Vendor Y.
        Vendor says fix in progress. Retest before moving back.
Watch:
  Customer Y – night traffic to 99990 (premium) blocked at 11:20 after spike.
        Customer informed; waiting for confirmation.
Changes today:
  New rates for Country Z fixed effective 1 Nov 00:00 (loaded, checked).
Platform: all normal. Disk 41%. Backups OK.

Weekly extras

• Compare this week’s KPIs per top route with last week’s and with the last month.
• Review the list of open and recurring tickets per vendor; a vendor with repeated problems may need a routing decision, not another ticket.
• Check that firewall and whitelist entries still match active partners (see Lesson 10).
• Run your standard test-call list on key routes (see Lesson 8).

Examples in this course use invented names (Vendor X, Customer Y, Country Z) and documentation IP ranges. Menu names and options differ between softswitch versions; always check your own system and vendor documentation.