Voice over IP (VoIP) networks, with their reliance on real-time communication protocols like SIP (Session Initiation Protocol) and RTP (Real-time Transport Protocol), must maintain high availability and low latency. Issues that arise need to be quickly identified and resolved to prevent service disruptions.

A common problem is incompatibility issues where there are over 100 Requests for Comments (RFCs) related to SIP with a lot of “SHOULD” statements instead of “MUST”. This often leads to users being unable to make outgoing or incoming calls.

This article presents a troubleshooting approach using IOTA, a real-time traffic capture and analysis tool that simplifies the identification of root causes in complex VoIP network problems. The article will highlight how IOTA can help efficiently troubleshoot network issues, identify anomalies, and resolve problems that traditional log analysis and basic traffic capture might miss.

Problem Statement

VoIP networks are susceptible to various issues that can affect service quality. A typical user complaint might involve being unable to make outgoing calls to external numbers, which can quickly escalate into a high-priority support issue. In such cases, troubleshooting must be done efficiently to restore service as quickly as possible. 

VoIP administrators often begin by reviewing logs or performing basic packet captures via SPAN ports and analyze manually afterward, but these methods may not always provide the clarity needed for a swift resolution.

User Perspective

From a user perspective, the issue is immediately noticeable: users cannot make outgoing calls to external numbers. This causes frustration and can lead to the creation of a high-priority support ticket. The support team must respond quickly and effectively to avoid further disruptions.

Figure 1:  Affected user. Source: Pixabay, Pexels, Unsplashs

Log Perspective

Figure 2: Log view with 403 forbidden but without detailed TCP information.

When the issue is first detected, VoIP administrators will typically start by

reviewing logs from affected clients. In this case, logs show SIP 403 "Forbidden" response codes from the PBX to the softphone. Additionally, the logs hint at an authentication error, which leads administrators to investigate potential causes related to SIP registration and authentication.

Upon further investigation, administrators may find that registration data appears to be correct. However, even after verifying that the registration process is functioning normally, the 403 response persists, prompting them to dig deeper. At this point, administrators often capture network traffic during an affected call to gain more insight.

Network Perspective

Network-level troubleshooting involves capturing traffic at relevant points in the network to observe SIP signaling and RTP transmission. This is where the problem often becomes more complex. VoIP networks consist of several interconnected components, including the PBX, softphones, SBCs (Session Border Controllers), and ITSPs (Internet Telephony Service Providers). 

The next challenge arises from possibly different data flows in SIP signaling compared to voice data in RTP flows. To effectively capture the relevant traffic, administrators need to ensure they are capturing at the right network points, including between the softphone and PBX, PBX and SBC, and SBC and ITSP.

Traditional packet capture methods using SPAN ports on network switches can become bottlenecks, affecting the accuracy of captured data. TAPs or Capture Appliances with an in-line capture feature can eliminate this. However, even with correct capture tools in place, identifying the root cause of a SIP 403 message can be time-consuming and complicated.

Troubleshooting Focus: How IOTA Improves VoIP Root Cause Analysis

Traffic Capture with IOTA

IOTA addresses many of the challenges faced by network administrators in troubleshooting VoIP issues. IOTA allows administrators to efficiently gather data during an affected call by providing real-time traffic capture and analysis. It can be deployed in-line across multiple network segments, including:

• Between the softphone and PBX
• Between the PBX and SBC
• Between the SBC and ITSP

Figure 3: IOTA placement for troubleshooting.

IOTA’s ability to capture traffic across all call legs—internal, DMZ, and external—even on SPAN ports if needed, ensures comprehensive coverage of the entire communication flow, helping administrators identify where the issue arises.

Analyzing SIP 403 Errors

Once traffic is captured, IOTA’s VoIP dashboard provides a detailed overview of SIP response codes. In the case of a SIP 403 error, administrators can immediately identify an increase in the frequency of these response codes when the issue occurs. By comparing this data with their baseline of previous instances when outgoing calls were functioning, administrators can observe any significant differences in the signaling patterns, especially around the time of the failed calls.

IOTA’s user-friendly interface allows for filtering by SIP URIs in From—or To-Header, as well as VoIP/SIP Call-ID or User-Agent, using a simple dropdown list. SIP Registrations have the same From—and To-URI, so they can be filtered by this pattern. In our example, we found that SBC is sending VOIP_FROM_URI without the suffix “;user=phone” in Register Requests and with this suffix on Invite Requests, which is used in outgoing calls, so we can differentiate them in filters.

Figure 4: Filter by SIP URI in From-Header via VOIP_FROM_URI filter.

Afterward, we narrowed down the affected calls, making it easier to focus on the specific issues related to the 403 response.

Figure 5: Filter by VoIP Call-ID.

Deep Packet Inspection and TCP Analysis

A critical aspect of troubleshooting VoIP issues is examining the details of the captured packets. In this case, administrators can observe that SIP registrations and INVITE requests are using TCP as the transport protocol if they look at the list of flows in the Overview dashboard. It’s visible in the “Protocol Stack” column.

Figure 6: List of Flows with an affected call at the Overview dashboard.

The TCP Analysis dashboard can help examine TCP flows more deeply. At first glance, everything appears to be functioning correctly because all TCP sockets have a completed 3-way handshake, and iRTT is fine. 

Figure 7: TCP Flows of Register Requests at the TCP Analysis dashboard.

Figure 8: TCP Flows of affected calls on the TCP Analysis dashboard.

We compared TCP flows from SIP Registers and SIP Invites afterward. IOTA reveals that different TCP source ports are being used for registration and call setup requests (Invite), as shown in Figures 7 and 8. Upon further investigation, it becomes clear that the ITSP in question rejects calls where TCP sessions are not reused, which aligns with its specific interface requirements. This finding can be critical in diagnosing why the 403 response is being returned.

Streamlined Troubleshooting with Visualized Data

Traditional troubleshooting approaches often require administrators to sift through massive amounts of log data, looking for clues and inconsistencies. IOTA simplifies this process by offering visualized data on its dashboards, allowing administrators to quickly see where issues are occurring and what needs further investigation. 

For example, by using IOTA’s SIP response code analysis, administrators can see spikes in 403 responses at specific times, making it easier to identify the root cause. The TCP Analysis dashboard can help determine specific details of a Socket like Handshake status, iRTT or source and destination ports. This visual approach enables faster decision-making and minimizes the time spent troubleshooting.

Key Benefits of Using IOTA for VoIP Troubleshooting

Increased Accuracy in Capturing

• Capturing data at multiple points in the network with high accuracy can help gather all required data and ensure that no critical detail is overlooked. In-line and SPAN options can help in multiple scenarios. If traffic is captured at remote sites without knowledge workers, it can be started by a simple hardware “click” with zero knowledge.
  
  

Faster and Efficient Analysis

• IOTA’s real-time and detailed analysis dashboards enable administrators and analysts to identify issues quickly, reducing downtime and service disruptions.
• Through Deep Packet Inspection and correlation of data like SIP and TCP flows in this example, IOTA helps pinpoint the root causes of issues like SIP 403 responses, such as misconfigured TCP flow handling or authentication mismatches.
  
  

Baselining 

• By using IOTA for baselining through capturing traffic patterns, administrators and analysts can identify issues by comparing failed traffic patterns with “known good” situations.

Conclusion

Troubleshooting VoIP network issues can be a complex and time-sensitive task, particularly when users are unable to make outgoing calls due to SIP 403 errors. By integrating IOTA into the troubleshooting process, network administrators can significantly improve their ability to identify the root cause of issues quickly and accurately. IOTA’s ability to capture real-time traffic, analyze SIP response codes, and inspect TCP flows provides a comprehensive and effective approach to diagnosing VoIP problems. Ultimately, IOTA helps streamline the troubleshooting process, reduces downtime, and ensures that VoIP services remain operational with minimal disruption.