This blog was written in collaboration with magellan netzwerke GmbH, A Profitap Partner.
DNS - OR ALSO: WHY IS MY INTERNET NOT WORKING?
Why does a visit to a website fail? Why can't I send my e-mail? DNS, the mother of all Internet services, is often the cause of the problem.
Loss of packets or an incorrectly configured firewall has an impact on a wide range of protocols. However, DNS as the basic protocol is the most vulnerable to packet loss and a misconfigured firewall. Knowing these effects supports the troubleshooting process so that any problems that may arise can be solved more quickly.
FUNCTION OF THE DNS
The main task of the Domain Name Service (DNS) is to answer requests for name resolution. Due to the provision of a worldwide distributed directory service, the Domain Name System has been an essential part of the functionality of the Internet since 1985.
PACKET LOSS
However, DNS is by design a vulnerable protocol for packet loss. Before an e-mail can be sent, a website visited or a stream started, a domain is first resolved into an IP address via DNS. With the corresponding IP address, the various protocols can establish a connection to the respective server. However, if the DNS name resolution is disturbed by packet loss, all other processes are also affected. DNS is therefore in many cases the cause of errors and inefficiency. DNS primarily uses UDP (User Datagram Protocol) as its transport protocol and thus a non-connection-oriented protocol.
Figure 1: DNS query with lost packet
Even one lost packet leads to massive delays. In the example shown above, the client 192.168.2.120 only sends a new request to the DNS server after a five second delay.
If connections to and from DNS servers are disturbed to a certain degree and cannot be repaired, check whether DNS requests via TCP are an alternative. Where in figure 1 a lost packet already leads to five seconds delays, the DNS request in figure 2 is handled with 20% packet loss in 846ms.
Figure 2: DNS request via TCP with packet loss
DNS OVER TCP
Primary DNS uses UDP as transport protocol. However, DNS was designed from the beginning to be able to use both UDP and TCP (Transmission Control Protocol). TCP is only used when DNS is not able to communicate over UDP.
When does DNS change from UDP to TCP?
Typically when the packet size exceeds the maximum length of a single UDP packet of 512 bytes.
When is the maximum length exceeded?
In fact, this happens quite often in today's environment.
When DNS was first implemented, the only application where the maximum length was exceeded was a zone transfer.
In today's DNS systems, the maximum length can be exceeded for example with a TXT record and other spam prevention features.
What happens when TCP is blocked?
If the packet size exceeds 512 bytes, the 'TC' bit (truncation) is set, informing the client that the message length has exceeded the maximum allowed size. In these situations the client must retransmit the request over TCP.
However, if TCP is blocked, the client request must be answered via UDP. This results in the packet being either fragmented or discarded. This results in slow DNS resolution or unresolvable domain names.
Impact on the user experience:
- websites are not or only slowly accessible
- non-functioning services in applications
Why the limitation to 512 bytes?
The size of 512 byte UDP payload is a dependency of IPv4. The IPv4 standard specifies that each network participant must be able to handle packets of 576 bytes or less. If control data (headers) are subtracted, pure 512 byte payload remains.
This limitation was recognized as a problem, so the maximum size was increased to 4096 bytes in the Extension Mechanism for DNS (EDNS). Current DNS servers contain the EDNS implementation and are therefore able to answer queries.
Although EDNS has been in existence since 1999, not all network components guarantee smooth operation. Firewalls are a source of error, as DNS-UDP packets over a total of 576 bytes can be dropped for various reasons. This behavior may not cause any directly visible problems, but it must be ensured that all network devices support large DNS packets. If the network environment does not fully support large DNS messages, this may cause DNS packets to be fragmented and/or partially dropped. For the end user, this has the same effect as blocked DNS packets via TCP. DNS queries remain unanswered or take a very long time.
ANALYZING DNS PROBLEMS WITH PROFITAP IOTA
With the DNS protocol and some of the most common DNS issues discussed, it now comes to the next question. How to keep track of this all in an easy way?
This is where the Profitap IOTA comes in. IOTA is a complete and secure network analysis and troubleshooting solution.
By capturing raw packet data to disk and extracting metadata for use in the dashboards, IOTA can give a fast overview of what is happening on the network, without losing any detail. There are a number of customizable dashboards available to help monitor essential performance metrics, retransmissions, packet loss, latency, throughput, availability, connectivity and more.
Using IOTA to find DNS issues
The DNS dashboard gives you several views of your DNS traffic for whatever date and time range you have selected to see of your available traffic.
The first section shows the volume of DNS traffic. The pie charts show
TOP DNS SERVERS in use and the MOST QUERIED Servers and their respective bandwidths.
The Bottom section shows all the DNS queries, client and server IP addresses of those queries including the DNS REPLY code, transaction ID and date/time seen in the traffic. Under the Details Section, you can see the total number of DNS requests and the most popular DNS queries being used.
Figure 3: DNS dashboard in IOTA web interface
Without IOTA, the 1TB capture disk and the dashboards, troubleshooting these issues would take considerably more time or even become near impossible. Imagine digging through a 500GB trace file with Wireshark and the load times for each time you apply a filter.
With IOTA, the dashboards give an instant overview of what is happening to the DNS traffic. For example, in the below screenshot from the DNS dashboard, the Server Fail DNS REPLY is clearly displayed, along with what clients and servers are associated with this traffic.
Figure 4: IOTA latest DNS request via TCP with packet loss
If you want to narrow down your search, it’s possible to filter by selecting the ‘Filter By’ magnifying glass next to the ServFail error. This applies a filter looking for all of these specific errors. By applying this filter, IOTA drills into the DNS Server responses and metadata for the particular Server. You can see the client under IP Source that is making the DNS queries and the IP Destination as the Server that is receiving and responding to the DNS queries.
You can see below that the IOTA has identified DNS Reply Codes 0, “OK”, Reply Code 2, “Server Failure”, and Reply Code 3 “Non-existent domain” occurring in the last 30 minutes of traffic, requested from client 192.168.50.57, and replied from DNS server 207.91.5.20
Figure 5: DNS reply code identification
What is the latency of the traffic with DNS Server located at 207.91.5.20?
For this, we’ll Goto Dashboard Server Latency. By using Goto>>, the active filter we setup from the DNS Dashboard will be preserved, and implemented on the Server Latency Dashboard.
As you can see below, the top graph shows latency related in milliseconds. The Max Application Latency in the last 30 minutes is 163.41ms. I can further see the exact times of the spikes of the higher latency DNS calls.
Figure 6: IOTA server latency dashboard
From here, the underlined server IP address (207.91.5.20) can be selected to export a PCAP of the traffic of interest and analyze it further, as shown below.
Figure 7: Filtered PCAP export from IOTA shown in Wireshark
Secure access and remote analysis
With an integrated TAP and internal storage capabilities, IOTA can be deployed at any key capture points in the network to provide full visibility into the network safely. The monitored network is isolated from the management interface to avoid any risk of injection of MITM attack through the device.
IOTA is fully managed over HTTPS and features a built-in VPN, which means that the device can be deployed anywhere and accessed easily from behind your desk.
Do you have any questions? Please don´t hesitate to contact us.
Author:
Carl Jonathan Schubert
Data Access & Network Telemetry
magellan netzwerke GmbH
John Modlin WCNA
Senior Systems Engineer North America
Profitap