Lesson 12 - Layer 2 Connectivity Troubleshooting Part 2

By

Lesson 12 - Layer 2 Connectivity Troubleshooting Part 2

In the previous lesson I attempted to show you how to go about a performance problems we might experience due to the duplex mismatch. It was our first trouble ticket in the series on layer 2 problems.

In this lesson I am going to create another issue to show you how your current skills can be practically useful in diagnosing network connectivity problems.

NOTICE
The steps presented in this lesson are not the ALL possible diagnostics you can do. And they do not have to be done in this specific order. I am merely listing some logical steps which might be useful in order to 'nail down' the root cause of the problem.

Trouble Ticket 2
Some client computers have problems accessing FTP server located in the same network 192.168.1.0/24. This behavior is very intermittent. 

We need to collect a bit more data related to this problem. A good starting point might be to take down the following pieces of information:


FTP Server
IP address  = 192.168.1.4
MAC address = 00:10:5a:d3:e4:e0

FTP Client1 (with no connectivity to FTP server)
IP address = 192.168.1.2
MAC address = 00:10:4f:b0:b2:fc

Here is one way of doing basic diagnostics.

Step 1
We want to make sure we have layer 1-3 connectivity between the Client1 and the server first (remember 'divide and conquer' method from the previous lesson?).

jr@mandala:~$ ping 192.168.1.4
PING 192.168.1.4 (192.168.1.4) 56(84) bytes of data.
64 bytes from 192.168.1.4: icmp_seq=1 ttl=64 time=0.372 ms

64 bytes from 192.168.1.4: icmp_seq=2 ttl=64 time=0.349 ms
64 bytes from 192.168.1.4: icmp_seq=3 ttl=64 time=0.371 ms
64 bytes from 192.168.1.4: icmp_seq=4 ttl=64 time=0.374 ms

It seems we have layer 3 connectivity working just fine.

Here's an interesting question for you. How do you know, you have received the replies from the FTP server in question (192.168.1.4)?

Now, you may have very confused expression on you face as in 'what do you mean?'. Have I not just gotten the reply from it?

Well, let me show you something to address this question.

Step 2
In order to be absolutely sure I got the reply from the FTP server (192.168.1.4) I need to verify my computer's ARP cache (Client1). Recall, that every time a host sends the packets, it encapsulates them in layer 2 frames (here Ethernet ones). In order to do that, the Client must have a valid destination MAC address (00:10:5a:d3:e4:e0) mapped to its IP address (192.168.1.4). If this mapping entry is not found in the APR cache, the Client1 will send ARP request to learn MAC address of 192.168.1.4. Knowing it let's check the ARP cache after we have sent ping packets. Here's what we find in (Pic. 1):

Pic. 1 - ARP Cache Entries
Pay attention to the highlighted entry. Is the MAC address mapped to our FTP server's IP address correct? 

NO! 

This MAC address does not belong to the server. FTP server's MAC address is: 00:10:5a:d3:e4:e0.

That test proved that getting echo replies to our echo packets (ping) must also be verified in the sender's ARP cache. 

So, which device does this MAC address 00:10:0f:a3:3b:e6 belong to? And how on earth did it end up in our Client's ARP cache?

Well, in order to answer the second question, the reasons for this wrong mapping might be different. There could be the device with the duplicate address (same as the FTP server). Then, when the clients send ARP request for the MAC address of the server, this 'rouge' device (not the legitimate FTP server), also responds to the query. And if its answers arrives later than from the legitimate server, the override the legitimate MAC address mapping in the ARP cache. If some other clients get the reply for their ARP query from the 'rouge' device first and then from the FTP server, they create the mapping correctly. That would explain why some computers can still FTP to the server and others can't. Another reason for this wrong mac-to-ip mappings might be ARP poisoning attack in the network (eg. using Ettercap tool). 

As for the answer to the first question, if this is not an ARP poisoning attack, you can use your knowledge from the lesson 9 about switching which helps you understand CAM table creation. Then, send uninterrupted ping towards the 'rouge' device and try to trace the mac addresses from switch to switch to find out where the device with duplicate IP address is located. MAC address table should help you find it relatively quickly.

There might be one more question I would like to clarify. Why the duplicate address was not detected by the 'rouge' device? Answer to that question may surprise you. The duplicate address detection uses so called 'gratuitous arp'. This is an unsolicited advertising of the MAC address upon computer startup. If some computers use the same MAC or IP, the newly computer cannot use its IP in the network. Unfortunately, some operating systems may not use this mechanism. They 'trust' what the operator does. If she or he wants this address, there will be no protest on their part.

I hope you are now beginning to see how much you already know!

In my next lesson you are going to analyze the third trouble ticket for layer 2 connectivity. This is going to be the last troubleshooting lesson. The upcoming lessons will describe other tools and interesting technologies such as VLANs, Spanning-Tree Protocol. Once we finish with foundations related to layer 2, we'll start discussing layer 3 technologies and concepts such as IP addressing, subnetting, routing etc.

Comments

Post a Comment

Popular posts from this blog

Lesson 27 - IPv4 Address Dissected - Part 1

By
Image
Lesson 27 - IPv4 Address Dissected - Part 1 If you already feel comfortable with binary-to-decimal and decimal-to-binary conversions you're up to this post's challenge: learning more about IP version 4 address (IPv4). If you need to brush upon the numeral conversions, check my  previous post . In one of the earlier posts on  TCP/IP traffic flow , I briefly described the importance of the addresses. In order for devices to communicate they must be uniquely identified in the network after all. There are typically three such identifiers we use to distinguish between devices: Names  - These are used by humans. Applications use addresses not names. Hence, DNS services out there (DNS service is used to resolve names-to-ip, and ip-to-names).  Layer 3 addresses  -  logical  and  hierarchical  unique identifiers of devices in the network. All layer 3 protocols such as IPX, Apple-Talk, IP etc. use different forms of layer 3 addresses.  Layer 2 addresses  -  flat  and most ofte
Read more

Lesson 21 - Spanning-Tree Protocol in Practice

By
Image
Lesson 21 - Spanning-Tree Protocol in Practice Previous post was designed to present in a nutshell the STP operation. However, without some practice it's just academic knowledge. I think it is a good idea to look at the same concepts using real equipment. Here goes... The below topology (pic. 1) uses redundant links which create the loops. Pic. 1 - Network Topology Icons designed by: Andrzej Szoblik -   http://www.newo.pl If there's one thing the administrator should do with such design, that would be configuring the root bridge. Typically, the most powerful switch in the center of the network plays that role. You do not want some access switch to be transmitting the frames between other switches. Access switches are designed to connect your computers to the network, and not to handle the majority of the traffic between the switches which root bridge must deal with. If you do not configure root bridge yourself, the switch with the lowest MAC address becomes th
Read more

Lesson 4 - Introduction to TCP/IP Layers

By
Image
Lesson 4 - Introduction to TCP/IP Layers In this lesson we take a sneak peek at the fundamentals regarding TCP/IP. This is one of the most important aspects to understand in order to follow the upcoming lessons. It is not my ambition to explain everything here as this would turn into a hefty book to read. You can find a lot of them on the market anyway. Instead, I will try to focus on some basic aspects of TCP/IP. They will constitute the minimum knowledge to help us understand how computers communicate. If you feel the urge to learn more on TCP/IP right now please visit the following site:  http://www.tcpipguide.com/free/t_toc.htm NOTICE! Familiarize yourself with ALL terms in red. They will be used throughout the classes. Introduction Computer communication follows some well defined rules and guidelines which we call  protocols . In order for the computers to exchange data they have to agree on using the same rules, otherwise they become incompatible. That was the case in t
Read more