How To Clear Incomplete Arp Entries Cisco

Incomplete ARP Entry or Firewall Responds to Every ARP Request on the Network

81002

Created On 09/25/18 19:36 PM - Last Modified 04/28/20 23:20 PM

Symptom

There can be two symptoms:

The Palo Alto Networks firewall has an incomplete ARP entry for a host on the network (for example, default gateway):

> show arp all maximum of entries supported :      2500 default timeout:                    1800 seconds total ARP entries in table :        1 total ARP entries shown :           1 status: s - static, c - complete, e - expiring, i - incomplete  interface         ip address      hw address        port              status   ttl -------------------------------------------------------------------------------- ethernet1/4       10.108.121.1    (incomplete)      ethernet1/4         i      1

The firewall is responding to every ARP request on the network. On the endpoint, select any random IP address, try to ping it and you'll see an ARP entry with the firewall's IP MAC:

$ ping -w 2  10.108.121.251 $ ping -w 2  10.108.121.252 $ arp -an ? (10.108.121.251) at 00:1b:17:00:04:13 [ether] on eth4 ? (10.108.121.252) at 00:1b:17:00:04:13 [ether] on eth4

Cause
It is likely there is an incorrectly configured source NAT policy with a mask length that is not /32. For example, if an interface is configured with IP address 10.108.121.2/24, then the NAT IP should be configured as 10.108.121.3/32 (with /32 mask). The NAT IP in this example should not be configured as 10.108.121.3/24. Resolution

> show running global-ippool Idx  Type             From                             To              Num   Ref. Cnt ---- ---------------- -------------------------------- --------------- ---   ---------- 3    Dynamic IP/Port  0.0.0.0-255.255.255.255          10.108.121.5    1     1 6    Dynamic IP/Port  0.0.0.0-255.255.255.255          10.108.121.0    256   1

Note: Pool 6 is using 256 addresses in 10.108.121.0 network.

To determine which policy, run the following command, then press "/" (slash), then type in: "idx: 6" (there is space between double colon and 6 and if needed go back a little bit by pressing the up arrow key).

> show running nat-policy [...] dmz_Out {         from dmz;         source any;         to outside;         to-interface ethernet1/4 ;         destination any;         service  any/any/any;         translate-to "src: 10.108.121.0-10.108.121.255 (dynamic-ip-and-port) (pool idx: 6)";         terminal no; }

See the incorrectly configured rule is dmz_out.

Method 2

Run a single command, which basically tells the firewall to output all rule names and src NAT translations, where a range of IPs is used. In this case, the rule name that precedes the translation is the offending rule.

> show running nat-policy | match {\|src:[^\(]*-  "Rule 1" { smtp04-in { smtp04-out { smtp03-out { "Internet outbound" { dmz_Out {         translate-to "src: 10.108.121.0-10.108.121.255 (dynamic-ip-and-port) (pool idx: 6)"; "Rule 4" { "Rule 5" { smtp03-in {

Method 3

Important! Use cautiously, because this method enables debugs on the dataplane.

Enable debug on DP:

> debug dataplane packet-diag clear all > debug dataplane packet-diag set filter match non-ip only > debug dataplane packet-diag set filter on > debug dataplane packet-diag set log feature flow arp > debug dataplane packet-diag set log on > debug dataplane packet-diag clear log log

After trying to send the communication through the firewall (or pinging from the firewall default gateway):

> ping source 10.108.121.253 host 10.108.121.1

Review the DP debug files:

> less dp-log pan_task_*

It is possible to go to the next file by pressing "n"

At some point, the following appears, which may be similar to:

Received ARP packet from port ethernet1/4 Packet decoded dump: L2:     00:50:56:a3:10:5a->00:1b:17:00:04:13, type 0x0806 ARP:    hardware type 0x0001         protocol type 0x0800         hardware size 6         protocol size 4         opcode REPLY         sender mac address 00:50:56:a3:10:5a         sender ip address 10.108.121.1         target mac address 00:1b:17:00:04:13         target ip address 10.108.121.253 ARP packet sent from translated IP in NAT rule index 5 in vsys 1 ARP packet sent to interface ethernet1/4 IP ARP packet parse complete, learn: no, target myself: yes, gratuitous ARP: no

In the example above, the firewall states that someone is using the IP address, which firewall it is using in NAT rule index 5.

Note: Index 5 means only active policies (disabled policies do not count) and it starts from 0. The easiest way is to again run command "> show running nat-policy" and count policies.

Fix Details

This is how the rule looked in the WebGUI and CLI:

> show config running [...] dmz_Out {      to outside;      from dmz;      source any;      destination any;      service any;      nat-type ipv4;      source-translation {           dynamic-ip-and-port {                translated-address 10.108.121.211;           }      }      to-interface ethernet1/4; }

At a glance, there is nothing wrong with this rule, however after some investigation see that the 10.108.121.211 is really an object:

10.108.121.211 {               ip-netmask 10.108.121.211/24;             }

This would be difficult to find just by browsing through the WebUI.

Change the netmask to the appropriate one (most likely /32) and verify. See how the rule looks after the change:

> show running nat-policy dmz_Out {         from dmz;         source any;         to outside;         to-interface ethernet1/4 ;         destination any;         service  any/any/any;         translate-to "src: 10.108.121.211 (dynamic-ip-and-port) (pool idx: 7)";         terminal no; }  > show running global-ippool Idx  Type             From                             To              Num   Ref. Cnt ---- ---------------- -------------------------------- --------------- ---   ---------- 3    Dynamic IP/Port  0.0.0.0-255.255.255.255          10.108.121.5    1     1 7    Dynamic IP/Port  0.0.0.0-255.255.255.255          10.108.121.211  1     1

In the example above, note a different pool ID used by the very same rule, but only a single IP address is used.

Attachments