vPC is not working on NX9k emulation

Before posting something, READ the changelog, WATCH the videos, howto and provide following:
Your install is: Bare metal, ESXi, what CPU model, RAM, HD, what EVE version you have, output of the uname -a and any other info that might help us faster.

Moderator: mike

Post Reply
adrianfer
Posts: 4
Joined: Sat Jun 08, 2019 2:37 am

vPC is not working on NX9k emulation

Post by adrianfer » Thu Sep 05, 2019 4:11 pm

Good afternoon,

I can not flip my vpc traffic when reloading my primary vpc switch. Here is the config:

LAB:

Image

NXOS1 config:

NXOS1# sh run vpc
!Command: show running-config vpc
!Time: Tue Aug 6 21:48:45 2019
version 7.0(3)I7(4)

feature vpc
vpc domain 1
peer-switch
role priority 20
peer-keepalive destination 10.1.2.2 source 10.1.2.1 vrf default
peer-gateway
auto-recovery
ip arp synchronize

interface port-channel20
vpc peer-link

interface port-channel30
vpc 30

interface port-channel40
vpc 40

NXOS1# sh vpc brief

Legend:
(*) - local vPC is down, forwarding via vPC peer-link

vPC domain id : 1
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is alive

Configuration consistency status : success
Per-vlan consistency status : success
Type-2 consistency status : success
vPC role : primary
Number of vPCs configured : 2
Peer Gateway : Enabled
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
Auto-recovery status : Enabled, timer is off.(timeout = 240s)
Delay-restore status : Timer is off.(timeout = 30s)
Delay-restore SVI status : Timer is off.(timeout = 10s)
Operational Layer3 Peer-router : Disabled


vPC Peer-link status
---------------------------------------------------------------------
id Port Status Active vlans
-- ---- ------ -------------------------------------------------
1 Po20 up 1-2

vPC status
----------------------------------------------------------------------------
Id Port Status Consistency Reason Active vlans
-- ------------ ------ ----------- ------ ---------------
30 Po30 up success success 1-2

40 Po40 up success success 1-2


NXOS1# sh vpc role

vPC Role status
----------------------------------------------------
vPC role : primary
Dual Active Detection Status : 0
vPC system-mac : 00:23:04:ee:be:01
vPC system-priority : 32667
vPC local system-mac : 50:00:00:01:00:07
vPC local role-priority : 20
vPC local config role-priority : 20
vPC peer system-mac : 50:00:00:02:00:07
vPC peer role-priority : 30
vPC peer config role-priority : 30

NXOS1# sh spanning-tree summary
Switch is in rapid-pvst mode
Root bridge for: VLAN0001-VLAN0002
L2 Gateway STP is disabled
Port Type Default is disable
Edge Port [PortFast] BPDU Guard Default is disabled
Edge Port [PortFast] BPDU Filter Default is disabled
Bridge Assurance is enabled
Loopguard Default is disabled
Pathcost method used is short
vPC peer-switch is enabled (operational)
STP-Lite is disabled

Name Blocking Listening Learning Forwarding STP Active
---------------------- -------- --------- -------- ---------- ----------
VLAN0001 0 0 0 4 4
VLAN0002 0 0 0 4 4
---------------------- -------- --------- -------- ---------- ----------
2 vlans 0 0 0 8 8

NXOS2 config:

NXOS2# sh run vpc

!Command: show running-config vpc

!Time: Tue Aug 6 21:49:46 2019

version 7.0(3)I7(4)
feature vpc

vpc domain 1
peer-switch
role priority 30
peer-keepalive destination 10.1.2.1 source 10.1.2.2 vrf default
peer-gateway
auto-recovery
ip arp synchronize

interface port-channel20
vpc peer-link

interface port-channel30
vpc 30

interface port-channel40
vpc 40

NXOS2# sh vpc brief
Legend:

(*) - local vPC is down, forwarding via vPC peer-link

vPC domain id : 1
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is alive

Configuration consistency status : success
Per-vlan consistency status : success
Type-2 consistency status : success
vPC role : secondary
Number of vPCs configured : 2
Peer Gateway : Enabled
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
Auto-recovery status : Enabled, timer is off.(timeout = 240s)
Delay-restore status : Timer is off.(timeout = 30s)
Delay-restore SVI status : Timer is off.(timeout = 10s)
Operational Layer3 Peer-router : Disabled

vPC Peer-link status
---------------------------------------------------------------------
id Port Status Active vlans
-- ---- ------ -------------------------------------------------
1 Po20 up 1-2

vPC status
----------------------------------------------------------------------------
Id Port Status Consistency Reason Active vlans
-- ------------ ------ ----------- ------ ---------------
30 Po30 up success success 1-2

40 Po40 up success success 1-2


NXOS2# sh vpc role
vPC Role status
----------------------------------------------------
vPC role : secondary
Dual Active Detection Status : 0
vPC system-mac : 00:23:04:ee:be:01
vPC system-priority : 32667
vPC local system-mac : 50:00:00:02:00:07
vPC local role-priority : 30
vPC local config role-priority : 30
vPC peer system-mac : 50:00:00:01:00:07
vPC peer role-priority : 20
vPC peer config role-priority : 20
NXOS2#
NXOS2#
NXOS2# sh spanning-tree summary
Switch is in rapid-pvst mode
Root bridge for: VLAN0001-VLAN0002
L2 Gateway STP is disabled
Port Type Default is disable
Edge Port [PortFast] BPDU Guard Default is disabled
Edge Port [PortFast] BPDU Filter Default is disabled
Bridge Assurance is enabled
Loopguard Default is disabled
Pathcost method used is short
vPC peer-switch is enabled (operational)
STP-Lite is disabled

Name Blocking Listening Learning Forwarding STP Active
---------------------- -------- --------- -------- ---------- ----------
VLAN0001 0 0 0 4 4
VLAN0002 0 0 0 4 4
---------------------- -------- --------- -------- ---------- ----------
2 vlans 0 0 0 8 8
NXOS2#

SITUATION:

Step 1 - R3 pings R8:

R3#ping 10.1.1.4 repeat 100000 source 10.1.1.1

Type escape sequence to abort.
Sending 100000, 100-byte ICMP Echos to 10.1.1.4, timeout is 2 seconds:
Packet sent with a source address of 10.1.1.1
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!


step 2 - I reload NXOS1:

NXOS1#
NXOS1# reload
This command will reboot the system. (y/n)? [n] y
2019 Aug 6 21:39:14 NXOS1 %$ VDC-1 %$ %PLATFORM-2-PFM_SYSTEM_RESET: Manual system restart from Command Line Interface

step 3 - I loose pings on R3 and NXOS2 lose connection with NXOS1 :
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!................

NXOS2# sh vpc brief
Legend:
(*) - local vPC is down, forwarding via vPC peer-link

vPC domain id : 1
Peer status : peer link is down
vPC keep-alive status : Suspended (Destination IP not reachable)

Configuration consistency status : success
Per-vlan consistency status : success
Type-2 consistency status : success
vPC role : secondary, operational primary
Number of vPCs configured : 2
Peer Gateway : Enabled
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
Auto-recovery status : Enabled, timer is off.(timeout = 240s)
Delay-restore status : Timer is off.(timeout = 30s)
Delay-restore SVI status : Timer is off.(timeout = 10s)
Operational Layer3 Peer-router : Disabled

vPC Peer-link status
---------------------------------------------------------------------
id Port Status Active vlans
-- ---- ------ -------------------------------------------------
1 Po20 down -

vPC status
----------------------------------------------------------------------------
Id Port Status Consistency Reason Active vlans
-- ------------ ------ ----------- ------ ---------------
30 Po30 up success Type checks were 1-2

bypassed for the vPC

40 Po40 up success Type checks were 1-2

bypassed for the vPC

step 4 - Pings only come back almost 4 minutes after the reload, some seconds after the keep-alive link and peer link go up.

What I am doing wrong? This needs to be transparent.

Here is some analysis I did looking at the MAC address tables:

My test is a ping from R3 to R8 ip 10.1.1.4, so here is the destination ip's MAC:

R3#sh ip arp
Protocol Address Age (min) Hardware Addr Type Interface
Internet 10.1.1.1 - aabb.cc00.3000 ARPA Ethernet0/0
Internet 10.1.1.2 43 aabb.cc00.4000 ARPA Ethernet0/0
Internet 10.1.1.3 43 aabb.cc00.6000 ARPA Ethernet0/0
Internet 10.1.1.4 43 aabb.cc00.8000 ARPA Ethernet0/0

R8#sh arp
Protocol Address Age (min) Hardware Addr Type Interface
Internet 10.1.1.1 11 aabb.cc00.3000 ARPA Ethernet0/0
Internet 10.1.1.2 11 aabb.cc00.4000 ARPA Ethernet0/0
Internet 10.1.1.3 11 aabb.cc00.6000 ARPA Ethernet0/0
Internet 10.1.1.4 - aabb.cc00.8000 ARPA Ethernet0/0

Do SW1 see the MAC?

SW1#sh mac address-table
Mac Address Table
-------------------------------------------
Vlan Mac Address Type Ports
---- ----------- -------- -----
2 0000.0c07.ac01 DYNAMIC Po10
2 aabb.cc00.3000 DYNAMIC Et0/2
2 aabb.cc00.4000 DYNAMIC Po10
2 aabb.cc00.6000 DYNAMIC Po10
2 aabb.cc00.8000 DYNAMIC Po10


Yes, it knows about the MAC and it is reachable through the right port Po10.

Running the test:

I start the ping between R3 and R8.
Once I reload NXOS1, e0/0 on SW1 goes down, but Port-Channel stay up:

SW1#sh ip int b
Interface IP-Address OK? Method Status Protocol
Ethernet0/0 unassigned YES unset up down
Ethernet0/1 unassigned YES unset up up
Ethernet0/2 unassigned YES unset up up
Ethernet0/3 unassigned YES unset up up
Port-channel10 unassigned YES unset up up

SW1#sh etherchannel summary
Flags: D - down P - bundled in port-channel
I - stand-alone s - suspended
H - Hot-standby (LACP only)
R - Layer3 S - Layer2
U - in use N - not in use, no aggregation
f - failed to allocate aggregator
M - not in use, minimum links not met
m - not in use, port not aggregated due to minimum links not met
u - unsuitable for bundling
w - waiting to be aggregated
d - default port
A - formed by Auto LAG

Number of channel-groups in use: 1
Number of aggregators: 1

Group Port-channel Protocol Ports
------+-------------+-----------+-----------------------------------------------
10 Po10(SU) LACP Et0/0(s) Et0/1(P)


Everything is fine here, but when I look at SW1 MAC address table I got the following:

SW1#sh mac address-table
Mac Address Table
-------------------------------------------
Vlan Mac Address Type Ports
---- ----------- -------- -----
2 0000.0c07.ac01 DYNAMIC Et0/2
2 aabb.cc00.3000 DYNAMIC Et0/2
2 aabb.cc00.4000 DYNAMIC Po10

Total Mac Addresses for this criterion: 3

aabb.cc00.8000 is not there any more. Obviously there is a problem with the ARP propagation between SW1 and NXOS2.

Uldis (UD)
Posts: 5067
Joined: Wed Mar 15, 2017 4:44 pm
Location: London
Contact:

Re: vPC is not working on NX9k emulation

Post by Uldis (UD) » Sat Sep 07, 2019 2:37 pm

Works 100 %
Used image in your lab does vPC 100%..
Try again..

adrianfer
Posts: 4
Joined: Sat Jun 08, 2019 2:37 am

Re: vPC is not working on NX9k emulation

Post by adrianfer » Sun Sep 08, 2019 2:53 pm

CLI works as expected, show commands are OK, but fail-over does not work.

Uldis (UD)
Posts: 5067
Joined: Wed Mar 15, 2017 4:44 pm
Location: London
Contact:

Re: vPC is not working on NX9k emulation

Post by Uldis (UD) » Sun Sep 08, 2019 6:42 pm

virtual image interfaces issue, it has no L1 at all
try shut no shut links after you second nxos rebooted,,,
Its not eve issue mate, sorry

adrianfer
Posts: 4
Joined: Sat Jun 08, 2019 2:37 am

Re: vPC is not working on NX9k emulation

Post by adrianfer » Sun Jan 10, 2021 2:00 am

Found the problem:

Topology: IOL switch (L2 iron 15.2 image) doing portchannel towards 2 NX9K (QEMU) on vPC mode.

Issue: when the primary NX9K goes down the IOL switch loose connectivity (even if the second NX9K is up).

Solution: I change the iOl switch with a NX9K (QEMU) on the topology.

Uldis (UD)
Posts: 5067
Joined: Wed Mar 15, 2017 4:44 pm
Location: London
Contact:

Re: vPC is not working on NX9k emulation

Post by Uldis (UD) » Sun Jan 10, 2021 9:29 am

Between IOL and NX9k working only legacy mode ON
LACP no
If you want one sw to vpc pair user nx9k instead

Post Reply