Tejas Jain

Cisco ACI - Port Tracking One of the techniques to speed up convergence in case of internal fabric connectivity failures, "port-tracking" feature addresses an outage where a leaf node loses connectivity to "all" the spine nodes in the Cisco ACI fabric. In such a scenario, the hosts that are connected to such a leaf in active-standby setup are usually not aware of such an outage and continue to send traffic to the now isolated leaf. This is where the port-tracking feature brings down all the host facing ports of the isolated leaf node. For the servers that are dual homed to different leafs, this action would ensure that the uplink to the isolated leaf is not considered for forwarding the traffic. The changes can be made as below: System >> System Settings >> Port Tracking

Cisco ACI has the concept of Anycast Gateway where the default gateway of the subnet (configured with the Bridge domain) exists on the Leaf devices. Now, more importantly, the anycast gateway / SVI (Switched Virtual Interface) is configured (rather programmed) on only those Leaf switches which have endpoints belonging to that bridge domain. How does Cisco ACI determine whether it should configure an SVI on a particular Leaf? It does this via CDP, LLDP or OpFlex (if the endpoints support it). This would imply, that CDP / LLDP is not just there for operational purposes, but rather, it actually holds a powerful influence on the actual traffic forwarding, unlike traditional switches. CDP uses the usual Cisco CDP timers with an interval of 60s and a holdtime of 120s. LLDP uses the usual LLDP timers with an interval of 30s and a holdtime of 120s. CDP support for Fabric Extenders has started from ACI 2.2 release. For older releases, LLDP should do the trick.

For network guys coming from the traditional switching world, the interface configuration on Cisco ACI is not as simple as putting "switchport xxx" command under interface x/x. Rather there is a huge list of interface policies which needs to be configured, which is then referenced in the Interface policy group and then stitched with the actual interface (Interface selector). The list of interface policies are as follows: LLDP - Link Layer Discovery Protocol CDP - Cisco Discovery Protocol LACP - Link Aggregation Control Protocol Port Speed Storm Control MCP - Mis-Cabling Protocol Now each policy type is already configured with the default configuration. The best practice is to not touch this default configuration but create an explicit policy. For example, I always have an LLDP_ON, LLDP_OFF, CDP_ON, CDP_OFF and so on, configured explicitly for my setup. Explicit policies for each of these policy types also enables you to configure other parameters such as the CDP, LLDP i...

One of the most intriguing (of course, if you get the hang of it) or depressing concepts of Cisco ACI is how the traffic forwarding takes place inside Cisco ACI. Let's start with an endpoint sending the frame to the connected leaf: The leaf checks the destination MAC address of the frame. The leaf will do a layer 2 lookup to find the destination MAC. If the leaf knows the location of the destination MAC (either local to the leaf or some other leaf), it will determine the destination's EPG. Depending on the EPG, it would determine if a contract is required to allow the frame to forward.. If yes, it would look into the L3 and L4 contents of the packet to determine if the contract exists. If it does, allow the traffic, if not drop. If the frame has the destination MAC address of that of the leaf, it will be routed. This will be the standard destination IP based routing. If a route exists for the destination in the VRF of the source, it is routed. If not, it will be dropped. With r...

Unlike the traditional Cisco switching world, where there existed only 3 VLAN types (standard, extended, private), Cisco ACI is definitely supposed to have several of them, to ensure that multitudes of Network Professionals get their brains wired! Luckily, I came across the following ones which seem to make sense about their respective roles to have the traffic forwarding in place: VLAN ID ( VlanID ) - Platform independent VLAN that is locally significant to each switch. This VLAN is automatically bound to the port-group VLAN existing on the DVS. It is derived from the VLAN pool that is configured in the Fabric Access Policies. Hardware VLAN ID ( HW_VlanId ) - In order to switch traffic locally, most leaf switches comprise of Broadcom ASIC. This VLAN type is utilized by the Broadcom ASIC chip. Connect to Broadcom ASIC on the leaf : vsh_lc To generate the list of endpoints connected : show system internal eltmc info vlan brief The various VLAN types are: BD_CTRL_VLAN - Infrastructure...

During the fabric registration process, an ACI node usually transitions across different states. These states are usually recorded in the Fabric Node Vector (FNV) table, which can be checked using the below command on the APIC acidiag fnvread States and descriptions: Unknown – Node has been discovered but no Node ID policy has been configured Undiscovered – Node ID has been configured but the node is yet to be discovered Discovering – Node has been discovered but VTEP IP has not yet been assigned Unsupported – Node is not a supported model Disabled – Node has been decommissioned Inactive – There is no IP connectivity Active – Node is active

First post of the several ones to follow where I will attempt to decipher the inner workings of the packet forwarding in Application Centric Infrastructure (ACI). To the uninitiated, of the other tables that a leaf switch maintains, we will be focussing on the "Global Station table" If a leaf receives a packet with a destination IP, the host route (/32) for which already exists in its global station table, it would imply that the leaf switch is aware of that IP's existence in the ACI fabric and the it knows where to forward the packet. If the leaf receives a packet with a destination IP, the host route (/32) for which doesn't exist in its global station table, it would check if the IP belongs to the IP address range of the tenant. If the IP address range exists in the ACI fabric but the leaf doesn't know how to reach the destination IP (obviously since it doesn't have the entry in its global station table), it would encapsulate the packet with the VXLAN hea...