Tejas Jain

Cisco ACI - Port Tracking One of the techniques to speed up convergence in case of internal fabric connectivity failures, "port-tracking" feature addresses an outage where a leaf node loses connectivity to "all" the spine nodes in the Cisco ACI fabric. In such a scenario, the hosts that are connected to such a leaf in active-standby setup are usually not aware of such an outage and continue to send traffic to the now isolated leaf. This is where the port-tracking feature brings down all the host facing ports of the isolated leaf node. For the servers that are dual homed to different leafs, this action would ensure that the uplink to the isolated leaf is not considered for forwarding the traffic. The changes can be made as below: System >> System Settings >> Port Tracking

Cisco ACI has the concept of Anycast Gateway where the default gateway of the subnet (configured with the Bridge domain) exists on the Leaf devices. Now, more importantly, the anycast gateway / SVI (Switched Virtual Interface) is configured (rather programmed) on only those Leaf switches which have endpoints belonging to that bridge domain. How does Cisco ACI determine whether it should configure an SVI on a particular Leaf? It does this via CDP, LLDP or OpFlex (if the endpoints support it). This would imply, that CDP / LLDP is not just there for operational purposes, but rather, it actually holds a powerful influence on the actual traffic forwarding, unlike traditional switches. CDP uses the usual Cisco CDP timers with an interval of 60s and a holdtime of 120s. LLDP uses the usual LLDP timers with an interval of 30s and a holdtime of 120s. CDP support for Fabric Extenders has started from ACI 2.2 release. For older releases, LLDP should do the trick.

For network guys coming from the traditional switching world, the interface configuration on Cisco ACI is not as simple as putting "switchport xxx" command under interface x/x. Rather there is a huge list of interface policies which needs to be configured, which is then referenced in the Interface policy group and then stitched with the actual interface (Interface selector). The list of interface policies are as follows: LLDP - Link Layer Discovery Protocol CDP - Cisco Discovery Protocol LACP - Link Aggregation Control Protocol Port Speed Storm Control MCP - Mis-Cabling Protocol Now each policy type is already configured with the default configuration. The best practice is to not touch this default configuration but create an explicit policy. For example, I always have an LLDP_ON, LLDP_OFF, CDP_ON, CDP_OFF and so on, configured explicitly for my setup. Explicit policies for each of these policy types also enables you to configure other parameters such as the CDP, LLDP i...

BGP communities are optional transitive attributes They are used mainly to associate an administrative tag to a BGP route In spite of the communities being transitive, Cisco IOS routers do not pass them across the BGP sessions by default. The feature needs to be activated using the command “neighbor send-community” To display the community in the structured notation, you need to enter the global configuration command “ip bgp-community new-format” In order to set a community value, use the route-map command “ set community etc” or to add the community value on the existing community values use “set community additive ” The routes with communities assigned to them can be matched using “community-list” and which can then be referenced in a route-map later. There are two types of community lists : standard and extended. For eg. ip community-list 1 permit 101:1 101:2 ip community list 1 deny no-export Std community list NO-ADVERTISE community attribute: The well-known NO_ADVERTISE...

The ultimate variables that decide whether a particular metric component should contribute towards EIGRP metric calculation or not are the EIGRP K-values. There are 5 in all, K1 to K5 with K1 (in conjunction with Bandwidth) and K3 (in conjunction with Delay) used primarily for EIGRP metric calculation. The K-values are not interface specific but router specific, meaning, the same set of K-values would be applicable to all the interfaces of a particular EIGRP router. Of course, changes can be done under "router eigrp ASN" command and not under interface. The command "show ip protocols" would display the current K-values. show ip protocols (EIGRP K-values) Let us now modify the K-values.. EIGRP metric weights The excerpt shows that the moment K values are modified, the EIGRP adjacency goes down due to the mismatch of K-values with the neighboring router. The changes are reflected again by using the command : show ip protocols

When your neighbor's router is on fire, your internet is not at stake! Wish the same were true in the EIGRP universe as well (unless you have a well designed redundant network, which unfortunately is not very often the case!) EIGRP works if and only if your router establishes an 'amicable' EIGRP neighborship with the adjacent EIGRP router. Well, this is not an issue in most cases. Of importance, here, is how do you quantify the 'amicability' of the EIGRP adjacency. Let's inspect various fields in the "show ip eigrp neighbor" output show ip eigrp neighbor Autonomous System (AS) number - Here, visible as "Process 100" - EIGRP uses the concept of  autonomous systems . An autonomous system is simply a group of EIGRP-enabled routers that should become EIGRP neighbors and exchange routes. Address: IP address of the EIGRP neighbor Interface : Which interface is this EIGRP adjacency is formed upon Hold timer: Hold down timer is the number of secon...

While configuring Cisco router you must have run into issues where the console tried to mess with your commands. (I have faced this more often than I am ready to accept!) Something like below: Cisco IOS - logging synchronous At a first glance, everything seems OK.. but wait.. where are my last three octets of the subnet mask?? The "link" and "line protocol" logs have kicked my three octets out.. What do I need to ensure that this doesn't happen? Disable console logging -- This, of course works! However, for someone who likes to feel the heat of his deeds, in the real time, it might not be very enjoying. Enable "logging synchronous" -- This needs to be done below the "line <console/vty> The latter is definitely desirable as it allows me to get the real-time logs along with, not messing up with my commands. Cisco IOS - logging synchronous configuration Of course, if you are telnet ing the device remotely, do this configuration under "line...

Cisco Software Defined Access is a new paradigm towards building Enterprise Networks. Primarily built upon the Cisco's DNA Center software, it leverages several next-generation components to design, provision and apply policy to create an intelligent wired and wireless network infrastructure. Cisco DNA Assurance which turns the network devices into sensors, giving visibility to everything on the network; guided remediation which automates resolution to keep the network at its optimal performance and thereby improve the end user experience, are some of Cisco's USPs. Cisco SDA accomplishes this intent based networking via the following components: Fabric Control Plane nodes: Based on LISP map-server and map-resolver functionality combined together on the same node, control plane node (database) tracks the endpoints in the fabric site and associates the endpoints to fabric nodes. Border nodes and edge nodes register with the control plane nodes. Control Plane node can be dedicate...

One of the most intriguing (of course, if you get the hang of it) or depressing concepts of Cisco ACI is how the traffic forwarding takes place inside Cisco ACI. Let's start with an endpoint sending the frame to the connected leaf: The leaf checks the destination MAC address of the frame. The leaf will do a layer 2 lookup to find the destination MAC. If the leaf knows the location of the destination MAC (either local to the leaf or some other leaf), it will determine the destination's EPG. Depending on the EPG, it would determine if a contract is required to allow the frame to forward.. If yes, it would look into the L3 and L4 contents of the packet to determine if the contract exists. If it does, allow the traffic, if not drop. If the frame has the destination MAC address of that of the leaf, it will be routed. This will be the standard destination IP based routing. If a route exists for the destination in the VRF of the source, it is routed. If not, it will be dropped. With r...

Unlike the traditional Cisco switching world, where there existed only 3 VLAN types (standard, extended, private), Cisco ACI is definitely supposed to have several of them, to ensure that multitudes of Network Professionals get their brains wired! Luckily, I came across the following ones which seem to make sense about their respective roles to have the traffic forwarding in place: VLAN ID ( VlanID ) - Platform independent VLAN that is locally significant to each switch. This VLAN is automatically bound to the port-group VLAN existing on the DVS. It is derived from the VLAN pool that is configured in the Fabric Access Policies. Hardware VLAN ID ( HW_VlanId ) - In order to switch traffic locally, most leaf switches comprise of Broadcom ASIC. This VLAN type is utilized by the Broadcom ASIC chip. Connect to Broadcom ASIC on the leaf : vsh_lc To generate the list of endpoints connected : show system internal eltmc info vlan brief The various VLAN types are: BD_CTRL_VLAN - Infrastructure...

What is a switch fabric module? Fabric module connects the supervisor engine and line cards of the switch Fabric Module Types Fabric Module - 1 - Generation 1 (provides 46 Gbps of backplane capacity) Fabric Module - 2 - Generation 2 (provides 110 Gbps of backplane capacity) Fabric Module - 3 - Generation 3 (provides 2.8 Tbps of backplane capacity) What do the above numbers indicate? The original M1 series I/O modules eg. N7K-M132XP-12 have 80 Gbps of capacity per slot. So, two Generation 1 fabric modules would be required (2*46Gbps = 92 Gbps) for providing sufficient fabric capacity for the M1 series 10GE line card. Ordinarily, Cisco would recommend using 3 fabric modules in this case, for redundancy purposes, since one fabric module going down would translate to reduced capacity / backplane in the production environment. This implies, the overall fabric module capacity should exceed the line card capacity. Hardware specific details Nexus 7004 has no fabric modules Nexus 7000 series su...

During the fabric registration process, an ACI node usually transitions across different states. These states are usually recorded in the Fabric Node Vector (FNV) table, which can be checked using the below command on the APIC acidiag fnvread States and descriptions: Unknown – Node has been discovered but no Node ID policy has been configured Undiscovered – Node ID has been configured but the node is yet to be discovered Discovering – Node has been discovered but VTEP IP has not yet been assigned Unsupported – Node is not a supported model Disabled – Node has been decommissioned Inactive – There is no IP connectivity Active – Node is active

First post of the several ones to follow where I will attempt to decipher the inner workings of the packet forwarding in Application Centric Infrastructure (ACI). To the uninitiated, of the other tables that a leaf switch maintains, we will be focussing on the "Global Station table" If a leaf receives a packet with a destination IP, the host route (/32) for which already exists in its global station table, it would imply that the leaf switch is aware of that IP's existence in the ACI fabric and the it knows where to forward the packet. If the leaf receives a packet with a destination IP, the host route (/32) for which doesn't exist in its global station table, it would check if the IP belongs to the IP address range of the tenant. If the IP address range exists in the ACI fabric but the leaf doesn't know how to reach the destination IP (obviously since it doesn't have the entry in its global station table), it would encapsulate the packet with the VXLAN hea...

Yes.. CDP still very much exists (though LLDP is doing a fairly decent job).. and most probably, would continue to, so long as Cisco is in the big game! And if you are an old Cisco techie, like me, you would be disappointed to know that it is not as simple as "cdp run" or "cdp enable" etc. Since you would be enabling CDP on the leaf interfaces which connect to the devices that are not the part of the fabric, you would find this configuration under Fabric >> Access Policies. Here are the steps for the configuration: Fabric >> Access Policies >> Interface policies Policies >> CDP Interface >> Right click on CDP interface and select "Create CDP Interface Policy" Name it and select the Admin state. 4. Next create Leaf Policy Group from Interface Policies and reference the above CDP policy in it as shown below: 5. Time to stitch the Leaf policy group with the actual Leaf Interface. 6. Go to Interface Policies >> Interface Ove...