Play with MACSEC on copper interface with 3750X

The Cisco documentation is not clear on the switch-to-switch (via copper) macsec feasibility.
Also, I have decided to test it between two 3750x:

• 3750X-24TS (without service module)
• 3750X-48TS (without service module)

I have applied the following configuration on each switch:

I have plugged the cable between this 2 switchs and checked that SAP 'succeeded':

Interface is up and configuration looks fine. But I have prefer checked by myself that the traffic is well encrypted. The best way to do this is to use a hub. But this equipment has disappeared from IT services and is very rare!! Also we have designed our own RJ45 TAP:).

I have used this magic TAP and wireshark to sniff the traffic between our both switchs.
Below, you can see the result of a packet when it's encrypted by MACSEC (802.1ae). We can see the Ethertype (88e5) used by this protocol.

To resume, MACSEC is available on Cisco Switch (switch-to-switch) on copper interface without Service Module. This configuration is not available on 3560X. I guess, the service module is mandatory for it.

TCN received from stackport1 on 3750

I was troubleshooting a network issue and during this time, I have done the following command on a 3750 stack:

show spanning-tree vlan 200 detail

Bridge Identifier has priority 20480, sysid 200, address a8b1.d36f.c670

  Configured hello time 2, max age 20, forward delay 15

  Current root has priority 8392, address 0021.e809.0670

  Root port is 568 (Port-channel11), cost of root path is 3

  Topology change flag not set, detected flag not set

  Number of topology changes 27 last change occurred 1w2d ago

          from StackPort1

As you can see, the source of the TCN is the StackPort1 on the switch!!!

But what is the meaning of this StackPort1 and how to determinate the real source of the topology change ?

In fact as my SSH session is open on the master and the TCN come from another switch of the stack, the source for my session is the stack port of the master switch. In order to determine which interface is really the source I have to open a session on each switch in order to find the TCN source.

For example, I have opened a session on the 2nd switch of the stack:

C3750#session 2

C3750-2# show spanning-tree detail | i from|exec|occur

 VLAN0001 is executing the ieee compatible Spanning Tree protocol

  Number of topology changes 26 last change occurred 1w2d ago

          from GigabitEthernet2/0/52

 VLAN0040 is executing the ieee compatible Spanning Tree protocol

  Number of topology changes 26 last change occurred 1w2d ago

          from GigabitEthernet2/0/52

As you can see above, we have found the real source of the TCN.

Install Netflow collector on Cacti

This tuto explains how to install the flowview plugin on Cacti. I have worked with Cacti 0.8.8b installed on an Ubuntu server.

Install and configure flow-capture

In order to capture netflow traffic, I have used flow-capture. In order to install it on Ubuntu, you can use apt:
apt-get install flow-capture

Once flow-capture is installed, you can configure the flow-capture.conf file:
vim /etc/flow-tools/flow-capture.conf

# Example 1:
# Capture flows from router at 10.1.1.10, listening at port 3000.
# Store flows in /var/netflow/flows/myrouter.
-w /var/netflow/flows/myrouter 0/10.1.1.10/3000

Add the file in dedicated folder:
mkdir /var/netflow/flows/myrouter

Configure a Router in order to export netflow

This configuration is different for each constructor. For example, I have configured an netflow export on a Cisco 4500:

flow record R1
 match ipv4 protocol
 match ipv4 source address
 match ipv4 destination address
 match transport source-port
 match transport destination-port
 collect counter bytes
!
flow exporter CACTI
 destination 10.10.10.10
 export-protocol netflow-v5 => flow-capture is only v5 capable
!
flow monitor M1
 exporter CACTI
 cache entries 1000
 record R1
!
interface Port-channel1
 ip flow monitor M1 input

Install and configure flowview on Cacti

Download the flowview plugin (http://docs.cacti.net/plugin:flowview) and untar it in:
/usr/share/cacti/site/plugins

Go to the Cacti console
Configuration>Plugin Management
And enable Flowview

Configure the path in order to read the netflow file created by flow-capture:
Go to the Cacti console
Configuration>Settings>Misc
Under Flows directory, specified your folder (for example /var/netflow/flows/)

Not able to execute 'copy running-config startup-config' command

Today, I have encountered the following problem:

MY-SWITCH#copy running-config startup-config
startup-config file open failed (Device or resource busy)

In a first step, I was thinking that the nvram: was corrupted because I was not able to see files in the nvram (dir nvram:). In fact, we were 2 users connected on the switch.


MY-SWITCH#show users
    Line       User       Host(s)              Idle       Location
*  1 vty 0     admin      idle                 00:00:00 10.10.10.10 => My session
   2 vty 1     admin      idle                 00:07:09 10.10.20.10

 Also, I have just ejected my colleague with the following command:

MY-SWITCH#clear line 2

After that, I was able to backup my configuration.

Troubleshoot OSPF neighbors (Hellos check)

In order to become neighbors, routers perform several checks. If this check fails, we have to troubleshoot and find the cause of this issue. You will find below several examples of neighbor failed. I have added logs messages and debug messages in order to easily find the cause.

• Area mismatch:

• Authentication key mismatch:

• Duplicate Router-id:

• Subnet/mask Mismatch:

• Area Type Mismatch:

How a phone choose the voice vlan ?

Just by listening the CDP announce coming from the switch:

ICMP Fragmentation and Firewall

Following an installation of new firewalls, I was facing an issue of communication between a CMC (Central Management Console) and a Riverbed Steelhead.
In a normal way, the CMC open a SSH session to push rules to the Riverbed. But after the installation, it was not working anymore. I have investigated and found the root cause of this issue.

Packets between the CMC and the Riverbed are fragmented. Without firewalls, CMC and Riverbed use PMTU in order to discover the MTU small enough to traverse the entire path without fragmentation.
In a first step, Path MTU uses the option Don't Fragment (DF). When a packet is sent with this option, it cannot been fragmented. If a router with a smaller MTU receives this packet, it will drop it and send to the sender an ICMP Fragmentation Needed. This step is repeated until the source has reached the destination without fragmentation.

As you can see below, our problem come from this ICMP Fragmentation Needed packet.
When the steelhead sends a packet (with option Don't fragment) to the CMC, a router on the path sends a ICMP fragmentation Needed. However, the new firewall filters this packet and now the handshake failed.

Workaround:

• Allow ICMP option on the firewall.
• Reduce MTU on primary interface of the Riverbed.