Friday, May 29, 2015

ACS 5.6 install notes

Cisco Access Control Server (ACS) 5.6 install

I had a project to install ACS 5.6 for a customer..and I just wanted to go into a bit of detail of what I did for them!

ACS 5.6 can be installed on a VM or a physical chassis.  With regards to the physical chassis, there are a few options: Cisco's SNS 3415 chassis (who the customer used in this case) or the legacy Cisco 1113 chassis.  In the past, I'd always used the 1113 chassis or VMs--This was my first SNS 3415 install.

Before we dig into the ACS specifics, I'll discuss the setup of the device!

Racking:
The setup was fairly easy (with a bit of googling!)--as there were no instructions in the box.  The box comes with the physical chassis, a set of rails, mounting screws, power cables, ethernet cables, and a serial dongle.  The trick to mount the rails were to pull up on a tab on the end of the rail and pull back on the spring loaded mounts (1).  You then slide the square (or circular, depending on your rack) pegs into the rack (2).  Once they are in, pull on the same tab and the spring will close the gap and hold the bracket firmly against the rack (1).  Once you do the same for the backside of the rail (make sure they are on the same RU!!), there is a screw hole on the front of the rack to keep the rail from coming undone.  After you get the screws it, it is as simple as sliding the server onto the rails, as it already has the mounts on the server.  Once mounted, I ran ethernet to two different switches.  

The SNS 3415 supports bonding the physical ethernet ports so to provide upstream switch and cable redundancy.  When you bond the ports it creates a logical bond port.  One port is active while the secondary is passive; both ports will share the same MAC address so to avoid any bridging loops/mac address-table instability.  Just for verification sake, I took down one of the ports and at most saw a single packet lost.

 

Once you power on the device, it will immediately prompt you with the configuration items: hostname, IP address, gateway, name server, domain, CLI login, password, etc.  

Now!  Lets get to the meat of it all!

To start things off..lets get this baby integrated with AD!

This part is pretty darn easy.  Simply click on "join/test connection" from the "Active Directory" sub-section and enter the required information.  Remember, if you're using the name service, you must have configured the name-services in the CLI beforehand!  NOTE: When adding the ACS server to AD, The AD account required for domain access in ACS should have either of these privileges:

Add workstations to domain user right in corresponding domain.
Create Computer Objects or Delete Computer Objects permission on corresponding computers     container where ACS machine's account is created before joining ACS machine to the domain.


Integrating AD allows us to do two important things: Use our AD accounts for device management and ACS management.  For device management purposes, you can map certain AD user groups to certain rights/command sets.  Furthermore, you can map certain AD groups certain ACS rights (read/write/superadmin).

To select the AD groups you wish to use, click select under "Directory Groups."  If your device is joined and connected to AD, you should see a list of the groups available.  In this example I simply added a generic "Domain Admins."
Call me crazy, but I want both users in active directory and local users to be able to access the devices.  Who knows...maybe you have a vendor who you want to grant network access but don't want to create an AD account?  To do so, go to "Identity Store Sequences" and create a custom ID store.  Under the "Authentication and Attribute Retrieval Search List" choose both AD1 and Internal users.  In this order, it will first verify if the user is in AD, but if not, will check the local user store.

Implementation:

Time to create some Identity groups!  Identity groups are nothing more than a container in which you can associate certain rights/privileges.  In a basic ACS setup I'd create two identity groups: Read only users (users who can only run show commands) and Network admins (unlimited access).

Shell profiles!  Shell profiles are a place where you can assign attributes (default/maximum privilege, shell timeouts, etc).  Here I would create two shell profiles: Network Admins and Read Only.  The example below is pretty much the only values I'd put in for the Network Admins shell profile.  For the Read Only profile I'd set the default privilege to 1 and configure a timeout value of 10 minutes.   

Command sets!  Shell profiles are where you identify what commands can be run.  For example purposes, I'd create two shell profiles: Show only and Permit any.  Show only is limited to basic show commands while Permit any can run any command.  
By not listing any arguments, any parameter of the given commands are permitted.  For example, since the "show" command is allowed, it would also allow show run, show ip interface brief, etc.  If we were to have listed the argument "run" under arguments, then only the show run command would per permitted.

For the permit any, it is as simple as checking the "Permit any command that is not in the table below" checkbox.  Since we did not list any commands, all commands will be allowed!

Now that we have the identity groups, shell profiles, and command sets created, now we have to configure the device management.  Under the access policies section you will see access services.  Unless its a large scale deployment..I'm fairly comfortable using the default device admin service.  Under "Default Device Admin" you will see three fields: Identity, Group Mapping, and Authorization.  If you do not see one of these, click on "Default Device Admin"--there should be checkboxes for each area.  

Identity: This area is fairly simple.  When a username is entered, it needs to know where to lookup the username for access.  I've been successful leaving it in "Single result selection" and using the custom identity store we created at the start.  

Group Mapping: Here is where we map the AD group to a particular identity group.
If you do not see any of the fields above, click on "customize" in the bottom right hand corner.  For this particular customer, they weren't interested in the "Read only" user group...but it would be the same process tying the read only AD group to the predefined read only identity group.

Authorization: Here is where we put a bow on everything---we tie the identity group to the pre-defined shell profile and command sets!

While the read only users is not being used, I still created the authorization rule in the event they change their minds.  Furthermore, the customize bottom button at the bottom right corner can give you a TON of conditions (should you want to use them).  You can setup authorization rules based on device type, location, time, IP address, etc.

In addition to the AD mappings, we were also able to use the local users.  The key for this is to specify what identity group the user is in under the user settings.  In the example below, kyle.barnes would be part of the shell profile "Network Admins" and the command set "Permit any."

Adding network devices:

This part is super easy if you aren't being picky in your authorization rules.  In this customer's case, they didn't care about having location/device type specific rules, so I didn't have to be too granular in the device addition.  
I created 3 device types (Router, switch, and firewall) under network resources.  Was this necessary, given their requirements?  No.  I included the device type for logging purposes.  If one only cares for the bare bones, then he or she would only need to include the name, IP address, and TACACS shared secret!

AAA configurations:

Here is where things can get tricky.  Lets start easy...tacacs-server configs!

ip tacacs source-interface <management interface>
tacacs-server host <ACS IP address>
tacacs-server key 0 <shared secret configured in the network resources>

Now the fun part..

One thing I may add is more accounting on the commands.  I feel as if I can sometimes get more information than via the authorization logs.

Time for device administration!

As I said previously, we can now manage the ACS server with our AD credentials!  To do so, you need to configure the Administrative access control section as follows:
This rule basically says, "If you are trying to connect with a username, let me try and compare it against my AD ID store."


Lastly, the Authorization Policy says "Ok, let me see what group that particular user is in.  Once I find it, you will have the resulting role."  In the picture above, if your AD account is in the group "Domain Admins" then you'll have SuperAdmin rights.

Important notes!

  • Default ACS web login: U: acsadmin P: default -- by default acsadmin is the recovery account.  Regardless of how anything is setup it will let you in!  



Thursday, May 28, 2015

Auto-negotiation

I figured i'd put together some notes to concisely describe auto-negotiation, based off a particular client's requirements.

What is auto-negotiation?  It is an Ethernet procedure by which two connected devices choose common transmission parameters, such as speed, duplex mode, and flow control.  Basically, it is a mechanism for two devices to negotiate the BEST method of communicating across a common link.  The most common parameters negotiated are speed and duplex.

But why is this needed?  Well, in a word where end devices support half-duplex, full-duplex, 10 Mbps, 100 Mbps, 1 Gbps, etc...it would be a monumental task to maintain static duplex/speed configurations on a large enterprise network.  In addition to duplex and speed, auto-negotiation can also be used to detect physical layer issues.  

I like to compare half-duplex and full-duplex to a walkie-talkie and a telephone, respectively.  In half-duplex, only one end of the communication can send or receive at any given period of time.  With full-duplex, both nodes can send and receive simultaneously.  In an ideal scenario, full-duplex is the preferred means of communication..obviously.

Regardless of what is "best," both ends of the link go through the following list to determine the best possible mode of operation that is supported by both devices:


  1. 10GBASE-T full duplex
  2. 1000BASE-T full duplex
  3. 1000BASE-T half duplex
  4. 100BASE-T2 full duplex
  5. 100BASE-TX full duplex
  6. 100BASE-T2 half duplex
  7. 100BASE-T4 half duplex
  8. 100BASE-TX half duplex
  9. 10BASE-T full duplex
  10. 10BASE-T half duplex


In other words, among the modes that are supported by both devices, each device chooses the one that is the topmost in this list.

The real complications arise when either a switch port or a end device is hard-coded.  Why might something get hardcoded?  Well, there used to be a time when the auto-negotiation standards were interpreted wildly by every given vendor.  As a result of previous inconsistencies, many people today still fear auto-magically letting things negotiate.  The problem with hard-coding, is that there are often cases where doing so will "break" the link:


1 A duplex mismatch can result in performance issues, intermittent connectivity, and loss of communication. When you troubleshoot NIC issues, verify that the NIC and switch use a valid configuration.

2 Some third-party NIC cards can fall back to half-duplex operation mode, even though both the switch port and NIC configuration are manually configured for 100 Mbps, full-duplex. This is because NIC autonegotiation link detection still operates when the NIC is manually configured. This causes duplex inconsistency between the switch port and the NIC. Symptoms include poor port performance and frame check sequence (FCS) errors that increment on the switch port. In order to troubleshoot this issue, try to manually configure the switch port to 100 Mbps and half-duplex. If this action resolves the connectivity problems, this NIC issue is the possible cause. Try to update to the latest drivers for your NIC, or contact your NIC card vendor for additional support.

Wednesday, May 27, 2015

Random QoS Notes

My brain hurts



Ok, I fucking hate QoS.  I've taken a class on it, studied it, read the book...shit just drips out of my brain the second I'm not staring at it.  It's like the ghost from mario; When you aren't looking it's trying to kill me.

* Congestion avoidance (WRED or WTD)
* Congestion management (queuing)

Now, Cisco recommends classification and markings be done "As close to the source as possible."  Well, let me throw an ACL and match by IP address and mark these packets with the appropriate DSCP markings so that I can ensure my link isn't crushed by Candy Crush ©®℗™℠.  But what about those links up to the L3 device?  Those stupid ass L2 switches have their own queues and thresholds and other shit.  If we only implement MQC..then we aren't deploying QoS "End-to-end."  Shut up Cisco.

Now, I won't pretend to be an expert on this crap...even though I am technically certified as one.  I hate QoS, did I mention that?  Anyways, lets talk this thing out, friends.

Some basic points:

CoS (Class of Service) is L2

DSCP (Differentiated Service Code Point) is L3

There is a whole lotta stuff to talk about regarding these two points; how to convert the numbers from binary, what fields make up these numbers, etc.  We can talk about those if we want...but a lot of those points are googlaable.  One major point is that these values are by default not trusted..but we'll get back to that.

Now what makes up QoS:

* Classification
  * What is it, and what it do.
  * Do it as close to the edge as possible.
* Marking
  * "Coloring" packets with a certain color to be identified as it flows through the network.
  * Do it as close to the edge as possible.

We can classify and mark at L2 or L3..L2 would be using the CoS and L3 would be using DSCP.  CoS markings are obviously lost if it goes over a L3 link...we can mitigate this by mapping the L2 CoS value to a L3 DSCP value.  Once the traffic has been successfully marked, it can be accessible for QoS handling on the network!

Now that we know those basics..we have to actually find out the traffic on the network.  You can use NBAR, Netflow, Cisco IP SLA, etc.  The point is to learn what is on the network.  I know of some people that swear by aproduct called [LiveAction](http://liveaction.com/), while others have used something called [Scrutinizer](https://www.plixer.com/Scrutinizer-Netflow-Sflow/scrutinizer.html).

Once you understand what is on the network, it is recommended to try and group this into differing QoS service classes.  These classes are simply a logical grouping of packets that are to receive similar levels of applied quality.  A service class can be as granular as that of a single user (MAC/IP address) or by application (Port number/URL).  There are a number of service class templates to help facilitate the mapping, be it a 4- and 5-class model, 8-class model, or 11-class model..it really depends on how granular you wish to get.  The most important note to make about these different service classes is that they are consistent.  If you have a Voice service class with a DSCP marking of 46...you better use 46 everywhere in your environment (or good luck unraveling that shitball).

Now that we've discovered what is on the network we need to classify it and mark it appropriately so that it is correctly handled.  Where and how we do this is up to the environment.

* You CAN mark L2 frames on a PC if the NIC supports it.
* You PROBABLY will mark at the Cisco phone for voice (if there is one) and reclassify or mark the incoming PC frames.
* You PROBABLY will accept the markings or remap at the access switch.
* You PROBABLY remap the CoS values to DSCP..or accept the L2 markings.

The above points are the QoS trust boundaries.  By default...Cisco does not typically trust any markings.  Why would they?  Joe Shmoe could mark his own traffic as AF46 (The best service, typically)...so we would rather control where that point is configured.

*****

Remember why I hate L2 QoS?  **The number of hardware queues available differ in each switch model, supervisor, and line card.**  Some switches suck and ONLY support egress queuing..**cough 2960S cough**..so you have to know the capabilities of the switch.  The queuing mechanism is represented as 1PxQyT.  What the fuck did I just say?  P=Priority queue, Q=Non-priority queues, T=Thresholds *per* queue.  Now looking at the 3750 we have...we see that it has differing queuing mechanisms for input and output.  For input, it has a **1P1Q3T ingress/Receive queue** and a **1P3Q3T Egress/Transmit queue.**

What is a priority queue?  Well, it depends (Screw you QoS!!); Some switches (6500) service the priority queue whenever there is traffic in it...which may starve other queues.  Some switches support one or more drop thresholds to a queue (like our 3750 example).

What happens in these queues is FURTHER made difficult...ugh!  Our faithful 3750 uses SRR for both ingress/egress queues.  INGRESS IS ALWAYS SHARED WHILE EGRESS CAN BE EITHERendrant.  The 6500 uses WRR AND SRR...depending on the supervisor and line card.

Once we know the capabilities of the particular switch..we can start prioritizing traffic by placing packets with particular DSCP or CoS values into certain queues and threshold values.  When supported, the threshold values can be used to set the point where the switch congestion avoidance mechanisms will start to drop that traffic.

Now that we know the 3750 has a total of 2 ingress queues (1 priority and 1 non-priority) and 3 thresholds per queue...we can start carving it out!

WTF is a threshold?  It ultimately all comes down to port with a finite buffer space, broken out into independent queues, or figurative lines the incoming frames must wait in.  We have to identify the cutoff point on those individual lines in which to start tail dropping traffic.

What percentage/ratio is assigned to each independent queue is determined by the mls qos srr-queue input buffers <queue#1> <queue2> global command.  Using mls qos srr-queue input buffers 90 10 as an example, queue 1 has 90% of the available buffer space, while queue 2 has 10%.  Queue 2 is the priority queue by default!  Furthermore, we can identify the bandwidth amount per queue!  We do this by using the global command: mls qos srr-queue input bandwidth <queue1> <queue2>.  Furthermore, we can specify the priority queue bandwidth amount with the global command mls qos srr-queue input priority-queue <#> bandwidth <%>.  An example configuration of this could be "mls qos srr-queue input bandwidth 70 30" and "mls qos srr-queue input priority-queue 2 bandwidth 30."


If the potential maximum is 100% of a threshold until traffic starts dropping for the independent queues, thresholds can be identified with specific CoS/DSCP mappings.  To  expand on this, you can map certain values to a queue with different thresholds.  For example, the global command
mls qos srr-queue input cos-map queue 1 threshold 1 0 1 2 would map the cos of 0, 1, and 2 to queue 1 threshold 1 and mls qos srr-queue input cos-map queue 2 threshold 1  4 5 would map the cos of 4 and 5 to queue 2 threshold 1.  ALL THIS IS DOING IS ASSIGNING THE COS VALUES TO THE RESPECTIVE QUEUES AND IDENTIFYING THE QUEUE THRESHOLD!

ANOTHER Important note:
You can configure SRR on egress queues for sharing or shaping.  However, for ingress queues, sharing is the default mode, and it is the only mode supported.  Shaped mode guarantees bandwidth and rate limnits to that amount.  Inversely, shared mode guarantees bandwidth but can utilize unused bandwidth assigned to other queues.  Ultimately, shared round robin is a minimum bandwidth guarantee.

EXAMPLE TIME!!!

Now using the following cos-maps...
mls qos srr-queue input cos-map queue 1 threshold 1  0 1 2
mls qos srr-queue input cos-map queue 1 threshold 2  3
mls qos srr-queue input cos-map queue 1 threshold 3  6 7
mls qos srr-queue input cos-map queue 2 threshold 1  4 5

Since this is an INGRESS queue, we are using SHARED mode SRR.  To reiterate, that means that all of the queues (all 2 of the ingress queues) can utilize the available bandwidth presented to them.

Using the line analogy, we've essentially created 2 lines (queues) with different lengths (thresholds) in which they can grow before management starts kicking people out.

Now what about the egress queue?

The concepts are IDENTICAL, with the exception being that egress queues use shaped mode over shared mode, which guarantees bandwidth and rate limits to that amount.  Being that it is shaped mode, we can configure threshold limits!  Furthermore, each port supports four egress queues, one of which (queue 1) can be the egress expedite queue. These queues are assigned to a queue-set. All traffic exiting the switch flows through one of these four queues and is subjected to a threshold based on the QoS label assigned to the packet.

The switch uses a buffer allocation scheme to reserve a minimum amount of buffers for each egress queue, to prevent any queue or port from consuming all the buffers and depriving other queues, and to control whether to grant buffer space to a requesting queue. The switch detects whether the target queue has not consumed more buffers than its reserved amount (under-limit), whether it has consumed all of its maximum buffers (over limit), and whether the common pool is empty (no free buffers) or not empty (free buffers). If the queue is not over-limit, the switch can allocate buffer space from the reserved pool or from the common pool (if it is not empty). If there are no free buffers in the common pool or if the queue is over-limit, the switch drops the frame.

Lets pretend we have the following configuration...
mls qos srr-queue output cos-map queue 2 threshold 1 2
mls qos queue-set output 1 buffers 15 30 35 20
mls qos queue-set output 1 threshold 2 40 70 90 400

What are we doing?

Well, lets break each one out (remember, this is going to be very similar to the input queue).

mls qos srr-queue output cos-map queue 2 threshold 1 2
Here, we're simply mapping the cos value of 2 to queue 2, threshold 1 (3 per queue).

mls qos queue-set output 1 buffers 15 30 35 20
Each threshold value is a percentage of the queue’s allocated memory, which you specify by using the mls qos queue-set output qset-id buffers allocation1 ... allocation4 global configuration command. The sum of all the allocated buffers represents the reserved pool, and the remaining buffers are part of the common pool.  In this example, 15% for queue 1, 30% for queue 2, 35% for queue 3, and 20% for queue 4 (15+30+35+20=100).

mls qos queue-set output 1 threshold 2 40 70 90 400
This is where the egress queue gets squirrely.  What do these values mean?  We're saying that the threshold values for queue 2 are as follows: 40% for threshold1, 70% for threshold2, 100% for threshold3 (non-configurable), 90% for reserved, and 400% maximum.

Well, that was confusing as shit.

Another example to help drive it home with a look at queue 2:

mls qos srr-queue output cos-map queue 2 threshold 2  4
mls qos srr-queue output cos-map queue 2 threshold 3  6 7
mls qos queue-set output <1/2> threshold 2 40 70 90 400

When the buffer is filled with 70% of traffic it will start dropping traffic with cos 4 and not traffic with cos 6 or 7 because it's mapped to thresshold 3 (=100%)

But what about the reserved and max?

[buffer size:
mls qos queue-set output 1 buffers 15 30 35 20
mls qos queue-set output <1/2> threshold 2 40 70 90 400

If you take for example queue 2, we configured 90% as reserved. So you reserve 90% of the buffer size of queue 2 (which was 30% of the total buffer space).  So out of a theoretical 100%, we gave 30 of that to queue 2.  Of that 30 %, we configured 90% as reserved.

The remaining 10% from queue 2 is seen as common buffer space. Can be used by queue 1, 3, 4

Max reserved means, how much it may expanded and use buffer space from the common pool buffer space as explained above.  Of that 10% that is not reserved...it can be used to service the other queues.

Helpful show commands:
show mls qos: This will show you if QoS is enabled!
show mls qos queue-set: This will show you the queue-sets configured for the output queues.
show mls qos input-queue: This will show the configured buffer, bandwidth, priority%, and threshold values for the different queues.

Also, what is the queue-set number meaning?  1 is default...AKA you don't have to configure anything.  You CAN configure a queue-set 2, but it will require interface-level configs (queue-set 2) to use.  Having more than 1 could simply allow for more granularity.

Interface level configs:
interface giX/X/X
priority-queue out - Enables the priority queue; The expedite queue is a priority queue, and it is serviced until empty before the other queues are serviced.
mls qos trust device cisco-phone - Extends the trust zone to the phone.
mls qos trust dscp - Trusts the DSCP value; By default the DSCP value is not trusted.
srr-queue bandwidth share 1 30 35 5 - Here we are saying 1. Ignore queue 1's SRR sharing weight as this is the priority queue.  Furthermore, were allocating 30% to queue 2, 35% to queue 3, and 5% to queue 4.  Because we said share, the other queues can use bandwidth from other queues if under congestion.




Policing:  The point I want to notate is the amount + the burst.  Kevin Wallace explained this better than anyone I've ever heard: The specified rate is an AVERAGE.  Think of it as intercity traffic; you go as fast as you can, until you cant.  The rate is TECHNICALLY the speed limit, but you go as fast as the traffic allows.  While the speed is the line rate, the amount that goes is the burst rate (why it is in bytes not kbps, mbps, etc).

  • Optimally deployed as ingress tools as they make instantaneous send/drop decisions.
  • Do NOT delay traffic, they simply check the traffic and make a decision.
  • When supported, markdowns should be done according to standards-based rules (AFX1 to AFX2 for example).
Shaping: Uses buffers to smooth out spiking traffic flows for later re-transmission.
  • Only applied as egress tools.
  • Objective is to NOT drop the traffic.
  • Excess traffic is buffered and delayed until the offered traffic dips below the defined rate.
  • Real-time traffic should be shamed to 10ms intervals.  Since tc cannot be manipulated directly, use the formula Tc=Bc/CIR.  For example, if we have a 1Gbps link that we wish to shape to 120Mbps with 10ms intervals..how much data should go per interval?  If we wish to have 10ms intervals, then there will be 100 intervals per second (1000/10ms).  That would mean that we'd need to have a Bc value of 1.2 Mbps.  So for every interval, 1.2 Mbps will be sent at a time interval of 10ms.  


ASIC+Buffer:  The buffer amount is shared per ASIC.  The number of ports per ASIC is dependent on the hardware version of the switch.  The 2960S, for example, has 2 ASICs..or 24 ports per ASIC.  This can be identified by performing a show platform pm if-numbers; The first number in the port column identifies the asic/X.  For example, 0/X would represent a port that is in ASIC 0 and 1/X would represent a port that is in ASIC 1.  In an ideal world, we'd split traffic across the ASICs by connecting some hosts on the first set of 24 ports and some on the other set of 24 ports, to best utilize the buffer memory.

Buffer vs Bandwidth?  The biggest thing to know is that if its a 100Mb connection or a 10Gb connection...the transfer rate is still the same: How fast electricity is sent over copper.  That being said, what makes a 1Gb or 10Gb connection different?  The difference is the rate in which the scheduler can service the queues.  When we allocate bandwidth, we're essentially allocating a weighted value to how often the scheduler will service the queues.  When we reference the buffer, we're telling the hardware how much memory to allocate to the queues.

Buffer notes:
  • Give every queue at least 10 % buffer space with a thresholds of 40 %.
  • Setting thresholds too high may result in one highly congested port stealing buffer memory from all the other ports.
  • If every queue reserves the maximum possible very little will be left in the common buffer pool.
  • If a queue doesn’t reserve anything it may be starved by the other queues/ports.
Dropping: Tail drop or random drop
  • Tail drop is used by both policing and shaping; when the traffic exceeds a certain rate, every packet is dropped until traffic drops below that rate.  This has adverse effects on TCP re-transmission.
  • Random drop is a tool that randomly (go figure) drops packets in an effort to drop packets to AVOID congestion that may lead to tail dropping.  This works more closely with TCP retransmission logic.
  • Best used to regulate TCP-based data traffic...sucks for real-time traffic.


Input:

Cisco 3750
  • Queuing:
    • 1 Priority queue, 1 non-priority queue, and 3 thresholds per queue (1P1Q3T)
    • Queue 2 is the priority queue by default.
  • Buffer:
    • We specify the input buffer ratio with the following command: mls qos srr-queue input buffers <queue1> <queue2>.
  • Bandwidth: 
    • SHARED round robin.  This is applied globally.  Shared means that its a MINIMUM bandwidth guarantee.  Should other queues need additional bandwidth and it is available, they can "share" the bandwidth.  Only in times of congestion does the bandwidth guarantee take place.  
    • We can specify the bandwidth amount for both queues: mls qos srr-queue input bandwidth <queue1> <queue2>.
    • We can specify the bandwidth specifically to the priority queue: mls qos srr-queue input priority-queue <#> bandwidth <%>

Output:

Cisco 3750
  • Queuing:
    • 1 Priority queue, 3 non-priority queues, and 3 thresholds per queue (1P3Q3T)
    • Queue 1 is the priority queue by default.
  • Buffer: We specify the output buffer ratio with the following command: mls qos queue-set output 1 buffers <queue1> <queue2> <queue3> <queue4>. 
    • The egress buffer is shared PER ASIC.
  • Bandwidth: 
    • SHAPED or SHARED round robin.  This is applied at the interface level.  Unlike shared that is a minimum guarantee, shaped is a MAXIMUM bandwidth guarantee.  In other words, it gets a reserve portion of a port's bandwidth, and no more.
    • We can specify the bandwidth amount for all 4 queues...AT THE INTERFACE LEVEL!
      • srr-queue bandwidth share weight1 weight2 weight3 weight4.  This specifies the weighted values for shared mode.  These weights do NOT have to equal 100%  They are weighted values.  For example, if we had the weights 100 100 100 100, we'd have a total of 400.  We'd essentially be giving each queue a weighted value of 100/400, or 25%.  
      • srr-queue bandwidth shape weight1 weight2 weight3 weight4.  This specifies the inverse of the weight (1/weight) to determine the bandwidth for a queue.  For example, the weight values 50 50 0 0 applied to a 1Gbps interface would mean 1/50th of the bandwidth of the 1Gbps link..or ~20Mbps.  A value of 0 means...do not shape..or in other words..NO LIMIT--WOOHOOO!!!!
      • If an interface has BOTH shaping and sharing...the shaped mode configuration is applied.
      • srr-queue bandwidth limit weight can be used to specify the max amount of an interfaces bandwidth that can be used for outgoing traffic.  By default, there is no limit (i.e. a weight of 100).   AT THE INTERFACE LEVEL.

Scenarios:

1. We want to limit inbound traffic to 50 Mbps on a 1Gbps link.  We could/should use a policer.  We cannot used shaped round robin, as this is only available on egress interfaces.
2. We want to limit outbound traffic for a particular queue to 40 Mbps.  We could use shaped round robin.

Practical implementation:

  • Classify traffic as close to the trust boundary as possible.
    • Use class-maps to match the traffic types (Can use L3 addressing, L2/L3 markings, ports, or specific protocols).
    • Default class-map treatment is match-all.
    • NBAR (match protocol) is more CPU intensive than by matching traffic by DSCP, addresses, or ACLs.
  • Marking of traffic entails writing a value in the frame or packet header to document the class of traffic the packet has been determined to belong to.
    • If the packet markings are not trusted, they are re-marked to a DSCP value of 0.
      • By default trusting COS causes the DSCP value to be rewritten to the corresponding value based on the cos-dscp map.
    • Marking is often done at the end device for Cisco voice and video products.
    • Class-based marking occurs after classification.  Therefore, if used on an output policy the packet markings can be used by the next hop-node to classify the packets but cannot be used on this node for classification purposes.  If class-based markings are used on an ingress interface as an input policy, the markings applied to the packet can be used on the same device on its egress interface for classification purposes.
    • If you apply an output policy to tunnel interface, the marking is applied to the inner packet header.
    • If you apply an output policy to the physical interface, the marking is applied to the output packet header.
    • Mark voice traffic EF (DSCP 46).
    • Broadcast video should be marked to CS5 (DSCP 40).
    • Real-time interactive traffic should be marked to CS4 (DCSP32).
    • Multimedia conferencing should be marked to the AF4 class (AF41, 42, or 43..or DSCP 34, 36, or 38, respectively). 
    • Multimedia streaming should be marked to the AF3 class (AF31, 32, 33 or DSCP 26, 28, 30, respectively).
    • Transactional data (excessive latency in response times directly impacts user productivity--low-latency) should be marked to the AF2 class (AF21, 22, 23 or DSCP 18, 20, 22, respectively). 
    • Bulk Data (High throughput data service--non-interactive background data applications) should be marked AF1 class (AF11, 12, 13 or DSCP 10, 12, 14, respectively).  
    • Best Effort should be marked DF (DSCP 0)
    • Scavenger Should be marked to CS1 (DSCP 8).  
    • Network control should be marked to CS6 (DSCP 48).
    • Signaling should be marked to CS3 (DSCP 24). 
    • Use set dscp and set precedence, as using set ip dscp only matches IPv4 and not IPv6. 
  • Voice should be admission controlled.
    • Examples: G.711 and G.729a VoIP calls
  • Broadcast video should be admission controlled.
    • Example: Cisco IP Video Surveillance (IPVS), Cisco Digital Media Players (DMPs).
  • Real-time interactive should be admission controlled.
    • Example: Cisco TelePresence. 
  • Multimedia conferencing should be admission controlled.
    • Example: Cisco Unified Personal Communicator, Cisco Unified Video Advantage, etc.
  • Multimedia streaming MAY be admission controlled.
    • Example: Cisco Digital Media System VoD streams.
  • Transactional data should be treated with an AF PHB, with a guaranteed-bandwidth queue and DSCP-based WRED.
    • Examples: Database applications, data components of multimedia collaboration applications.
  • Bulk data should be treated with an AF PHB, provisioned with a guaranteed-bandwidth queue and DSCP-based WRED.
    • Usually provisioned in a moderately provisioned queue to provide a degree of bandwidth constraints during periods of congestion.
    • Examples: FTP, email, backup operations, etc.
  • Best effort should be provisioned with a dedicated queue and may be provisioned with a guaranteed-bandwidth allocation and WRED/RED.
  • Scavenger class should be assigned to a minimally provisioned queue.  
    • Examples: iTunes, YouTube, BitTorrent, etc.
  • Network control traffic may be assigned a moderately provisioned guaranteed-bandwidth queue.  
    • Examples: EIGRP, OSPF, BGP, HSRP, IKE, etc.
  • Signaling may be assigned to a moderately provisioned guaranteed-bandwidth queue.
    • Examples: SCCP, SIP, H.323, etc.
CBWFQ notes:
  • Use fair-queue presorter within a policy-map to manage fairly multiple flows contending for a single queue.  Without using the fair-queue policy-map configuration, the default is FIFO.  What does this mean?  By "fair queuing," we create "mini" queues within the queue, per session.  As seen on the right, the top picture indicates fair-queuing while the bottom one indicates FIFO.  With fair-queue, the router will give each "mini queue" equal shares of the allocated bandwidth.  Ultimately, the goal is to ensure that low volume talkers are not starved out of bandwidth by higher volume talkers.
  • 64 configurable queues.
  • Each queue is serviced in a weighted round robin (WRR) fashion based on the bandwidth assigned to each class.  The CBWFQ scheduler then forwards packets to the Tx-Ring.
  • Do NOT exceed 33% of the link capacity for the priority queue--you may starve other classes.  IF there are links dedicated to a particular traffic type...ignore this rule.
  • LLQ includes an implicit policer; a MAXIMUM bandwidth allocation.
  • Allocate at LEAST 25% of link bandwidth for the default class; Unclassified applications end up here.

Congestion avoidance:  Work BEST with TCP-based applications because selective dropping of packets causes the TCP windowing mechanism to throttle back and adjust the rate of the flows to manageable rates.
  • Little impact on UDP..as no retry logic.
  • Random early detection (RED): Randomly dropping of packets before it is all done at once (tail dropping) stops global synchronization of TCP streams.
  • Weighted random early detection (WRED--WHAT CISCO USES): Uses AF drop preference values to drop packets more aggressively if it has a higher AF drop precedence.  This is configured by using random-detect dscp-based.  Do not configure WRED on the LLQ, but should generally be used whenever the feature is available.  If you configure WRED on a policy map, make sure that it is not configured at the interface level.