This discussion is locked

ASK THE EXPERTS - WAAS MONITORING AND REPORTING

Unanswered Question
Aug 13th, 2010

Welcome to the Cisco Networking  Professionals Ask the Expert conversation. This is an opportunity to learn about Cisco Wide Area Application Services monitoring and reporting with Michael Holloway and Joe Merrill.  Michael is an escalation support engineer in the Application  Delivery Business Unit focusing on escalations to engineering related to  the Cisco Wide Area Application Services (WAAS) product. He has worked  with Cisco WAAS since its initial development, and with the first  product beta.

/* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-qformat:yes; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin:0in; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:11.0pt; font-family:"Calibri","sans-serif"; mso-ascii-font-family:Calibri; mso-ascii-theme-font:minor-latin; mso-fareast-font-family:"Times New Roman"; mso-fareast-theme-font:minor-fareast; mso-hansi-font-family:Calibri; mso-hansi-theme-font:minor-latin; mso-bidi-font-family:"Times New Roman"; mso-bidi-theme-font:minor-bidi;}

Joe Merrill is an escalation support engineer in the Application Delivery Business Unit focusing on escalations to engineering related to the Cisco Wide Area Application Services (WAAS) product. He has worked with Cisco WAAS since its initial development, and with the first product beta.

Remember to use the rating system to let Michael and Joe know if you have received an adequate response.

Michael and Joe might not be able to answer each question due to the volume expected   during this event. Our moderators will post many of the unanswered   questions in other discussion forums shortly after the event. This  event  lasts through August 27, 2010. Visit this forum often to view  responses  to your questions and the questions of other community  members.

I have this problem too.
0 votes
  • 1
  • 2
  • 3
  • 4
  • 5
Average Rating: 4 (2 ratings)
peggyjackson Tue, 08/17/2010 - 09:13

Hi Michael,


/* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-qformat:yes; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin:0in; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:11.0pt; font-family:"Calibri","sans-serif"; mso-ascii-font-family:Calibri; mso-ascii-theme-font:minor-latin; mso-fareast-font-family:"Times New Roman"; mso-fareast-theme-font:minor-fareast; mso-hansi-font-family:Calibri; mso-hansi-theme-font:minor-latin; mso-bidi-font-family:"Times New Roman"; mso-bidi-theme-font:minor-bidi;} When I bring up WCCP on my WAAS device, my switch shows in increase in fragmented packets and the CPU on the switch seems high. What could cause this?

Thanks.

nchidamb Tue, 08/17/2010 - 09:24

1. Do you have branches that don't have WAAS? If so, I suggest you apply ACL on your L3 switch (assuming Cat6k) to not redirect that non-WAAS branch. A scenario like this could cause fragmentation and very well increase the CPU on Cat6k as the fragmented packets will be processed in s/w.

2. What type of method are you using to return the traffic from WAE to Cat6k? If you use IP Forwarding (using Static Rotue or Default Gateway ) the traffic will return from WAE to Cat6k. If you use IP Forwarding to return the traffic then WAE will not add any additional header. So, even in a scenario like #1 the fragmentation will not happen.

Pl note that if your WAE optimizes the traffic then it will not fragment the packets even if you use WCCP-GRE or Generic GRE as your Egress-Method to return the traffic.

Regards

Nat

steve_vanburen Tue, 08/17/2010 - 15:29

Hi Michael and Joe,


/* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-qformat:yes; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin:0in; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:11.0pt; font-family:"Calibri","sans-serif"; mso-ascii-font-family:Calibri; mso-ascii-theme-font:minor-latin; mso-fareast-font-family:"Times New Roman"; mso-fareast-theme-font:minor-fareast; mso-hansi-font-family:Calibri; mso-hansi-theme-font:minor-latin; mso-bidi-font-family:"Times New Roman"; mso-bidi-theme-font:minor-bidi;} Much of our traffic shows up as ‘Other’ in the charts on the Central Manager.  We see OK optimization for ‘Other’ traffic, but suspect that some traffic is being optimized that should probably be pass-through.  How can we determine what traffic is being defined as ‘Other’?

--Steve

mhollowa Tue, 08/17/2010 - 16:01

Steve,

You can enable transaction logging and one of  the fields is classifier matched.  You can also look a CM GUI for each WAE and under Monitoring / Optomization / Connection Statiscics there is also a Classifer field.

Michael

jomerril Tue, 08/17/2010 - 21:34

Enabling transaction logs will help you identify which traffic is hitting which classifiers, but may not be an optimal solution in a very busy production environment.  You can also look at the "show statistics connection" output, and identify connections with Application Name of 'Other'.  You can create a new Application that has statistics enabled, and an associated Classifier that matches some aspect of a connection currently mapped to 'Other'.  With that new policy definition in place, watch that Application's statistics to see how much traffic goes through, and whether it sees any benefit from optimizations.  You may need to talk with the file server or application management staff there to determine what application is associated with that traffic, and whether you want to optimize it.  Some traffic may see good optimization because it is text, but not get any realistic benefit from the optimizations because of its transactional nature where only a few bytes are sent back and forth at a time.

dean.fitzpatrick Fri, 08/20/2010 - 13:08

Hi,

Maybe you can help me.  Since bringing another WAE into the WCCP cluster, we’ve noticed a dramatic increase in the packet incoming and outgoing rate on the interfaces.  But the number of users hasn’t changed, and they haven’t changed what network services they are using.  What would cause this?

---Dean

jomerril Fri, 08/20/2010 - 13:26

You may be seeing the results of flow-protection.  When a new WAE is introduced to an existing WCCP cluster, the WCCP buckets are reassigned with some buckets moving from previous WAEs to the new WAE so they all have an (nearly) equal number of buckets.  In order to avoid resetting connections on flows that are optimized by one WAE but now belong to a new WAE, WAAS will flow-protect those connections.  When the router redirects an in-progress connection to a WAE that is the new owner of the bucket, that WAE will re-redirect the packet to the WAE that previously owned that bucket so that WAE can continue the optimization.

That re-redirection adds traffic to the interfaces without actually increasing the number of flows.

In order to see whether flow protection is occurring, take a look at the "show wccp flow tcp-promiscuous details" output at each of the WAEs.  Any buckets that show IN mean that this WAE is now the owner of a bucket that appears to have in-progress flows and will redirect those to another WAE.  The IP address of the previous owner of the bucket, who will be receiving those packets, is listed in the output as well.  Any buckets that show OUT indicate that the bucket moved away from this WAE (to the WAE whose IP address is also listed) and that it will receive re-redirected packets from the new owner of the bucket.

Also, the "show wccp gre" output will show the number of packets redirected to another WAE, and that number would be increasing.

As attrition of the connections occurs, with those connections naturally terminating over time, the number of flow-protected connections will decrease and ultimately flow-protection will cease on its own.  However, during this time, any Pass-through traffic that may go idle for 10 or 15 seconds may also fall into flow-protection as the WAE who owns the bucket (shows IN) forgets the reason for the pass-through decision and lumps it in with the other InProgress connections.

If flow-protection is a concern, and you are willing to risk disrupting existing flows--perhaps during a change control window--you can temporarily disable flow-protection on the WAE that now owns the bucket, and shows the IN flag.  Leave it disabled for a few minutes, then re-enable it.

To disable flow-protection:

  config

  no wccp flow-redirect enable

  exit

To enable flow-protection again:

  config

  wccp flow-redirect enable

  exit

Mohammed Khair ... Sun, 08/22/2010 - 02:58

I am planning to install WAAS between our HQ and a branch, I have some concerns regarding the design of the solution.

At the HQ we have Cisco WAE 674-k9 connected to 6509.

At the branch we have NME-WAE 502 module connected to 2811 router.

What is the best way to setup the WAAS componenets in this scenario?


Shall I use WCCP between the 6509 and WAE 674 in the HQ?

How can I apply the WAAS only on specific traffic types since I have only 1 router "2811" with the module on the router itself.

Thanks in advance

jomerril Mon, 08/23/2010 - 07:10

You would probably use WCCP at both the data center and the branch to redirect packets to the WAEs.  On the routers, you could create ACLs so that only the traffic you want to optimize is redirected to the WAEs.  But take care to keep the ACL limited and simple.  As the ACL grows it be comes more complicated, more prone to configuration errors, and more likely to fill the tables used for hardware routing on the Cisco Catalyst 6500.

Mohammed Khair ... Tue, 08/24/2010 - 03:34

Hi Joseph,

Thanks for the reponse, what I am interested in is knowing how would I apply the NME configuration on the Branch Router 2811 in this case.

How can I use WCCP between the router and the NME module that is also installed on the same router?

Regards

jomerril Tue, 08/24/2010 - 07:53

You can either use the internal NME interface (links through the backplane in the router), or the external NME interface cabled to an interface on the router.  You would configure WCCP as normal, using either of those two interfaces and the IP address from the corresponding interface on the NME.

Usually, your basic configuration would look something like...

On the router:

ip wccp 61

ip wccp 62


interface Integrated-Service-Engine

ip address

service-module ip address

service-module ip default-gateway


interface

  ip wccp 62 redirect in


interface

  ip wccp 61 redirect in

This configures the router for WCCP, and configures the Integrated-Services-Engine with the internal interfaces for the router and NME. You would then need to configure the NME.  You can either telnet to it, or from the router's CLI you can open a console session to it in order to configure it.  On the NME, configure WCCP as you normally would, with the redirect list pointing to the internal interface we configured for the router (above).

You can find examples in the documentation.  For instance:

http://www.cisco.com/en/US/partner/docs/nsite/enterprise/wan/wan_optimization/chap09.html#wp1176095

kumar_vijay Thu, 08/26/2010 - 08:03

Hi,

My question is simple and short.

Is there any documentation available with the help of which it is understand the report of WAAS.

jomerril Thu, 08/26/2010 - 08:12

I'm afraid I'll need you to expand on your question a little bit.  For which report are you asking for documentation?

kumar_vijay Thu, 08/26/2010 - 08:37

Hi,

I need the explanation of the below terms.

1)When we pull bandwidth Optimization report, on Y-Axis the graphs says Effective Capacity .What is Effective Capacity?
2)what is reduction % excluding and including passthrough
3)What is effectivity capacity including and excluding passthrough ?
4)With the help of which report, we can show the customer that the file download which took 10 mins in first attempt, is downloaded in 10secons in next attempt?
5)How to show that the bandwidth utilization has decreased by which %.
6)Which report says that the applications have become this much time faster?

These questions are normally put forwarded by many customers ? Can you please help me with your expertise answer ?

Regards,

Vijay

jomerril Thu, 08/26/2010 - 13:06

Very good questions. Let me try and take them one at a time. Some of the answers you will likely find in the CM GUI help (upper-left corner is the Help button), or in the online documentation. But let's add a little more color and detail.

1)When we pull bandwidth Optimization report, on Y-Axis the graphs says Effective Capacity .What is Effective Capacity?

Basically, the "effective increased bandwidth capacity" is telling you how much additional WAN bandwidth you've gained because of the optimization. It will chart somewhere between 1 times and 100 times. Typically it charts all traffic, though you can configure it to chart traffic for specific Applications.

The CDM online help gives the formulas used to chart the graph:

Effective WAN Capacity = 1 / (1-% Reduction Excluding Pass-Through)

% Reduction Excluding Pass-Through = (Original Excluding Pass-Through - Optimized) / (Original Excluding Pass-Through)

2)what is reduction % excluding and including passthrough

Looking at the formulas given above might help you understand. The one is a reduction ratio compared to only the original traffic that is optimized. The other is a reduction ratio compared to all original traffic, whether it is optimized or not. So, if you want to know what kind of optimization you are getting for the traffic that you configured to have optimized, look at the "excluding pass-through" numbers. If you want to know the positive effect that optimization is having on your full traffic load, take a look at the "including pass-through" numbers.

3)What is effectivity capacity including and excluding passthrough ?

The effective capacity is what kind of throughput you can potentially realize on the WAN -- assuming you would fill it to 100% capacity -- because of the level of optimization you are seeing. The "including" numbers show you the effect of optimization compared to all the traffic passing through the WAE whether it is optimized or not. The "excluding" numbers show the effect of optimization compared only to the traffic that is receiving optimizations.

4)With the help of which report, we can show the customer that the file download which took 10 mins in first attempt, is downloaded in 10secons in next attempt?

This one is a little trickier. The reports are much broader than a single connection. They are for all traffic, or for traffic that matches specific defined Applications. You could create a separate Application and matching classifiers for the client and/or server IP addresses and/or ports, run the test, then configure the charts to only show you the data for that Application. By default, statistics for an Application aren’t charted unless you check the "Enable Statistics" box when defining/editing the Application.

5)How to show that the bandwidth utilization has decreased by which %.

You want to look at the % reduction numbers you asked about in #2 above.

6)Which report says that the applications have become this much time faster?

These questions are normally put forwarded by many customers ? Can you please help me with your expertise answer ?

This is probably the hardest question to answer.

"Faster" isn't always easy to define. You are probably talking about user experience rather than statistics found in a network device. What determines that experience? A web page fully populating with all the pictures? A CIFS-based application that saves a file? A custom application that collects data from different servers over different protocols to perform some operation? Much of that is subjective and based on multiple individual requests, sometimes over different protocols.

What we can provide are statistics to show the effect of WAN optimization and application acceleration for specific types of traffic. We can't show you that displaying a web page is N times faster with WAAS, because we don't know which of all the many HTTP requests that are made are specific to the user experience. But we can show that each of the requests received so much overall optimization, so much optimization from DRE, so much optimization from LZ, so much added benefit from HTTP acceleration.

What you would probably do is collect some base-line timings for performing certain user activities, then perform the same operations both cold (first pass) and warm (subsequent passes). Back up those timing numbers with reports from the CM GUI, and perhaps even the "show statistics connection connection-id ". Which reports to use? Start with those Optimization and Acceleration reports. Those are the reports we expect will give the most complete/accurate pictures of the benefit of WAAS. You can also create and even schedule custom reports as needed.

kumar_vijay Thu, 08/26/2010 - 22:14

Hi Joseph,

Thanks very much for the detailed answer.Much Appreciated.

We are going to integrate NetQoS RA with WAAS.It would be new to me.Can you just give a briefing regarding this integration or help me with the link to documentation stuff pertaining to this.

Regards,

Vijay

jomerril Fri, 08/27/2010 - 07:51

Essentially you will enable the FlowAgent feature on your WAE and point it at your NetQoS SuperAgent monitoring agent console device. The FlowAgent feature will collect statistics and send them over to the NetQoS SuperAgent. You can then use NetQoS to present that data in various ways, such as a chart showing bandwidth usage.

Enabling the FlowAgent on the WAE can be done either from the CM GUI device or device group configuration page, or from the CLI itself. The procedure is really two simple configuration lines. It looks like this from the CLI...

flow monitor tcpstat-v1 host

flow monitor tcpstat-v1 enable

You will need to follow the documentation for your NetQoS device in order to configure it as desired.

The Configuration Guide for WAAS 4.2.1 provides the basic instructions for configuring the FlowAgent on Cisco WAAS:

http://www.cisco.com/en/US/docs/app_ntwk_services/waas/waas/v421/configuration/guide/monitor.html#wpmkr1163525

This white paper may also be helpful:

http://www.cisco.com/en/US/prod/collateral/contnetw/ps5680/ps6870/prod_white_paper0900aecd80693006.html

Rodrigo Marchin... Thu, 08/26/2010 - 11:20

Hi Michael and Joe!

How are you?

I upgraded a WAVE-574 with the last WAAS version 4.2.3.

But, I am getting a alarm of "Memory Mismatch" (see the attachment).

My WAVE-574 has 4GB of RAM.

Do I think the version 4.2.3 is checking if WAVE-574 has 3GB or 6GB of RAM, right?

I downgrade it to version 4.1.7 and I didn´t get any problem.

Do you think CISCO will send a new release to fix that?

Thanks

Rodrigo Marchina Soares

jomerril Thu, 08/26/2010 - 13:17

The WAVE-574 device supports either 3 GB or 6 GB or RAM.  WAAS software expects one of those two configurations, and will allocate disk and memory for the optimizers and accelerators based on one of those two configurations.  With an unsupported configuration of 4 GB, there is a risk of unpredictable issues as maybe not enough memory would be allocated for the configured disk space.

This should have been caught in 4.1.7.  It is a good thing that we are catching it in 4.2.3.

Look for bad DRAM that dropped the discovered memory from 6 GB installed to 4 GB, and request RMA as needed.

mario-leitao Thu, 08/26/2010 - 11:50

I installed Waas Mobile but I want to configure the server´s address on the client software distribution package as its hostname instead of its ip address. Is it possible? In the waas mobile 3.5.1 admin manual (page 34) it says that I can configure as server ip or hostname but when I try to configure it then I just get a drop down list with the server ip address. The point of this configuration is that I have a customer that needs to access the waas mobile server via its Name because it will access from different locations and via its ip address is not a viable solution.

jomerril Thu, 08/26/2010 - 14:04

That drop-down list also contains a blank line as the first line in the list.  If you click that blank line, you are then given the opportunity to enter either a hostname or IP address.  But whatever you enter must be resolvable.  It is this hostname or IP address that a client will use when it is first installed and contacts the management server in order to determine which server the client will then use for acceleration.

kumar_vijay Fri, 08/27/2010 - 04:38

Hi,

Recently we upgraded the IOS of some 20 WAAS-574-WAVE devices from 4.1.7 to 4.2.1 .For all devices upgradation was following with the same process.

Here is what happened after reebot :

1)14 devices came up smoothly.

2)6 boxes were continusly rebooting.

3)We were able to recover 3 boxes with rescue CD.

4)1 boxes came up after hardbooting 3-4 times.

5)2 boxes didn't come up by any means and hardware had to be replaced.

Any idea what could be the reasons behind it ? The same method was followed for all devices and are of same model.

Regards,

Vijay

jomerril Fri, 08/27/2010 - 07:29

I'm afraid that without looking at data from the failed devices, it would not be possible to speculate on the causes. There are many possibilities ranging from various hardware failures to software problems. Likely we will find that not all failures occurred for the same reason.

In my experience, such a concentration of failed upgrades is not common and needs to be investigated. If you have not already done so, please involve TAC to collect data from the failed devices and have it analyzed for individual failures on each device as well as holistically to see whether there could be a common cause.

Actions

Login or Register to take actions

This Discussion

Posted August 13, 2010 at 1:58 PM
Stats:
Replies:24 Avg. Rating:4
Views:6991 Votes:0
Shares:0

Related Content

Discussions Leaderboard