System Upgrade Slow

Unanswered Question
Jul 27th, 2009

Is there any reason that the system upgrade takes hours? We have plenty of bandwidth, etc. I do not know the size of the upgrade, but it seems like it should be able to do this within 20 to 30 minutes instead of 5 or 6 hours. Is there a way to speed this up?

I have this problem too.
0 votes
  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 0 (0 ratings)
Loading.
dkoopman_ironport Tue, 07/28/2009 - 20:22

The first time I upgraded with the GUI, it was stuck on 5%, I was staring at it, thinking something must be wrong with my network. It wasn't moving. Minutes were going by, still at 5%. Then I realized it was waiting for me to click the "continue button". Once I did that, it finished the download in seconds, and upgraded completed in less than a minute.

In one case, I had to click "Continue" more than once.

Since this, I've used the CLI to do upgrades. It's a little more descriptive about what's going on.

If it's your network, a common problem is duplex mismatch. In the CLI, do a "etherconfig" "media". It'll list your Ethernet interfaces like this:

Ethernet interfaces:
1. Data 1 (Autoselect: <1000baseTX>) 00:24:e8:55:e5:ae
2. Data 2 (Autoselect: <1000baseTX>) 00:24:e8:55:e5:ae
3. Management (Autoselect: )) 00:10:18:55:4a:2b

Does the duplex match what you expect?

Bart_ironport Sun, 08/09/2009 - 10:28

Indeed, always check your network interface speed and duplex settings, especially if you have cisco switches.

You can always try upgrading from a local server - the manual has some basic instructions on how to set this up (but its not trivial). Even on systems with lots of bandwidth, this can make a huge difference. Last time I monitored the download speed i got, it was only 4mbit. Maybe they only have servers in the US or so.

Also, if your box is under heavy load, always suspend it before starting the upgrade. Its mentioned in the upgrade instructions and not without reason. I've seen what happens if a system is near its maximum capacity and you start an upgrade without suspending. It takes hours to recover - not something you want to do quickly at the end of a day :)

Several hours is a very long time for an upgrade. Installing the software from a local server only takes a few minutes - streaming upgrades take about half an hour around here. Then it reboots, which usually takes 10-20 minutes and which is the scariest part of the process because you have absolutely no information about what is going on.

shannon.hagan Thu, 08/13/2009 - 19:51

Local upgrades go much quicker. You can write a short script that will grab the right files for your serial number(s) and then put it on a web server locally and then do the upgrade. Every once in a while IronPort makes a change and the script needs a little tweeking but not often.

meyd45_ironport Wed, 08/19/2009 - 16:21

downloads.ironport.com is hosted on Akamai, so should be fairly speedy wherever you are.

AsyncOS upgrade files were circa 250Mb last time I checked

J..

gjones_ironport Mon, 08/24/2009 - 22:55

I got with support. 2 things I had to do to fix this issue -

1. Use the CLI (even after number 2 below it still doesn't work via the GUI and IronPort support strongly recommended the CLI for upgrades).

2. My firewall uses an outbound HTTP Proxy action to screen outbound traffic. I had to setup an separate route using only an HTTP path so that no deep packet inspections would be performed on it.

That seems to have done the trick.

mychrislo_ironport Fri, 08/28/2009 - 08:24

Today i got another strange problem (could be network...though).

one appliance is running 6.3.x and i am upgrading to 6.5.0 (it wont let me go to directly 6.5.x).

it wasn't success at all. I have to change download link from
downloads.ironport.com to downloads-static.ironport.com.

Upgrade took almost 1 hour 40min or so.

Then I upgrade to 6.5.1. That took 10 minutes....or 15 tops...

steven_geerts Wed, 09/16/2009 - 19:50

I have to change download link from 
downloads.ironport.com to downloads-static.ironport.com


YEP! I noticed that too.
We use the static upgrade server from the beginning (AsyncOS 4.7) since our firewall team did not want to open a range of download servers and we needed to request for opening ports to new download servers.
It's quite confusing since some of the use (only) HTTPS and others use (only) HTTP. Besides that you can configure the ports to use for some of het download settings, for others you can not even specify the protocol to use (http/https).

It would have been nice if these changes where announce some more widely, since now I had to find out the hard way something has changed and….. (maybe more important)…. WHAT has been changed?

Steven
pvdberg00 Fri, 09/18/2009 - 11:32

In the beginning of this year I had the following problem:
I want to upgrade our test C60 from 6.5.0 (405) to 6.5.1 (004). We received the error “error fetching manifest: Failed to connect to manifest server”. From your knowledge base I understand that something is changed in 6.5. We normally upgrade from http://downloads-static.ironport.com/asyncos/ from the default port (80) and I can dee that the new version will do the upgrade from https://update-manifests.ironport.com (port 443). Is there a way to change this, otherwise we have to do a complete reviewproces with our customer which take some time ?

Is this the same problem you are referring to or ... ?

If so ask the following to your fw team:

ETG-NG Ironport1_Externe interface (external_normal) Mail gateway xxx.xxx.xx.xxx ETG-NG Ironport Upgrade server Ironport Upgrade server 204.15.82.17 TCP https 443
ETG-NG Ironport1_Externe interface Mail gateway xxx.xxx.xx.xxx ETG-NG Ironport Upgrade server Ironport Upgrade server 204.15.82.16 TCP https 443
ETG-NG Ironport1_Externe interface Mail gateway xxx.xxx.xx.xxx ETG-NG Ironport Upgrade server Ironport Upgrade server 204.15.82.17 TCP http 80
ETG-NG Ironport1_Externe interface Mail gateway xxx.xxx.xx.xxx ETG-NG Ironport Upgrade server Ironport Upgrade server 204.15.82.16 TCP http 80


There is also a knowledge base article about this
994 Using downloads-static.ironport.com instead of downloads.ironport.com

Greetings Peter

steven_geerts Fri, 09/18/2009 - 12:41

Hi Peter,

This is exacly the problem!

For us a similar request to our firewall team solved the problem.
(allow the Ironport ineterface used for upgrading to access 204.15.82.16 and 204.15.82.17 on ports TCP 80 and TCP443)

Steven.

Actions

This Discussion