Storage network slow backups

carl_townshend · ‎04-07-2010

Hi all

I dont know if anyone can help on here, We are having issues backing up to some LTO 4 drives on our SAN, the san switches are old brocades running at 2 Gig, the devices are backed up via a HP G5 server running netvault, the servers sit on the LAN and are mainly teamed to 2 GIG using the HP software. We had a consultant come in last week and say the speed is ridiculously slow, someservers are backing up at no more than 10Meg per second data rate, he said he normally see's around 100meg when backing up using LTO 4 drives,

he used a utility called fstfill which created a 1 gig file then wrote to disk, he said the disk performance on the old servers was terrible, some at 8Meg a second, could this be the reason??

Can anyone shed any light on why they would go slow slow??

hope somone can help

Cheers

Carl

marksmithinnz · ‎04-07-2010

Are the servers backingup directly to the tape drives ? Or are they sending their data over the network to a backup server which is the driving the tape drives ?

If the data is going to a backup server is it backingup to a pool of disk and then on to tape or trying to feed the tape drives directly from the network feeds ?

- If your are running straight from a network stream, how many streams at a time is your backup software configured to send to each drive ? (ie. are you backingup 1 server at a time, or doing 5 at a time ?)

- How congested is your network between your servers and the backup server ?

Also you mention "old servers" ... how fragmented are the filesystems on those servers ? They could be spending a lot of time seeking the data.

Just some thoughts from the top of my head while I wait for someone to answer my question

carl_townshend · ‎04-07-2010

Hi There

As far as I know, the backups run via the backup server then straight to the tape, they stream straight to tape I believe, I think there is maybe 2-3 backups running at the same time at most. There is no congestion between the network at all, and when I check the servers, they are using no network at all!!

any ideas ?

chenwan2 · ‎04-13-2010

hi Carl,

Did you install some tools based SNMP, like Cacti. Using those tools, you can see the network flow, and your server CPU uage.

The backup work flow is like this, right?

Server --->FC switch ---> LTO

Using monitor tools, you can see where is the bottleneck.

And you can do the next step to do the troubleshooting work.

carl_townshend · ‎04-13-2010

Hi,

Ok, I have done some LAN testing to the backup server, and im getting between 400 and 700Mb per second from most of my HP servers to the backup server, this was done using jperf. So this rules any bottlenecks out of the lan.

the backup server to the SAN is also fine as when we run direct from local disk on the backup server to tape it is very fast.

So, what could the issues be??

we are getting transfer rates when backing up I see at about 10Megabytes which is 80Megabits per second,

any thoughts anyone?

cheers

Carl

chenwan2 · ‎04-13-2010

Hi

It seems that all your HP servers sent backup data via backup agent to your backup server. Your backup server recieved the backup data and then wrote to the tap or something, right?

If your arch is like this, maybe the backup server is bottleneck. The recieved data over load.

mikram · ‎03-12-2011

hi

just a quick question, Are you using FC or iSCSI between Server and storage?.

I know on MDS switches you could use accelerate parameters etc.