i am university student. i am required to develop an system, which is a network monitoring system. im having a hard time trying to get started with it. let me brief about my system. the system should monitor the network in my college(in a particular building), and then identify which server or channel is congested, and then using the load balance technique to divert the data flow from channel or server which is congested to the other channel or server which is available and least congested or used. i am not sure of which language to use todevelpe this, maybe C i guess. java or perl im not sure bout that.please do brief about me. would be much obliged with your help. thank you.
I think this would be very difficult because you are talking about two different things.
You can easily implement something to monitor the network in your building and identify which server / ports are congested. However, loadbalancing is a different animal all together. I am not sure how you would try to have an application you write divert traffic to less congested servers without building your own loadbalancer.
ok..so would u give me a brief on how to start on developing a monitor system to monitor thr port or servers which are congested..give me suggestions on what language to build on, protocols, tools etc...just a simple monitoring system will do.. and what would you say bout diverting traffic to other ports or servers..can i use a different algorithm or technique to do that..do tell..
You can check MRTG source code for a start http://oss.oetiker.ch/mrtg/. MRTG is built in PERL. Using SNMP, it can monitor network equipments (routers, switches, and firewalls) and systems (linux, unix, and windows) CPU, Memory, Interfaces, and also services (for systems).
i think i got it wrong here. messed it up a bit. load balancing is to distribute process evenly to all servers, ports, chanels so that no one server is overwhelmed with request, rite. so can i build a network monitoring system, monitor the network as what usual monitors does. and as an additional feature, i would use the load balanccer to distribute the data evenly to all nodes when congestion occurs in any one of the node. can this be implemented?, please reply.
Monitoring is just that monitoring. You might be able to trip a piece of code and change something around depending on the device on the other side. But these really are to separate products. IMHO load balancing should be setup before the issue ever occurs, monitoring can show you a base line of traffic CPU cycles etc.... You set the pain tolerances for that circuit, application etc...
Example here, You have 5 web servers, you can monitor all 5 of them, and they are all different platforms etc.... So server 1 is old 10 mb connection, 2-4 are newer with 100 mb connections, and 5 is a new box full gig. You will monitor all of these and setup a base line of there capacity. I would recommend testing loads against each to find where it breaks down or becomes unacceptable. Once you have you base lines for each server, and all errors fixed. You can then start to think about load balancing. Now for you to write a program that changes the load balancing on the fly into your monitoring system can be done, but why? It will screw up all your bases, all your justification numbers, etc unless you set each interface to the same through put with a rate limiting command, shooting 4 out of your 5 servers in the foot.
So you buy create a load balancer, setup your rules in the load balancer completely for the servers based on there capabilities.
So server 1 should only get X of traffic
Server 2-4 should get y
Server 5 should get Z
Now server 5 could get as much as 100 times the hits as server 1 and still be functional it is the rules that your base line will tell you and testing.
Simplest way to put this is a good network admin will anticipate congestion have the data to backup the decision and plan and act before the congestion is apparent to the customers. Yes you do this with monitoring, but not via dynamic load balancing. Even if it was a case of Gig link, 100mb links and a T-1 or VPN across a DSL, you could prioritize that traffic through costs or other ways depending on your routing protocols and make the routers / switches dynamic in nature but it happens much faster than a monitoring system could react.
Monitoring system reaction times are based off the polling cycles of the devices, if the device is under heavy load, I prefer it to pass mission critical data, and drop a few pings, or SNMP requests from my polling engines. Once you get the system set and tweaked, it becomes reliable, but you might have polling times in the 3-5 minute ranges, depending on the network etc.... So your load balancer would rely on these polling times and be reactive over a long period of time instead of over a few nanoseconds in the case of a switch. You can set your polling times down into the second range but then you congest the network gear with polling request creating your own form of DOS attack via your monitoring system. Worse case scenario is the dynamic IDS/IPS functions you build into V2.0 would shut down your server due to the DOS attacks and leave your monitoring dynamic load balancing system off line then what?
Its never wrong to have ideas, try them, and see the effects.
ok..so monitoring and load balancing is a total different thing, rite...wel, im still a student..still doing my reaserch on this..and i dont have much idea about this.. so what would you actually suggest me to do as in developing a monitoring system?? i intend to do a monitoring system, with my specal feature of the load balancing..but after reading and resaerching..i got to know that load balancing is something different to what network monitoring is..is load balancing a device that should be bought and installed in the network?..i wz thinking that load balacing as an algorithm to be implemented in the monitoring system..ok, so what do you think is a good idea, developing a network monitoring system, to monitor the network data transfer to and from the server, the congestion and what kinda special feature that would be easy to implement in it..im having a nightmare thinking about the load balancing as i don't know what is it at all..my lecturer told me to research on it and try implementing it..please brief me in this?? what additional feature can i, as student implement in a monitoring system that i am going to develop..and what is the language i should be using?? i have learned C, bit of object oriented prog..thats about it..this is my final year project that i am doin as a requiremnt to fulfill my course..can you or anyone there help me out here?? im desperately in need of your help..thank you..
I HIGHLY recommend you download some free existing network monitoring tools and get an idea of what the do and the "look and feel" of them.
If you are going to implement a new tool, you need to know what the other well accepted and highly used tools do already.
As for load balancing, I don't know of anyone that has used a software based solution. All the load balancers I have used are appliances, like F5 or Cisco's CSS
Load balancing can come in many different animals. The simplest and least effective is DNS round robin. So you assign a name to the interface. And then you create alias to that name for your servers. But it is not very effective as it is just a look up an no verification as to either the server exists or not.
Some applications have load balancing built into them. Citrix for example will allow load balancing, and it does work requires tweaking through testing and then monitoring. But it does work, rather well.
On the heavy side you can use a hardware load balancer, Cisco has a few of them, as does Foundry ( the best one for simple setup, IMHO)
I have never thought about writing a software version or though that is all the hardware devises are is software on an OS specific for that job.
Check http://lcic.org/load_balancing.html for some of the Linux version out there. Mainly this is clustering references but you get the idea.
Monitoring download a version of Solarwinds as a Demo, and Check out Kiwi stuff for sysloging, and base you ideas from there. SNMP allows you to connect to most major devices, you can also use Netflow on Ciscos and Foundries.