MARS v6 Data Archiving

Unanswered Question
Jul 20th, 2009
User Badges:

In the data archiving section (admin - data archiving), what does Remote Storage Capacity in Days mean or what impact does the number of days have on MARS and the data archive location?


I set my storage capacity to 90 days. From MARS documentation, MARS begins archiving from the enable point going forward. So if I start archiving today, I will have to wait 90 days to get 90 days of archived data. The point of my NetPro is what happens on the 91st day? Does MARS stop archiving? Does MARS automatically remove the 1st archived day? I do I need to remove the 1st archived day?


MARS documentation…The data in the NFS or SFTP file share has a life span specified in days. Therefore, to keep a year's worth of data, you would specify 365 days as the value for the Remote Storage Capacity (in Days) field. All data older than 365 days is purged from the archive file.


How is the data purged?


Thanks

  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 5 (1 ratings)
Loading.

Data purge operation done by automatically.

Currently CS-MARS keeps events on a rotating buffer so newer events will wipe out the older events after a certain period of time. Basically events are purged as per FIFO.


Each MARS Appliance seamlessly archives data using an expiration date that you specify. When the MARS internal storage reaches capacity, it automatically purges the data in the oldest partition of the local database, roughly 10% of the stored event and session data. The data in the NFS file share has a life span specified in days.


Dynamic and event data is initially stored in two locations: the NFS archive and MARS database. Later, when the MARS database partition becomes full, the database purge operation occurs to make room for new events-but those events and incidents were archived prior to the purge operation.


rmeans Mon, 07/27/2009 - 10:48
User Badges:

Thank you for the replay.


MARS is data archiving with SFTP. I have MARS set to archive for 90 days. My archive computer shows a directory for each day. At the 91st day, does MARS delete the oldest entry?

pmccubbin Wed, 08/05/2009 - 04:36
User Badges:
  • Silver, 250 points or more

Rick,

The answer to your question is yes. You could set the archive for a larger number, too. The limitation is the size of the box.


Here is a replay from the MARS config guide on the subject, which you may have already read:


"Each MARS Appliance seamlessly archives data using an expiration date that you specify. When the MARS internal storage reaches capacity, it automatically purges the data in the oldest partition of the local database, roughly 10% of the stored event and session data. The data in the NFS or SFTP file share has a life span specified in days. Therefore, to keep a year's worth of data, you would specify 365 days as the value for the Remote Storage Capacity (in Days) field. All data older than 365 days is purged from the archive file.


When planning for space requirements, use the following guidance: Estimate 6 GB of storage space for one year's worth of data, received at a sustained 10 events/second. This estimate assumes an average of 200 Bytes/event and a compression factor of 10, both realistic mean values. In addition to capacity planning, plan the placement of your archive server to ensure a reliable network connection that can transmit 10 MB/second exists between the archive server and the MARS Appliance. You should consider using the eth1 interface to avoid high-traffic networks that might introduce latency and to ensure that the backup operation is not competing with other operations in the MARS Appliance. Also, define a default route to the archive server on the MARS Appliance and that you verify any intermediate routers and firewalls allow for multi-hour NFS or SFTP connections to prevent session timeouts during the backup operation."



Hope this helps.

Actions

This Discussion