| |
|
|
Introduction
Logging is the recording of subsystem generated information for the purpose of monitoring Avici router operation.
All Avici router subsystems generate logging messages to ensure that:
- Unexpected events report to system administration for analysis.
- Program tracing and debug aid system troubleshooters in the search for, and resolution of, problems.
- Critical errors generate appropriate indications to the operations center for network management consideration.
For a listing of Avici router subsystem see the subsystems table in the "Logging Commands" section of the IPriori Command-Line Interface Reference.
IPriori uses the same message format across all subsystems. View the recorded information in the form of text messages. These messages indicate the occurrence of notable events. Each message contains up to six distinct fields as outlined in Table 11-1 on page 194.
Each logging message is assigned a severity level. The severity level has two important related functions: to identify the nature of the event, and to provide the user with a basis for filtering messages that do not rise to the level of current interest. IPriori provides six severity levels. Examples of the criteria for separating out messages by severity level are:
- Informational messages that do not indicate an error
- Unexpected subsystem events that should not affect normal operation
- Problems that will affect subsystem operation
- Errors that will or have already resulted in a subsystem reset
Log messages reside in the server file system, and can be optionally redirected to a remote host's file system.
Messages display to the console or current telnet session. Messages can either be displayed as they arrive at the server or displayed from saved files. Message-filtering options are the same for either display source.
syslogd provides for remote host distribution of messages. Avici router severity levels map with standard syslogd priority levels.
Input filtering determines which messages are forwarded to the server at message generation, depending upon the default or user configured filtering criteria. Input filtering commands allow for filtering by subsystem and severity level. The highest level (critical) messages are always logged, regardless of the input filtering configuration.
Display filtering determines which messages already logged on the server will display to the console or monitor. Display filtering is by platform, subsystem, and severity level. When searching log files, a start and end time stamp is also available. Any message below the configured severity level will not display. The default setting displays all logged messages.
Logging files may be searched and sorted by any allowed filtering configuration, with the output sent either to the console or to another file.
Message Format
There are six Avici router logging message fields. Avici router logging message fields are described in Table 11-1.
Table 11-1. Logging Message Fields Field Description Platform
The specific platform that generates the message. There are three Avici router platforms: server, module, and bay.
Date and Time
A stamp indicating the system date and time the logging event occurred.
System/Level Code
A series of eight hexadecimal characters used by the server to aid in filtering the log files on the hard disk for output to the console.
Level
An indication of the severity of the message. See Table 11-2 on page 195. There are seven Avici router severity levels.
System
The subsystem that generated the message, for example IS-IS, BGP or SNMP.
Message
The text of the message.
Figure 11-1 displays a sample logging message format as it displays on the console or is written to file. All Avici router logging messages follow this format.
Figure 11-1. Logging Message Format
Severity Levels
IPriori provides eight levels of severity. Critical messages are always logged regardless of the filtering configuration.
Table 11-2 on page 195 lists the logging subsystem severity levels.
Message Filtering
Message filtering provides the user with a means of limiting the messages that either display to console, or are written to disk. Filtering is based upon the platform, subsystem, and message severity level.
There are a number of reasons why filtering is important:
- The quantity or diversity of messages can be difficult to cope with if you are looking for a specific type of message in a display or stored message search.
- Unrestrained message generation can deplete the Ethernet link bandwidth between a platform and the server.
- Your log file message search can be confined to a specific time period.
There are two types of message filtering. The two filtering types can best be understood by asking the following two questions:
- Which messages should get logged, (given the trade-off between your need to know the status of the subsystem, and the subsystem resources required to transport the message from the generating platform to the control)? This question is answered by how you configure the first type of message filtering: input filtering. This type of filtering primarily concerns itself with subsystem resources and which messages you are willing to do without ever seeing, because they never get logged.
- Which messages should display to a console or monitor (given your current priorities concerning the status of the Avici router)? This question is answered by how you configure the second type of message filtering: display filtering. This type of message filtering primarily concerns itself with the actual review of messages and the filtering out of the clutter of messages that are not of current interest.
See Figure 11-2 for an overview of the relationship of platforms and commands to the two types of message filtering.
Figure 11-2. Message Filtering
Input Filtering
IPriori allows you to filter messages by severity level at the time of message generation. Only messages with a severity level at or above the configured setting will be sent to the server. Input filtering not only reduces the size of the resultant log files, but also frees bandwidth between the platform and server, that can then be used for operations of a higher system priority.
To setup input filtering on a platform, you must first enter the command mode for that platform. Entry is provided by three commands. These commands are server, module, and bay.
The logging-filter command sets the minimum severity level for messages logged to the logging system. When a platform generates a logging message, IPriori checks the severity of the message against the setting of the logging filter for that platform. If this check fails, indicating that the message severity is below the severity set in the filter command, the message is discarded; if the check passes, the message information is placed in protected RAM for transport to the server.
NOTE Critical level messages can not be filtered. They are always sent to the server without any delay.
Figure 11-3 presents a graphic display of severity level filtering set for warning. Messages at severity level warning and above are passed to the server; messages at severity level cleared and below are blocked and discarded. Warning is the default severity level for input filtering.
Figure 11-3. Severity Filtering
RAM Storage
When a message is generated, if it passes the input filtering criteria, it is immediately placed in RAM on the local platform, before being sent on to the server.
NOTE RAM maintains its contents between system reboots and when there is a loss of platform power.
Messages are sent from RAM to the server based upon the following criteria:
- It is a critical (highest) level message and therefore is sent to the server immediately.
- When a block of memory will overflow due to the arrival of the current message, the content of the full block is sent on to the server.
When a predefined amount of time has passed since the generation of the first message in the current memory block, the content of the block is sent on to the server.
Platform Reset
When a platform resets, IPriori resends messages to the server as part of the initialization process:
- All messages in RAM are searched.
- The message with the latest time stamp is identified.
- All messages found within 5 minutes of the latest timestamp are sent to the server.
- The server compares the received messages with the last timestamp received from that platform.
- Any messages that occurred after the last timestamp received from the platform are stored as normal in the appropriate logging file. If the server also resets, all received messages are stored in the new logging file.
The Server
The server is responsible for:
- Logging configuration
- Message storage to the disk
- Message display to the console
- Remote host communications using syslogd
Message Storage
File names are determined by platform (server, router module or bay controller.) The number of files total is either 5 (default) or the maximum number of log files set using the logging-max-history command. File names indicate the platform number and file number (from 0 up to one less than the configured maximum number of log files allowed. The default is 0-4.)
File name formats are described in Table 11-3.
NOTE When a system is first booted and the server ID has not been set using the server-id command, the server ID defaults to 0. If you see that the server files are named SRV0000.yyy, you will need to give the server a unique ID using the server-id command.
The save file will roll-over to the next file in the file name extension sequence based upon the following criteria:
- At reset, the server examines the latest time on all log files for a platform. It then chooses the next log number in the sequence. If the maximum file value is reached, then 0 is chosen. The logging-max-history command is used to set the maximum saved files allowed.
- Maximum file size of 500K is reached. Log-file rotation rolls back to 0 when the maximum save file value is reached. The old file is overwritten. See Figure 11-4 for an example of a log file rotation based upon a maximum save file value of 5.
Figure 11-4. Log File Rotation
Display Filtering
Display filtering redirects log messages to the console, telnet session or to a user created file, depending upon the criteria set in filtering configuration. The source of the log messages depends upon the command used:
- It can be the server as the messages arrive from the originating platform.
- It can be the local log files on the server.
There are three types of display filtering:
- Logging to console or the current telnet session as the messages arrive.
- Logging to console or the current telnet session from the log files.
- Logging the contents of log files to a new file.
Logging to Console as Messages Arrive
The logging monitor command is used to display log messages to the console or a current telnet session as they arrive and are processed at the server. These commands provide the quickest method of viewing events as they happen.
Options exist to select the platforms, subsystems, and levels to display to the console or monitor.
NOTE The displayed messages may not be in time order due to the delays associated with queueing and sending messages from the router modules and bay controllers.
Logging to Console from the Log Files
The show logging command displays log messages to the console or current telnet session from all log files. This method offers the user the ability to coordinate among all the various platforms as well as view system-wide events in time order. The drawback to this command is that messages displayed are not in real-time (as messages arrive to the server.) The command is only capable of accessing messages up to the last message stored in the log files on the hard drive.
Options exist to select: platforms, subsystems, levels, and starting and ending time stamps.
Logging Contents of Log Files to New File
The copy logging command redirects log messages from all log files to a user-created file. This command provides for the creation of a new file that contains a sub-set of the log messages from all log files currently on the disk, based upon the options selected. As with the show logging command, messages displayed are not in real-time.
Options exist to select: platforms, subsystems, levels, and starting and ending time stamps.
NOTE Input filtering takes place when messages are generated. At this time, the settings of the logging-filter command determine which messages to place in protective RAM for eventual transmission to the server. Display filtering takes place at the server and determines which messages will display to console or be written to file.
Logging Configuration Tasks
Setup is performed from the CLI prompt. You must be in the appropriate command mode for the task to configure IPriori for logging.
There are four logging steps to consider when configuring IPriori:
- Setup input filtering
- Configure the number of log files for each platform
- Configure remote host display of messages using syslogd
- Configure display filtering
For most users, the default settings for input filtering, display filtering and the number of log files per platform will be satisfactory. Remote host message display requires connection configuration for each host. The following sections address configuration considerations for each of the four areas.
Configuring Input Filtering
Input filtering determines, by severity level, which messages get forwarded to the server per subsystem. The default setting for all subsystems (with the exception of user-command) is to forward all messages to the server with severity level warning and above. User-command messages default to the information severity level, because user command messages are always informational in nature. These defaults represent a reasonable compromise between your need to know the status of the Avici router platforms and the Ethernet bandwidth usage required to log messages to the server.
There are two important considerations when changing the input filtering defaults:
- Lowering the threshold for messages logged to the server means that more of the Ethernet bandwidth between non-control server platforms and the server will be used for transporting logging messages, and there will be less bandwidth for other Avici router requirements. If you raise the input filtering threshold, this becomes less an issue.
- Raising the threshold for messages logged to the server means that all messages below that new threshold will never be available to you.
With these two considerations in mind, caution should be used when changing input filtering defaults.
You must be in the associated platform mode to configure input filtering. Platform mode commands are: server, module, and bay.
For each platform, use the logging-filter command to change the minimum severity level of messages logged to the server.
For example, the following command line sets input filtering to discard all BGP system messages at severity levels minor and below, with the end result that only messages with severity levels major and critical will be logged to the server:
logging-filter bgp major
Configuring the Number of Log Files
Log files have the characteristics of being fixed in size, dedicated to a specific platform, and reside on a hard disk partition. The fixed file size requires that you configure for multiple files for each platform. The number of platforms, combined with the fixed size of the hard disk partition, requires that an upper limit be placed upon the total number of files.
The default setting is for five log files for each platform. When changing this setting, keep the following in mind:
- Each file is 500K in size.
- A dedicated logging hard disk partition size is 2Gbytes.
- Each of the three platforms requires multiple logging files.
What follows is an example log file configuration that shows the disk resource calculation:
The example Avici router has one server, two bay controllers, and thirty-nine modules:
- The route server is stable; server one is set to five log files.
- The active bay controller (1) has recently been unstable; the number of log files has been raised to eight for bay controller 1.
- ten modules 1/1 through 1/10 are connected to routers within your control and are considered to be stable. The default setting of 5 is satisfactory.
- 30 modules are connected to the outside world. With the other end of the link outside of your control, you perceive these modules to be less stable. The number of log files is set to 8.
The disk requirement for this example is as follows:
Figure 11-5. Logging Disk Space Requirement Example
You must be in the associated platform mode to configure logging-max-history. Platform commands are: server, module, and bay.
The following command line sets the number of log files to 8:
logging-max-history 8
In most cases, the default values satisfy log file requirements. However, to detect a sporadic error on one router module, increase the number of log files for that module, so that you have a picture from a longer period of time. Likewise, if you need to decrease the granularity of a platform filter, and accept more messages, increase the number of log files for that platform while you monitor the increased number of messages.
Configuring for Remote Host Message Display
The logging command allows you to configure a remote host for logging message display. You must be in the privileged command line mode to use the logging command. The command line requires the following:
- The remote host IP address
- syslogd facility (IPriori supports local0 through local7, default of local7)
- Host must be properly setup to accept remote messages
Example: For example, the following command line enables a remote session on the host IP address 89.2.2.1 using facility local7 (default) for all platforms, to show all subsystem messages, at all severity levels:
logging 89.2.2.1 platform all
Configuring for Display Filtering
Display filtering allows you to filter messages:
- As messages arrive at the server from the originating platform
- From the log file
Any messages that pass input filtering are written to the log files regardless of the display filtering setting. Display filtering further reduces the messages sent to the console or remote host.
NOTE Display filtering is a second level of filtering of logging messages. Even if you display all messages, you will not see messages that were filtered before they were stored to the disk.
Current Console or Telnet Session Monitor
Use the logging monitor command when setting up display filtering for the current console or telnet session monitor. You must be in privileged mode when using the logging monitor command.
Specify a specific platform(s) or the command applies to all platforms. If you do not specify at least one subsystem, the command is applied to all subsystems. If you do not specify a minimum severity level, all levels are displayed for the subsystem specified.
For example, the following command line will display log messages to the console from all platforms and for all subsystem messages from severity level minor and above:
logging monitor platform all level minor
Current Log File to the Console
Use the show logging command when setting up display filtering for log file messages to the console. You must be in privileged mode when using the show logging command.
You may specify a starting and ending date or default to all messages in the log file. You must specify:
- A specific platform(s) type.
- The command applies to all platforms of that type.
If you do not specify at least one subsystem, the command is applied to all subsystems. If you do not specify a minimum severity level, all levels are displayed for the subsystem.
Example: For example, the following command line searches the current log file and displays all messages between January 4th and January 12th for all platforms, for all subsystem messages at all severity levels:
show logging from January 04 1999 to January 12 1999
| |
|
|
Source
File Name: Logging.fm
HTML File Name: Logging.html
Last Updated: 05/30/02 at 13:54:42
Please email suggestions and comments to: doc@avici.com