Troubleshooting Digital Voice Interfaces to the IP Network
Digital voice ports are found at the intersection of a packet voice network and a digital, circuit-switched telephone network. Digital voice telephony interfaces include T1 or E1 channel-associated signaling (CAS), ISDN primary-rate interface (PRI) or basic rate interface (BRI), and E1 R2 signaling.
To troubleshoot digital voice interfaces, see the following sections:
If you are troubleshooting a connection to a PBX, you might the PBX interoperability notes useful. These notes contain configuration information for Cisco gateways and several types of PBXs. To access these notes, use the following website:
Checking the Hardware
Digital voice interface hardware connects a router or access server to a line from a circuit-switched telephony device in a PBX or the public switched telephone network (PSTN).
Troubleshoot digital voice hardware by checking the following:
Software Compatibility
To ensure that your card is compatible with your software, check the following:
•
For network modules inserted into Cisco modular access routers, refer to the compatibility tables in the "Overview of Cisco Network Modules" chapterin the Cisco Network Modules Hardware Installation Guide.
•
For interface cards inserted into Cisco modular access routers, refer to the compatibility tables in the "Overview of Cisco Interface Cards" chapter in the Cisco Interface Cards Installation Guide.
Cabling
Cabling for the digital ports varies by platform:
•
Cisco 1600 series, Cisco 1700 series, Cisco 2600 series, Cisco 3600 series, Cisco 3700 series, and Cisco ICS 7750 platforms that use the Multiflex Trunk Interface card use an RJ-48C cable. Refer to the Cisco Interface Card Hardware Installation Guide for information about digital Cisco interface cards.
•
Cisco 7200 VXR platforms use RJ-48C cables for the port adapter. See the MIX-Multichannel T1/E1 Port Adapter Installation and Configuration Guidefor more information.
•
Cisco AS5300 universal access servers use RJ-45 cables for the T1 or E1 interface. A VoIP feature is also required for voice traffic. See the Cisco AS5300 Module Installation Guide.
•
Cisco AS5350 and 5400 universal gateways use RJ-45 cables for the four-port card and a 36-pin cable to RJ-45 interface for the eight-port card. For more information about cabling these platforms, see the "Cabling Specifications" chapter of the Cisco AS5350 and AS5400 Universal Gateway Card Installation Guide.
T1/E1 Trunk and Digital Voice Port Pinouts (RJ-48)
Figure 32 shows the RJ-48 connector wiring for the T1/E1 trunk cable and the digital voice port cable; Table 32 lists the pinouts.
Figure 32 RJ-48-to-RJ-48 T1/E1 Cable Wiring
Pin 1
|
Signal
|
|---|---|
1
|
RX (input)
|
2
|
RX (input)
|
3
|
-
|
4
|
TX (output)
|
5
|
TX (output)
|
6
|
-
|
7
|
-
|
8
|
-
|
1 Any pin not referenced on a connector is not connected.
|
T1/E1 Trunk and Digital Voice Port Pinouts (RJ-45)
RJ-45 Pin
|
Description
|
|---|---|
1
|
RX tip
|
2
|
RX ring
|
3
|
RX shield
|
4
|
TX tip
|
5
|
TX ring
|
6
|
TX shield
|
7
|
-
|
8
|
-
|
Shutdown Port
If the port is not operational, check to make sure the port is not shut down. Enter the show voice port command with the voice port number that you are troubleshooting, which tells you:
•
If the voice port is up. If it is not, use the no shutdown command to make it active.
•
What parameter values have been set for the voice port, including default values. (these do not appear in the output from the show running-config command.) If these values do not match those of the telephony connection you are making, reconfigure the voice port.
Checking the Digital Signal Processors
Digital signal processors (DSPs) enable Cisco platforms to efficiently process digital voice traffic. The following symptoms can be attributed to DSP hardware or software issues:
•
No audio heard by either party or one-way audio on the voice path after the call is connected.
•
Call setup failure, such as the inability to detect or transmit proper Channel Associated Signaling (CAS) state transitions.
•
Channels are stuck in the PARK state and cannot be used.
•
Error messages on the console or in the router log complain of DSP timeouts.
To check your DSPs, use the following sections:
Voice DSP Control Message Logger
This section contains the following information:
Caution
Using the logger feature in a production network environment increases CPU and memory usage on the gateway.
Note
We recommend that you work closely with your Cisco representative to use this feature. If you are experiencing problems with certain voice calls, the engineering team at Cisco might ask you to capture the control messages using the voice DSP logger. You can capture these messages by turning on the logger, repeating the problematic calls, and capturing the logs. Only Cisco engineers can determine if you should send the logs in for further review.
Message Logger Overview
The Voice DSP Control Message Logger feature provides improved debugging capabilities through Cisco IOS software and allows loggin of control messages that pass through the voice DSP firmware on the host port interface (HPI). The logged messages can later be examined for diagnosis of voice problems.
There are two main types of HPI messages that flow through the HPI interface: control messages and data messages. Control messages carry control information between Cisco IOS software and the DSP. Data messages carry voice data.
The Voice DSP Contol Message Logger feature captures control messages sent between the platform-independent portions of Cisco IOS software and the DSP. The HPI subsystem that is in Cisco IOS software contains the platform-independent portion of Cisco IOS software. This feature addresses the sequence and contents of the control messages. The logged messages can be checked for parameters that might cause undesirable DSP behavior including the following:
•
Incorrect parameters
•
Out-of-sequence function calls
•
Interactions between parameters of different HPI calls
In many cases, DSP problems have been the result of bad control messages. By logging all of these messages for offline analysis, you can better integrate and debug at-speed issues for analysis.
Message Capture
Message capture occurs when voice control messages are captured and passed between the Cisco IOS software and the DSP to a ring buffer. Some of these messages are sent in fast-path routines that run at a high priority, so the capture of the message must be done as quickly as possible. After the fast-path routine messages have been sent, a normal priority process sends the messages that are waiting in the ring buffer to off-router data storage through the Cisco IOS File System (IFS).
The size of the ring buffer is configurable through the use of the voice hpi capture command. If the ring buffer fills up faster than the normal priority process can move the messages off the router, some of the control messages are dropped.
Counters keep track of the number of messages that are waiting in the ring buffer, the number of messages that are sent, and the number of messages that are dropped. When message capture is enabled and a message arrives for which there is no buffer space, a missed-message count is started. The next time there is room for a message on the ring, the dropped-message count is included with the message data. This alerts the software that processes the messages to the missed messages, and it provides data capture feedback that helps you configure the ring buffer size to your specifications.
If messages are dropped during the capture, the ability to check the messages becomes limited. A complete capture is required for analysis.
Benefits
Improved DSP Reliability
This feature improves the reliability of DSPs by improving debug capabilities. Unexpected sequences of calls or parameters that cause DSP problems are difficult to debug because many calls can be made to the DSP before any ill effects are noticed. Systems that are running under load are more likely to encounter subtle timing-related issues that occur infrequently and are very hard to reproduce and debug. These parameters are marked as bad in HPI calls to the DSP, and they can cause undesirable DSP behavior. There fore, the logger intercepts those parameters that pass between Cisco IOS software and the DSP that can later be checked for errors.
Robust Firmware
This feature makes the T1-based DSP firmware more robust, adding debug capabilities and enabling better field support.
Restrictions
The Voice DSP Contol Message Logger feature is supported only on systems that use the HPI interface.
Configuration Tasks
See the following sections for configuration tasks for the Voice DSP Control Message Logger feature. Each task in the list is identified as either required or optional.
•
Configuring the Voice DSP Control Message Logger (required)
•
Verifying the Voice DSP Control Message Logger (optional)
Configuring the Voice DSP Control Message Logger
You can start the message logger by choosing the amount of memory (greater than 324 bytes) that the buffer-queueing system can allocate to the free message pool. HPI messages are captured until buffer space runs out. Once the buffer-queueing system is running, the transport process attempts to connect to a new or existing capture destination URL. A version message is written to the URL, and if the version message is accepted, any messages placed into the message queue are written to the URL. If a new URL is entered using command-line interface (CLI), an open URL is closed, and the system tries to write to the new URL. If the new URL fails, the transport process exits. The transport process is restarted when another URL is entered or the system is restarted.
To configure the message logger, use the following commands beginning in privileged EXEC mode.
SUMMARY STEPS
1.
enable
2.
show voice hpi capture
3.
debug hpi capture
4.
configure terminal
5.
voice hpi capture buffer size
6.
voice hpi capture destination url
7.
exit
8.
show voice hpi capture
9.
configure terminal
10.
no voice hpi capture buffer 0
11.
exit
12.
show voice hpi capture
DETAILED STEPS
Verifying the Voice DSP Control Message Logger
To verify and print capture status and statistics, use the show voice hpi capture privileged EXEC command. This command displays the capture status and statistics and checks that the message counter is incrementing. If messages are being dropped consistently, try increasing the buffer size.
Note
If you want to stop the logger or change the buffer to another size, first set the buffer size to zero.
Troubleshooting Tips
Use the debug hpi capture command in privileged EXEC mode to turn on the debug output for the logger. Enable the debug output for the logger by using the CLI.
Configuration Examples
This section provides configuration examples for the Voice DSP Control Message Logger feature. This section contains the following examples:
Starting the Logger Feature Example
In the following example, the voice hpi capture buffer command is used in global configuration mode to start the logger by giving it a buffer size of 700000:
Router(config)# voice hpi capture buffer 700000
*Mar 1 00:24:47.090:caplog:caplog_cli_interface:hpi capture buffer size set to 700000 bytes
*Mar 1 00:24:47.090:caplog:caplog_logger_init:TRUE, Started task HPI Logger (PID 140 *Mar 1 00:24:47.150:caplog:caplog_cache_init:TRUE, malloc_named(699952), 2134 elements (each 328 bytes big)
*Mar 1 00:24:47.154:caplog:caplog_logger_proc:Terminating...
In the following example, the show voice hpi capture command is used in privileged EXEC mode to examine the logger status output now that the logger is enabled:
Router# show voice hpi capture
HPI Capture is on and is logging to URL <www.company.com>0 messages sent to URL, 0 messages droppedMessage Buffer (total:inuse:free) 2134:0000:2134Buffer Memory:699952 bytes, Message size:328 bytes
Setting up an FTP Destination Example
In the following example, the voice hpi capture destination command is used in global configuration mode to set up an FTP destination file where the logged data can be sent:
Router(config)# voice hpi capture destination ftp://100.00.100.200/d:\test_data.dat
*Mar 1 00:26:54.617:caplog:caplog_cli_interface:hpi capture destination:ftp://100.00.100.200/d:\test_data.dat *Mar 1 00:26:54.621:caplog:caplog_logger_init:TRUE, Started task HPI Logger (PID 140) *Mar 1 00:26:54.621:caplog:caplog_logger_proc:Attempting to open ftp://100.00.100.200/d:\test_data.dat *Mar 1 00:26:57.091:caplog:caplog_logger_proc:Logging to ftp://100.00.100.200/d:\test_data.dat
In the following example, the show voice hpi capture command is used in privileged EXEC mode to examine the logger status output:
Router# show voice hpi capture
HPI Capture is on and is logging to URL ftp://172.23.184.216/d:\test_data.dat1 messages sent to URL, 0 messages droppedMessage Buffer (total:inuse:free) 2134:0000:2134Buffer Memory:699952 bytes, Message size:328 bytes
Verifying Configuration Example
In the following example, the show voice hpi capture command is used in privileged EXEC mode to check on the status of the logger before, during, and after configuration:
Router# show voice hpi capture
HPI Capture is off and is logging to URL <no URL>
0 messages sent to URL, 0 messages dropped
Message Buffer (total:inuse:free) 0000:0000:0000
Buffer Memory:0 bytes, Message size:328 bytes
Voice Call Tuning
The Voice Call Tuning feature monitors the interface between Cisco IOS software and a system's digital signaling processors (DSPs) in real time and reports status on the following: packet flow, DSP state, echo-cancellation state, and jitter state. The feature also allows you to manipulate echo-cancellation and jitter-buffer parameters in real time. For details on this feature, see the "Voice Call Tuning" section on page 309 in the "Troubleshooting Quality of Service for VoIP" chapter.
Voice DSP Crash Dump File Analysis
The Voice Crash Dump File Analysis feature allows Cisco IOS voice platforms using Texas Instruments DSPs the ability to capture the contents of the DSP memory into a file in the event of a DSP crash. By making this crash dump file available for offline analysis, engineers can better and more quickly determine and fix the cause of the crash.
DSP crash dump analysis allows to you do the following:
•
Detect when control messages have been lost between Cisco IOS software and the DSP
•
Detect when the DSP has crashed
•
Collect an image of the DSP memory after a DSP crash and put it into a file for analysis later by an engineer
When these events have been detected, they are announced by console alarms. You can enable and disable this feature and specify where the crash dump is to be written using Cisco IOS command-line interface (CLI). The active part of the stack is written to the console, while the entire contents of the DSP memory is written to the crash dump file. You can request that a dump file be written into a "smart" slot 0 or slot 1 flash card, or sent to a server using TFTP or FTP, or it may be written directly to Flash.
How to Configure Voice DSP Crash Dump File Analysis
To configure Voice DSP crash dump file analysis, use the following steps:
SUMMARY STEPS
1.
enable
2.
configure {terminal | memory | network}
3.
voice dsp crash-dump destination url
4.
voice dsp crash-dump file-limit limit-number
5.
exit
6.
show voice dsp crash-dump
DETAILED STEPS
Troubleshooting Voice DSP Crash Dump File Analysis
To troubleshoot the Voice DSP Crash Dump File Analysis feature, use the debug voice dsp crash-dump command in privileged EXEC mode. Thiscommand is intended only for troubleshooting purposes because the volume of output generated by the software can result in severe performance degradation on the router.
SUMMARY STEPS
1.
enable
2.
debug voice dsp crash-dump keepalive
3.
undebug all
4.
debug voice dsp crash detail
5.
exit
DETAILED STEPS
Verifying DSP Crash Dump File Analysis
To verify crash dump statistics, use the show voice dsp crash-dump command in privileged EXEC mode.
Configuration Examples for Voice DSP Crash Dump File Analysis
The following example shows that crash dump analysis is enabled:
Router(config)# voice dsp crash-dump destination 172.29.248.12
Router(config)# voice dsp crash-dump file-limit 10
voice dsp crash-dump destination tftp://172.29.248.12/tester/crash-152-t
voice dsp crash-dump file-limit 10
end
1w0d:%SYS-5-CONFIG_I:Configured from console by consoleoice dsp crash
Verifying Voice DSP Crash Dump File Analysis
The following example shows output information that verifies the status of the crash dump:
Router# show voice dsp crash-dump
Voice DSP Crash-dump status:
Destination file url is
tftp://172.29.248.12/tester/crash-152-t
File limit is 10
Last DSP dump file written was
tftp://172.29.248.12/zongshan/crash/26-152-t2
Next DSP dump file written will be
tftp://172.29.248.12/tester/crash-152-t1
Troubleshooting Universal Port SPEs
A universal port card is a hardware card that processes digital signals for the Cisco AS5350 and Cisco AS5400 universal gateways. The service-processing element (SPE) works as a DSP for the universal port card.
This section provides troubleshooting information that apply to modems regardless of service type mode. It describes how to perform diagnostic tests on installed ports or SPEs, configure automatic recovery of ports on an SPE, and configure a scheduled recovery of SPEs.
Configure SPE Diagnostic Tests
You can perform three types of diagnostic tests on the SPE modem:
SPE Startup Test
To perform diagnostic testing on all the installed SPE ports during the system's initial startup or rebooting process, use theport modem startup-testcommand in global configuration mode.
The results of the SPE port startup test are displayed in the show port modem test command output. SPE ports that pass the diagnostic test are marked as Pass, Fail, and Unkn. Ports that fail the diagnostic test are marked as Bad. These ports cannot be used for call connections. Depending on how many ports are installed, this diagnostic test may take from 5 to 10 minutes to complete. Perform additional testing on an inoperative SPE port by executing thetest port modem back-to-back command. The no port modem startup-test command disables startup testing.
SPE Auto-Test
To perform diagnostic testing on all the installed SPE ports during the system's initial startup or rebooting process, or during service, use the port modem autotest command in global configuration mode.
The results of the SPE port auto-test are displayed in the show port modem test command's output. Ports that pass the diagnostic test are marked asIdle, Busy, Downloading, and Reset, and are put into service. Ports that fail the diagnostic test are marked as Bad, and are not put into service or tested again until they are no longer marked as Bad. If all the ports of an SPE are bad, the corresponding SPE is also marked bad. These ports cannot be used for call connections. Depending on how many ports are present and not marked Bad, this diagnostic test may take from 5 to 10 minutes to complete. You may perform additional testing on an inoperative port by executing the test port modem back-to-back command. The no port modem autotestcommand disables testing.
You may optionally configure the following commands:
•
port modem autotest minimum ports—Define the minimum number of free ports available for autotest to begin.
•
port modem autotest time hh:mm {interval}—Enable autotesting time and interval.A sample diagnostic autotest setting the time at 12:45 and at 8 hour intervals looks like the following:
AS5400(config)# port modem autotest time 12:45 8
AS5400(config)#
•
port modem autotest error threshold—Define the maximum number of errors detected for autotest to begin.
SPE Back-to-Back Test
When an SPE port tests as Bad, perform additional testing by conducting a series of internal back-to-back connections and data transfers between two SPE ports. All port test connections occur inside the gateway. For example, if mobile users cannot dial into port 2/5 (the sixth port on the universal port card in the second chassis slot), attempt a back-to-back test with port 2/5 and a known-functioning port such as port 2/6.
Enter the following command in privileged EXEC mode (the prompt is displayed as
AS5350# or AS5400#) to perform internal back-to-back port tests between two ports:
test port modem back-to-back slot/port slot/port {num-packets}—Perform internal back-to-back port tests between two ports, sending test packets of the specified size.
You might need to enable this command on several different combinations of ports to determine which one is not functioning properly. A pair of operable ports successfully connect and complete transmitting data in both directions. An operable port and an inoperable port do not successfully connect with each other.
A sample back-to-back test might look like the following:
AS5400# test port modem back-to-back 2/10 3/20
Repetitions (of 10-byte packets) [1]:
*Mar 02 12:13:51.743:%PM_MODEM_MAINT-5-B2BCONNECT:Modems (2/10) and (3/20) connected in back-to-back test:CONNECT33600/V34/LAP
*Mar 02 12:13:52.783:%PM_MODEM_MAINT-5-B2BMODEMS:Modems (3/20) and (2/10) completed back-to-back test:success/packets = 2/2
A port that has been confirmed to have problems can often be fixed using the clear spe command.
The results of the test port modem back-to-back command are displayed in the show port modem test command's output:
AS5400# show port modem test
Date Time Modem Test Reason State Result
3/02 12:00:57 PM 2/01 Back-To-Back :STARTUP TEST Idle PASS
3/02 12:00:57 PM 2/00 Back-To-Back :STARTUP TEST Idle PASS
3/02 12:00:58 PM 2/02 Back-To-Back :STARTUP TEST Idle PASS
3/02 12:00:58 PM 2/03 Back-To-Back :STARTUP TEST Idle PASS
3/02 12:00:58 PM 2/04 Back-To-Back :STARTUP TEST Idle PASS
3/02 12:00:58 PM 2/05 Back-To-Back :STARTUP TEST Idle PASS
...
3/02 12:01:14 PM 3/95 Back-To-Back :STARTUP TEST Idle PASS
3/02 12:01:14 PM 3/94 Back-To-Back :STARTUP TEST Idle PASS
3/02 12:01:15 PM 3/75 Back-To-Back :STARTUP TEST Idle PASS
3/02 12:01:15 PM 3/74 Back-To-Back :STARTUP TEST Idle PASS
3/02 12:13:52 PM 3/20 Back-To-Back :USER INITIATED Idle PASS
3/02 12:13:52 PM 2/10 Back-To-Back :USER INITIATED Idle PASS
...
3/02 12:44:00 PM 3/102 No Test (Time) :MIN IDLE MODEMS Idle NOTST
3/02 12:44:00 PM 3/103 No Test (Time) :MIN IDLE MODEMS Idle NOTST
3/02 12:44:00 PM 3/104 No Test (Time) :MIN IDLE MODEMS Idle NOTST
3/02 12:44:00 PM 3/105 No Test (Time) :MIN IDLE MODEMS Idle NOTST
3/02 12:44:00 PM 3/106 No Test (Time) :MIN IDLE MODEMS Idle NOTST
3/02 12:44:00 PM 3/107 No Test (Time) :MIN IDLE MODEMS Idle NOTST
3/02 12:44:21 PM 2/73 Back-To-Back :TIME INTERVAL Idle PASS
3/02 12:44:21 PM 2/72 Back-To-Back :TIME INTERVAL Idle PASS
3/02 12:44:21 PM 2/33 Back-To-Back :TIME INTERVAL Idle PASS
3/02 12:44:21 PM 2/32 Back-To-Back :TIME INTERVAL Idle PASS
3/02 12:44:21 PM 3/37 Back-To-Back :TIME INTERVAL Idle PASS
Note
The Reason column indicates why the test was started. The TIME INTERVAL is one of the triggers under autotest; the other is the error threshold.
SPE Disconnect Reason Codes
This section describes how to interpret the call disconnect reason codes reported by Cisco universal port card SPEs. Whenever a call using the universal port SPEs is cleared or disconnected, the SPE records the reason for the disconnect. This disconnect reason code can be used to determine whether the disconnect was normal or an error occurred. This reason code can be used to track down possible sources of failure. Modems can be disconnected due to a variety of factors such as client disconnects, telco errors, and call drops at the network access server (NAS). A "good" disconnect reason is that the DTE (client modem or NAS) at one end or the other wanted to terminate the call. Such "normal" disconnects indicate that the disconnect was not a result of modem or transmission level errors.
Note
The disconnect reason is managed in a first-come-first-serve fashion. This means that the first disconnect reason generated is the only disconnect reason recorded. If the modem and the NAS attempt to terminate the session simultaneously and the modem happens to save the disconnect reason before the LINK_TERMINATE message from the NAS is processed, then the NAS disconnect reason is ignored.
Determining the disconnect Reason
When evaluating whether you are experiencing "good" or "bad" disconnects, it is important to obtain the history of disconnects that a particular port has experienced. In most environments, the disconnect reason is obtained using modem call records or call tracker syslog messages. This disconnect code can then be interpreted using the table provided in this document. Use the following commands to determine the disconnect reason:
•
The show spe modem disconnect-reason command does not display the disconnect reason code as a hexadecimal value. However, it does indicate the disconnect reason as a name. The name and class of the disconnect reason can be found inTable 35 and Table 36 respectively.
•
The show port modem log command displays the disconnect reason code as a hexadecimal value. Refer to Table 34 for the hexadecimal values.
Using the show port modem log command
Use the show port modem log slot/port command to obtain the disconnect cause code (in Hex) for a particular call on a specific port. This disconnect code is identical to the cause code obtained from modem call-record and call-tracker syslog outputs. An example is shown:
*Jan 1 00:53:56.867: Modem State event: State: Terminate
*Jan 1 00:53:56.879: Modem End Connect event:
Call Timer : 195 secs
Disconnect Reason Info : 0x220
Type (=0 ):
Class (=2 ): EC condition - locally detected
Reason (=32 ): received DISC frame -- normal LAPM termination
From the example above, note that the disconnect code is 0x220.
Using the show spe modem disconnect-reason Command
Use the show spe modem disconnect-reason {summary | slot | slot/spe} command to determine the distribution of disconnect reasons that the particular port has experienced. A sample summary output of all the ports is shown below:








































































































No comments:
Post a Comment