【资料名称】：A9130 BSC紧急情况的处理方法

【资料作者】：ASB

【资料日期】：2012

【资料语言】：中文

【资料格式】：DOC

【资料目录和简介】：

A9130 BSC紧急情况的处理方法

第一部分：状态检查
1，SBL 状态检查
通过OMC，或者BSC Terminal检查
1．1观察各个硬件模块的状态灯，初步判断故障模块。
1．2通过OMC终端，可以直接检查当前告警和历史告警，通过图形界面直接查看各个SBL的状态。
1．3通过BSC Terminal可以使用commands－》Equipment handling－》SBL status List查看各个SBL状态。
1．4 Mx BSC主要有以下的告警种类，具体的操作可以参考alarm dictionary。
告警的种类列表：
29 Class 97: ATCA-PROCESSOR Alarms
29.1 [97,0] RESET
29.2 [97,1] TAKEOVER
29.3 [97,2] RLC-OV
29.4 [97,3] ACCESS-FAIL
29.5 [97,4] APPLICATION-FATAL-TERMINATION
29.6 [97,5] SYSTEM-FATAL-TERMINATION
29.7 [97,6] UNPLUGGED
29.8 [97,8] FW-UPGRADE
29.9 [97,10] FATAL-HW-FAILURE
29.10 [97,11] TEMPERATURE-CRITICAL
29.11 [97,12] TEMPERATURE-MAJOR
29.12 [97,13] TEMPERATURE-MINOR
29.13 [97,14] VOLTAGE-FATAL
29.14 [97,15] VOLTAGE-MAJOR
29.15 [97,16] VOLTAGE-MINOR
29.16 [97,17] ETHERNET-LINK-FAILURE
29.17 [97,20] CONFIGURATION-FAILURE
29.18 [97,21] TAKEOVER-REFUSED
29.19 [97,30] N7-DEVICE
29.20 [97,31] HDLC-HANDLER
29.21 [97,32] Q1-HANDLER
29.22 [97,33] RW-HANDLER
29.23 [97,34] TDM-FAILURE
29.24 [97,35] E1-FRAMER
29.25 [97,36] TP-MAIN-SW-ERROR
29.26 [97,37] TP-MAIN-FILE-ERROR
29.27 [97,38] TP-MAIN-ACCESS-PB
29.28 [97,39] SS7-SW-ERROR
29.29 [97,50] NE1OE-FAILURE
29.30 [97,51] E1-SL-FAILURE
30 Class 98: ATCA-SWITCH Alarms
30.1 [98,0] RESET
30.2 [98,3] ACCESS-FAIL
30.3 [98,6] UNPLUGGED
30.4 [98,8] FW-UPGRADE
30.5 [98,10] FATAL-HW-FAILURE
30.6 [98,11] TEMPERATURE-CRITICAL
30.7 [98,12] TEMPERATURE-MAJOR
30.8 [98,13] TEMPERATURE-MINOR
30.9 [98,14] VOLTAGE-FATAL
30.10 [98,15] VOLTAGE-MAJOR
30.11 [98,16] VOLTAGE-MINOR
30.12 [98,18] IP-ROUTER-LINK-FAIL
31 Class 99: ATCA-SHELF Alarms
31.1 [99,0] RESET
31.2 [99,3] ACCESS-FAIL
31.3 [99,6] UNPLUGGED
31.4 [99,8] FW-UPGRADE
31.5 [99,10] FATAL-HW-FAILURE
31.6 [99,11] TEMPERATURE-CRITICAL
31.7 [99,12] TEMPERATURE-MAJOR
31.8 [99,13] TEMPERATURE-MINOR
31.9 [99,14] VOLTAGE-FATAL
31.10 [99,15] VOLTAGE-MAJOR
31.11 [99,16] VOLTAGE-MINOR
31.12 [99,17] ETHERNET-LINK-FAILURE
31.13 [99,18] SHELF-MANAGER-ACCESS-PB
31.14 [99,19] FAN-FAILURE
32 Class 100: LIU-SHELF Alarms
32.1 [100,0] RESET
32.2 [100,1] MUX-TAKEOVER
32.3 [100,4] ACCESS-FAIL
32.4 [100,6] UNPLUGGED
32.5 [100,8] FW-UPGRADE
32.6 [100,17] PEM-FAILURE
32.7 [100,18] 48V-REDUNDANCY-LOSS
32.8 [100,50] NE1OE-FAILURE
32.9 [100,51] E1-SL-FAILURE


2，进程状态的检查
通过script检查或者pstree的命令检查，下面是通过script检查的方法。
2．1通过BSC terminal查看当前HW mapping
Commands－》BSS Operations－》Display HW Log Mapping Type查看当前各个active的模块。
2．2．1连接active的OMCP，以root 、alcatel的用户名和密码登陆。
2．2．2 进入 usr/local/bin目录，运行./mx_chkSanity.
2．2．3检查var/log/MX/trace/files/AuditMxBSC.trace或者var/log/MX/logs/ AuditMxBSC.log，查看当前各个进程的状态。



第二部分：A9130 Trace和现场数据的收集
1，A9130 trace的收集。
紧急情理时请根据TAC和ISE总部的要求按照以下方法收集trace。
1．1 Introduction
In MXBSC, some trace are enable automatic, some other trace need manually to open them. The table show all the trace available for MXBSC
BoardTrace typeEnableComments
OMCP/CCP/TPcore dump fileAutomatic-VOS core, nonVOS core and selfRelient core files=/var/log/MX/trace/dump/files
--
OMCP/CCP/TPLinux system log,
Automatic-dmesg command to see the startup result
-all the log file located in /var/log/
OMCP/CCP/TPSelfRelient logAutomatic-selfRelient log file are stored in /var/log/MX/trace/srk/
OMCP/CCPNTP logAutomatic-/var/log/MX/logs/NTP
OMCP/CCPVCE realtime trace(SCPR,OCPR, TCU,DTC,TSC)Request from trace terminal-trace file will be finally stored in OMCP disk /var/log/MX/trace/realtime/files
OMCP/CCPVCE daily traceRequest from trace terminal-trace file will be finally be stored in OMCP disk: /var/log/MX/trace/daily/files
OMCPMxPF current logAutomatic-trace file will be finally be stored in OMCP disk: /var/log/MX/logs/
OMCP/CCPVCE Interactive traceRequest from trace terminal-trace file will be finally be send to trace facility terminal, use du
TP/OMCP/CCPNon-VCE trace(CMW,EIM,CPI, SWMGT,TPMAIN,SLH …)Manually active from each board-trace file will be finally be stored in OMCP disk: /var/log/MX/trace/files
1．2 Trace class and sub-class
For each trace active, we can define the class and subclass, these two parameters are used to filter the trace, display only the trace we interested.
For VCE realtime trace, the detail meaning of the class and subclass can be found in trace facility terminal menu. Trace-> Realtime trace-> Get/set configuration data. The information can also been found in Trace facility terminal user guides.
1．3 Default trace
When MXBSC is coming up, the default trace already been opened background. For each VCE, when there’s non-recoverable error happens, maximum 512K trace will be save to disk. The default VCE trace include the following class: ERROR,MSG,API,FST with priority 3 and dump lever 1.Default trace is only activated for the VCE on the OMCP board, not for the VCE on the CCP board.
1．4 Active VCE realtime trace
MXBSC uses a trace config file to control all the trace in all the VCEs. The following command can show the trace file name:
root@OMCP1-n3s30#ls /common/bsc/SCPRDISK/|grep BSC
BSCTRACE.01F
The following is the way to active VCE realtime trace:
Active trace from trace facility terminal, after connected to BSC, trace facility terminal support the following operation
a.Get/Set/disable the trace for one VCE.
b.Get/Set/disable trace for more than one VCE.
c.Forcelly dump trace to OMCP disk.
1．5 Active NonVOS trace
To open a non-VCE trace, we need to modify the config file in all the boards.
vi /usr/local/trace/conf_custom/trace_dyn_mxpf.conf
vi /usr/local/trace/conf_custom/trace_dyn_appli.conf
/usr/local/trace/bin/snd2ctm trigger 127.0.0.1 send the trigger to own board to active trace
/usr/local/trace/bin/snd2ctm get_shm 127.0.0.1 check the traces enabled successfully or not

1．6TP和CCP的realtime内容。
由于TP和CCP没有DISK，在reset后相关的内存里的trace将丢失。所以在reset前对于TP板需要登录到TP板上（110和130），在usr/local/tpgsm/tools下，运行./tp_debug 5，在把/var/log/MX/dump/common下的文件备份下来。对于CCP板也需要将/var/log/MXtrace/common下的内容备份，再对板子reset。

2，跟踪相关的A口信令，并保存记录。

3，保存bsc terminal log


4，备份问题解决前后的DLS

5，收集二进制历史告警
注意事项：对MX各个模块的操作命令，尽量BSC侧进行，不要通过OMCR执行。

第三部分：问题的简单处理
1，在保存好相关的trace后，根据前面的检查情况。可以对相关的TP，CCP和OMCP进行reset的尝试操作。
2，如果reset后仍然没有效果，可以尝试reset（或者开关电）A9130 BSC。