Start a Conversation

Unsolved

C

1 Rookie

 • 

1 Message

18

May 8th, 2024 18:09

Switch rebooting when system reboots

I have 2 Dell Switches S5212F-ON

These are configured as a storage network for our Hyper-V hosts to our storage system.

We've been getting alerts that when we reboot a host on the network that other hosts lost a network connection.  We finally had a chance to look into it and we're seeing that a switch completely reboots.  So far they've been 1 at a time and it has not caused any downtime for our VM's but switch reboots are certainly not wanted.

show logging log-file

<165>1 2024-05-08T22:05:57.138028+00:00 switch2 dn_alm 764 - - Node.1-Unit.1:PRI [event], Dell (OS10) %IFM_OSTATE_DN: Interface operational state is down :ethernet1/1/4<165>1 2024-05-08T22:10:17.018475+00:00 switch2 dn_alm 764 - - Node.1-Unit.1:PRI [event], Dell (OS10) %IFM_OSTATE_UP: Interface operational state is up :ethernet1/1/4<165>1 2024-05-08T22:11:07.344117+00:00 switch2 dn_alm 764 - - Node.1-Unit.1:PRI [event], Dell (OS10) %IFM_OSTATE_DN: Interface operational state is down :ethernet1/1/4<165>1 2024-05-08T22:11:37.786650+00:00 switch2 dn_alm 764 - - Node.1-Unit.1:PRI [event], Dell (OS10) %IFM_OSTATE_UP: Interface operational state is up :ethernet1/1/4<165>1 2024-05-08T23:04:41.070166+00:00 switch2 dn_alm 764 - - Node.1-Unit.1:PRI [event], Dell (OS10) %ALM_AUTH_EVENT: Authentication event was raised MESSAGE=pam_unix(sshd:session): session closed for user admin<165>1 2024-05-08T23:04:41.070855+00:00 switch2 dn_alm 764 - - Node.1-Unit.1:PRI [event], Dell (OS10) %ALM_AUTH_EVENT: Authentication event was raised MESSAGE=pam_unix(sshd:session): session closed for user admin<165>1 2024-05-08T23:48:09.820378+00:00 switch2 dn_alm 764 - - Node.1-Unit.1:PRI [event], Dell (OS10) %ALM_AUTH_EVENT: Authentication event was raised MESSAGE=pam_unix(sshd:session): session opened for user admin by (uid=0)<165>1 2024-05-08T23:48:11.570354+00:00 switch2 dn_alm 764 - - Node.1-Unit.1:PRI [event], Dell (OS10) %ALM_AUTH_EVENT: Authentication event was raised MESSAGE=pam_unix(sshd:session): session opened for user admin by (uid=0)<165>1 2024-05-08T23:56:42.069868+00:00 switch2 dn_alm 764 - - Node.1-Unit.1:PRI [event], Dell (OS10) %ALM_AUTH_EVENT: Authentication event was raised MESSAGE=pam_unix(sshd:session): session closed for user admin<165>1 2024-05-08T23:56:42.070907+00:00 switch2 dn_alm 764 - - Node.1-Unit.1:PRI [event], Dell (OS10) %ALM_AUTH_EVENT: Authentication event was raised MESSAGE=pam_unix(sshd:session): session closed for user admin<165>1 2024-05-08T23:59:41.320326+00:00 switch2 dn_alm 764 - - Node.1-Unit.1:PRI [event], Dell (OS10) %ALM_AUTH_EVENT: Authentication event was raised MESSAGE=pam_unix(sshd:session): session opened for user admin by (uid=0)<165>1 2024-05-08T23:59:43.211911+00:00 switch2 dn_alm 764 - - Node.1-Unit.1:PRI [event], Dell (OS10) %ALM_AUTH_EVENT: Authentication event was raised MESSAGE=pam_unix(sshd:session): session opened for user admin by (uid=0)<165>1 2024-05-09T00:07:27.765741+00:00 switch2 dn_alm 764 - - Node.1-Unit.1:PRI [event], Dell (OS10) %IFM_OSTATE_DN: Interface operational state is down :ethernet1/1/3<165>1 2024-05-09T00:13:41.647687+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %ALM_SYSTEM_RESTART: OS10 Event and Alarm management system restarted<161>1 2024-05-09T00:13:52.628155+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [alarm], Dell (OS10) %PM_SYS_UTIL_HI:  CPU Utilization 1-minute High Threshold alarm Raised: 100.00 (85-75) %%ACTION=RAISED %%SEQUENCE=1414<164>1 2024-05-09T00:13:52.631261+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [alarm], Dell (OS10) %PM_SYS_UTIL_LO:  CPU Utilization 1-minute Low Threshold alarm Raised: 100.00 (85-75) %%ACTION=RAISED %%SEQUENCE=1415<165>1 2024-05-09T00:14:06.275122+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %EQM_UNIT_DETECTED: Unit present unit 1<165>1 2024-05-09T00:14:06.277565+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %EQM_PSU_DETECTED: Power supply unit present PSU 1<165>1 2024-05-09T00:14:06.278105+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %EQM_PSU_DETECTED: Power supply unit present PSU 2<165>1 2024-05-09T00:14:06.279075+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %EQM_FAN_TRAY_DETECTED: Fan tray present fan tray 1<165>1 2024-05-09T00:14:06.279653+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %EQM_FAN_TRAY_DETECTED: Fan tray present fan tray 2<165>1 2024-05-09T00:14:06.280189+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %EQM_FAN_TRAY_DETECTED: Fan tray present fan tray 3<165>1 2024-05-09T00:14:06.282445+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %EQM_FAN_TRAY_DETECTED: Fan tray present fan tray 4<165>1 2024-05-09T00:14:08.141482+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %EQM_UNIT_CHECKIN: Unit check-in detected unit 1 (type S5212F-ON 12x25GbE SFP28, 3x100GbE QSFP28 Interface Module)<165>1 2024-05-09T00:14:08.144107+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %EQM_UNIT_UP: Unit is up unit 1<165>1 2024-05-09T00:14:09.307843+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %CMS_INIT_STATE: init state enum state 5<165>1 2024-05-09T00:14:10.579101+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %IFM_ASTATE_UP: Interface admin state up :ethernet1/1/1<165>1 2024-05-09T00:14:10.765532+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %IFM_ASTATE_UP: Interface admin state up :ethernet1/1/2<165>1 2024-05-09T00:14:10.943589+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %IFM_ASTATE_UP: Interface admin state up :ethernet1/1/3<165>1 2024-05-09T00:14:11.139676+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %IFM_ASTATE_UP: Interface admin state up :ethernet1/1/4<165>1 2024-05-09T00:14:11.303654+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %IFM_ASTATE_UP: Interface admin state up :ethernet1/1/5<165>1 2024-05-09T00:14:11.448086+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %IFM_ASTATE_UP: Interface admin state up :ethernet1/1/6<165>1 2024-05-09T00:14:11.573316+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %IFM_ASTATE_UP: Interface admin state up :ethernet1/1/7<165>1 2024-05-09T00:14:11.722684+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %IFM_ASTATE_UP: Interface admin state up :ethernet1/1/8<165>1 2024-05-09T00:14:11.882950+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %IFM_ASTATE_UP: Interface admin state up :ethernet1/1/9<165>1 2024-05-09T00:14:12.018548+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %IFM_ASTATE_UP: Interface admin state up :ethernet1/1/10<165>1 2024-05-09T00:14:12.173943+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %IFM_ASTATE_UP: Interface admin state up :ethernet1/1/11<165>1 2024-05-09T00:14:12.367740+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %IFM_ASTATE_UP: Interface admin state up :ethernet1/1/12<165>1 2024-05-09T00:14:12.526428+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %IFM_ASTATE_UP: Interface admin state up :ethernet1/1/13<165>1 2024-05-09T00:14:12.674774+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %IFM_ASTATE_UP: Interface admin state up :ethernet1/1/14<165>1 2024-05-09T00:14:12.861026+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %IFM_ASTATE_UP: Interface admin state up :ethernet1/1/15<165>1 2024-05-09T00:14:12.926237+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %IFM_ASTATE_UP: Interface admin state up :mgmt1/1/1<165>1 2024-05-09T00:14:12.952217+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %IFM_ASTATE_UP: Interface admin state up :null0<165>1 2024-05-09T00:14:12.952422+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %IFM_OSTATE_UP: Interface operational state is up :null0<165>1 2024-05-09T00:14:14.148760+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %IP_ADDRESS_ADD: IP Address add is successful. IP 192.168.X.X/20 in VRF:default added successfully<165>1 2024-05-09T00:14:14.210646+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %IFM_ASTATE_UP: Interface admin state up :port-channel50<165>1 2024-05-09T00:14:14.210965+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %IFM_OSTATE_DN: Interface operational state is down :port-channel50<165>1 2024-05-09T00:14:14.393392+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %IFM_ASTATE_UP: Interface admin state up :vlan1<165>1 2024-05-09T00:14:16.578413+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %IFM_OSTATE_UP: Interface operational state is up :mgmt1/1/1<165>1 2024-05-09T00:14:20.564306+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [alarm], Dell (OS10) %PM_SYS_UTIL_HI:  CPU Utilization 1-minute High Threshold alarm Cleared: 84.62 (85-75) %%ACTION=CLEARED %%SEQUENCE=1414<165>1 2024-05-09T00:14:32.689360+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %SYSTEM_MODE_CHNG: System mode changed : default to full-switch<165>1 2024-05-09T00:14:34.678469+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [alarm], Dell (OS10) %PM_SYS_UTIL_LO:  CPU Utilization 1-minute Low Threshold alarm Cleared: 74.86 (85-75) %%ACTION=CLEARED %%SEQUENCE=1415<165>1 2024-05-09T00:14:40.015457+00:00 switch2 dn-app-uds 945 - - Node.1-Unit.1:PRI [event], Dell (OS10) %UDS: UDS feature disabled<165>1 2024-05-09T00:14:40.056899+00:00 switch2 dn-app-uds 945 - - Node.1-Unit.1:PRI [event], Dell (OS10) %UDS: Docker service stopped  successfully<165>1 2024-05-09T00:14:40.100529+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %CMS_INIT_STATE: init state enum state 6<165>1 2024-05-09T00:14:40.107850+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %ISCSI_GLB_DISABLE: ISCSI_OPT: ISCSI Globally disabled. .<165>1 2024-05-09T00:14:40.553623+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %IFM_OSTATE_UP: Interface operational state is up :ethernet1/1/1<165>1 2024-05-09T00:14:40.553936+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %IFM_OSTATE_UP: Interface operational state is up :vlan1<165>1 2024-05-09T00:14:40.560733+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %IFM_OSTATE_UP: Interface operational state is up :ethernet1/1/2<165>1 2024-05-09T00:14:40.652753+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %IFM_OSTATE_UP: Interface operational state is up :ethernet1/1/3<165>1 2024-05-09T00:14:40.789202+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %IFM_OSTATE_UP: Interface operational state is up :ethernet1/1/4<165>1 2024-05-09T00:14:40.808419+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %IFM_OSTATE_UP: Interface operational state is up :ethernet1/1/11<165>1 2024-05-09T00:14:40.836272+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %IFM_OSTATE_UP: Interface operational state is up :ethernet1/1/15<165>1 2024-05-09T00:14:40.946464+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %IFM_OSTATE_UP: Interface operational state is up :ethernet1/1/12<165>1 2024-05-09T00:14:42.854121+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %IFM_OSTATE_UP: Interface operational state is up :port-channel50<165>1 2024-05-09T00:14:42.857159+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %LACP_PORT_GROUPED: Interface joined port-channel  port-channel50 : ethernet1/1/15<165>1 2024-05-09T00:14:46.514507+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %DCBX_PFC_PORT_OPER_DOWN: PFC Parameters MISMATCH on interface ethernet1/1/11<165>1 2024-05-09T00:14:46.515013+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %DCBX_ETS_PORT_OPER_DOWN: ETS Parameters MISMATCH on interface ethernet1/1/11<165>1 2024-05-09T00:15:03.497271+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %DCBX_PFC_PORT_OPER_DOWN: PFC Parameters MISMATCH on interface ethernet1/1/12<165>1 2024-05-09T00:15:03.497688+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %DCBX_ETS_PORT_OPER_DOWN: ETS Parameters MISMATCH on interface ethernet1/1/12<165>1 2024-05-09T00:17:09.093088+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %ALM_AUTH_EVENT: Authentication event was raised MESSAGE=pam_unix(sshd:session): session opened for user admin by (uid=0)<165>1 2024-05-09T00:17:11.044580+00:00 switch2 dn_alm 765 - - Node.1-Unit.1:PRI [event], Dell (OS10) %ALM_AUTH_EVENT: Authentication event was raised MESSAGE=pam_unix(sshd:session): session opened for user admin by (uid=0)

In the output of the logs 
Host01 = ethernet1/1/1
Host02 = ethernet1/1/2

Host03 = ethernet1/1/3

Host04 = ethernet1/1/4

We rebooted Host04 and it went down/up without issue.  However when we rebooted Host03 it forced a reboot and when it returned had the message "%ALM_SYSTEM_RESTART: OS10 Event and Alarm management system restarted" in the logs.  Any ideas on what I can research to try to resolve this?

Moderator

 • 

2.2K Posts

May 9th, 2024 07:29

Hello,

I reviewed the logs. While researching the warning, I found below information. Ensure you are running the latest firmware version for the switch. You can check the switch configuration and resource utilization to identify any bottlenecks. There are a configuration issue, a bug in the switch firmware, or a high volume of traffic. 

ALM messages

This section lists the event and alarm management system log messages.

Table 1. ALM system log messagesALM system log messages
System log message Severity Description Recommended action
ALM_ACCNT_EVENT: Accounting event was raised INFORMATIONAL Indicates a user login. When AAA accounting is configured, these logs are displayed for each user login. No action is required.
ALM_AUTH_EVENT: Authentication event was raised INFORMATIONAL Indicates that user authentication was successful. No action is required.
ALM_CLOCK_UPDATE: Clock changed INFORMATIONAL Indicates that system clock is changed. No action is required.
ALM_HISTORY_PURGED: Older version of the alarm history detected, will be purged INFORMATIONAL Indicates that OS10 does not persist alarm history beyond Release 10.4.3.0. No action is required.
ALM_SEQUENCE_ROLLOVER: OS10 Event sequence number rollover occured INFORMATIONAL Indicates that the event sequence is rolled over. No action is required.
ALM_SYSTEM_RESTART:OS10 Event and Alarm management system restarted INFORMATIONAL Indicates that OS10 event and alarm management system is restarted. No action is required.

 

Hope that helps!

No Events found!

Top