stonithd[5815]: 2007/09/20_22:32:53 info: 10.0.0.166 uses sysOID '.1.3.6.1.4.1.318.1.3.4.5', compatible with 'masterSwitchrPDU' stonithd[5815]: 2007/09/20_22:32:53 info: 10.0.0.166 has sysName 'RackPDU Archivage' heartbeat[2826]: 2007/09/20_22:32:59 info: killing /usr/lib/heartbeat/mgmtd -v process group 3779 with signal 15 mgmtd[3779]: 2007/09/20_22:32:59 info: mgmtd is shutting down mgmtd[3779]: 2007/09/20_22:32:59 debug: [mgmtd] stopped heartbeat[2826]: 2007/09/20_22:32:59 info: killing /usr/lib/heartbeat/crmd process group 3778 with signal 15 crmd[3778]: 2007/09/20_22:32:59 info: crm_shutdown: Requesting shutdown crmd[3778]: 2007/09/20_22:32:59 info: do_state_transition: State transition S_IDLE -> S_POLICY_ENGINE [ input=I_SHUTDOWN cause=C_SHUTDOWN origin=crm_shutdown ] crmd[3778]: 2007/09/20_22:32:59 info: do_state_transition: All 1 cluster nodes are eligible to run resources. crmd[3778]: 2007/09/20_22:32:59 info: do_shutdown_req: Sending shutdown request to DC: host-1.awale.qc.ca crmd[3778]: 2007/09/20_22:32:59 info: do_shutdown_req: Processing shutdown locally crmd[3778]: 2007/09/20_22:32:59 info: handle_shutdown_request: Creating shutdown request for host-1.awale.qc.ca tengine[3795]: 2007/09/20_22:32:59 info: extract_event: Aborting on shutdown attribute for 1da8804b-8a93-4145-9d40-1133699fe377 tengine[3795]: 2007/09/20_22:32:59 info: update_abort_priority: Abort priority upgraded to 1000000 cibmon[3772]: 2007/09/20_22:32:59 info: cibmon_diff: [cib_diff_notify] cib_update confirmed cibmon[3772]: 2007/09/20_22:32:59 info: log_cib_diff: cib_update: Diff: --- 0.12.1 cibmon[3772]: 2007/09/20_22:32:59 info: log_cib_diff: cib_update: Diff: +++ 0.12.2 cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_update: - cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_update: - cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_update: - cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_update: - cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_update: - cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_update: + heartbeat[2826]: 2007/09/20_22:32:59 WARN: G_SIG_dispatch: Dispatch function for SIGCHLD took too long to execute: 50 ms (> 30 ms) (GSource: 0x99b2fa0) pengine[3796]: 2007/09/20_22:32:59 notice: cluster_option: Using default value 'stop' for cluster option 'no-quorum-policy' pengine[3796]: 2007/09/20_22:32:59 notice: cluster_option: Using default value 'true' for cluster option 'symmetric-cluster' pengine[3796]: 2007/09/20_22:32:59 notice: cluster_option: Using default value 'false' for cluster option 'stonith-enabled' pengine[3796]: 2007/09/20_22:32:59 notice: cluster_option: Using default value 'reboot' for cluster option 'stonith-action' pengine[3796]: 2007/09/20_22:32:59 notice: cluster_option: Using default value '0' for cluster option 'default-resource-stickiness' pengine[3796]: 2007/09/20_22:32:59 notice: cluster_option: Using default value '0' for cluster option 'default-resource-failure-stickiness' pengine[3796]: 2007/09/20_22:32:59 notice: cluster_option: Using default value 'true' for cluster option 'is-managed-default' pengine[3796]: 2007/09/20_22:32:59 notice: cluster_option: Using default value '60s' for cluster option 'cluster-delay' pengine[3796]: 2007/09/20_22:32:59 notice: cluster_option: Using default value '20s' for cluster option 'default-action-timeout' pengine[3796]: 2007/09/20_22:32:59 notice: cluster_option: Using default value 'true' for cluster option 'stop-orphan-resources' pengine[3796]: 2007/09/20_22:32:59 notice: cluster_option: Using default value 'true' for cluster option 'stop-orphan-actions' pengine[3796]: 2007/09/20_22:32:59 notice: cluster_option: Using default value 'false' for cluster option 'remove-after-stop' pengine[3796]: 2007/09/20_22:32:59 notice: cluster_option: Using default value '-1' for cluster option 'pe-error-series-max' pengine[3796]: 2007/09/20_22:32:59 notice: cluster_option: Using default value '-1' for cluster option 'pe-warn-series-max' pengine[3796]: 2007/09/20_22:32:59 notice: cluster_option: Using default value '-1' for cluster option 'pe-input-series-max' pengine[3796]: 2007/09/20_22:32:59 notice: cluster_option: Using default value 'true' for cluster option 'startup-fencing' pengine[3796]: 2007/09/20_22:32:59 info: determine_online_status: Node host-1.awale.qc.ca is shutting down pengine[3796]: 2007/09/20_22:32:59 info: clone_print: Master/Slave Set: ms-drbd_0 pengine[3796]: 2007/09/20_22:32:59 info: native_print: share-drbd_0:0 (heartbeat::ocf:drbd): Master host-1.awale.qc.ca pengine[3796]: 2007/09/20_22:32:59 info: native_print: share-drbd_0:1 (heartbeat::ocf:drbd): Stopped pengine[3796]: 2007/09/20_22:32:59 info: group_print: Resource Group: DW pengine[3796]: 2007/09/20_22:32:59 info: native_print: msg_first (heartbeat:msg): Started host-1.awale.qc.ca pengine[3796]: 2007/09/20_22:32:59 info: native_print: share-FS (heartbeat::ocf:Filesystem): Started host-1.awale.qc.ca pengine[3796]: 2007/09/20_22:32:59 info: native_print: share-drbdlinks (heartbeat:drbdlinks): Started host-1.awale.qc.ca pengine[3796]: 2007/09/20_22:32:59 info: native_print: IPaddr_10_0_0_59 (heartbeat::ocf:IPaddr): Started host-1.awale.qc.ca pengine[3796]: 2007/09/20_22:32:59 info: native_print: mysql (heartbeat::ocf:mysql): Started host-1.awale.qc.ca pengine[3796]: 2007/09/20_22:32:59 info: native_print: dw-httpd (awale::ocf:daemontools): Started host-1.awale.qc.ca pengine[3796]: 2007/09/20_22:32:59 info: native_print: msg_last (heartbeat:msg): Started host-1.awale.qc.ca pengine[3796]: 2007/09/20_22:32:59 info: clone_print: Clone Set: vmstat-mon pengine[3796]: 2007/09/20_22:32:59 info: native_print: vmstat-mon-child:0 (awale::ocf:daemontools): Started host-1.awale.qc.ca pengine[3796]: 2007/09/20_22:32:59 info: native_print: vmstat-mon-child:1 (awale::ocf:daemontools): Stopped pengine[3796]: 2007/09/20_22:32:59 info: clone_print: Clone Set: stonith pengine[3796]: 2007/09/20_22:32:59 info: native_print: stonith-child:0 (stonith:apcsnmp): Started host-1.awale.qc.ca pengine[3796]: 2007/09/20_22:32:59 info: native_print: stonith-child:1 (stonith:apcsnmp): Stopped pengine[3796]: 2007/09/20_22:32:59 WARN: native_color: Resource share-drbd_0:0 cannot run anywhere pengine[3796]: 2007/09/20_22:32:59 WARN: native_color: Resource share-drbd_0:1 cannot run anywhere pengine[3796]: 2007/09/20_22:32:59 info: master_color: Promoted 0 instances of a possible 1 to master pengine[3796]: 2007/09/20_22:32:59 notice: DemoteRsc: host-1.awale.qc.ca Demote share-drbd_0:0 pengine[3796]: 2007/09/20_22:32:59 notice: StopRsc: host-1.awale.qc.ca Stop share-drbd_0:0 pengine[3796]: 2007/09/20_22:32:59 notice: DemoteRsc: host-1.awale.qc.ca Demote share-drbd_0:0 pengine[3796]: 2007/09/20_22:32:59 notice: StopRsc: host-1.awale.qc.ca Stop share-drbd_0:0 pengine[3796]: 2007/09/20_22:32:59 info: master_color: Promoted 0 instances of a possible 1 to master pengine[3796]: 2007/09/20_22:32:59 info: master_color: Promoted 0 instances of a possible 1 to master pengine[3796]: 2007/09/20_22:32:59 WARN: native_color: Resource msg_first cannot run anywhere pengine[3796]: 2007/09/20_22:32:59 info: master_color: Promoted 0 instances of a possible 1 to master pengine[3796]: 2007/09/20_22:32:59 info: master_color: Promoted 0 instances of a possible 1 to master pengine[3796]: 2007/09/20_22:32:59 WARN: native_color: Resource share-FS cannot run anywhere pengine[3796]: 2007/09/20_22:32:59 WARN: native_color: Resource share-drbdlinks cannot run anywhere crmd[3778]: 2007/09/20_22:32:59 info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=route_message ] pengine[3796]: 2007/09/20_22:32:59 WARN: native_color: Resource IPaddr_10_0_0_59 cannot run anywhere pengine[3796]: 2007/09/20_22:32:59 WARN: native_color: Resource mysql cannot run anywhere pengine[3796]: 2007/09/20_22:32:59 WARN: native_color: Resource dw-httpd cannot run anywhere pengine[3796]: 2007/09/20_22:32:59 WARN: native_color: Resource msg_last cannot run anywhere pengine[3796]: 2007/09/20_22:32:59 notice: StopRsc: host-1.awale.qc.ca Stop msg_first pengine[3796]: 2007/09/20_22:32:59 notice: StopRsc: host-1.awale.qc.ca Stop share-FS pengine[3796]: 2007/09/20_22:32:59 notice: StopRsc: host-1.awale.qc.ca Stop share-drbdlinks pengine[3796]: 2007/09/20_22:32:59 notice: StopRsc: host-1.awale.qc.ca Stop IPaddr_10_0_0_59 pengine[3796]: 2007/09/20_22:32:59 notice: StopRsc: host-1.awale.qc.ca Stop mysql pengine[3796]: 2007/09/20_22:32:59 notice: StopRsc: host-1.awale.qc.ca Stop dw-httpd pengine[3796]: 2007/09/20_22:32:59 notice: StopRsc: host-1.awale.qc.ca Stop msg_last pengine[3796]: 2007/09/20_22:32:59 WARN: native_color: Resource vmstat-mon-child:0 cannot run anywhere pengine[3796]: 2007/09/20_22:32:59 WARN: native_color: Resource vmstat-mon-child:1 cannot run anywhere pengine[3796]: 2007/09/20_22:32:59 notice: StopRsc: host-1.awale.qc.ca Stop vmstat-mon-child:0 pengine[3796]: 2007/09/20_22:32:59 WARN: native_color: Resource stonith-child:0 cannot run anywhere pengine[3796]: 2007/09/20_22:32:59 WARN: native_color: Resource stonith-child:1 cannot run anywhere pengine[3796]: 2007/09/20_22:32:59 notice: StopRsc: host-1.awale.qc.ca Stop stonith-child:0 pengine[3796]: 2007/09/20_22:32:59 info: stage6: Scheduling Node host-1.awale.qc.ca for shutdown pengine[3796]: 2007/09/20_22:32:59 WARN: process_pe_message: Transition 6: WARNINGs found during PE processing. PEngine Input stored in: /var/lib/heartbeat/pengine/pe-warn-484.bz2 pengine[3796]: 2007/09/20_22:32:59 info: process_pe_message: Configuration WARNINGs found during PE processing. Please run "crm_verify -L" to identify issues. tengine[3795]: 2007/09/20_22:32:59 info: unpack_graph: Unpacked transition 6: 33 actions in 33 synapses tengine[3795]: 2007/09/20_22:32:59 info: te_pseudo_action: Pseudo action 34 fired and confirmed tengine[3795]: 2007/09/20_22:32:59 info: te_pseudo_action: Pseudo action 47 fired and confirmed tengine[3795]: 2007/09/20_22:32:59 info: send_rsc_command: Initiating action 44: msg_last_stop_0 on host-1.awale.qc.ca tengine[3795]: 2007/09/20_22:32:59 info: te_pseudo_action: Pseudo action 52 fired and confirmed tengine[3795]: 2007/09/20_22:32:59 info: te_pseudo_action: Pseudo action 57 fired and confirmed tengine[3795]: 2007/09/20_22:32:59 info: send_rsc_command: Initiating action 80: share-drbd_0:0_pre_notify_demote_0 on host-1.awale.qc.ca crmd[3778]: 2007/09/20_22:32:59 info: do_lrm_rsc_op: Performing op=msg_last_stop_0 key=44:6:1805e0f1-8a22-46af-b444-fd7d9d4562af) tengine[3795]: 2007/09/20_22:32:59 info: send_rsc_command: Initiating action 49: vmstat-mon-child:0_stop_0 on host-1.awale.qc.ca tengine[3795]: 2007/09/20_22:32:59 info: send_rsc_command: Initiating action 54: stonith-child:0_stop_0 on host-1.awale.qc.ca crmd[3778]: 2007/09/20_22:32:59 info: do_lrm_rsc_op: Performing op=share-drbd_0:0_notify_0 key=80:6:1805e0f1-8a22-46af-b444-fd7d9d4562af) crmd[3778]: 2007/09/20_22:32:59 info: do_lrm_rsc_op: Performing op=vmstat-mon-child:0_stop_0 key=49:6:1805e0f1-8a22-46af-b444-fd7d9d4562af) crmd[3778]: 2007/09/20_22:32:59 info: do_lrm_rsc_op: Performing op=stonith-child:0_stop_0 key=54:6:1805e0f1-8a22-46af-b444-fd7d9d4562af) crmd[3778]: 2007/09/20_22:32:59 info: process_lrm_event: LRM operation msg_last_monitor_360000 (call=43, rc=-2) Cancelled crmd[3778]: 2007/09/20_22:32:59 info: process_lrm_event: LRM operation vmstat-mon-child:0_monitor_120000 (call=17, rc=-2) Cancelled crmd[3778]: 2007/09/20_22:32:59 info: process_lrm_event: LRM operation stonith-child:0_monitor_50000 (call=18, rc=-2) Cancelled cibmon[3772]: 2007/09/20_22:32:59 info: cibmon_diff: [cib_diff_notify] cib_delete confirmed cibmon[3772]: 2007/09/20_22:32:59 info: log_cib_diff: cib_delete: Diff: --- 0.12.2 cibmon[3772]: 2007/09/20_22:32:59 info: log_cib_diff: cib_delete: Diff: +++ 0.12.3 cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_delete: + cibmon[3772]: 2007/09/20_22:32:59 info: cibmon_diff: [cib_diff_notify] cib_delete confirmed cibmon[3772]: 2007/09/20_22:32:59 info: log_cib_diff: cib_delete: Diff: --- 0.12.3 cibmon[3772]: 2007/09/20_22:32:59 info: log_cib_diff: cib_delete: Diff: +++ 0.12.4 cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_delete: + cibmon[3772]: 2007/09/20_22:32:59 info: cibmon_diff: [cib_diff_notify] cib_delete confirmed cibmon[3772]: 2007/09/20_22:32:59 info: log_cib_diff: cib_delete: Diff: --- 0.12.4 cibmon[3772]: 2007/09/20_22:32:59 info: log_cib_diff: cib_delete: Diff: +++ 0.12.5 cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_delete: + lrmd[6098]: 2007/09/20_22:32:59 info: Try to stop STONITH resource : Device=apcsnmp crmd[3778]: 2007/09/20_22:32:59 info: process_lrm_event: LRM operation stonith-child:0_stop_0 (call=47, rc=0) complete tengine[3795]: 2007/09/20_22:32:59 info: match_graph_event: Action stonith-child:0_stop_0 (54) confirmed on host-1.awale.qc.ca tengine[3795]: 2007/09/20_22:32:59 info: te_pseudo_action: Pseudo action 58 fired and confirmed cibmon[3772]: 2007/09/20_22:32:59 info: cibmon_diff: [cib_diff_notify] cib_update confirmed cibmon[3772]: 2007/09/20_22:32:59 info: log_cib_diff: cib_update: Diff: --- 0.12.5 cibmon[3772]: 2007/09/20_22:32:59 info: log_cib_diff: cib_update: Diff: +++ 0.12.6 cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_update: - cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:32:59 info: log_data_element: cib_update: + lrmd[3775]: 2007/09/20_22:32:59 info: RA output: (share-drbd_0:0:notify:stdout) r0: ACTION=notify drbd[6093][6129]: 2007/09/20_22:33:00 DEBUG: r0 notify: pre for demote - counts: active 0 - starting 0 - stopping 1 msg[6092][6139]: 2007/09/20_22:33:00 INFO Starting to switch away from this node. crmd[3778]: 2007/09/20_22:33:00 info: process_lrm_event: LRM operation share-drbd_0:0_notify_0 (call=45, rc=0) complete msg[6092][6140]: 2007/09/20_22:33:00 INFO Node: host-1.awale.qc.ca cibmon[3772]: 2007/09/20_22:33:00 info: cibmon_diff: [cib_diff_notify] cib_update confirmed tengine[3795]: 2007/09/20_22:33:00 info: match_graph_event: Action share-drbd_0:0_pre_notify_demote_0 (80) confirmed on host-1.awale.qc.ca cibmon[3772]: 2007/09/20_22:33:00 info: log_cib_diff: cib_update: Diff: --- 0.12.6 tengine[3795]: 2007/09/20_22:33:00 info: te_pseudo_action: Pseudo action 35 fired and confirmed cibmon[3772]: 2007/09/20_22:33:00 info: log_cib_diff: cib_update: Diff: +++ 0.12.7 cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: - cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: + crmd[3778]: 2007/09/20_22:33:00 info: process_lrm_event: LRM operation msg_last_stop_0 (call=44, rc=0) complete cibmon[3772]: 2007/09/20_22:33:00 info: cibmon_diff: [cib_diff_notify] cib_update confirmed tengine[3795]: 2007/09/20_22:33:00 info: match_graph_event: Action msg_last_stop_0 (44) confirmed on host-1.awale.qc.ca cibmon[3772]: 2007/09/20_22:33:00 info: log_cib_diff: cib_update: Diff: --- 0.12.7 tengine[3795]: 2007/09/20_22:33:00 info: send_rsc_command: Initiating action 43: dw-httpd_stop_0 on host-1.awale.qc.ca cibmon[3772]: 2007/09/20_22:33:00 info: log_cib_diff: cib_update: Diff: +++ 0.12.8 cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: - cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: + crmd[3778]: 2007/09/20_22:33:00 info: do_lrm_rsc_op: Performing op=dw-httpd_stop_0 key=43:6:1805e0f1-8a22-46af-b444-fd7d9d4562af) crmd[3778]: 2007/09/20_22:33:00 info: process_lrm_event: LRM operation dw-httpd_monitor_120000 (call=41, rc=-2) Cancelled crmd[3778]: 2007/09/20_22:33:00 info: process_lrm_event: LRM operation vmstat-mon-child:0_stop_0 (call=46, rc=0) complete cibmon[3772]: 2007/09/20_22:33:00 info: cibmon_diff: [cib_diff_notify] cib_delete confirmed cibmon[3772]: 2007/09/20_22:33:00 info: log_cib_diff: cib_delete: Diff: --- 0.12.8 cibmon[3772]: 2007/09/20_22:33:00 info: log_cib_diff: cib_delete: Diff: +++ 0.12.9 cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_delete: + tengine[3795]: 2007/09/20_22:33:00 info: match_graph_event: Action vmstat-mon-child:0_stop_0 (49) confirmed on host-1.awale.qc.ca tengine[3795]: 2007/09/20_22:33:00 info: te_pseudo_action: Pseudo action 53 fired and confirmed cibmon[3772]: 2007/09/20_22:33:00 info: cibmon_diff: [cib_diff_notify] cib_update confirmed cibmon[3772]: 2007/09/20_22:33:00 info: log_cib_diff: cib_update: Diff: --- 0.12.9 cibmon[3772]: 2007/09/20_22:33:00 info: log_cib_diff: cib_update: Diff: +++ 0.12.10 cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: - cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: + crmd[3778]: 2007/09/20_22:33:00 info: process_lrm_event: LRM operation dw-httpd_stop_0 (call=48, rc=0) complete cibmon[3772]: 2007/09/20_22:33:00 info: cibmon_diff: [cib_diff_notify] cib_update confirmed tengine[3795]: 2007/09/20_22:33:00 info: match_graph_event: Action dw-httpd_stop_0 (43) confirmed on host-1.awale.qc.ca cibmon[3772]: 2007/09/20_22:33:00 info: log_cib_diff: cib_update: Diff: --- 0.12.10 tengine[3795]: 2007/09/20_22:33:00 info: send_rsc_command: Initiating action 42: mysql_stop_0 on host-1.awale.qc.ca cibmon[3772]: 2007/09/20_22:33:00 info: log_cib_diff: cib_update: Diff: +++ 0.12.11 cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: - cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: + crmd[3778]: 2007/09/20_22:33:00 info: do_lrm_rsc_op: Performing op=mysql_stop_0 key=42:6:1805e0f1-8a22-46af-b444-fd7d9d4562af) cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_update: + crmd[3778]: 2007/09/20_22:33:00 info: process_lrm_event: LRM operation mysql_monitor_120000 (call=39, rc=-2) Cancelled cibmon[3772]: 2007/09/20_22:33:00 info: cibmon_diff: [cib_diff_notify] cib_delete confirmed cibmon[3772]: 2007/09/20_22:33:00 info: log_cib_diff: cib_delete: Diff: --- 0.12.11 cibmon[3772]: 2007/09/20_22:33:00 info: log_cib_diff: cib_delete: Diff: +++ 0.12.12 cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:00 info: log_data_element: cib_delete: + mysql[6171][6191]: 2007/09/20_22:33:02 DEBUG: MySQL is not running mysql[6171][6193]: 2007/09/20_22:33:03 INFO: MySQL stopped crmd[3778]: 2007/09/20_22:33:03 info: process_lrm_event: LRM operation mysql_stop_0 (call=49, rc=0) complete tengine[3795]: 2007/09/20_22:33:03 info: match_graph_event: Action mysql_stop_0 (42) confirmed on host-1.awale.qc.ca tengine[3795]: 2007/09/20_22:33:03 info: send_rsc_command: Initiating action 41: IPaddr_10_0_0_59_stop_0 on host-1.awale.qc.ca cibmon[3772]: 2007/09/20_22:33:03 info: cibmon_diff: [cib_diff_notify] cib_update confirmed cibmon[3772]: 2007/09/20_22:33:03 info: log_cib_diff: cib_update: Diff: --- 0.12.12 cibmon[3772]: 2007/09/20_22:33:03 info: log_cib_diff: cib_update: Diff: +++ 0.12.13 cibmon[3772]: 2007/09/20_22:33:03 info: log_data_element: cib_update: - cibmon[3772]: 2007/09/20_22:33:03 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:03 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:03 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:03 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:03 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:03 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:03 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:03 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:03 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:03 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:03 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:03 info: log_data_element: cib_update: + crmd[3778]: 2007/09/20_22:33:03 info: do_lrm_rsc_op: Performing op=IPaddr_10_0_0_59_stop_0 key=41:6:1805e0f1-8a22-46af-b444-fd7d9d4562af) cibmon[3772]: 2007/09/20_22:33:03 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:03 info: cibmon_diff: [cib_diff_notify] cib_delete confirmed crmd[3778]: 2007/09/20_22:33:03 info: process_lrm_event: LRM operation IPaddr_10_0_0_59_monitor_5000 (call=37, rc=-2) Cancelled cibmon[3772]: 2007/09/20_22:33:03 info: log_cib_diff: cib_delete: Diff: --- 0.12.13 cibmon[3772]: 2007/09/20_22:33:03 info: log_cib_diff: cib_delete: Diff: +++ 0.12.14 cibmon[3772]: 2007/09/20_22:33:03 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:03 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:03 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:03 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:03 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:03 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:03 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:03 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:03 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:03 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:03 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:03 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:03 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:03 info: log_data_element: cib_delete: + lrmd[3775]: 2007/09/20_22:33:03 info: RA output: (IPaddr_10_0_0_59:stop:stdout) In IP Stop lrmd[3775]: 2007/09/20_22:33:03 info: RA output: (IPaddr_10_0_0_59:stop:stderr) SIOCDELRT: No such process IPaddr[6196][6211]: 2007/09/20_22:33:03 INFO: ifconfig eth0:0 down crmd[3778]: 2007/09/20_22:33:04 info: process_lrm_event: LRM operation IPaddr_10_0_0_59_stop_0 (call=50, rc=0) complete cibmon[3772]: 2007/09/20_22:33:04 info: cibmon_diff: [cib_diff_notify] cib_update confirmed cibmon[3772]: 2007/09/20_22:33:04 info: log_cib_diff: cib_update: Diff: --- 0.12.14 cibmon[3772]: 2007/09/20_22:33:04 info: log_cib_diff: cib_update: Diff: +++ 0.12.15 tengine[3795]: 2007/09/20_22:33:04 info: match_graph_event: Action IPaddr_10_0_0_59_stop_0 (41) confirmed on host-1.awale.qc.ca cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_update: - tengine[3795]: 2007/09/20_22:33:04 info: send_rsc_command: Initiating action 40: share-drbdlinks_stop_0 on host-1.awale.qc.ca cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_update: + crmd[3778]: 2007/09/20_22:33:04 info: do_lrm_rsc_op: Performing op=share-drbdlinks_stop_0 key=40:6:1805e0f1-8a22-46af-b444-fd7d9d4562af) cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_update: + crmd[3778]: 2007/09/20_22:33:04 info: process_lrm_event: LRM operation share-drbdlinks_monitor_120000 (call=35, rc=-2) Cancelled cibmon[3772]: 2007/09/20_22:33:04 info: cibmon_diff: [cib_diff_notify] cib_delete confirmed cibmon[3772]: 2007/09/20_22:33:04 info: log_cib_diff: cib_delete: Diff: --- 0.12.15 cibmon[3772]: 2007/09/20_22:33:04 info: log_cib_diff: cib_delete: Diff: +++ 0.12.16 cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_delete: + crmd[3778]: 2007/09/20_22:33:04 info: process_lrm_event: LRM operation share-drbdlinks_stop_0 (call=51, rc=0) complete cibmon[3772]: 2007/09/20_22:33:04 info: cibmon_diff: [cib_diff_notify] cib_update confirmed tengine[3795]: 2007/09/20_22:33:04 info: match_graph_event: Action share-drbdlinks_stop_0 (40) confirmed on host-1.awale.qc.ca cibmon[3772]: 2007/09/20_22:33:04 info: log_cib_diff: cib_update: Diff: --- 0.12.16 tengine[3795]: 2007/09/20_22:33:04 info: send_rsc_command: Initiating action 39: share-FS_stop_0 on host-1.awale.qc.ca cibmon[3772]: 2007/09/20_22:33:04 info: log_cib_diff: cib_update: Diff: +++ 0.12.17 cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_update: - cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_update: + crmd[3778]: 2007/09/20_22:33:04 info: do_lrm_rsc_op: Performing op=share-FS_stop_0 key=39:6:1805e0f1-8a22-46af-b444-fd7d9d4562af) cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_update: + crmd[3778]: 2007/09/20_22:33:04 info: process_lrm_event: LRM operation share-FS_monitor_120000 (call=33, rc=-2) Cancelled cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:04 info: cibmon_diff: [cib_diff_notify] cib_delete confirmed cibmon[3772]: 2007/09/20_22:33:04 info: log_cib_diff: cib_delete: Diff: --- 0.12.17 cibmon[3772]: 2007/09/20_22:33:04 info: log_cib_diff: cib_delete: Diff: +++ 0.12.18 cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_delete: + Filesystem[6220][6249]: 2007/09/20_22:33:04 INFO: Running stop for /dev/drbd0 on /share Filesystem[6220][6259]: 2007/09/20_22:33:04 INFO: Trying to unmount /share Filesystem[6220][6261]: 2007/09/20_22:33:04 INFO: unmounted /share successfully crmd[3778]: 2007/09/20_22:33:04 info: process_lrm_event: LRM operation share-FS_stop_0 (call=52, rc=0) complete tengine[3795]: 2007/09/20_22:33:04 info: match_graph_event: Action share-FS_stop_0 (39) confirmed on host-1.awale.qc.ca cibmon[3772]: 2007/09/20_22:33:04 info: cibmon_diff: [cib_diff_notify] cib_update confirmed tengine[3795]: 2007/09/20_22:33:04 info: send_rsc_command: Initiating action 38: msg_first_stop_0 on host-1.awale.qc.ca cibmon[3772]: 2007/09/20_22:33:04 info: log_cib_diff: cib_update: Diff: --- 0.12.18 cibmon[3772]: 2007/09/20_22:33:04 info: log_cib_diff: cib_update: Diff: +++ 0.12.19 crmd[3778]: 2007/09/20_22:33:04 info: do_lrm_rsc_op: Performing op=msg_first_stop_0 key=38:6:1805e0f1-8a22-46af-b444-fd7d9d4562af) cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_update: - cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:04 info: cibmon_diff: [cib_diff_notify] cib_delete confirmed cibmon[3772]: 2007/09/20_22:33:04 info: log_cib_diff: cib_delete: Diff: --- 0.12.19 cibmon[3772]: 2007/09/20_22:33:04 info: log_cib_diff: cib_delete: Diff: +++ 0.12.20 cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:04 info: log_data_element: cib_delete: + msg[6267][6288]: 2007/09/20_22:33:05 INFO Completed switching away from this node. msg[6267][6289]: 2007/09/20_22:33:05 INFO Node: host-1.awale.qc.ca crmd[3778]: 2007/09/20_22:33:05 info: process_lrm_event: LRM operation msg_first_stop_0 (call=53, rc=0) complete cibmon[3772]: 2007/09/20_22:33:05 info: cibmon_diff: [cib_diff_notify] cib_update confirmed tengine[3795]: 2007/09/20_22:33:05 info: match_graph_event: Action msg_first_stop_0 (38) confirmed on host-1.awale.qc.ca cibmon[3772]: 2007/09/20_22:33:05 info: log_cib_diff: cib_update: Diff: --- 0.12.20 tengine[3795]: 2007/09/20_22:33:05 info: te_pseudo_action: Pseudo action 48 fired and confirmed cibmon[3772]: 2007/09/20_22:33:05 info: log_cib_diff: cib_update: Diff: +++ 0.12.21 tengine[3795]: 2007/09/20_22:33:05 info: te_pseudo_action: Pseudo action 32 fired and confirmed cibmon[3772]: 2007/09/20_22:33:05 info: log_data_element: cib_update: - tengine[3795]: 2007/09/20_22:33:05 info: send_rsc_command: Initiating action 12: share-drbd_0:0_demote_0 on host-1.awale.qc.ca cibmon[3772]: 2007/09/20_22:33:05 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:05 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:05 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:05 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:05 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:05 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:05 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:05 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:05 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:05 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:05 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:05 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:05 info: log_data_element: cib_update: + crmd[3778]: 2007/09/20_22:33:05 info: do_lrm_rsc_op: Performing op=share-drbd_0:0_demote_0 key=12:6:1805e0f1-8a22-46af-b444-fd7d9d4562af) lrmd[3775]: 2007/09/20_22:33:05 info: RA output: (share-drbd_0:0:demote:stdout) r0: ACTION=demote drbd[6296][6307]: 2007/09/20_22:33:05 DEBUG: r0: Calling drbdadm -c /etc/drbd.conf secondary r0 drbd[6296][6314]: 2007/09/20_22:33:05 DEBUG: r0: Exit code 0 drbd[6296][6315]: 2007/09/20_22:33:05 DEBUG: r0: Command output: lrmd[3775]: 2007/09/20_22:33:05 info: RA output: (share-drbd_0:0:demote:stdout) drbd[6296][6316]: 2007/09/20_22:33:05 DEBUG: r0 demote: succeeded crmd[3778]: 2007/09/20_22:33:05 info: process_lrm_event: LRM operation share-drbd_0:0_demote_0 (call=54, rc=0) complete cibmon[3772]: 2007/09/20_22:33:05 info: cibmon_diff: [cib_diff_notify] cib_update confirmed tengine[3795]: 2007/09/20_22:33:05 info: match_graph_event: Action share-drbd_0:0_demote_0 (12) confirmed on host-1.awale.qc.ca cibmon[3772]: 2007/09/20_22:33:05 info: log_cib_diff: cib_update: Diff: --- 0.12.21 crmd[3778]: 2007/09/20_22:33:05 info: do_lrm_rsc_op: Performing op=share-drbd_0:0_notify_0 key=81:6:1805e0f1-8a22-46af-b444-fd7d9d4562af) tengine[3795]: 2007/09/20_22:33:05 info: te_pseudo_action: Pseudo action 33 fired and confirmed cibmon[3772]: 2007/09/20_22:33:05 info: log_cib_diff: cib_update: Diff: +++ 0.12.22 tengine[3795]: 2007/09/20_22:33:05 info: te_pseudo_action: Pseudo action 36 fired and confirmed cibmon[3772]: 2007/09/20_22:33:05 info: log_data_element: cib_update: - tengine[3795]: 2007/09/20_22:33:05 info: send_rsc_command: Initiating action 81: share-drbd_0:0_post_notify_demote_0 on host-1.awale.qc.ca cibmon[3772]: 2007/09/20_22:33:05 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:05 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:05 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:05 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:05 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:05 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:05 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:05 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:05 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:05 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:05 info: log_data_element: cib_update: + lrmd[3775]: 2007/09/20_22:33:05 info: RA output: (share-drbd_0:0:notify:stdout) r0: ACTION=notify drbd[6317][6328]: 2007/09/20_22:33:05 DEBUG: r0 notify: post for demote - counts: active 0 - starting 0 - stopping 1 drbd[6317][6330]: 2007/09/20_22:33:05 DEBUG: r0: Calling drbdadm -c /etc/drbd.conf state r0 drbd[6317][6337]: 2007/09/20_22:33:05 DEBUG: r0: Exit code 0 drbd[6317][6338]: 2007/09/20_22:33:05 DEBUG: r0: Command output: Secondary/Unknown drbd[6317][6346]: 2007/09/20_22:33:05 DEBUG: r0: Calling drbdadm -c /etc/drbd.conf cstate r0 drbd[6317][6359]: 2007/09/20_22:33:05 DEBUG: r0: Exit code 0 drbd[6317][6360]: 2007/09/20_22:33:06 DEBUG: r0: Command output: WFConnection drbd[6317][6361]: 2007/09/20_22:33:06 DEBUG: r0 status: Secondary/Unknown Secondary Unknown WFConnection drbd[6317][6362]: 2007/09/20_22:33:06 DEBUG: r0: Calling /usr/sbin/crm_master -v 10 -l reboot crm_master[6364]: 2007/09/20_22:33:06 info: Invoked: /usr/sbin/crm_master -v 10 -l reboot cibmon[3772]: 2007/09/20_22:33:06 info: cibmon_diff: [cib_diff_notify] cib_modify confirmed cibmon[3772]: 2007/09/20_22:33:06 info: log_cib_diff: cib_modify: Diff: --- 0.12.22 cibmon[3772]: 2007/09/20_22:33:06 info: log_cib_diff: cib_modify: Diff: +++ 0.12.23 cibmon[3772]: 2007/09/20_22:33:06 info: log_data_element: cib_modify: - cibmon[3772]: 2007/09/20_22:33:06 info: log_data_element: cib_modify: + drbd[6317][6365]: 2007/09/20_22:33:06 DEBUG: r0: Exit code 0 drbd[6317][6366]: 2007/09/20_22:33:06 DEBUG: r0: Command output: crm_master[6364]: 2007/09/20_22:33:06 info: Invoked: /usr/sbin/crm_master -v 10 -l reboot lrmd[3775]: 2007/09/20_22:33:06 info: RA output: (share-drbd_0:0:notify:stdout) crm_master[6364]: 2007/09/20_22:33:06 info: Invoked: /usr/sbin/crm_master -v 10 -l reboot crmd[3778]: 2007/09/20_22:33:06 info: process_lrm_event: LRM operation share-drbd_0:0_notify_0 (call=55, rc=0) complete tengine[3795]: 2007/09/20_22:33:06 info: match_graph_event: Action share-drbd_0:0_post_notify_demote_0 (81) confirmed on host-1.awale.qc.ca cibmon[3772]: 2007/09/20_22:33:06 info: cibmon_diff: [cib_diff_notify] cib_update confirmed tengine[3795]: 2007/09/20_22:33:06 info: te_pseudo_action: Pseudo action 37 fired and confirmed cibmon[3772]: 2007/09/20_22:33:06 info: log_cib_diff: cib_update: Diff: --- 0.12.23 crmd[3778]: 2007/09/20_22:33:06 info: do_lrm_rsc_op: Performing op=share-drbd_0:0_notify_0 key=79:6:1805e0f1-8a22-46af-b444-fd7d9d4562af) tengine[3795]: 2007/09/20_22:33:06 info: te_pseudo_action: Pseudo action 22 fired and confirmed cibmon[3772]: 2007/09/20_22:33:06 info: log_cib_diff: cib_update: Diff: +++ 0.12.24 tengine[3795]: 2007/09/20_22:33:06 info: send_rsc_command: Initiating action 79: share-drbd_0:0_pre_notify_stop_0 on host-1.awale.qc.ca cibmon[3772]: 2007/09/20_22:33:06 info: log_data_element: cib_update: - cibmon[3772]: 2007/09/20_22:33:06 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:06 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:06 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:06 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:06 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:06 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:06 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:06 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:06 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:06 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:06 info: log_data_element: cib_update: + lrmd[3775]: 2007/09/20_22:33:06 info: RA output: (share-drbd_0:0:notify:stdout) r0: ACTION=notify drbd[6367][6378]: 2007/09/20_22:33:06 DEBUG: r0 notify: pre for stop - counts: active 0 - starting 0 - stopping 1 crmd[3778]: 2007/09/20_22:33:06 info: process_lrm_event: LRM operation share-drbd_0:0_notify_0 (call=56, rc=0) complete tengine[3795]: 2007/09/20_22:33:06 info: match_graph_event: Action share-drbd_0:0_pre_notify_stop_0 (79) confirmed on host-1.awale.qc.ca tengine[3795]: 2007/09/20_22:33:06 info: te_pseudo_action: Pseudo action 23 fired and confirmed tengine[3795]: 2007/09/20_22:33:06 info: te_pseudo_action: Pseudo action 20 fired and confirmed tengine[3795]: 2007/09/20_22:33:06 info: send_rsc_command: Initiating action 13: share-drbd_0:0_stop_0 on host-1.awale.qc.ca cibmon[3772]: 2007/09/20_22:33:06 info: cibmon_diff: [cib_diff_notify] cib_update confirmed cibmon[3772]: 2007/09/20_22:33:06 info: log_cib_diff: cib_update: Diff: --- 0.12.24 cibmon[3772]: 2007/09/20_22:33:06 info: log_cib_diff: cib_update: Diff: +++ 0.12.25 cibmon[3772]: 2007/09/20_22:33:06 info: log_data_element: cib_update: - cibmon[3772]: 2007/09/20_22:33:06 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:06 info: log_data_element: cib_update: + crmd[3778]: 2007/09/20_22:33:06 info: do_lrm_rsc_op: Performing op=share-drbd_0:0_stop_0 key=13:6:1805e0f1-8a22-46af-b444-fd7d9d4562af) cibmon[3772]: 2007/09/20_22:33:06 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:06 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:06 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:06 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:06 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:06 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:06 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:06 info: log_data_element: cib_update: + cibmon[3772]: 2007/09/20_22:33:06 info: log_data_element: cib_update: + lrmd[3775]: 2007/09/20_22:33:06 info: RA output: (share-drbd_0:0:stop:stdout) r0: ACTION=stop drbd[6379][6391]: 2007/09/20_22:33:06 DEBUG: r0: Calling drbdadm -c /etc/drbd.conf state r0 drbd[6379][6398]: 2007/09/20_22:33:06 DEBUG: r0: Exit code 0 drbd[6379][6399]: 2007/09/20_22:33:06 DEBUG: r0: Command output: Secondary/Unknown drbd[6379][6407]: 2007/09/20_22:33:06 DEBUG: r0: Calling drbdadm -c /etc/drbd.conf cstate r0 drbd[6379][6420]: 2007/09/20_22:33:06 DEBUG: r0: Exit code 0 drbd[6379][6421]: 2007/09/20_22:33:06 DEBUG: r0: Command output: WFConnection drbd[6379][6422]: 2007/09/20_22:33:06 DEBUG: r0 status: Secondary/Unknown Secondary Unknown WFConnection drbd[6379][6423]: 2007/09/20_22:33:06 DEBUG: r0: Calling /usr/sbin/crm_master -D crm_master[6425]: 2007/09/20_22:33:06 info: Invoked: /usr/sbin/crm_master -D cibmon[3772]: 2007/09/20_22:33:06 info: cibmon_diff: [cib_diff_notify] cib_delete confirmed cibmon[3772]: 2007/09/20_22:33:06 info: log_cib_diff: cib_delete: Diff: --- 0.12.25 cibmon[3772]: 2007/09/20_22:33:06 info: log_cib_diff: cib_delete: Diff: +++ 0.12.26 cibmon[3772]: 2007/09/20_22:33:06 info: log_data_element: cib_delete: - cibmon[3772]: 2007/09/20_22:33:06 info: log_data_element: cib_delete: + drbd[6379][6426]: 2007/09/20_22:33:06 DEBUG: r0: Exit code 0 drbd[6379][6427]: 2007/09/20_22:33:07 DEBUG: r0: Command output: crm_master[6425]: 2007/09/20_22:33:06 info: Invoked: /usr/sbin/crm_master -D lrmd[3775]: 2007/09/20_22:33:07 info: RA output: (share-drbd_0:0:stop:stdout) crm_master[6425]: 2007/09/20_22:33:06 info: Invoked: /usr/sbin/crm_master -D drbd[6379][6428]: 2007/09/20_22:33:07 DEBUG: r0: Calling drbdadm -c /etc/drbd.conf down r0 drbd[6379][6436]: 2007/09/20_22:33:07 DEBUG: r0: Exit code 0 drbd[6379][6437]: 2007/09/20_22:33:07 DEBUG: r0: Command output: lrmd[3775]: 2007/09/20_22:33:07 info: RA output: (share-drbd_0:0:stop:stdout) drbd[6379][6438]: 2007/09/20_22:33:07 DEBUG: r0 stop: drbdadm down succeeded. drbd[6379][6440]: 2007/09/20_22:33:07 DEBUG: r0: Calling drbdadm -c /etc/drbd.conf state r0 drbd[6379][6447]: 2007/09/20_22:33:07 DEBUG: r0: Exit code 0 drbd[6379][6448]: 2007/09/20_22:33:07 DEBUG: r0: Command output: Not configured drbd[6379][6456]: 2007/09/20_22:33:07 DEBUG: r0: Calling drbdadm -c /etc/drbd.conf cstate r0 drbd[6379][6469]: 2007/09/20_22:33:07 DEBUG: r0: Exit code 0 drbd[6379][6470]: 2007/09/20_22:33:07 DEBUG: r0: Command output: Not configured drbd[6379][6471]: 2007/09/20_22:33:07 DEBUG: r0 status: Not configured Not configured Not configured Unconfigured crmd[3778]: 2007/09/20_22:33:07 info: process_lrm_event: LRM operation share-drbd_0:0_stop_0 (call=57, rc=0) complete cibmon[3772]: 2007/09/20_22:33:07 info: cibmon_diff: [cib_diff_notify] cib_update confirmed tengine[3795]: 2007/09/20_22:33:07 info: match_graph_event: Action share-drbd_0:0_stop_0 (13) confirmed on host-1.awale.qc.ca cibmon[3772]: 2007/09/20_22:33:07 info: log_cib_diff: cib_update: Diff: --- 0.12.26 tengine[3795]: 2007/09/20_22:33:07 info: te_pseudo_action: Pseudo action 21 fired and confirmed cibmon[3772]: 2007/09/20_22:33:07 info: log_cib_diff: cib_update: Diff: +++ 0.12.27 tengine[3795]: 2007/09/20_22:33:07 info: te_pseudo_action: Pseudo action 24 fired and confirmed cibmon[3772]: 2007/09/20_22:33:07 info: log_data_element: cib_update: - tengine[3795]: 2007/09/20_22:33:07 info: te_pseudo_action: Pseudo action 25 fired and confirmed cibmon[3772]: 2007/09/20_22:33:07 info: log_data_element: cib_update: + tengine[3795]: 2007/09/20_22:33:07 info: te_crm_command: Executing crm-event (59): do_shutdown on host-1.awale.qc.ca cibmon[3772]: 2007/09/20_22:33:07 info: log_data_element: cib_update: + crmd[3778]: 2007/09/20_22:33:07 info: handle_request: Shutting ourselves down (DC) tengine[3795]: 2007/09/20_22:33:07 info: te_crm_command: Skipping wait for 59 cibmon[3772]: 2007/09/20_22:33:07 info: log_data_element: cib_update: + crmd[3778]: 2007/09/20_22:33:07 info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_STOPPING [ input=I_STOP cause=C_IPC_MESSAGE origin=route_message ] tengine[3795]: 2007/09/20_22:33:07 info: run_graph: Transition 6: (Complete=33, Pending=0, Fired=0, Skipped=0, Incomplete=0) cibmon[3772]: 2007/09/20_22:33:07 info: log_data_element: cib_update: + crmd[3778]: 2007/09/20_22:33:07 info: do_dc_release: DC role released tengine[3795]: 2007/09/20_22:33:07 info: notify_crmd: Transition 6 status: te_complete - pengine[3796]: 2007/09/20_22:33:07 info: pengine_shutdown: Exiting PEngine (SIGTERM) cibmon[3772]: 2007/09/20_22:33:07 info: log_data_element: cib_update: + crmd[3778]: 2007/09/20_22:33:07 info: stop_subsystem: Sent -TERM to pengine: [3796] cibmon[3772]: 2007/09/20_22:33:07 info: log_data_element: cib_update: + crmd[3778]: 2007/09/20_22:33:07 info: stop_subsystem: Sent -TERM to pengine: [3796] cibmon[3772]: 2007/09/20_22:33:07 info: log_data_element: cib_update: + crmd[3778]: 2007/09/20_22:33:07 info: do_shutdown: Terminating the tengine tengine[3795]: 2007/09/20_22:33:07 info: update_abort_priority: Abort priority upgraded to 1000000 cibmon[3772]: 2007/09/20_22:33:07 info: log_data_element: cib_update: + crmd[3778]: 2007/09/20_22:33:07 info: stop_subsystem: Sent -TERM to tengine: [3795] tengine[3795]: 2007/09/20_22:33:07 info: update_abort_priority: Abort action 2 superceeded by 3 cibmon[3772]: 2007/09/20_22:33:07 info: log_data_element: cib_update: + crmd[3778]: 2007/09/20_22:33:07 info: do_shutdown: Waiting for subsystems to exit tengine[3795]: 2007/09/20_22:33:07 info: notify_crmd: Exiting after transition cibmon[3772]: 2007/09/20_22:33:07 info: log_data_element: cib_update: + crmd[3778]: 2007/09/20_22:33:07 WARN: register_fsa_input_adv: do_shutdown stalled the FSA with pending inputs tengine[3795]: 2007/09/20_22:33:07 info: te_init: Exiting tengine cibmon[3772]: 2007/09/20_22:33:07 info: log_data_element: cib_update: + crmd[3778]: 2007/09/20_22:33:07 WARN: do_log: [[FSA]] Input I_TE_SUCCESS from route_message() received in state (S_STOPPING) crmd[3778]: 2007/09/20_22:33:07 info: do_shutdown: Terminating the pengine crmd[3778]: 2007/09/20_22:33:07 info: stop_subsystem: Sent -TERM to pengine: [3796] crmd[3778]: 2007/09/20_22:33:07 info: do_shutdown: Terminating the tengine crmd[3778]: 2007/09/20_22:33:07 info: stop_subsystem: Sent -TERM to tengine: [3795] crmd[3778]: 2007/09/20_22:33:07 info: do_shutdown: Waiting for subsystems to exit crmd[3778]: 2007/09/20_22:33:07 WARN: register_fsa_input_adv: do_shutdown stalled the FSA with pending inputs crmd[3778]: 2007/09/20_22:33:07 info: crmdManagedChildDied: Process tengine:[3795] exited (signal=0, exitcode=0) crmd[3778]: 2007/09/20_22:33:07 info: crmdManagedChildDied: Process pengine:[3796] exited (signal=0, exitcode=0) crmd[3778]: 2007/09/20_22:33:07 WARN: do_log: [[FSA]] Input I_RELEASE_SUCCESS from do_dc_release() received in state (S_STOPPING) crmd[3778]: 2007/09/20_22:33:07 info: do_state_transition: State transition S_STOPPING -> S_TERMINATE [ input=I_TERMINATE cause=C_FSA_INTERNAL origin=do_shutdown ] crmd[3778]: 2007/09/20_22:33:07 info: verify_stopped: Checking for active resources before exit crmd[3778]: 2007/09/20_22:33:07 info: verify_stopped: Checking for active resources before exit ccm[3773]: 2007/09/20_22:33:07 info: client (pid=3778) removed from ccm crmd[3778]: 2007/09/20_22:33:07 info: do_lrm_control: Disconnected from the LRM crmd[3778]: 2007/09/20_22:33:07 info: do_ha_control: Disconnected from Heartbeat crmd[3778]: 2007/09/20_22:33:07 info: do_cib_control: Disconnecting CIB crmd[3778]: 2007/09/20_22:33:07 info: crmd_cib_connection_destroy: Connection to the CIB terminated... cib[3774]: 2007/09/20_22:33:07 info: cib_process_readwrite: We are now in R/O mode crmd[3778]: 2007/09/20_22:33:07 info: do_exit: Performing A_EXIT_0 - gracefully exiting the CRMd cib[3774]: 2007/09/20_22:33:07 WARN: send_via_callback_channel: Cannot find client for token ab90f9bb-042d-4a8b-9671-c30d99c780c7 crmd[3778]: 2007/09/20_22:33:07 info: free_mem: Dropping I_TERMINATE: [ state=S_TERMINATE cause=C_FSA_INTERNAL origin=verify_stopped ] cib[3774]: 2007/09/20_22:33:07 WARN: do_local_notify: A-Sync reply to 3778 failed: client left before we could send reply crmd[3778]: 2007/09/20_22:33:07 info: free_mem: Dropping I_TERMINATE: [ state=S_TERMINATE cause=C_FSA_INTERNAL origin=verify_stopped ] crmd[3778]: 2007/09/20_22:33:07 info: do_exit: [crmd] stopped (0) heartbeat[2826]: 2007/09/20_22:33:07 info: killing /usr/lib/heartbeat/attrd process group 3777 with signal 15 attrd[3777]: 2007/09/20_22:33:07 info: attrd_shutdown: Exiting attrd[3777]: 2007/09/20_22:33:07 info: main: Exiting... heartbeat[2826]: 2007/09/20_22:33:07 info: killing /usr/lib/heartbeat/stonithd process group 3776 with signal 15 stonithd[3776]: 2007/09/20_22:33:07 notice: /usr/lib/heartbeat/stonithd normally quit. heartbeat[2826]: 2007/09/20_22:33:07 info: killing /usr/lib/heartbeat/lrmd -r process group 3775 with signal 15 lrmd[3775]: 2007/09/20_22:33:07 info: lrmd is shutting down heartbeat[2826]: 2007/09/20_22:33:07 info: killing /usr/lib/heartbeat/cib process group 3774 with signal 15 cib[3774]: 2007/09/20_22:33:07 WARN: disconnect_cib_client: Disconnecting /cib_ro... cib[3774]: 2007/09/20_22:33:07 WARN: disconnect_cib_client: Disconnecting cibmon/cib_callback... cibmon[3772]: 2007/09/20_22:33:07 ERROR: cib_native_msgready: Message pending on command channel [3774] cib[3774]: 2007/09/20_22:33:07 info: cib_shutdown: Disconnected 2 clients cibmon[3772]: 2007/09/20_22:33:07 ERROR: crm_log_message_adv: #========= cib:cmd message start ==========# cib[3774]: 2007/09/20_22:33:07 info: cib_process_disconnect: All clients disconnected... cibmon[3772]: 2007/09/20_22:33:07 ERROR: MSG: No message to dump cib[3774]: 2007/09/20_22:33:07 info: terminate_ha_connection: initiate_exit: Disconnecting heartbeat cibmon[3772]: 2007/09/20_22:33:07 info: cib_native_msgready: Lost connection to the CIB service [3774]. cibmon[3772]: 2007/09/20_22:33:07 CRIT: cib_native_dispatch: Lost connection to the CIB service [3774/callback]. cibmon[3772]: 2007/09/20_22:33:07 ERROR: cib_connection_destroy: Connection to the CIB terminated... exiting cib[3774]: 2007/09/20_22:33:07 info: cib_ha_connection_destroy: Heartbeat disconnection complete... exiting cib[3774]: 2007/09/20_22:33:07 info: main: Done ccm[3773]: 2007/09/20_22:33:07 info: client (pid=3774) removed from ccm heartbeat[2826]: 2007/09/20_22:33:07 info: killing /usr/lib/heartbeat/ccm process group 3773 with signal 15 ccm[3773]: 2007/09/20_22:33:07 info: received SIGTERM, going to shut down heartbeat[2826]: 2007/09/20_22:33:07 info: client [/usr/lib/heartbeat/cibmon -d] is not running. heartbeat[2826]: 2007/09/20_22:33:08 info: killing HBWRITE process 2871 with signal 15 heartbeat[2826]: 2007/09/20_22:33:08 info: killing HBREAD process 2872 with signal 15 heartbeat[2826]: 2007/09/20_22:33:08 info: killing HBFIFO process 2862 with signal 15 heartbeat[2826]: 2007/09/20_22:33:08 info: killing HBWRITE process 2869 with signal 15 heartbeat[2826]: 2007/09/20_22:33:08 info: killing HBREAD process 2870 with signal 15 heartbeat[2826]: 2007/09/20_22:33:08 info: Core process 2870 exited. 5 remaining heartbeat[2826]: 2007/09/20_22:33:08 info: Core process 2862 exited. 4 remaining heartbeat[2826]: 2007/09/20_22:33:08 info: Core process 2872 exited. 3 remaining heartbeat[2826]: 2007/09/20_22:33:08 info: Core process 2869 exited. 2 remaining heartbeat[2826]: 2007/09/20_22:33:08 info: Core process 2871 exited. 1 remaining heartbeat[2826]: 2007/09/20_22:33:08 info: host-1.awale.qc.ca Heartbeat shutdown complete. logd[2804]: 2007/09/20_22:33:09 debug: logd_term_action: received SIGTERM logd[2804]: 2007/09/20_22:33:09 debug: logd_term_action: waiting for 0 messages to be read by write process logd[2804]: 2007/09/20_22:33:09 debug: logd_term_action: sending SIGTERM to write process logd[2809]: 2007/09/20_22:33:09 info: logd_term_write_action: received SIGTERM logd[2809]: 2007/09/20_22:33:09 debug: Writing out 0 messages then quitting logd[2809]: 2007/09/20_22:33:09 info: Exiting write process logd[2790]: 2007/09/20_22:34:18 info: logd started with /etc/logd.cf. logd[2795]: 2007/09/20_22:34:18 info: G_main_add_SignalHandler: Added signal handler for signal 15 logd[2790]: 2007/09/20_22:34:18 info: G_main_add_SignalHandler: Added signal handler for signal 15 heartbeat[2811]: 2007/09/20_22:34:18 info: No log entry found in ha.cf -- use logd heartbeat[2811]: 2007/09/20_22:34:18 info: Enabling logging daemon heartbeat[2811]: 2007/09/20_22:34:18 info: logfile and debug file are those specified in logd config file (default /etc/logd.cf) heartbeat[2811]: 2007/09/20_22:34:18 info: ************************** heartbeat[2811]: 2007/09/20_22:34:18 info: Configuration validated. Starting heartbeat 2.1.2 heartbeat[2812]: 2007/09/20_22:34:18 info: heartbeat: version 2.1.2 heartbeat[2812]: 2007/09/20_22:34:18 info: Heartbeat generation: 1188360888 heartbeat[2812]: 2007/09/20_22:34:18 info: G_main_add_TriggerHandler: Added signal manual handler heartbeat[2812]: 2007/09/20_22:34:18 info: G_main_add_TriggerHandler: Added signal manual handler heartbeat[2812]: 2007/09/20_22:34:18 info: Removing /var/run/heartbeat/rsctmp failed, recreating. heartbeat[2812]: 2007/09/20_22:34:18 info: glib: UDP Broadcast heartbeat started on port 694 (694) interface eth0 heartbeat[2812]: 2007/09/20_22:34:18 info: glib: UDP Broadcast heartbeat closed on port 694 interface eth0 - Status: 1 heartbeat[2812]: 2007/09/20_22:34:18 info: glib: ping group heartbeat started. heartbeat[2812]: 2007/09/20_22:34:18 info: G_main_add_SignalHandler: Added signal handler for signal 17 heartbeat[2812]: 2007/09/20_22:34:18 info: Local status now set to: 'up' heartbeat[2812]: 2007/09/20_22:34:20 info: Link local:local up. heartbeat[2812]: 2007/09/20_22:34:20 info: Status update for node local: status ping heartbeat[2812]: 2007/09/20_22:34:20 info: Link host-1.awale.qc.ca:eth0 up. heartbeat[2812]: 2007/09/20_22:34:34 WARN: node host-2.awale.qc.ca: is dead heartbeat[2812]: 2007/09/20_22:34:34 info: Comm_now_up(): updating status to active heartbeat[2812]: 2007/09/20_22:34:34 info: Local status now set to: 'active' heartbeat[2812]: 2007/09/20_22:34:34 info: Starting child client "/usr/lib/heartbeat/cibmon -d" (90,90) heartbeat[2812]: 2007/09/20_22:34:34 info: Starting child client "/usr/lib/heartbeat/ccm" (90,90) heartbeat[2812]: 2007/09/20_22:34:34 info: Starting child client "/usr/lib/heartbeat/cib" (90,90) heartbeat[2812]: 2007/09/20_22:34:34 info: Starting child client "/usr/lib/heartbeat/lrmd -r" (0,0) heartbeat[3657]: 2007/09/20_22:34:34 info: Starting "/usr/lib/heartbeat/cibmon -d" as uid 90 gid 90 (pid 3657) heartbeat[2812]: 2007/09/20_22:34:34 info: Starting child client "/usr/lib/heartbeat/stonithd" (0,0) heartbeat[2812]: 2007/09/20_22:34:34 info: Starting child client "/usr/lib/heartbeat/attrd" (90,90) heartbeat[2812]: 2007/09/20_22:34:34 info: Starting child client "/usr/lib/heartbeat/crmd" (90,90) heartbeat[2812]: 2007/09/20_22:34:34 info: Starting child client "/usr/lib/heartbeat/mgmtd -v" (0,0) heartbeat[3658]: 2007/09/20_22:34:34 info: Starting "/usr/lib/heartbeat/ccm" as uid 90 gid 90 (pid 3658) heartbeat[3659]: 2007/09/20_22:34:34 info: Starting "/usr/lib/heartbeat/cib" as uid 90 gid 90 (pid 3659) heartbeat[3660]: 2007/09/20_22:34:34 info: Starting "/usr/lib/heartbeat/lrmd -r" as uid 0 gid 0 (pid 3660) cibmon[3657]: 2007/09/20_22:34:34 info: G_main_add_SignalHandler: Added signal handler for signal 15 heartbeat[3661]: 2007/09/20_22:34:34 info: Starting "/usr/lib/heartbeat/stonithd" as uid 0 gid 0 (pid 3661) heartbeat[3664]: 2007/09/20_22:34:34 info: Starting "/usr/lib/heartbeat/mgmtd -v" as uid 0 gid 0 (pid 3664) heartbeat[3662]: 2007/09/20_22:34:34 info: Starting "/usr/lib/heartbeat/attrd" as uid 90 gid 90 (pid 3662) heartbeat[3663]: 2007/09/20_22:34:34 info: Starting "/usr/lib/heartbeat/crmd" as uid 90 gid 90 (pid 3663) cib[3659]: 2007/09/20_22:34:34 info: G_main_add_SignalHandler: Added signal handler for signal 15 cib[3659]: 2007/09/20_22:34:34 info: G_main_add_TriggerHandler: Added signal manual handler cib[3659]: 2007/09/20_22:34:34 info: G_main_add_SignalHandler: Added signal handler for signal 17 cib[3659]: 2007/09/20_22:34:34 info: main: Retrieval of a per-action CIB: disabled cib[3659]: 2007/09/20_22:34:34 info: readCibXmlFile: Reading cluster configuration from: /var/lib/heartbeat/crm/cib.xml lrmd[3660]: 2007/09/20_22:34:34 info: G_main_add_SignalHandler: Added signal handler for signal 15 lrmd[3660]: 2007/09/20_22:34:34 info: G_main_add_SignalHandler: Added signal handler for signal 17 cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] ccm[3658]: 2007/09/20_22:34:34 info: Hostname: host-1.awale.qc.ca cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: log_data_element: readCibXmlFile: [on-disk] cib[3659]: 2007/09/20_22:34:34 info: startCib: CIB Initialization completed successfully cib[3659]: 2007/09/20_22:34:34 info: cib_register_ha: Signing in with Heartbeat cib[3659]: 2007/09/20_22:34:34 info: cib_register_ha: FSA Hostname: host-1.awale.qc.ca cib[3659]: 2007/09/20_22:34:34 WARN: cib_init: CCM Activation failed cib[3659]: 2007/09/20_22:34:34 WARN: cib_init: CCM Connection failed 1 times (30 max) lrmd[3660]: 2007/09/20_22:34:34 info: G_main_add_SignalHandler: Added signal handler for signal 10 lrmd[3660]: 2007/09/20_22:34:34 info: G_main_add_SignalHandler: Added signal handler for signal 12 lrmd[3660]: 2007/09/20_22:34:34 info: Started. crmd[3663]: 2007/09/20_22:34:34 info: main: CRM Hg Version: 9a995a8a7052 tip crmd[3663]: 2007/09/20_22:34:34 info: crmd_init: Starting crmd crmd[3663]: 2007/09/20_22:34:34 info: G_main_add_SignalHandler: Added signal handler for signal 15 crmd[3663]: 2007/09/20_22:34:34 info: G_main_add_TriggerHandler: Added signal manual handler crmd[3663]: 2007/09/20_22:34:34 info: G_main_add_SignalHandler: Added signal handler for signal 17 stonithd[3661]: 2007/09/20_22:34:34 info: G_main_add_SignalHandler: Added signal handler for signal 10 stonithd[3661]: 2007/09/20_22:34:34 info: G_main_add_SignalHandler: Added signal handler for signal 12 attrd[3662]: 2007/09/20_22:34:34 info: G_main_add_SignalHandler: Added signal handler for signal 15 attrd[3662]: 2007/09/20_22:34:34 info: register_with_ha: Hostname: host-1.awale.qc.ca stonithd[3661]: 2007/09/20_22:34:34 info: Signing in with heartbeat. attrd[3662]: 2007/09/20_22:34:34 info: register_with_ha: UUID: 1da8804b-8a93-4145-9d40-1133699fe377 mgmtd[3664]: 2007/09/20_22:34:34 info: G_main_add_SignalHandler: Added signal handler for signal 15 mgmtd[3664]: 2007/09/20_22:34:34 debug: Enabling coredumps mgmtd[3664]: 2007/09/20_22:34:34 info: G_main_add_SignalHandler: Added signal handler for signal 10 mgmtd[3664]: 2007/09/20_22:34:34 info: G_main_add_SignalHandler: Added signal handler for signal 12 stonithd[3661]: 2007/09/20_22:34:34 notice: /usr/lib/heartbeat/stonithd start up successfully. stonithd[3661]: 2007/09/20_22:34:34 info: G_main_add_SignalHandler: Added signal handler for signal 17 ccm[3658]: 2007/09/20_22:34:34 debug: quorum plugin: majority ccm[3658]: 2007/09/20_22:34:34 debug: cluster:linux-ha, member_count=1, member_quorum_votes=100 mgmtd[3664]: 2007/09/20_22:34:34 info: init_crm ccm[3658]: 2007/09/20_22:34:34 debug: total_node_count=2, total_quorum_votes=200 ccm[3658]: 2007/09/20_22:34:34 debug: quorum plugin: twonodes ccm[3658]: 2007/09/20_22:34:34 debug: cluster:linux-ha, member_count=1, member_quorum_votes=100 ccm[3658]: 2007/09/20_22:34:34 debug: total_node_count=2, total_quorum_votes=200 ccm[3658]: 2007/09/20_22:34:34 info: Break tie for 2 nodes cluster ccm[3658]: 2007/09/20_22:34:34 info: G_main_add_SignalHandler: Added signal handler for signal 15 cib[3659]: 2007/09/20_22:34:35 info: cib_init: Starting cib mainloop cib[3659]: 2007/09/20_22:34:35 info: mem_handle_event: Got an event OC_EV_MS_NEW_MEMBERSHIP from ccm cib[3659]: 2007/09/20_22:34:35 info: mem_handle_event: instance=1, nodes=1, new=1, lost=0, n_idx=0, new_idx=0, old_idx=3 cib[3659]: 2007/09/20_22:34:35 info: cib_ccm_msg_callback: PEER: host-1.awale.qc.ca cibmon[3657]: 2007/09/20_22:34:35 info: main: Starting mainloop cib[3659]: 2007/09/20_22:34:35 info: cib_null_callback: Setting cib_diff_notify callbacks for cibmon: on cib[3659]: 2007/09/20_22:34:35 info: cib_null_callback: Setting cib_diff_notify callbacks for mgmtd: on crmd[3663]: 2007/09/20_22:34:35 info: do_cib_control: CIB connection established crmd[3663]: 2007/09/20_22:34:35 info: register_with_ha: Hostname: host-1.awale.qc.ca cib[3659]: 2007/09/20_22:34:35 info: cib_client_status_callback: Status update: Client host-1.awale.qc.ca/cib now has status [join] cib[3659]: 2007/09/20_22:34:35 info: cib_client_status_callback: Status update: Client host-1.awale.qc.ca/cib now has status [online] cib[3659]: 2007/09/20_22:34:35 info: cib_null_callback: Setting cib_refresh_notify callbacks for crmd: on cib[3666]: 2007/09/20_22:34:35 info: write_cib_contents: Wrote version 0.12.1 of the CIB to disk (digest: 0d4380905f465e084bc2c68cb2bc8036) crmd[3663]: 2007/09/20_22:34:36 info: register_with_ha: UUID: 1da8804b-8a93-4145-9d40-1133699fe377 crmd[3663]: 2007/09/20_22:34:36 info: populate_cib_nodes: Requesting the list of configured nodes mgmtd[3664]: 2007/09/20_22:34:37 debug: main: run the loop... mgmtd[3664]: 2007/09/20_22:34:37 info: Started. crmd[3663]: 2007/09/20_22:34:37 WARN: get_uuid: Could not calculate UUID for host-2.awale.qc.ca crmd[3663]: 2007/09/20_22:34:37 WARN: populate_cib_nodes: Node host-2.awale.qc.ca: no uuid found crmd[3663]: 2007/09/20_22:34:37 notice: populate_cib_nodes: Node: host-1.awale.qc.ca (uuid: 1da8804b-8a93-4145-9d40-1133699fe377) crmd[3663]: 2007/09/20_22:34:37 info: do_ha_control: Connected to Heartbeat crmd[3663]: 2007/09/20_22:34:37 info: do_ccm_control: CCM connection established... waiting for first callback crmd[3663]: 2007/09/20_22:34:37 info: do_started: Delaying start, CCM (0000000000100000) not connected crmd[3663]: 2007/09/20_22:34:37 info: crmd_init: Starting crmd's mainloop crmd[3663]: 2007/09/20_22:34:37 notice: cluster_option: Using default value '10s' for cluster option 'dc_deadtime' crmd[3663]: 2007/09/20_22:34:37 notice: cluster_option: Using default value '0' for cluster option 'cluster_recheck_interval' crmd[3663]: 2007/09/20_22:34:37 notice: cluster_option: Using default value '2min' for cluster option 'election_timeout' crmd[3663]: 2007/09/20_22:34:37 notice: cluster_option: Using default value '20min' for cluster option 'shutdown_escalation' crmd[3663]: 2007/09/20_22:34:37 notice: cluster_option: Using default value '3min' for cluster option 'crmd-integration-timeout' crmd[3663]: 2007/09/20_22:34:37 notice: cluster_option: Using default value '10min' for cluster option 'crmd-finalization-timeout' crmd[3663]: 2007/09/20_22:34:37 notice: crmd_client_status_callback: Status update: Client host-1.awale.qc.ca/crmd now has status [online] crmd[3663]: 2007/09/20_22:34:38 notice: crmd_client_status_callback: Status update: Client host-1.awale.qc.ca/crmd now has status [online] cibmon[3657]: 2007/09/20_22:34:38 info: cibmon_diff: [cib_diff_notify] cib_update confirmed cibmon[3657]: 2007/09/20_22:34:38 info: log_cib_diff: cib_update: Local-only Change: 0.12.1 cibmon[3657]: 2007/09/20_22:34:38 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:38 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:38 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:38 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:38 info: log_data_element: cib_update: + crmd[3663]: 2007/09/20_22:34:38 info: mem_handle_event: Got an event OC_EV_MS_NEW_MEMBERSHIP from ccm crmd[3663]: 2007/09/20_22:34:38 info: mem_handle_event: instance=1, nodes=1, new=1, lost=0, n_idx=0, new_idx=0, old_idx=3 crmd[3663]: 2007/09/20_22:34:38 info: crmd_ccm_msg_callback: Quorum (re)attained after event=NEW MEMBERSHIP (id=1) crmd[3663]: 2007/09/20_22:34:38 info: ccm_event_detail: NEW MEMBERSHIP: trans=1, nodes=1, new=1, lost=0 n_idx=0, new_idx=0, old_idx=3 crmd[3663]: 2007/09/20_22:34:38 info: ccm_event_detail: CURRENT: host-1.awale.qc.ca [nodeid=0, born=1] crmd[3663]: 2007/09/20_22:34:38 info: ccm_event_detail: NEW: host-1.awale.qc.ca [nodeid=0, born=1] crmd[3663]: 2007/09/20_22:34:38 info: do_started: The local CRM is operational crmd[3663]: 2007/09/20_22:34:38 info: do_state_transition: State transition S_STARTING -> S_PENDING [ input=I_PENDING cause=C_CCM_CALLBACK origin=do_started ] crmd[3663]: 2007/09/20_22:34:38 info: update_dc: Set DC to () cibmon[3657]: 2007/09/20_22:34:38 info: cibmon_diff: [cib_diff_notify] cib_update confirmed cibmon[3657]: 2007/09/20_22:34:38 info: log_cib_diff: cib_update: Local-only Change: 0.12.1 cibmon[3657]: 2007/09/20_22:34:38 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:38 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:38 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:38 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:38 info: log_data_element: cib_update: + attrd[3662]: 2007/09/20_22:34:40 info: main: Starting mainloop... crmd[3663]: 2007/09/20_22:34:49 info: crm_timer_popped: Election Trigger (I_DC_TIMEOUT) just popped! crmd[3663]: 2007/09/20_22:34:49 WARN: do_log: [[FSA]] Input I_DC_TIMEOUT from crm_timer_popped() received in state (S_PENDING) crmd[3663]: 2007/09/20_22:34:49 info: do_state_transition: State transition S_PENDING -> S_ELECTION [ input=I_DC_TIMEOUT cause=C_TIMER_POPPED origin=crm_timer_popped ] crmd[3663]: 2007/09/20_22:34:49 info: update_dc: Set DC to () crmd[3663]: 2007/09/20_22:34:49 info: do_election_count_vote: Updated voted hash for host-1.awale.qc.ca to vote crmd[3663]: 2007/09/20_22:34:49 info: do_election_count_vote: Election ignore: our vote (host-1.awale.qc.ca) crmd[3663]: 2007/09/20_22:34:49 info: do_state_transition: State transition S_ELECTION -> S_INTEGRATION [ input=I_ELECTION_DC cause=C_FSA_INTERNAL origin=do_election_check ] crmd[3663]: 2007/09/20_22:34:49 info: start_subsystem: Starting sub-system "tengine" crmd[3663]: 2007/09/20_22:34:49 info: start_subsystem: Starting sub-system "pengine" crmd[3663]: 2007/09/20_22:34:49 info: do_dc_takeover: Taking over DC status for this partition crmd[3663]: 2007/09/20_22:34:49 info: update_dc: Set DC to () crmd[3663]: 2007/09/20_22:34:49 info: join_make_offer: Making join offers based on membership 1 crmd[3663]: 2007/09/20_22:34:49 info: do_dc_join_offer_all: join-1: Waiting on 1 outstanding join acks cib[3659]: 2007/09/20_22:34:49 info: cib_process_readwrite: We are now in R/W mode tengine[3682]: 2007/09/20_22:34:49 info: G_main_add_SignalHandler: Added signal handler for signal 15 crmd[3663]: 2007/09/20_22:34:49 info: update_dc: Set DC to host-1.awale.qc.ca (1.0.9) tengine[3682]: 2007/09/20_22:34:49 info: G_main_add_TriggerHandler: Added signal manual handler cibmon[3657]: 2007/09/20_22:34:49 info: cibmon_diff: [cib_diff_notify] cib_update confirmed cibmon[3657]: 2007/09/20_22:34:49 info: log_cib_diff: cib_update: Diff: --- 0.12.1 cibmon[3657]: 2007/09/20_22:34:49 info: log_cib_diff: cib_update: Diff: +++ 0.12.2 cibmon[3657]: 2007/09/20_22:34:49 info: log_data_element: cib_update: - cibmon[3657]: 2007/09/20_22:34:49 info: log_data_element: cib_update: + cib[3659]: 2007/09/20_22:34:49 info: cib_null_callback: Setting cib_diff_notify callbacks for tengine: on tengine[3682]: 2007/09/20_22:34:49 info: te_init: Registering TE UUID: d66e5bec-f673-4648-85f1-de6226a15a97 tengine[3682]: 2007/09/20_22:34:49 info: set_graph_functions: Setting custom graph functions tengine[3682]: 2007/09/20_22:34:49 info: unpack_graph: Unpacked transition -1: 0 actions in 0 synapses tengine[3682]: 2007/09/20_22:34:49 info: te_init: Starting tengine pengine[3683]: 2007/09/20_22:34:49 info: G_main_add_SignalHandler: Added signal handler for signal 15 pengine[3683]: 2007/09/20_22:34:49 info: pe_init: Starting pengine cib[3659]: 2007/09/20_22:34:49 info: sync_our_cib: Syncing CIB to all peers crmd[3663]: 2007/09/20_22:34:49 info: do_state_transition: State transition S_INTEGRATION -> S_FINALIZE_JOIN [ input=I_INTEGRATED cause=C_FSA_INTERNAL origin=check_join_state ] crmd[3663]: 2007/09/20_22:34:49 info: do_state_transition: All 1 cluster nodes responded to the join offer. crmd[3663]: 2007/09/20_22:34:49 info: update_attrd: Connecting to attrd... attrd[3662]: 2007/09/20_22:34:49 info: attrd_local_callback: Sending full refresh cibmon[3657]: 2007/09/20_22:34:49 info: cibmon_diff: [cib_diff_notify] cib_update confirmed cibmon[3657]: 2007/09/20_22:34:49 info: log_cib_diff: cib_update: Diff: --- 0.12.2 cibmon[3657]: 2007/09/20_22:34:49 info: log_cib_diff: cib_update: Diff: +++ 0.12.3 cibmon[3657]: 2007/09/20_22:34:49 info: log_data_element: cib_update: - cibmon[3657]: 2007/09/20_22:34:49 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:49 info: cibmon_diff: [cib_diff_notify] cib_update confirmed cibmon[3657]: 2007/09/20_22:34:49 info: log_cib_diff: cib_update: Diff: --- 0.12.3 cibmon[3657]: 2007/09/20_22:34:49 info: log_cib_diff: cib_update: Diff: +++ 0.12.4 cibmon[3657]: 2007/09/20_22:34:49 info: log_data_element: cib_update: - cibmon[3657]: 2007/09/20_22:34:49 info: log_data_element: cib_update: + crmd[3663]: 2007/09/20_22:34:50 info: update_dc: Set DC to host-1.awale.qc.ca (1.0.9) crmd[3663]: 2007/09/20_22:34:50 info: do_dc_join_ack: join-1: Updating node state to member for host-1.awale.qc.ca crmd[3663]: 2007/09/20_22:34:50 info: do_state_transition: State transition S_FINALIZE_JOIN -> S_POLICY_ENGINE [ input=I_FINALIZED cause=C_FSA_INTERNAL origin=check_join_state ] crmd[3663]: 2007/09/20_22:34:50 info: do_state_transition: All 1 cluster nodes are eligible to run resources. tengine[3682]: 2007/09/20_22:34:50 info: update_abort_priority: Abort priority upgraded to 1000000 cibmon[3657]: 2007/09/20_22:34:50 info: cibmon_diff: [cib_diff_notify] cib_update confirmed tengine[3682]: 2007/09/20_22:34:50 info: update_abort_priority: 'DC Takeover' abort superceeded cibmon[3657]: 2007/09/20_22:34:50 info: log_cib_diff: cib_update: Diff: --- 0.12.4 cibmon[3657]: 2007/09/20_22:34:50 info: log_cib_diff: cib_update: Diff: +++ 0.12.5 cibmon[3657]: 2007/09/20_22:34:50 info: log_data_element: cib_update: - cibmon[3657]: 2007/09/20_22:34:50 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:50 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:50 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:50 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:50 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:50 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:50 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:50 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:50 info: log_data_element: cib_update: + pengine[3683]: 2007/09/20_22:34:50 notice: cluster_option: Using default value 'stop' for cluster option 'no-quorum-policy' pengine[3683]: 2007/09/20_22:34:50 notice: cluster_option: Using default value 'true' for cluster option 'symmetric-cluster' pengine[3683]: 2007/09/20_22:34:50 notice: cluster_option: Using default value 'false' for cluster option 'stonith-enabled' pengine[3683]: 2007/09/20_22:34:50 notice: cluster_option: Using default value 'reboot' for cluster option 'stonith-action' pengine[3683]: 2007/09/20_22:34:50 notice: cluster_option: Using default value '0' for cluster option 'default-resource-stickiness' pengine[3683]: 2007/09/20_22:34:50 notice: cluster_option: Using default value '0' for cluster option 'default-resource-failure-stickiness' pengine[3683]: 2007/09/20_22:34:50 notice: cluster_option: Using default value 'true' for cluster option 'is-managed-default' pengine[3683]: 2007/09/20_22:34:50 notice: cluster_option: Using default value '60s' for cluster option 'cluster-delay' pengine[3683]: 2007/09/20_22:34:50 notice: cluster_option: Using default value '20s' for cluster option 'default-action-timeout' pengine[3683]: 2007/09/20_22:34:50 notice: cluster_option: Using default value 'true' for cluster option 'stop-orphan-resources' pengine[3683]: 2007/09/20_22:34:50 notice: cluster_option: Using default value 'true' for cluster option 'stop-orphan-actions' pengine[3683]: 2007/09/20_22:34:50 notice: cluster_option: Using default value 'false' for cluster option 'remove-after-stop' pengine[3683]: 2007/09/20_22:34:50 notice: cluster_option: Using default value '-1' for cluster option 'pe-error-series-max' pengine[3683]: 2007/09/20_22:34:50 notice: cluster_option: Using default value '-1' for cluster option 'pe-warn-series-max' pengine[3683]: 2007/09/20_22:34:50 notice: cluster_option: Using default value '-1' for cluster option 'pe-input-series-max' pengine[3683]: 2007/09/20_22:34:50 notice: cluster_option: Using default value 'true' for cluster option 'startup-fencing' pengine[3683]: 2007/09/20_22:34:50 info: determine_online_status: Node host-1.awale.qc.ca is online pengine[3683]: 2007/09/20_22:34:50 info: clone_print: Master/Slave Set: ms-drbd_0 pengine[3683]: 2007/09/20_22:34:50 info: native_print: share-drbd_0:0 (heartbeat::ocf:drbd): Stopped pengine[3683]: 2007/09/20_22:34:50 info: native_print: share-drbd_0:1 (heartbeat::ocf:drbd): Stopped pengine[3683]: 2007/09/20_22:34:50 info: group_print: Resource Group: DW pengine[3683]: 2007/09/20_22:34:50 info: native_print: msg_first (heartbeat:msg): Stopped pengine[3683]: 2007/09/20_22:34:50 info: native_print: share-FS (heartbeat::ocf:Filesystem): Stopped pengine[3683]: 2007/09/20_22:34:50 info: native_print: share-drbdlinks (heartbeat:drbdlinks): Stopped pengine[3683]: 2007/09/20_22:34:50 info: native_print: IPaddr_10_0_0_59 (heartbeat::ocf:IPaddr): Stopped pengine[3683]: 2007/09/20_22:34:50 info: native_print: mysql (heartbeat::ocf:mysql): Stopped pengine[3683]: 2007/09/20_22:34:50 info: native_print: dw-httpd (awale::ocf:daemontools): Stopped pengine[3683]: 2007/09/20_22:34:50 info: native_print: msg_last (heartbeat:msg): Stopped pengine[3683]: 2007/09/20_22:34:50 info: clone_print: Clone Set: vmstat-mon pengine[3683]: 2007/09/20_22:34:50 info: native_print: vmstat-mon-child:0 (awale::ocf:daemontools): Stopped pengine[3683]: 2007/09/20_22:34:50 info: native_print: vmstat-mon-child:1 (awale::ocf:daemontools): Stopped pengine[3683]: 2007/09/20_22:34:50 info: clone_print: Clone Set: stonith pengine[3683]: 2007/09/20_22:34:50 info: native_print: stonith-child:0 (stonith:apcsnmp): Stopped pengine[3683]: 2007/09/20_22:34:50 info: native_print: stonith-child:1 (stonith:apcsnmp): Stopped pengine[3683]: 2007/09/20_22:34:50 WARN: native_color: Resource share-drbd_0:1 cannot run anywhere pengine[3683]: 2007/09/20_22:34:50 info: master_color: Promoted 0 instances of a possible 1 to master pengine[3683]: 2007/09/20_22:34:50 notice: StartRsc: host-1.awale.qc.ca Start share-drbd_0:0 pengine[3683]: 2007/09/20_22:34:50 notice: RecurringOp: host-1.awale.qc.ca share-drbd_0:0_monitor_120000 pengine[3683]: 2007/09/20_22:34:50 notice: StartRsc: host-1.awale.qc.ca Start share-drbd_0:0 pengine[3683]: 2007/09/20_22:34:50 notice: RecurringOp: host-1.awale.qc.ca share-drbd_0:0_monitor_120000 pengine[3683]: 2007/09/20_22:34:50 info: master_color: Promoted 0 instances of a possible 1 to master pengine[3683]: 2007/09/20_22:34:50 info: master_color: Promoted 0 instances of a possible 1 to master pengine[3683]: 2007/09/20_22:34:50 WARN: native_color: Resource msg_first cannot run anywhere pengine[3683]: 2007/09/20_22:34:50 info: master_color: Promoted 0 instances of a possible 1 to master pengine[3683]: 2007/09/20_22:34:50 info: master_color: Promoted 0 instances of a possible 1 to master pengine[3683]: 2007/09/20_22:34:50 WARN: native_color: Resource share-FS cannot run anywhere pengine[3683]: 2007/09/20_22:34:50 WARN: native_color: Resource share-drbdlinks cannot run anywhere pengine[3683]: 2007/09/20_22:34:50 WARN: native_color: Resource IPaddr_10_0_0_59 cannot run anywhere pengine[3683]: 2007/09/20_22:34:50 WARN: native_color: Resource mysql cannot run anywhere pengine[3683]: 2007/09/20_22:34:50 WARN: native_color: Resource dw-httpd cannot run anywhere pengine[3683]: 2007/09/20_22:34:50 WARN: native_color: Resource msg_last cannot run anywhere pengine[3683]: 2007/09/20_22:34:50 WARN: native_color: Resource vmstat-mon-child:1 cannot run anywhere pengine[3683]: 2007/09/20_22:34:50 notice: StartRsc: host-1.awale.qc.ca Start vmstat-mon-child:0 pengine[3683]: 2007/09/20_22:34:50 notice: RecurringOp: host-1.awale.qc.ca vmstat-mon-child:0_monitor_120000 pengine[3683]: 2007/09/20_22:34:50 WARN: native_color: Resource stonith-child:1 cannot run anywhere pengine[3683]: 2007/09/20_22:34:50 notice: StartRsc: host-1.awale.qc.ca Start stonith-child:0 pengine[3683]: 2007/09/20_22:34:50 notice: RecurringOp: host-1.awale.qc.ca stonith-child:0_monitor_50000 pengine[3683]: 2007/09/20_22:34:50 WARN: process_pe_message: Transition 0: WARNINGs found during PE processing. PEngine Input stored in: /var/lib/heartbeat/pengine/pe-warn-485.bz2 pengine[3683]: 2007/09/20_22:34:50 info: process_pe_message: Configuration WARNINGs found during PE processing. Please run "crm_verify -L" to identify issues. crmd[3663]: 2007/09/20_22:34:50 info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=route_message ] tengine[3682]: 2007/09/20_22:34:50 info: unpack_graph: Unpacked transition 0: 31 actions in 31 synapses tengine[3682]: 2007/09/20_22:34:50 info: send_rsc_command: Initiating action 3: share-drbd_0:0_monitor_0 on host-1.awale.qc.ca tengine[3682]: 2007/09/20_22:34:50 info: te_pseudo_action: Pseudo action 19 fired and confirmed tengine[3682]: 2007/09/20_22:34:50 info: te_pseudo_action: Pseudo action 20 fired and confirmed tengine[3682]: 2007/09/20_22:34:50 info: send_rsc_command: Initiating action 4: msg_first_monitor_0 on host-1.awale.qc.ca tengine[3682]: 2007/09/20_22:34:50 info: send_rsc_command: Initiating action 5: share-FS_monitor_0 on host-1.awale.qc.ca crmd[3663]: 2007/09/20_22:34:50 info: do_lrm_rsc_op: Performing op=share-drbd_0:0_monitor_0 key=3:0:d66e5bec-f673-4648-85f1-de6226a15a97) tengine[3682]: 2007/09/20_22:34:50 info: send_rsc_command: Initiating action 6: share-drbdlinks_monitor_0 on host-1.awale.qc.ca tengine[3682]: 2007/09/20_22:34:50 info: send_rsc_command: Initiating action 7: IPaddr_10_0_0_59_monitor_0 on host-1.awale.qc.ca tengine[3682]: 2007/09/20_22:34:50 info: send_rsc_command: Initiating action 8: mysql_monitor_0 on host-1.awale.qc.ca tengine[3682]: 2007/09/20_22:34:50 info: send_rsc_command: Initiating action 9: dw-httpd_monitor_0 on host-1.awale.qc.ca tengine[3682]: 2007/09/20_22:34:50 info: send_rsc_command: Initiating action 10: msg_last_monitor_0 on host-1.awale.qc.ca tengine[3682]: 2007/09/20_22:34:50 info: send_rsc_command: Initiating action 11: vmstat-mon-child:0_monitor_0 on host-1.awale.qc.ca tengine[3682]: 2007/09/20_22:34:50 info: send_rsc_command: Initiating action 12: vmstat-mon-child:1_monitor_0 on host-1.awale.qc.ca tengine[3682]: 2007/09/20_22:34:50 info: send_rsc_command: Initiating action 13: stonith-child:0_monitor_0 on host-1.awale.qc.ca tengine[3682]: 2007/09/20_22:34:50 info: send_rsc_command: Initiating action 14: stonith-child:1_monitor_0 on host-1.awale.qc.ca crmd[3663]: 2007/09/20_22:34:50 info: do_lrm_rsc_op: Performing op=msg_first_monitor_0 key=4:0:d66e5bec-f673-4648-85f1-de6226a15a97) crmd[3663]: 2007/09/20_22:34:50 info: do_lrm_rsc_op: Performing op=share-FS_monitor_0 key=5:0:d66e5bec-f673-4648-85f1-de6226a15a97) crmd[3663]: 2007/09/20_22:34:50 info: do_lrm_rsc_op: Performing op=share-drbdlinks_monitor_0 key=6:0:d66e5bec-f673-4648-85f1-de6226a15a97) crmd[3663]: 2007/09/20_22:34:50 info: do_lrm_rsc_op: Performing op=IPaddr_10_0_0_59_monitor_0 key=7:0:d66e5bec-f673-4648-85f1-de6226a15a97) crmd[3663]: 2007/09/20_22:34:50 info: do_lrm_rsc_op: Performing op=mysql_monitor_0 key=8:0:d66e5bec-f673-4648-85f1-de6226a15a97) crmd[3663]: 2007/09/20_22:34:50 info: do_lrm_rsc_op: Performing op=dw-httpd_monitor_0 key=9:0:d66e5bec-f673-4648-85f1-de6226a15a97) crmd[3663]: 2007/09/20_22:34:50 info: do_lrm_rsc_op: Performing op=msg_last_monitor_0 key=10:0:d66e5bec-f673-4648-85f1-de6226a15a97) crmd[3663]: 2007/09/20_22:34:50 info: do_lrm_rsc_op: Performing op=vmstat-mon-child:0_monitor_0 key=11:0:d66e5bec-f673-4648-85f1-de6226a15a97) crmd[3663]: 2007/09/20_22:34:50 info: do_lrm_rsc_op: Performing op=vmstat-mon-child:1_monitor_0 key=12:0:d66e5bec-f673-4648-85f1-de6226a15a97) crmd[3663]: 2007/09/20_22:34:50 info: do_lrm_rsc_op: Performing op=stonith-child:0_monitor_0 key=13:0:d66e5bec-f673-4648-85f1-de6226a15a97) crmd[3663]: 2007/09/20_22:34:51 info: do_lrm_rsc_op: Performing op=stonith-child:1_monitor_0 key=14:0:d66e5bec-f673-4648-85f1-de6226a15a97) crmd[3663]: 2007/09/20_22:34:51 info: process_lrm_event: LRM operation share-drbdlinks_monitor_0 (call=5, rc=7) complete cibmon[3657]: 2007/09/20_22:34:51 info: cibmon_diff: [cib_diff_notify] cib_update confirmed tengine[3682]: 2007/09/20_22:34:51 info: match_graph_event: Action share-drbdlinks_monitor_0 (6) confirmed on host-1.awale.qc.ca cibmon[3657]: 2007/09/20_22:34:51 info: log_cib_diff: cib_update: Diff: --- 0.12.5 cibmon[3657]: 2007/09/20_22:34:51 info: log_cib_diff: cib_update: Diff: +++ 0.12.6 cibmon[3657]: 2007/09/20_22:34:51 info: log_data_element: cib_update: - cibmon[3657]: 2007/09/20_22:34:51 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:51 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:51 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:51 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:51 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:51 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:51 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:51 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:51 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:51 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:51 info: log_data_element: cib_update: + drbd[3685][3711]: 2007/09/20_22:34:51 WARNING: r0 monitor: drbd module not loaded crmd[3663]: 2007/09/20_22:34:51 info: process_lrm_event: LRM operation share-drbd_0:0_monitor_0 (call=2, rc=7) complete tengine[3682]: 2007/09/20_22:34:51 info: match_graph_event: Action share-drbd_0:0_monitor_0 (3) confirmed on host-1.awale.qc.ca cibmon[3657]: 2007/09/20_22:34:51 info: cibmon_diff: [cib_diff_notify] cib_update confirmed cibmon[3657]: 2007/09/20_22:34:51 info: log_cib_diff: cib_update: Diff: --- 0.12.6 cibmon[3657]: 2007/09/20_22:34:51 info: log_cib_diff: cib_update: Diff: +++ 0.12.7 cibmon[3657]: 2007/09/20_22:34:51 info: log_data_element: cib_update: - cibmon[3657]: 2007/09/20_22:34:51 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:51 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:51 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:51 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:51 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:51 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:51 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:51 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:51 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:51 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:51 info: log_data_element: cib_update: + crmd[3663]: 2007/09/20_22:34:51 info: process_lrm_event: LRM operation msg_first_monitor_0 (call=3, rc=7) complete Filesystem[3687][3720]: 2007/09/20_22:34:51 WARNING: Couldn't find device [/dev/drbd0]. Expected /dev/??? to exist cibmon[3657]: 2007/09/20_22:34:51 info: cibmon_diff: [cib_diff_notify] cib_update confirmed tengine[3682]: 2007/09/20_22:34:51 info: match_graph_event: Action msg_first_monitor_0 (4) confirmed on host-1.awale.qc.ca cibmon[3657]: 2007/09/20_22:34:51 info: log_cib_diff: cib_update: Diff: --- 0.12.7 cibmon[3657]: 2007/09/20_22:34:51 info: log_cib_diff: cib_update: Diff: +++ 0.12.8 cibmon[3657]: 2007/09/20_22:34:51 info: log_data_element: cib_update: - cibmon[3657]: 2007/09/20_22:34:51 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:51 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:51 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:51 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:51 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:51 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:51 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:51 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:51 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:51 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:51 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:51 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:51 info: log_data_element: cib_update: + crmd[3663]: 2007/09/20_22:34:51 info: process_lrm_event: LRM operation share-FS_monitor_0 (call=4, rc=7) complete tengine[3682]: 2007/09/20_22:34:51 info: match_graph_event: Action share-FS_monitor_0 (5) confirmed on host-1.awale.qc.ca cibmon[3657]: 2007/09/20_22:34:51 info: cibmon_diff: [cib_diff_notify] cib_update confirmed cibmon[3657]: 2007/09/20_22:34:51 info: log_cib_diff: cib_update: Diff: --- 0.12.8 cibmon[3657]: 2007/09/20_22:34:51 info: log_cib_diff: cib_update: Diff: +++ 0.12.9 cibmon[3657]: 2007/09/20_22:34:51 info: log_data_element: cib_update: - cibmon[3657]: 2007/09/20_22:34:51 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:51 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:51 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:51 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:51 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:51 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:51 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:51 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:51 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:51 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:51 info: log_data_element: cib_update: + mysql[3751][3790]: 2007/09/20_22:34:52 DEBUG: MySQL is not running crmd[3663]: 2007/09/20_22:34:52 info: process_lrm_event: LRM operation mysql_monitor_0 (call=7, rc=7) complete tengine[3682]: 2007/09/20_22:34:52 info: match_graph_event: Action mysql_monitor_0 (8) confirmed on host-1.awale.qc.ca cibmon[3657]: 2007/09/20_22:34:52 info: cibmon_diff: [cib_diff_notify] cib_update confirmed cibmon[3657]: 2007/09/20_22:34:52 info: log_cib_diff: cib_update: Diff: --- 0.12.9 cibmon[3657]: 2007/09/20_22:34:52 info: log_cib_diff: cib_update: Diff: +++ 0.12.10 cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: - cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + crmd[3663]: 2007/09/20_22:34:52 info: process_lrm_event: LRM operation msg_last_monitor_0 (call=9, rc=7) complete cibmon[3657]: 2007/09/20_22:34:52 info: cibmon_diff: [cib_diff_notify] cib_update confirmed tengine[3682]: 2007/09/20_22:34:52 info: match_graph_event: Action msg_last_monitor_0 (10) confirmed on host-1.awale.qc.ca cibmon[3657]: 2007/09/20_22:34:52 info: log_cib_diff: cib_update: Diff: --- 0.12.10 cibmon[3657]: 2007/09/20_22:34:52 info: log_cib_diff: cib_update: Diff: +++ 0.12.11 cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: - cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + crmd[3663]: 2007/09/20_22:34:52 info: process_lrm_event: LRM operation IPaddr_10_0_0_59_monitor_0 (call=6, rc=7) complete cibmon[3657]: 2007/09/20_22:34:52 info: cibmon_diff: [cib_diff_notify] cib_update confirmed tengine[3682]: 2007/09/20_22:34:52 info: match_graph_event: Action IPaddr_10_0_0_59_monitor_0 (7) confirmed on host-1.awale.qc.ca cibmon[3657]: 2007/09/20_22:34:52 info: log_cib_diff: cib_update: Diff: --- 0.12.11 cibmon[3657]: 2007/09/20_22:34:52 info: log_cib_diff: cib_update: Diff: +++ 0.12.12 cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: - cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + crmd[3663]: 2007/09/20_22:34:52 info: process_lrm_event: LRM operation dw-httpd_monitor_0 (call=8, rc=7) complete cibmon[3657]: 2007/09/20_22:34:52 info: cibmon_diff: [cib_diff_notify] cib_update confirmed cibmon[3657]: 2007/09/20_22:34:52 info: log_cib_diff: cib_update: Diff: --- 0.12.12 tengine[3682]: 2007/09/20_22:34:52 info: match_graph_event: Action dw-httpd_monitor_0 (9) confirmed on host-1.awale.qc.ca cibmon[3657]: 2007/09/20_22:34:52 info: log_cib_diff: cib_update: Diff: +++ 0.12.13 cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: - cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:52 info: log_data_element: cib_update: + crmd[3663]: 2007/09/20_22:34:53 info: process_lrm_event: LRM operation stonith-child:0_monitor_0 (call=12, rc=7) complete cibmon[3657]: 2007/09/20_22:34:53 info: cibmon_diff: [cib_diff_notify] cib_update confirmed tengine[3682]: 2007/09/20_22:34:53 info: match_graph_event: Action stonith-child:0_monitor_0 (13) confirmed on host-1.awale.qc.ca cibmon[3657]: 2007/09/20_22:34:53 info: log_cib_diff: cib_update: Diff: --- 0.12.13 cibmon[3657]: 2007/09/20_22:34:53 info: log_cib_diff: cib_update: Diff: +++ 0.12.14 cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: - cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + crmd[3663]: 2007/09/20_22:34:53 info: process_lrm_event: LRM operation stonith-child:1_monitor_0 (call=13, rc=7) complete cibmon[3657]: 2007/09/20_22:34:53 info: cibmon_diff: [cib_diff_notify] cib_update confirmed tengine[3682]: 2007/09/20_22:34:53 info: match_graph_event: Action stonith-child:1_monitor_0 (14) confirmed on host-1.awale.qc.ca cibmon[3657]: 2007/09/20_22:34:53 info: log_cib_diff: cib_update: Diff: --- 0.12.14 cibmon[3657]: 2007/09/20_22:34:53 info: log_cib_diff: cib_update: Diff: +++ 0.12.15 cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: - cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + crmd[3663]: 2007/09/20_22:34:53 info: process_lrm_event: LRM operation vmstat-mon-child:1_monitor_0 (call=11, rc=7) complete cibmon[3657]: 2007/09/20_22:34:53 info: cibmon_diff: [cib_diff_notify] cib_update confirmed tengine[3682]: 2007/09/20_22:34:53 info: match_graph_event: Action vmstat-mon-child:1_monitor_0 (12) confirmed on host-1.awale.qc.ca cibmon[3657]: 2007/09/20_22:34:53 info: log_cib_diff: cib_update: Diff: --- 0.12.15 cibmon[3657]: 2007/09/20_22:34:53 info: log_cib_diff: cib_update: Diff: +++ 0.12.16 cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: - cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + crmd[3663]: 2007/09/20_22:34:53 info: process_lrm_event: LRM operation vmstat-mon-child:0_monitor_0 (call=10, rc=7) complete cibmon[3657]: 2007/09/20_22:34:53 info: cibmon_diff: [cib_diff_notify] cib_update confirmed tengine[3682]: 2007/09/20_22:34:53 info: match_graph_event: Action vmstat-mon-child:0_monitor_0 (11) confirmed on host-1.awale.qc.ca cibmon[3657]: 2007/09/20_22:34:53 info: log_cib_diff: cib_update: Diff: --- 0.12.16 tengine[3682]: 2007/09/20_22:34:53 info: send_rsc_command: Initiating action 2: probe_complete on host-1.awale.qc.ca cibmon[3657]: 2007/09/20_22:34:53 info: log_cib_diff: cib_update: Diff: +++ 0.12.17 tengine[3682]: 2007/09/20_22:34:53 info: te_pseudo_action: Pseudo action 1 fired and confirmed cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: - tengine[3682]: 2007/09/20_22:34:53 info: te_pseudo_action: Pseudo action 17 fired and confirmed cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + tengine[3682]: 2007/09/20_22:34:53 info: te_pseudo_action: Pseudo action 47 fired and confirmed cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + tengine[3682]: 2007/09/20_22:34:53 info: te_pseudo_action: Pseudo action 53 fired and confirmed cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + tengine[3682]: 2007/09/20_22:34:53 info: send_rsc_command: Initiating action 15: share-drbd_0:0_start_0 on host-1.awale.qc.ca cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + tengine[3682]: 2007/09/20_22:34:53 info: send_rsc_command: Initiating action 45: vmstat-mon-child:0_start_0 on host-1.awale.qc.ca cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + tengine[3682]: 2007/09/20_22:34:53 info: send_rsc_command: Initiating action 51: stonith-child:0_start_0 on host-1.awale.qc.ca cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + crmd[3663]: 2007/09/20_22:34:53 info: do_lrm_rsc_op: Performing op=share-drbd_0:0_start_0 key=15:0:d66e5bec-f673-4648-85f1-de6226a15a97) tengine[3682]: 2007/09/20_22:34:53 info: extract_event: Aborting on transient_attributes changes for 1da8804b-8a93-4145-9d40-1133699fe377 tengine[3682]: 2007/09/20_22:34:53 info: update_abort_priority: Abort priority upgraded to 1000000 tengine[3682]: 2007/09/20_22:34:53 info: update_abort_priority: Abort action 0 superceeded by 2 cibmon[3657]: 2007/09/20_22:34:53 info: cibmon_diff: [cib_diff_notify] cib_modify confirmed crmd[3663]: 2007/09/20_22:34:53 info: do_lrm_rsc_op: Performing op=vmstat-mon-child:0_start_0 key=45:0:d66e5bec-f673-4648-85f1-de6226a15a97) cibmon[3657]: 2007/09/20_22:34:53 info: log_cib_diff: cib_modify: Diff: --- 0.12.17 cibmon[3657]: 2007/09/20_22:34:53 info: log_cib_diff: cib_modify: Diff: +++ 0.12.18 cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_modify: - cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_modify: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_modify: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_modify: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_modify: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_modify: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_modify: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_modify: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_modify: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_modify: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_modify: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_modify: + crmd[3663]: 2007/09/20_22:34:53 info: do_lrm_rsc_op: Performing op=stonith-child:0_start_0 key=51:0:d66e5bec-f673-4648-85f1-de6226a15a97) cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_modify: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_modify: + lrmd[3830]: 2007/09/20_22:34:53 info: Try to start STONITH resource : Device=apcsnmp daemontools[3829][3842]: 2007/09/20_22:34:53 INFO vmstat-mon: Starting stonithd[3843]: 2007/09/20_22:34:53 info: 10.0.0.166 uses sysOID '.1.3.6.1.4.1.318.1.3.4.5', compatible with 'masterSwitchrPDU' stonithd[3843]: 2007/09/20_22:34:53 info: 10.0.0.166 has sysName 'RackPDU Archivage' crmd[3663]: 2007/09/20_22:34:53 info: process_lrm_event: LRM operation stonith-child:0_start_0 (call=16, rc=0) complete crmd[3663]: 2007/09/20_22:34:53 info: build_operation_update: Digest for 0:0;51:0:d66e5bec-f673-4648-85f1-de6226a15a97 (stonith-child:0_start_0) was a48b2b8308a00f9fb8a169e964b6419c crmd[3663]: 2007/09/20_22:34:53 info: log_data_element: build_operation_update: digest:source lrmd[3660]: 2007/09/20_22:34:53 debug: stonithRA plugin: provider attribute is not needed and will be ignored. lrmd[3660]: 2007/09/20_22:34:53 WARN: stonithRA plugin: cannot get shortdesc segment of apcsnmp's metadata. tengine[3682]: 2007/09/20_22:34:53 info: match_graph_event: Action stonith-child:0_start_0 (51) confirmed on host-1.awale.qc.ca tengine[3682]: 2007/09/20_22:34:53 info: te_pseudo_action: Pseudo action 54 fired and confirmed cibmon[3657]: 2007/09/20_22:34:53 info: cibmon_diff: [cib_diff_notify] cib_update confirmed cibmon[3657]: 2007/09/20_22:34:53 info: log_cib_diff: cib_update: Diff: --- 0.12.18 cibmon[3657]: 2007/09/20_22:34:53 info: log_cib_diff: cib_update: Diff: +++ 0.12.19 cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: - cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:53 info: log_data_element: cib_update: + lrmd[3660]: 2007/09/20_22:34:53 info: RA output: (share-drbd_0:0:start:stdout) r0: ACTION=start stonithd[3661]: 2007/09/20_22:34:53 info: 10.0.0.166 uses sysOID '.1.3.6.1.4.1.318.1.3.4.5', compatible with 'masterSwitchrPDU' stonithd[3661]: 2007/09/20_22:34:53 info: 10.0.0.166 has sysName 'RackPDU Archivage' daemontools[3829][3859]: 2007/09/20_22:34:53 INFO vmstat-mon: Giving vmstat 8 seconds drbd[3828][3863]: 2007/09/20_22:34:53 DEBUG: r0: Calling modprobe -s drbd minor_count=1 drbd[3828][3872]: 2007/09/20_22:34:53 DEBUG: r0: Exit code 0 drbd[3828][3874]: 2007/09/20_22:34:53 DEBUG: r0: Command output: lrmd[3660]: 2007/09/20_22:34:53 info: RA output: (share-drbd_0:0:start:stdout) drbd[3828][3876]: 2007/09/20_22:34:53 DEBUG: r0 start: Module loaded. daemontools[3829][3879]: 2007/09/20_22:34:53 INFO vmstat-mon: Waiting for vmstat drbd[3828][3878]: 2007/09/20_22:34:53 DEBUG: r0: Calling drbdadm -c /etc/drbd.conf state r0 stonithd[3661]: 2007/09/20_22:34:53 WARN: G_SIG_dispatch: Dispatch function for SIGCHLD took too long to execute: 380 ms (> 30 ms) (GSource: 0x8107cf0) drbd[3828][3887]: 2007/09/20_22:34:54 DEBUG: r0: Exit code 0 drbd[3828][3888]: 2007/09/20_22:34:54 DEBUG: r0: Command output: can not open /dev/drbd0: No such file or directory Command 'drbdsetup /dev/drbd0 state' terminated with exit code 20 drbdadm aborting drbd[3828][3896]: 2007/09/20_22:34:54 DEBUG: r0: Calling drbdadm -c /etc/drbd.conf cstate r0 drbd[3828][3909]: 2007/09/20_22:34:54 DEBUG: r0: Exit code 0 drbd[3828][3910]: 2007/09/20_22:34:54 DEBUG: r0: Command output: can not open /dev/drbd0: No such file or directory Command 'drbdsetup /dev/drbd0 cstate' terminated with exit code 20 drbdadm aborting drbd[3828][3911]: 2007/09/20_22:34:54 DEBUG: r0 status: can not open /dev/drbd0: No such file or directory Command 'drbdsetup /dev/drbd0 state' terminated with exit code 20 drbdadm aborting Not configured Not configured Unconfigured drbd[3828][3912]: 2007/09/20_22:34:54 DEBUG: r0: Calling drbdadm -c /etc/drbd.conf up r0 drbd[3828][3925]: 2007/09/20_22:34:54 DEBUG: r0: Exit code 0 drbd[3828][3926]: 2007/09/20_22:34:54 DEBUG: r0: Command output: can not open /dev/drbd0: No such file or directory Command 'drbdsetup /dev/drbd0 disk /dev/sdb1 internal -1 --on-io-error=panic' terminated with exit code 20 lrmd[3660]: 2007/09/20_22:34:54 info: RA output: (share-drbd_0:0:start:stdout) can not open /dev/drbd0: No such file or directory Command 'drbdsetup /dev/drbd0 disk /dev/sdb1 internal -1 --on-io-error=panic' terminated with exit code 20 drbd[3828][3928]: 2007/09/20_22:34:54 DEBUG: r0: Calling drbdadm -c /etc/drbd.conf state r0 drbd[3828][3935]: 2007/09/20_22:34:54 DEBUG: r0: Exit code 0 drbd[3828][3936]: 2007/09/20_22:34:54 DEBUG: r0: Command output: can not open /dev/drbd0: No such file or directory Command 'drbdsetup /dev/drbd0 state' terminated with exit code 20 drbdadm aborting drbd[3828][3945]: 2007/09/20_22:34:54 DEBUG: r0: Calling drbdadm -c /etc/drbd.conf cstate r0 drbd[3828][3958]: 2007/09/20_22:34:54 DEBUG: r0: Exit code 0 drbd[3828][3959]: 2007/09/20_22:34:54 DEBUG: r0: Command output: can not open /dev/drbd0: No such file or directory Command 'drbdsetup /dev/drbd0 cstate' terminated with exit code 20 drbdadm aborting drbd[3828][3960]: 2007/09/20_22:34:54 DEBUG: r0 status: can not open /dev/drbd0: No such file or directory Command 'drbdsetup /dev/drbd0 state' terminated with exit code 20 drbdadm aborting Not configured Not configured Unconfigured drbd[3828][3961]: 2007/09/20_22:34:54 ERROR: r0 start: not in Secondary mode after start. crmd[3663]: 2007/09/20_22:34:54 ERROR: process_lrm_event: LRM operation share-drbd_0:0_start_0 (call=14, rc=1) Error unknown error crmd[3663]: 2007/09/20_22:34:54 info: build_operation_update: Digest for 4:1;15:0:d66e5bec-f673-4648-85f1-de6226a15a97 (share-drbd_0:0_start_0) was c0e018b73fdf522b6cdd355e125af15e crmd[3663]: 2007/09/20_22:34:54 info: log_data_element: build_operation_update: digest:source cibmon[3657]: 2007/09/20_22:34:55 info: cibmon_diff: [cib_diff_notify] cib_update confirmed tengine[3682]: 2007/09/20_22:34:55 WARN: status_from_rc: Action start on host-1.awale.qc.ca failed (target: vs. rc: 1): Error cibmon[3657]: 2007/09/20_22:34:55 info: log_cib_diff: cib_update: Diff: --- 0.12.19 tengine[3682]: 2007/09/20_22:34:55 info: match_graph_event: Action share-drbd_0:0_start_0 (15) confirmed on host-1.awale.qc.ca cibmon[3657]: 2007/09/20_22:34:55 info: log_cib_diff: cib_update: Diff: +++ 0.12.20 cibmon[3657]: 2007/09/20_22:34:55 info: log_data_element: cib_update: - cibmon[3657]: 2007/09/20_22:34:55 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:55 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:55 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:55 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:55 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:55 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:55 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:55 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:55 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:55 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:55 info: log_data_element: cib_update: + daemontools[3829][3973]: 2007/09/20_22:34:55 INFO vmstat-mon: Waiting for vmstat crmd[3663]: 2007/09/20_22:34:56 info: process_lrm_event: LRM operation vmstat-mon-child:0_start_0 (call=15, rc=0) complete crmd[3663]: 2007/09/20_22:34:56 info: build_operation_update: Digest for 0:0;45:0:d66e5bec-f673-4648-85f1-de6226a15a97 (vmstat-mon-child:0_start_0) was 9691e01599b1b00c74019931daf4554d crmd[3663]: 2007/09/20_22:34:56 info: log_data_element: build_operation_update: digest:source tengine[3682]: 2007/09/20_22:34:56 info: match_graph_event: Action vmstat-mon-child:0_start_0 (45) confirmed on host-1.awale.qc.ca tengine[3682]: 2007/09/20_22:34:56 info: te_pseudo_action: Pseudo action 48 fired and confirmed cibmon[3657]: 2007/09/20_22:34:56 info: cibmon_diff: [cib_diff_notify] cib_update confirmed tengine[3682]: 2007/09/20_22:34:56 info: run_graph: ==================================================== cibmon[3657]: 2007/09/20_22:34:56 info: log_cib_diff: cib_update: Diff: --- 0.12.20 tengine[3682]: 2007/09/20_22:34:56 notice: run_graph: Transition 0: (Complete=24, Pending=0, Fired=0, Skipped=3, Incomplete=4) cibmon[3657]: 2007/09/20_22:34:56 info: log_cib_diff: cib_update: Diff: +++ 0.12.21 cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: - crmd[3663]: 2007/09/20_22:34:56 info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_IPC_MESSAGE origin=route_message ] cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: + crmd[3663]: 2007/09/20_22:34:56 info: do_state_transition: All 1 cluster nodes are eligible to run resources. cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: + pengine[3683]: 2007/09/20_22:34:56 notice: cluster_option: Using default value 'stop' for cluster option 'no-quorum-policy' pengine[3683]: 2007/09/20_22:34:56 notice: cluster_option: Using default value 'true' for cluster option 'symmetric-cluster' pengine[3683]: 2007/09/20_22:34:56 notice: cluster_option: Using default value 'false' for cluster option 'stonith-enabled' pengine[3683]: 2007/09/20_22:34:56 notice: cluster_option: Using default value 'reboot' for cluster option 'stonith-action' pengine[3683]: 2007/09/20_22:34:56 notice: cluster_option: Using default value '0' for cluster option 'default-resource-stickiness' pengine[3683]: 2007/09/20_22:34:56 notice: cluster_option: Using default value '0' for cluster option 'default-resource-failure-stickiness' pengine[3683]: 2007/09/20_22:34:56 notice: cluster_option: Using default value 'true' for cluster option 'is-managed-default' pengine[3683]: 2007/09/20_22:34:56 notice: cluster_option: Using default value '60s' for cluster option 'cluster-delay' pengine[3683]: 2007/09/20_22:34:56 notice: cluster_option: Using default value '20s' for cluster option 'default-action-timeout' pengine[3683]: 2007/09/20_22:34:56 notice: cluster_option: Using default value 'true' for cluster option 'stop-orphan-resources' pengine[3683]: 2007/09/20_22:34:56 notice: cluster_option: Using default value 'true' for cluster option 'stop-orphan-actions' pengine[3683]: 2007/09/20_22:34:56 notice: cluster_option: Using default value 'false' for cluster option 'remove-after-stop' pengine[3683]: 2007/09/20_22:34:56 notice: cluster_option: Using default value '-1' for cluster option 'pe-error-series-max' pengine[3683]: 2007/09/20_22:34:56 notice: cluster_option: Using default value '-1' for cluster option 'pe-warn-series-max' pengine[3683]: 2007/09/20_22:34:56 notice: cluster_option: Using default value '-1' for cluster option 'pe-input-series-max' pengine[3683]: 2007/09/20_22:34:56 notice: cluster_option: Using default value 'true' for cluster option 'startup-fencing' pengine[3683]: 2007/09/20_22:34:56 info: determine_online_status: Node host-1.awale.qc.ca is online pengine[3683]: 2007/09/20_22:34:56 WARN: unpack_rsc_op: Processing failed op (share-drbd_0:0_start_0) on host-1.awale.qc.ca pengine[3683]: 2007/09/20_22:34:56 WARN: unpack_rsc_op: Handling failed start for share-drbd_0:0 on host-1.awale.qc.ca pengine[3683]: 2007/09/20_22:34:56 info: clone_print: Master/Slave Set: ms-drbd_0 pengine[3683]: 2007/09/20_22:34:56 info: native_print: share-drbd_0:0 (heartbeat::ocf:drbd): Started host-1.awale.qc.ca FAILED pengine[3683]: 2007/09/20_22:34:56 info: native_print: share-drbd_0:1 (heartbeat::ocf:drbd): Stopped pengine[3683]: 2007/09/20_22:34:56 info: group_print: Resource Group: DW pengine[3683]: 2007/09/20_22:34:56 info: native_print: msg_first (heartbeat:msg): Stopped crmd[3663]: 2007/09/20_22:34:56 info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=route_message ] pengine[3683]: 2007/09/20_22:34:56 info: native_print: share-FS (heartbeat::ocf:Filesystem): Stopped pengine[3683]: 2007/09/20_22:34:56 info: native_print: share-drbdlinks (heartbeat:drbdlinks): Stopped pengine[3683]: 2007/09/20_22:34:56 info: native_print: IPaddr_10_0_0_59 (heartbeat::ocf:IPaddr): Stopped pengine[3683]: 2007/09/20_22:34:56 info: native_print: mysql (heartbeat::ocf:mysql): Stopped pengine[3683]: 2007/09/20_22:34:56 info: native_print: dw-httpd (awale::ocf:daemontools): Stopped pengine[3683]: 2007/09/20_22:34:56 info: native_print: msg_last (heartbeat:msg): Stopped pengine[3683]: 2007/09/20_22:34:56 info: clone_print: Clone Set: vmstat-mon pengine[3683]: 2007/09/20_22:34:56 info: native_print: vmstat-mon-child:0 (awale::ocf:daemontools): Started host-1.awale.qc.ca pengine[3683]: 2007/09/20_22:34:56 info: native_print: vmstat-mon-child:1 (awale::ocf:daemontools): Stopped pengine[3683]: 2007/09/20_22:34:56 info: clone_print: Clone Set: stonith tengine[3682]: 2007/09/20_22:34:56 info: unpack_graph: Unpacked transition 1: 19 actions in 19 synapses pengine[3683]: 2007/09/20_22:34:56 info: native_print: stonith-child:0 (stonith:apcsnmp): Started host-1.awale.qc.ca tengine[3682]: 2007/09/20_22:34:56 info: te_pseudo_action: Pseudo action 14 fired and confirmed pengine[3683]: 2007/09/20_22:34:56 info: native_print: stonith-child:1 (stonith:apcsnmp): Stopped tengine[3682]: 2007/09/20_22:34:56 info: send_rsc_command: Initiating action 36: vmstat-mon-child:0_monitor_120000 on host-1.awale.qc.ca pengine[3683]: 2007/09/20_22:34:56 WARN: native_color: Resource share-drbd_0:0 cannot run anywhere tengine[3682]: 2007/09/20_22:34:56 info: send_rsc_command: Initiating action 43: stonith-child:0_monitor_50000 on host-1.awale.qc.ca pengine[3683]: 2007/09/20_22:34:56 info: master_color: Promoted 0 instances of a possible 1 to master tengine[3682]: 2007/09/20_22:34:56 info: send_rsc_command: Initiating action 72: share-drbd_0:0_pre_notify_stop_0 on host-1.awale.qc.ca pengine[3683]: 2007/09/20_22:34:56 notice: StopRsc: host-1.awale.qc.ca Stop share-drbd_0:0 crmd[3663]: 2007/09/20_22:34:56 info: do_lrm_rsc_op: Performing op=vmstat-mon-child:0_monitor_120000 key=36:1:d66e5bec-f673-4648-85f1-de6226a15a97) pengine[3683]: 2007/09/20_22:34:56 notice: StartRsc: host-1.awale.qc.ca Start share-drbd_0:1 pengine[3683]: 2007/09/20_22:34:56 notice: RecurringOp: host-1.awale.qc.ca share-drbd_0:1_monitor_120000 pengine[3683]: 2007/09/20_22:34:56 notice: StopRsc: host-1.awale.qc.ca Stop share-drbd_0:0 pengine[3683]: 2007/09/20_22:34:56 notice: StartRsc: host-1.awale.qc.ca Start share-drbd_0:1 pengine[3683]: 2007/09/20_22:34:56 notice: RecurringOp: host-1.awale.qc.ca share-drbd_0:1_monitor_120000 pengine[3683]: 2007/09/20_22:34:56 info: master_color: Promoted 0 instances of a possible 1 to master pengine[3683]: 2007/09/20_22:34:56 info: master_color: Promoted 0 instances of a possible 1 to master pengine[3683]: 2007/09/20_22:34:56 WARN: native_color: Resource msg_first cannot run anywhere pengine[3683]: 2007/09/20_22:34:56 info: master_color: Promoted 0 instances of a possible 1 to master pengine[3683]: 2007/09/20_22:34:56 info: master_color: Promoted 0 instances of a possible 1 to master pengine[3683]: 2007/09/20_22:34:56 WARN: native_color: Resource share-FS cannot run anywhere pengine[3683]: 2007/09/20_22:34:56 WARN: native_color: Resource share-drbdlinks cannot run anywhere pengine[3683]: 2007/09/20_22:34:56 WARN: native_color: Resource IPaddr_10_0_0_59 cannot run anywhere pengine[3683]: 2007/09/20_22:34:56 WARN: native_color: Resource mysql cannot run anywhere pengine[3683]: 2007/09/20_22:34:56 WARN: native_color: Resource dw-httpd cannot run anywhere crmd[3663]: 2007/09/20_22:34:56 info: do_lrm_rsc_op: Performing op=stonith-child:0_monitor_50000 key=43:1:d66e5bec-f673-4648-85f1-de6226a15a97) pengine[3683]: 2007/09/20_22:34:56 WARN: native_color: Resource msg_last cannot run anywhere pengine[3683]: 2007/09/20_22:34:56 WARN: native_color: Resource vmstat-mon-child:1 cannot run anywhere pengine[3683]: 2007/09/20_22:34:56 notice: NoRoleChange: Leave resource vmstat-mon-child:0 (host-1.awale.qc.ca) pengine[3683]: 2007/09/20_22:34:56 notice: RecurringOp: host-1.awale.qc.ca vmstat-mon-child:0_monitor_120000 pengine[3683]: 2007/09/20_22:34:56 WARN: native_color: Resource stonith-child:1 cannot run anywhere pengine[3683]: 2007/09/20_22:34:56 notice: NoRoleChange: Leave resource stonith-child:0 (host-1.awale.qc.ca) pengine[3683]: 2007/09/20_22:34:56 notice: RecurringOp: host-1.awale.qc.ca stonith-child:0_monitor_50000 pengine[3683]: 2007/09/20_22:34:56 WARN: process_pe_message: Transition 1: WARNINGs found during PE processing. PEngine Input stored in: /var/lib/heartbeat/pengine/pe-warn-486.bz2 pengine[3683]: 2007/09/20_22:34:56 info: process_pe_message: Configuration WARNINGs found during PE processing. Please run "crm_verify -L" to identify issues. crmd[3663]: 2007/09/20_22:34:56 info: do_lrm_rsc_op: Performing op=share-drbd_0:0_notify_0 key=72:1:d66e5bec-f673-4648-85f1-de6226a15a97) stonithd[3997]: 2007/09/20_22:34:56 info: 10.0.0.166 uses sysOID '.1.3.6.1.4.1.318.1.3.4.5', compatible with 'masterSwitchrPDU' stonithd[3997]: 2007/09/20_22:34:56 info: 10.0.0.166 has sysName 'RackPDU Archivage' crmd[3663]: 2007/09/20_22:34:56 info: process_lrm_event: LRM operation stonith-child:0_monitor_50000 (call=18, rc=0) complete cibmon[3657]: 2007/09/20_22:34:56 info: cibmon_diff: [cib_diff_notify] cib_update confirmed tengine[3682]: 2007/09/20_22:34:56 info: match_graph_event: Action stonith-child:0_monitor_50000 (43) confirmed on host-1.awale.qc.ca cibmon[3657]: 2007/09/20_22:34:56 info: log_cib_diff: cib_update: Diff: --- 0.12.21 cibmon[3657]: 2007/09/20_22:34:56 info: log_cib_diff: cib_update: Diff: +++ 0.12.22 cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: - cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: + lrmd[3660]: 2007/09/20_22:34:56 info: RA output: (share-drbd_0:0:notify:stdout) r0: ACTION=notify crmd[3663]: 2007/09/20_22:34:56 info: process_lrm_event: LRM operation vmstat-mon-child:0_monitor_120000 (call=17, rc=0) complete drbd[3999][4021]: 2007/09/20_22:34:56 DEBUG: r0 notify: pre for stop - counts: active 0 - starting 1 - stopping 1 crmd[3663]: 2007/09/20_22:34:56 info: process_lrm_event: LRM operation share-drbd_0:0_notify_0 (call=19, rc=0) complete cibmon[3657]: 2007/09/20_22:34:56 info: cibmon_diff: [cib_diff_notify] cib_update confirmed tengine[3682]: 2007/09/20_22:34:56 info: match_graph_event: Action vmstat-mon-child:0_monitor_120000 (36) confirmed on host-1.awale.qc.ca cibmon[3657]: 2007/09/20_22:34:56 info: log_cib_diff: cib_update: Diff: --- 0.12.22 cibmon[3657]: 2007/09/20_22:34:56 info: log_cib_diff: cib_update: Diff: +++ 0.12.23 cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: - cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:56 info: cibmon_diff: [cib_diff_notify] cib_update confirmed tengine[3682]: 2007/09/20_22:34:56 info: match_graph_event: Action share-drbd_0:0_pre_notify_stop_0 (72) confirmed on host-1.awale.qc.ca cibmon[3657]: 2007/09/20_22:34:56 info: log_cib_diff: cib_update: Diff: --- 0.12.23 tengine[3682]: 2007/09/20_22:34:56 info: te_pseudo_action: Pseudo action 15 fired and confirmed cibmon[3657]: 2007/09/20_22:34:56 info: log_cib_diff: cib_update: Diff: +++ 0.12.24 tengine[3682]: 2007/09/20_22:34:56 info: te_pseudo_action: Pseudo action 12 fired and confirmed cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: - tengine[3682]: 2007/09/20_22:34:56 info: send_rsc_command: Initiating action 1: share-drbd_0:0_stop_0 on host-1.awale.qc.ca cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:56 info: log_data_element: cib_update: + crmd[3663]: 2007/09/20_22:34:56 info: do_lrm_rsc_op: Performing op=share-drbd_0:0_stop_0 key=1:1:d66e5bec-f673-4648-85f1-de6226a15a97) lrmd[3660]: 2007/09/20_22:34:56 info: RA output: (share-drbd_0:0:stop:stdout) r0: ACTION=stop drbd[4022][4034]: 2007/09/20_22:34:56 DEBUG: r0: Calling drbdadm -c /etc/drbd.conf state r0 drbd[4022][4041]: 2007/09/20_22:34:57 DEBUG: r0: Exit code 0 drbd[4022][4042]: 2007/09/20_22:34:57 DEBUG: r0: Command output: can not open /dev/drbd0: No such file or directory Command 'drbdsetup /dev/drbd0 state' terminated with exit code 20 drbdadm aborting drbd[4022][4050]: 2007/09/20_22:34:57 DEBUG: r0: Calling drbdadm -c /etc/drbd.conf cstate r0 drbd[4022][4063]: 2007/09/20_22:34:57 DEBUG: r0: Exit code 0 drbd[4022][4064]: 2007/09/20_22:34:57 DEBUG: r0: Command output: can not open /dev/drbd0: No such file or directory Command 'drbdsetup /dev/drbd0 cstate' terminated with exit code 20 drbdadm aborting drbd[4022][4065]: 2007/09/20_22:34:57 DEBUG: r0 status: can not open /dev/drbd0: No such file or directory Command 'drbdsetup /dev/drbd0 state' terminated with exit code 20 drbdadm aborting Not configured Not configured Unconfigured drbd[4022][4066]: 2007/09/20_22:34:57 DEBUG: r0: Calling /usr/sbin/crm_master -D crm_master[4068]: 2007/09/20_22:34:57 info: Invoked: /usr/sbin/crm_master -D cibmon[3657]: 2007/09/20_22:34:57 info: cibmon_diff: [cib_diff_notify] cib_delete confirmed cibmon[3657]: 2007/09/20_22:34:57 info: log_cib_diff: cib_delete: Diff: --- 0.12.24 cibmon[3657]: 2007/09/20_22:34:57 info: log_cib_diff: cib_delete: Diff: +++ 0.12.25 cibmon[3657]: 2007/09/20_22:34:57 info: log_data_element: cib_delete: - cibmon[3657]: 2007/09/20_22:34:57 info: log_data_element: cib_delete: + drbd[4022][4069]: 2007/09/20_22:34:57 DEBUG: r0: Exit code 0 drbd[4022][4070]: 2007/09/20_22:34:57 DEBUG: r0: Command output: crm_master[4068]: 2007/09/20_22:34:57 info: Invoked: /usr/sbin/crm_master -D lrmd[3660]: 2007/09/20_22:34:57 info: RA output: (share-drbd_0:0:stop:stdout) crm_master[4068]: 2007/09/20_22:34:57 info: Invoked: /usr/sbin/crm_master -D drbd[4022][4071]: 2007/09/20_22:34:57 DEBUG: r0 stop: already unconfigured. crmd[3663]: 2007/09/20_22:34:57 info: process_lrm_event: LRM operation share-drbd_0:0_stop_0 (call=20, rc=0) complete cibmon[3657]: 2007/09/20_22:34:57 info: cibmon_diff: [cib_diff_notify] cib_update confirmed tengine[3682]: 2007/09/20_22:34:57 info: match_graph_event: Action share-drbd_0:0_stop_0 (1) confirmed on host-1.awale.qc.ca cibmon[3657]: 2007/09/20_22:34:57 info: log_cib_diff: cib_update: Diff: --- 0.12.25 tengine[3682]: 2007/09/20_22:34:57 info: te_pseudo_action: Pseudo action 13 fired and confirmed cibmon[3657]: 2007/09/20_22:34:57 info: log_cib_diff: cib_update: Diff: +++ 0.12.26 tengine[3682]: 2007/09/20_22:34:57 info: te_pseudo_action: Pseudo action 16 fired and confirmed cibmon[3657]: 2007/09/20_22:34:57 info: log_data_element: cib_update: - tengine[3682]: 2007/09/20_22:34:57 info: te_pseudo_action: Pseudo action 17 fired and confirmed cibmon[3657]: 2007/09/20_22:34:57 info: log_data_element: cib_update: + tengine[3682]: 2007/09/20_22:34:57 info: te_pseudo_action: Pseudo action 8 fired and confirmed cibmon[3657]: 2007/09/20_22:34:57 info: log_data_element: cib_update: + tengine[3682]: 2007/09/20_22:34:57 info: te_pseudo_action: Pseudo action 9 fired and confirmed cibmon[3657]: 2007/09/20_22:34:57 info: log_data_element: cib_update: + tengine[3682]: 2007/09/20_22:34:57 info: te_pseudo_action: Pseudo action 6 fired and confirmed cibmon[3657]: 2007/09/20_22:34:57 info: log_data_element: cib_update: + tengine[3682]: 2007/09/20_22:34:57 info: send_rsc_command: Initiating action 4: share-drbd_0:1_start_0 on host-1.awale.qc.ca cibmon[3657]: 2007/09/20_22:34:57 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:57 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:57 info: log_data_element: cib_update: + crmd[3663]: 2007/09/20_22:34:57 info: do_lrm_rsc_op: Performing op=share-drbd_0:1_start_0 key=4:1:d66e5bec-f673-4648-85f1-de6226a15a97) cibmon[3657]: 2007/09/20_22:34:57 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:57 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:57 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:57 info: log_data_element: cib_update: + lrmd[3660]: 2007/09/20_22:34:57 info: RA output: (share-drbd_0:1:start:stdout) r0: ACTION=start drbd[4072][4085]: 2007/09/20_22:34:57 DEBUG: r0: Calling drbdadm -c /etc/drbd.conf state r0 drbd[4072][4092]: 2007/09/20_22:34:57 DEBUG: r0: Exit code 0 drbd[4072][4093]: 2007/09/20_22:34:57 DEBUG: r0: Command output: can not open /dev/drbd0: No such file or directory Command 'drbdsetup /dev/drbd0 state' terminated with exit code 20 drbdadm aborting drbd[4072][4101]: 2007/09/20_22:34:57 DEBUG: r0: Calling drbdadm -c /etc/drbd.conf cstate r0 drbd[4072][4114]: 2007/09/20_22:34:58 DEBUG: r0: Exit code 0 drbd[4072][4115]: 2007/09/20_22:34:58 DEBUG: r0: Command output: can not open /dev/drbd0: No such file or directory Command 'drbdsetup /dev/drbd0 cstate' terminated with exit code 20 drbdadm aborting drbd[4072][4116]: 2007/09/20_22:34:58 DEBUG: r0 status: can not open /dev/drbd0: No such file or directory Command 'drbdsetup /dev/drbd0 state' terminated with exit code 20 drbdadm aborting Not configured Not configured Unconfigured drbd[4072][4117]: 2007/09/20_22:34:58 DEBUG: r0: Calling drbdadm -c /etc/drbd.conf up r0 drbd[4072][4130]: 2007/09/20_22:34:58 DEBUG: r0: Exit code 0 drbd[4072][4131]: 2007/09/20_22:34:58 DEBUG: r0: Command output: can not open /dev/drbd0: No such file or directory Command 'drbdsetup /dev/drbd0 disk /dev/sdb1 internal -1 --on-io-error=panic' terminated with exit code 20 lrmd[3660]: 2007/09/20_22:34:58 info: RA output: (share-drbd_0:1:start:stdout) can not open /dev/drbd0: No such file or directory Command 'drbdsetup /dev/drbd0 disk /dev/sdb1 internal -1 --on-io-error=panic' terminated with exit code 20 drbd[4072][4133]: 2007/09/20_22:34:58 DEBUG: r0: Calling drbdadm -c /etc/drbd.conf state r0 drbd[4072][4140]: 2007/09/20_22:34:58 DEBUG: r0: Exit code 0 drbd[4072][4141]: 2007/09/20_22:34:58 DEBUG: r0: Command output: can not open /dev/drbd0: No such file or directory Command 'drbdsetup /dev/drbd0 state' terminated with exit code 20 drbdadm aborting drbd[4072][4149]: 2007/09/20_22:34:58 DEBUG: r0: Calling drbdadm -c /etc/drbd.conf cstate r0 drbd[4072][4162]: 2007/09/20_22:34:58 DEBUG: r0: Exit code 0 drbd[4072][4163]: 2007/09/20_22:34:58 DEBUG: r0: Command output: can not open /dev/drbd0: No such file or directory Command 'drbdsetup /dev/drbd0 cstate' terminated with exit code 20 drbdadm aborting drbd[4072][4164]: 2007/09/20_22:34:58 DEBUG: r0 status: can not open /dev/drbd0: No such file or directory Command 'drbdsetup /dev/drbd0 state' terminated with exit code 20 drbdadm aborting Not configured Not configured Unconfigured drbd[4072][4165]: 2007/09/20_22:34:58 ERROR: r0 start: not in Secondary mode after start. crmd[3663]: 2007/09/20_22:34:58 ERROR: process_lrm_event: LRM operation share-drbd_0:1_start_0 (call=21, rc=1) Error unknown error crmd[3663]: 2007/09/20_22:34:58 info: build_operation_update: Digest for 4:1;4:1:d66e5bec-f673-4648-85f1-de6226a15a97 (share-drbd_0:1_start_0) was c0e018b73fdf522b6cdd355e125af15e crmd[3663]: 2007/09/20_22:34:58 info: log_data_element: build_operation_update: digest:source tengine[3682]: 2007/09/20_22:34:58 WARN: status_from_rc: Action start on host-1.awale.qc.ca failed (target: vs. rc: 1): Error cibmon[3657]: 2007/09/20_22:34:58 info: cibmon_diff: [cib_diff_notify] cib_update confirmed cibmon[3657]: 2007/09/20_22:34:58 info: log_cib_diff: cib_update: Diff: --- 0.12.26 cibmon[3657]: 2007/09/20_22:34:58 info: log_cib_diff: cib_update: Diff: +++ 0.12.27 cibmon[3657]: 2007/09/20_22:34:58 info: log_data_element: cib_update: - cibmon[3657]: 2007/09/20_22:34:58 info: log_data_element: cib_update: + tengine[3682]: 2007/09/20_22:34:58 info: update_abort_priority: Abort priority upgraded to 1 cibmon[3657]: 2007/09/20_22:34:58 info: log_data_element: cib_update: + tengine[3682]: 2007/09/20_22:34:58 info: update_abort_priority: Abort action 0 superceeded by 2 cibmon[3657]: 2007/09/20_22:34:58 info: log_data_element: cib_update: + crmd[3663]: 2007/09/20_22:34:58 info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_IPC_MESSAGE origin=route_message ] tengine[3682]: 2007/09/20_22:34:58 info: match_graph_event: Action share-drbd_0:1_start_0 (4) confirmed on host-1.awale.qc.ca cibmon[3657]: 2007/09/20_22:34:58 info: log_data_element: cib_update: + crmd[3663]: 2007/09/20_22:34:58 info: do_state_transition: All 1 cluster nodes are eligible to run resources. tengine[3682]: 2007/09/20_22:34:58 info: run_graph: ==================================================== cibmon[3657]: 2007/09/20_22:34:58 info: log_data_element: cib_update: + tengine[3682]: 2007/09/20_22:34:58 notice: run_graph: Transition 1: (Complete=14, Pending=0, Fired=0, Skipped=1, Incomplete=4) cibmon[3657]: 2007/09/20_22:34:58 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:58 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:58 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:58 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:58 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:58 info: log_data_element: cib_update: + pengine[3683]: 2007/09/20_22:34:58 notice: cluster_option: Using default value 'stop' for cluster option 'no-quorum-policy' pengine[3683]: 2007/09/20_22:34:58 notice: cluster_option: Using default value 'true' for cluster option 'symmetric-cluster' pengine[3683]: 2007/09/20_22:34:58 notice: cluster_option: Using default value 'false' for cluster option 'stonith-enabled' pengine[3683]: 2007/09/20_22:34:58 notice: cluster_option: Using default value 'reboot' for cluster option 'stonith-action' pengine[3683]: 2007/09/20_22:34:58 notice: cluster_option: Using default value '0' for cluster option 'default-resource-stickiness' pengine[3683]: 2007/09/20_22:34:58 notice: cluster_option: Using default value '0' for cluster option 'default-resource-failure-stickiness' pengine[3683]: 2007/09/20_22:34:58 notice: cluster_option: Using default value 'true' for cluster option 'is-managed-default' pengine[3683]: 2007/09/20_22:34:58 notice: cluster_option: Using default value '60s' for cluster option 'cluster-delay' pengine[3683]: 2007/09/20_22:34:58 notice: cluster_option: Using default value '20s' for cluster option 'default-action-timeout' pengine[3683]: 2007/09/20_22:34:58 notice: cluster_option: Using default value 'true' for cluster option 'stop-orphan-resources' pengine[3683]: 2007/09/20_22:34:58 notice: cluster_option: Using default value 'true' for cluster option 'stop-orphan-actions' pengine[3683]: 2007/09/20_22:34:58 notice: cluster_option: Using default value 'false' for cluster option 'remove-after-stop' pengine[3683]: 2007/09/20_22:34:58 notice: cluster_option: Using default value '-1' for cluster option 'pe-error-series-max' pengine[3683]: 2007/09/20_22:34:58 notice: cluster_option: Using default value '-1' for cluster option 'pe-warn-series-max' pengine[3683]: 2007/09/20_22:34:58 notice: cluster_option: Using default value '-1' for cluster option 'pe-input-series-max' pengine[3683]: 2007/09/20_22:34:58 notice: cluster_option: Using default value 'true' for cluster option 'startup-fencing' pengine[3683]: 2007/09/20_22:34:58 info: determine_online_status: Node host-1.awale.qc.ca is online pengine[3683]: 2007/09/20_22:34:58 WARN: unpack_rsc_op: Processing failed op (share-drbd_0:0_start_0) on host-1.awale.qc.ca pengine[3683]: 2007/09/20_22:34:58 WARN: unpack_rsc_op: Handling failed start for share-drbd_0:0 on host-1.awale.qc.ca crmd[3663]: 2007/09/20_22:34:58 info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=route_message ] pengine[3683]: 2007/09/20_22:34:58 WARN: unpack_rsc_op: Processing failed op (share-drbd_0:1_start_0) on host-1.awale.qc.ca pengine[3683]: 2007/09/20_22:34:58 WARN: unpack_rsc_op: Handling failed start for share-drbd_0:1 on host-1.awale.qc.ca pengine[3683]: 2007/09/20_22:34:58 info: clone_print: Master/Slave Set: ms-drbd_0 tengine[3682]: 2007/09/20_22:34:58 info: unpack_graph: Unpacked transition 2: 8 actions in 8 synapses pengine[3683]: 2007/09/20_22:34:58 info: native_print: share-drbd_0:0 (heartbeat::ocf:drbd): Stopped tengine[3682]: 2007/09/20_22:34:58 info: te_pseudo_action: Pseudo action 14 fired and confirmed pengine[3683]: 2007/09/20_22:34:58 info: native_print: share-drbd_0:1 (heartbeat::ocf:drbd): Started host-1.awale.qc.ca FAILED tengine[3682]: 2007/09/20_22:34:58 info: send_rsc_command: Initiating action 71: share-drbd_0:1_pre_notify_stop_0 on host-1.awale.qc.ca pengine[3683]: 2007/09/20_22:34:58 info: group_print: Resource Group: DW pengine[3683]: 2007/09/20_22:34:58 info: native_print: msg_first (heartbeat:msg): Stopped pengine[3683]: 2007/09/20_22:34:58 info: native_print: share-FS (heartbeat::ocf:Filesystem): Stopped pengine[3683]: 2007/09/20_22:34:58 info: native_print: share-drbdlinks (heartbeat:drbdlinks): Stopped pengine[3683]: 2007/09/20_22:34:58 info: native_print: IPaddr_10_0_0_59 (heartbeat::ocf:IPaddr): Stopped pengine[3683]: 2007/09/20_22:34:58 info: native_print: mysql (heartbeat::ocf:mysql): Stopped pengine[3683]: 2007/09/20_22:34:58 info: native_print: dw-httpd (awale::ocf:daemontools): Stopped pengine[3683]: 2007/09/20_22:34:58 info: native_print: msg_last (heartbeat:msg): Stopped pengine[3683]: 2007/09/20_22:34:58 info: clone_print: Clone Set: vmstat-mon pengine[3683]: 2007/09/20_22:34:58 info: native_print: vmstat-mon-child:0 (awale::ocf:daemontools): Started host-1.awale.qc.ca pengine[3683]: 2007/09/20_22:34:58 info: native_print: vmstat-mon-child:1 (awale::ocf:daemontools): Stopped pengine[3683]: 2007/09/20_22:34:58 info: clone_print: Clone Set: stonith pengine[3683]: 2007/09/20_22:34:58 info: native_print: stonith-child:0 (stonith:apcsnmp): Started host-1.awale.qc.ca crmd[3663]: 2007/09/20_22:34:58 info: do_lrm_rsc_op: Performing op=share-drbd_0:1_notify_0 key=71:2:d66e5bec-f673-4648-85f1-de6226a15a97) pengine[3683]: 2007/09/20_22:34:58 info: native_print: stonith-child:1 (stonith:apcsnmp): Stopped pengine[3683]: 2007/09/20_22:34:58 WARN: native_color: Resource share-drbd_0:1 cannot run anywhere pengine[3683]: 2007/09/20_22:34:58 WARN: native_color: Resource share-drbd_0:0 cannot run anywhere pengine[3683]: 2007/09/20_22:34:58 info: master_color: Promoted 0 instances of a possible 1 to master pengine[3683]: 2007/09/20_22:34:58 notice: StopRsc: host-1.awale.qc.ca Stop share-drbd_0:1 pengine[3683]: 2007/09/20_22:34:58 notice: StopRsc: host-1.awale.qc.ca Stop share-drbd_0:1 pengine[3683]: 2007/09/20_22:34:58 info: master_color: Promoted 0 instances of a possible 1 to master pengine[3683]: 2007/09/20_22:34:58 info: master_color: Promoted 0 instances of a possible 1 to master pengine[3683]: 2007/09/20_22:34:58 WARN: native_color: Resource msg_first cannot run anywhere pengine[3683]: 2007/09/20_22:34:58 info: master_color: Promoted 0 instances of a possible 1 to master pengine[3683]: 2007/09/20_22:34:58 info: master_color: Promoted 0 instances of a possible 1 to master pengine[3683]: 2007/09/20_22:34:58 WARN: native_color: Resource share-FS cannot run anywhere pengine[3683]: 2007/09/20_22:34:58 WARN: native_color: Resource share-drbdlinks cannot run anywhere pengine[3683]: 2007/09/20_22:34:58 WARN: native_color: Resource IPaddr_10_0_0_59 cannot run anywhere pengine[3683]: 2007/09/20_22:34:58 WARN: native_color: Resource mysql cannot run anywhere pengine[3683]: 2007/09/20_22:34:58 WARN: native_color: Resource dw-httpd cannot run anywhere pengine[3683]: 2007/09/20_22:34:58 WARN: native_color: Resource msg_last cannot run anywhere pengine[3683]: 2007/09/20_22:34:58 WARN: native_color: Resource vmstat-mon-child:1 cannot run anywhere pengine[3683]: 2007/09/20_22:34:58 notice: NoRoleChange: Leave resource vmstat-mon-child:0 (host-1.awale.qc.ca) pengine[3683]: 2007/09/20_22:34:58 WARN: native_color: Resource stonith-child:1 cannot run anywhere pengine[3683]: 2007/09/20_22:34:58 notice: NoRoleChange: Leave resource stonith-child:0 (host-1.awale.qc.ca) pengine[3683]: 2007/09/20_22:34:58 WARN: process_pe_message: Transition 2: WARNINGs found during PE processing. PEngine Input stored in: /var/lib/heartbeat/pengine/pe-warn-487.bz2 pengine[3683]: 2007/09/20_22:34:58 info: process_pe_message: Configuration WARNINGs found during PE processing. Please run "crm_verify -L" to identify issues. lrmd[3660]: 2007/09/20_22:34:58 info: RA output: (share-drbd_0:1:notify:stdout) r0: ACTION=notify drbd[4166][4178]: 2007/09/20_22:34:58 DEBUG: r0 notify: pre for stop - counts: active 0 - starting 1 - stopping 1 crmd[3663]: 2007/09/20_22:34:58 info: process_lrm_event: LRM operation share-drbd_0:1_notify_0 (call=22, rc=0) complete cibmon[3657]: 2007/09/20_22:34:58 info: cibmon_diff: [cib_diff_notify] cib_update confirmed tengine[3682]: 2007/09/20_22:34:58 info: match_graph_event: Action share-drbd_0:1_pre_notify_stop_0 (71) confirmed on host-1.awale.qc.ca cibmon[3657]: 2007/09/20_22:34:58 info: log_cib_diff: cib_update: Diff: --- 0.12.27 tengine[3682]: 2007/09/20_22:34:58 info: te_pseudo_action: Pseudo action 15 fired and confirmed cibmon[3657]: 2007/09/20_22:34:58 info: log_cib_diff: cib_update: Diff: +++ 0.12.28 tengine[3682]: 2007/09/20_22:34:58 info: te_pseudo_action: Pseudo action 12 fired and confirmed cibmon[3657]: 2007/09/20_22:34:58 info: log_data_element: cib_update: - tengine[3682]: 2007/09/20_22:34:58 info: send_rsc_command: Initiating action 3: share-drbd_0:1_stop_0 on host-1.awale.qc.ca cibmon[3657]: 2007/09/20_22:34:58 info: log_data_element: cib_update: + crmd[3663]: 2007/09/20_22:34:58 info: do_lrm_rsc_op: Performing op=share-drbd_0:1_stop_0 key=3:2:d66e5bec-f673-4648-85f1-de6226a15a97) cibmon[3657]: 2007/09/20_22:34:58 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:58 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:58 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:58 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:58 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:58 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:58 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:58 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:58 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:58 info: log_data_element: cib_update: + lrmd[3660]: 2007/09/20_22:34:59 info: RA output: (share-drbd_0:1:stop:stdout) r0: ACTION=stop drbd[4179][4200]: 2007/09/20_22:34:59 DEBUG: r0: Calling drbdadm -c /etc/drbd.conf state r0 drbd[4179][4207]: 2007/09/20_22:34:59 DEBUG: r0: Exit code 0 drbd[4179][4208]: 2007/09/20_22:34:59 DEBUG: r0: Command output: Not configured drbd[4179][4216]: 2007/09/20_22:34:59 DEBUG: r0: Calling drbdadm -c /etc/drbd.conf cstate r0 drbd[4179][4229]: 2007/09/20_22:34:59 DEBUG: r0: Exit code 0 drbd[4179][4230]: 2007/09/20_22:34:59 DEBUG: r0: Command output: Not configured drbd[4179][4231]: 2007/09/20_22:34:59 DEBUG: r0 status: Not configured Not configured Not configured Unconfigured drbd[4179][4232]: 2007/09/20_22:34:59 DEBUG: r0: Calling /usr/sbin/crm_master -D crm_master[4234]: 2007/09/20_22:34:59 info: Invoked: /usr/sbin/crm_master -D cibmon[3657]: 2007/09/20_22:34:59 info: cibmon_diff: [cib_diff_notify] cib_delete confirmed cibmon[3657]: 2007/09/20_22:34:59 info: log_cib_diff: cib_delete: Diff: --- 0.12.28 cibmon[3657]: 2007/09/20_22:34:59 info: log_cib_diff: cib_delete: Diff: +++ 0.12.29 cibmon[3657]: 2007/09/20_22:34:59 info: log_data_element: cib_delete: - cibmon[3657]: 2007/09/20_22:34:59 info: log_data_element: cib_delete: + drbd[4179][4235]: 2007/09/20_22:34:59 DEBUG: r0: Exit code 0 drbd[4179][4236]: 2007/09/20_22:34:59 DEBUG: r0: Command output: crm_master[4234]: 2007/09/20_22:34:59 info: Invoked: /usr/sbin/crm_master -D lrmd[3660]: 2007/09/20_22:34:59 info: RA output: (share-drbd_0:1:stop:stdout) crm_master[4234]: 2007/09/20_22:34:59 info: Invoked: /usr/sbin/crm_master -D drbd[4179][4237]: 2007/09/20_22:34:59 DEBUG: r0 stop: already unconfigured. crmd[3663]: 2007/09/20_22:34:59 info: process_lrm_event: LRM operation share-drbd_0:1_stop_0 (call=23, rc=0) complete tengine[3682]: 2007/09/20_22:34:59 info: match_graph_event: Action share-drbd_0:1_stop_0 (3) confirmed on host-1.awale.qc.ca cibmon[3657]: 2007/09/20_22:34:59 info: cibmon_diff: [cib_diff_notify] cib_update confirmed tengine[3682]: 2007/09/20_22:34:59 info: te_pseudo_action: Pseudo action 13 fired and confirmed cibmon[3657]: 2007/09/20_22:34:59 info: log_cib_diff: cib_update: Diff: --- 0.12.29 tengine[3682]: 2007/09/20_22:34:59 info: te_pseudo_action: Pseudo action 16 fired and confirmed cibmon[3657]: 2007/09/20_22:34:59 info: log_cib_diff: cib_update: Diff: +++ 0.12.30 tengine[3682]: 2007/09/20_22:34:59 info: te_pseudo_action: Pseudo action 17 fired and confirmed cibmon[3657]: 2007/09/20_22:34:59 info: log_data_element: cib_update: - crmd[3663]: 2007/09/20_22:34:59 info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_IPC_MESSAGE origin=route_message ] tengine[3682]: 2007/09/20_22:34:59 info: run_graph: Transition 2: (Complete=8, Pending=0, Fired=0, Skipped=0, Incomplete=0) cibmon[3657]: 2007/09/20_22:34:59 info: log_data_element: cib_update: + tengine[3682]: 2007/09/20_22:34:59 info: notify_crmd: Transition 2 status: te_complete - cibmon[3657]: 2007/09/20_22:34:59 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:59 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:59 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:59 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:59 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:59 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:59 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:59 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:59 info: log_data_element: cib_update: + cibmon[3657]: 2007/09/20_22:34:59 info: log_data_element: cib_update: + stonithd[3661]: 2007/09/20_22:35:34 WARN: Performed 2 more non-realtime malloc calls. stonithd[3661]: 2007/09/20_22:35:34 info: Total non-realtime malloc bytes: 274432 stonithd[4285]: 2007/09/20_22:35:46 info: 10.0.0.166 uses sysOID '.1.3.6.1.4.1.318.1.3.4.5', compatible with 'masterSwitchrPDU' stonithd[4285]: 2007/09/20_22:35:46 info: 10.0.0.166 has sysName 'RackPDU Archivage' stonithd[4336]: 2007/09/20_22:36:36 info: 10.0.0.166 uses sysOID '.1.3.6.1.4.1.318.1.3.4.5', compatible with 'masterSwitchrPDU' stonithd[4336]: 2007/09/20_22:36:36 info: 10.0.0.166 has sysName 'RackPDU Archivage' stonithd[4400]: 2007/09/20_22:37:26 info: 10.0.0.166 uses sysOID '.1.3.6.1.4.1.318.1.3.4.5', compatible with 'masterSwitchrPDU' stonithd[4400]: 2007/09/20_22:37:26 info: 10.0.0.166 has sysName 'RackPDU Archivage' stonithd[4457]: 2007/09/20_22:38:16 info: 10.0.0.166 uses sysOID '.1.3.6.1.4.1.318.1.3.4.5', compatible with 'masterSwitchrPDU' stonithd[4457]: 2007/09/20_22:38:16 info: 10.0.0.166 has sysName 'RackPDU Archivage' stonithd[4528]: 2007/09/20_22:39:06 info: 10.0.0.166 uses sysOID '.1.3.6.1.4.1.318.1.3.4.5', compatible with 'masterSwitchrPDU' stonithd[4528]: 2007/09/20_22:39:06 info: 10.0.0.166 has sysName 'RackPDU Archivage'