• 當前位置:首頁 > IT技術 > 系統服務 > 正文

    rabbitmq進程崩潰導致服務器宕機
    2022-02-14 10:37:53

    rabbitmq進程崩潰導致服務器宕機問題


    背景

    操作系統是 centos 6,RabbitMQ 3.8.1.

    服務器突然無法連接,重啟后查看??/var/log/messages??,報錯日志如下:

    epmd: erlang mapper daemon cant fork 2'nd time Cannot allocate memory
    epmd: erlang mapper daemon cant fork 2'nd time Cannot allocate memory
    epmd: erlang mapper daemon cant fork 2'nd time Cannot allocate memory
    epmd: erlang mapper daemon cant fork 2'nd time Cannot allocate memory
    sshd[1121]: error: fork: Cannot allocate memory
    sshd[1121]: error: fork: Cannot allocate memory
    sshd[1121]: error: fork: Cannot allocate memory
    sshd[1121]: error: fork: Cannot allocate memory
    sshd[1121]: error: fork: Cannot allocate memory
    sshd[1121]: error: fork: Cannot allocate memory
    sshd[1121]: error: fork: Cannot allocate memory
    sshd[1121]: error: fork: Cannot allocate memory

    使用??locate erl_crash.dump???找到崩潰日志 erl_crash.dump,使用 ??https://github.com/ferd/recon/edit/master/script/erl_crashdump_analyzer.sh?? 分析結果如下:

    analyzing erl_crash.dump, generated on:  Wed Apr 22 16:39:10 2020 

    Slogan: init terminating in do_boot ({error,{cannot_read_enabled_plugins_file,/etc/rabbitmq/enabled_plugins,eacces}})

    Memory:
    ===
    processes: 17 Mb
    processes_used: 17 Mb
    system: 16 Mb
    atom: 0 Mb
    atom_used: 0 Mb
    binary: 0 Mb
    code: 6 Mb
    ets: 1 Mb
    ---
    total: 34 Mb

    Different message queue lengths (5 largest different):
    ===
    89 0

    Error logger queue length:
    ===
    0

    File descriptors open:
    ===
    UDP: 0
    TCP: 2
    Files: 0
    ---
    Total: 2

    Number of processes:
    ===
    89

    Processes Heap+Stack memory sizes (words) used in the VM (5 largest different):
    ===
    1 46422
    1 10958
    2 6772
    2 4185
    2 2586

    Processes OldHeap memory sizes (words) used in the VM (5 largest different):
    ===
    1 318187
    1 75113
    1 46422
    1 10958
    2 2586

    Process States when crashing (sum):
    ===
    1 CONNECTED
    2 CONNECTED|BINARY_IO
    2 CONNECTED|BINARY_IO|PORT_LOCK
    1 Current Process Internal ACT_PRIO_NORMAL | USR_PRIO_NORMAL | PRQ_PRIO_NORMAL | ACTIVE | RUNNING | ACTIVE_SYS
    1 Current Process Running
    1 Internal ACT_PRIO_HIGH | USR_PRIO_HIGH | PRQ_PRIO_HIGH | OFF_HEAP_MSGQ
    3 Internal ACT_PRIO_MAX | USR_PRIO_MAX | PRQ_PRIO_MAX
    1 Internal ACT_PRIO_MAX | USR_PRIO_MAX | PRQ_PRIO_MAX | OFF_HEAP_MSGQ
    1 Internal ACT_PRIO_MAX | USR_PRIO_MAX | PRQ_PRIO_NORMAL
    76 Internal ACT_PRIO_NORMAL | USR_PRIO_NORMAL | PRQ_PRIO_NORMAL
    1 Internal ACT_PRIO_NORMAL | USR_PRIO_NORMAL | PRQ_PRIO_NORMAL | ACTIVE | RUNNING | ACTIVE_SYS
    6 Internal ACT_PRIO_NORMAL | USR_PRIO_NORMAL | PRQ_PRIO_NORMAL | OFF_HEAP_MSGQ
    1 Running
    88 Waiting

    沒排查出問題來,這臺機器是8c32g的機器,按道理應該不該出現內存不足的問題。

    后面開發說RabbitMQ不用了,服務停掉。。。問題消失了。



    本文摘自 :https://blog.51cto.com/u

    開通會員,享受整站包年服務
    国产呦精品一区二区三区网站|久久www免费人咸|精品无码人妻一区二区|久99久热只有精品国产15|中文字幕亚洲无线码