ZABBIX BUGS AND ISSUES
  1. ZABBIX BUGS AND ISSUES
  2. ZBX-7790

zabbix_server_m[13777]: segfault at 18 ip 00007fe6639a8fe0 sp 00007fffbb66e498 error 4 in libpthread-2.18 - crash in "main" process

    Details

    • Type: Incident report Incident report
    • Status: Closed
    • Priority: Major Major
    • Resolution: Won't Fix
    • Affects Version/s: 2.0.10
    • Fix Version/s: None
    • Component/s: Server (S)
    • Labels:
    • Environment:
      Linux fedora-virtualdomain 3.12.9-301.fc20.x86_64 #1 SMP Wed Jan 29 15:56:22 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

      Description

      [root@fedora-virtualdomain zabbix-server]# systemctl start zabbix-server
      Job for zabbix-server-mysql.service failed. See 'systemctl status zabbix-server-mysql.service' and 'journalctl -xn' for details.

      [root@fedora-virtualdomain zabbix-server]# journalctl -xn 30
      – Logs begin at Fri 2014-02-07 13:03:49 CST, end at Tue 2014-02-11 16:31:11 CST. –
      Feb 11 16:19:09 fedora-virtualdomain kernel: zabbix_server_m[13777]: segfault at 18 ip 00007fe6639a8fe0 sp 00007fffbb66e498 error 4 in libpthread-2.18

      1. ccpp-2014-02-10-18_52_03-10724.7z
        300 kB
        Christian Servin
      2. ccpp-2014-02-10-18_52_03-10724.tar.bz2
        427 kB
        Christian Servin
      3. config.7z
        4 kB
        Christian Servin
      4. config.tar.bz2
        4 kB
        Christian Servin
      5. zabbix_server.log.7z
        54 kB
        Christian Servin
      6. zabbix_server.log.bz2
        47 kB
        richlv

        Issue Links

          Activity

          Christian Servin created issue -
          Hide
          Christian Servin added a comment -

          systemd kills the server after the SIGSEGV.

          zabbix-server.log doesnt show the problem.

          When I make zabbixsvr user logeable then login with it:

          bash-4.2$ zabbix_server_mysql
          Segmentation fault (core dumped)

          bash-4.2$ ps -aux | grep zabbix
          root 8845 0.0 0.4 151756 5084 pts/3 S+ 13:35 0:00 vim zabbix_server.conf
          root 9153 0.0 0.5 202220 5272 pts/4 S 13:39 0:00 su zabbixsrv
          zabbixs+ 9162 0.0 0.1 115344 1948 pts/4 S 13:40 0:00 bash
          zabbixs+ 9173 0.0 0.3 247360 3748 ? S 13:40 0:00 zabbix_server_mysql
          zabbixs+ 9176 0.0 0.2 247368 2680 ? S 13:40 0:00 zabbix_server_mysql
          zabbixs+ 9177 0.0 0.2 247360 2252 ? S 13:40 0:00 zabbix_server_mysql
          zabbixs+ 9178 0.0 0.4 351780 4668 ? S 13:40 0:00 zabbix_server_mysql
          zabbixs+ 9179 0.0 0.4 351780 4656 ? S 13:40 0:00 zabbix_server_mysql
          zabbixs+ 9180 0.0 0.4 351780 4644 ? S 13:40 0:00 zabbix_server_mysql
          zabbixs+ 9181 0.0 0.4 351780 4644 ? S 13:40 0:00 zabbix_server_mysql
          zabbixs+ 9182 0.0 0.4 351780 4628 ? S 13:40 0:00 zabbix_server_mysql
          zabbixs+ 9183 0.0 0.4 351780 4672 ? S 13:40 0:00 zabbix_server_mysql
          zabbixs+ 9184 0.0 0.2 247380 2300 ? S 13:40 0:00 zabbix_server_mysql
          zabbixs+ 9185 0.0 0.2 247360 2084 ? S 13:40 0:00 zabbix_server_mysql
          zabbixs+ 9186 0.0 0.2 247380 2300 ? S 13:40 0:00 zabbix_server_mysql
          zabbixs+ 9191 0.0 0.2 247380 2300 ? S 13:40 0:00 zabbix_server_mysql
          zabbixs+ 9192 0.0 0.2 247380 2300 ? S 13:40 0:00 zabbix_server_mysql
          zabbixs+ 9194 0.0 0.2 247708 2296 ? S 13:40 0:00 zabbix_server_mysql
          zabbixs+ 9195 0.0 0.2 247360 2236 ? S 13:40 0:00 zabbix_server_mysql
          zabbixs+ 9197 0.0 0.2 247360 2196 ? S 13:40 0:00 zabbix_server_mysql
          zabbixs+ 9198 0.0 0.2 247380 2740 ? S 13:40 0:00 zabbix_server_mysql
          zabbixs+ 9199 0.0 0.2 247360 2284 ? S 13:40 0:00 zabbix_server_mysql
          zabbixs+ 9205 0.0 0.4 351720 4432 ? S 13:40 0:00 zabbix_server_mysql
          zabbixs+ 9206 0.0 0.2 247380 2892 ? S 13:40 0:00 zabbix_server_mysql
          zabbixs+ 9207 0.0 0.2 247380 2888 ? S 13:40 0:00 zabbix_server_mysql
          zabbixs+ 9208 0.0 0.2 247380 2892 ? S 13:40 0:00 zabbix_server_mysql
          zabbixs+ 9209 0.0 0.2 247380 2888 ? S 13:40 0:00 zabbix_server_mysql
          zabbixs+ 9213 0.0 0.2 247360 2288 ? S 13:40 0:00 zabbix_server_mysql
          zabbixs+ 9214 0.0 0.2 247360 2292 ? S 13:40 0:00 zabbix_server_mysql
          zabbixs+ 9216 0.0 0.2 247360 2040 ? S 13:40 0:00 zabbix_server_mysql
          zabbixs+ 9354 0.0 0.1 123356 1392 pts/4 R+ 13:44 0:00 ps -aux
          zabbixs+ 9355 0.0 0.0 112664 952 pts/4 S+ 13:44 0:00 grep zabbix

          The servers stays running and 'working'

          Show
          Christian Servin added a comment - systemd kills the server after the SIGSEGV. zabbix-server.log doesnt show the problem. When I make zabbixsvr user logeable then login with it: bash-4.2$ zabbix_server_mysql Segmentation fault (core dumped) bash-4.2$ ps -aux | grep zabbix root 8845 0.0 0.4 151756 5084 pts/3 S+ 13:35 0:00 vim zabbix_server.conf root 9153 0.0 0.5 202220 5272 pts/4 S 13:39 0:00 su zabbixsrv zabbixs+ 9162 0.0 0.1 115344 1948 pts/4 S 13:40 0:00 bash zabbixs+ 9173 0.0 0.3 247360 3748 ? S 13:40 0:00 zabbix_server_mysql zabbixs+ 9176 0.0 0.2 247368 2680 ? S 13:40 0:00 zabbix_server_mysql zabbixs+ 9177 0.0 0.2 247360 2252 ? S 13:40 0:00 zabbix_server_mysql zabbixs+ 9178 0.0 0.4 351780 4668 ? S 13:40 0:00 zabbix_server_mysql zabbixs+ 9179 0.0 0.4 351780 4656 ? S 13:40 0:00 zabbix_server_mysql zabbixs+ 9180 0.0 0.4 351780 4644 ? S 13:40 0:00 zabbix_server_mysql zabbixs+ 9181 0.0 0.4 351780 4644 ? S 13:40 0:00 zabbix_server_mysql zabbixs+ 9182 0.0 0.4 351780 4628 ? S 13:40 0:00 zabbix_server_mysql zabbixs+ 9183 0.0 0.4 351780 4672 ? S 13:40 0:00 zabbix_server_mysql zabbixs+ 9184 0.0 0.2 247380 2300 ? S 13:40 0:00 zabbix_server_mysql zabbixs+ 9185 0.0 0.2 247360 2084 ? S 13:40 0:00 zabbix_server_mysql zabbixs+ 9186 0.0 0.2 247380 2300 ? S 13:40 0:00 zabbix_server_mysql zabbixs+ 9191 0.0 0.2 247380 2300 ? S 13:40 0:00 zabbix_server_mysql zabbixs+ 9192 0.0 0.2 247380 2300 ? S 13:40 0:00 zabbix_server_mysql zabbixs+ 9194 0.0 0.2 247708 2296 ? S 13:40 0:00 zabbix_server_mysql zabbixs+ 9195 0.0 0.2 247360 2236 ? S 13:40 0:00 zabbix_server_mysql zabbixs+ 9197 0.0 0.2 247360 2196 ? S 13:40 0:00 zabbix_server_mysql zabbixs+ 9198 0.0 0.2 247380 2740 ? S 13:40 0:00 zabbix_server_mysql zabbixs+ 9199 0.0 0.2 247360 2284 ? S 13:40 0:00 zabbix_server_mysql zabbixs+ 9205 0.0 0.4 351720 4432 ? S 13:40 0:00 zabbix_server_mysql zabbixs+ 9206 0.0 0.2 247380 2892 ? S 13:40 0:00 zabbix_server_mysql zabbixs+ 9207 0.0 0.2 247380 2888 ? S 13:40 0:00 zabbix_server_mysql zabbixs+ 9208 0.0 0.2 247380 2892 ? S 13:40 0:00 zabbix_server_mysql zabbixs+ 9209 0.0 0.2 247380 2888 ? S 13:40 0:00 zabbix_server_mysql zabbixs+ 9213 0.0 0.2 247360 2288 ? S 13:40 0:00 zabbix_server_mysql zabbixs+ 9214 0.0 0.2 247360 2292 ? S 13:40 0:00 zabbix_server_mysql zabbixs+ 9216 0.0 0.2 247360 2040 ? S 13:40 0:00 zabbix_server_mysql zabbixs+ 9354 0.0 0.1 123356 1392 pts/4 R+ 13:44 0:00 ps -aux zabbixs+ 9355 0.0 0.0 112664 952 pts/4 S+ 13:44 0:00 grep zabbix The servers stays running and 'working'
          Christian Servin made changes -
          Field Original Value New Value
          Attachment zabbix_server.log.7z [ 26607 ]
          Hide
          richlv added a comment -

          recompressed logfile with bz2 (and it's smaller as well...)

          Show
          richlv added a comment - recompressed logfile with bz2 (and it's smaller as well...)
          richlv made changes -
          Attachment zabbix_server.log.bz2 [ 26609 ]
          Hide
          richlv added a comment -

          possibly useful contents from the logfile :

           13223:20140211:155619.238 Starting Zabbix Server. Zabbix 2.0.10 (revision 40809).
          
           13223:20140211:155619.434 End of DCsync_configuration()
           13223:20140211:155619.435 Got signal [signal:15(SIGTERM),sender_pid:1,sender_uid:0,reason:0]. Exiting ...
          
           13223:20140211:155621.445 Zabbix Server stopped. Zabbix 2.0.10 (revision 40809).
           13223:20140211:155621.445 Got signal [signal:11(SIGSEGV),reason:1,refaddr:0x18]. Crashing ...
           13223:20140211:155621.446 ====== Fatal information: ======
           13223:20140211:155621.446 Program counter: 0x7fcb34902fe0
           13223:20140211:155621.446 === Registers: ===
           13223:20140211:155621.446 r8      =                0 =                    0 =                    0
           13223:20140211:155621.446 r9      =     7fcb34ed1ea8 =      140510743043752 =      140510743043752
           13223:20140211:155621.446 r10     =     7fffdcaa0350 =      140736895517520 =      140736895517520
           13223:20140211:155621.446 r11     =     7fcb34c18640 =      140510740186688 =      140510740186688
           13223:20140211:155621.446 r12     =     7fcb30fe4bc0 =      140510677060544 =      140510677060544
           13223:20140211:155621.446 r13     =               20 =                   32 =                   32
           13223:20140211:155621.446 r14     =                1 =                    1 =                    1
           13223:20140211:155621.446 r15     =     7fffdcaa05b0 =      140736895518128 =      140736895518128
           13223:20140211:155621.446 rdi     =                8 =                    8 =                    8
           13223:20140211:155621.446 rsi     =                1 =                    1 =                    1
           13223:20140211:155621.446 rbp     =     7fffdcaa07f0 =      140736895518704 =      140736895518704
           13223:20140211:155621.446 rbx     =     7fcb376c39a0 =      140510784928160 =      140510784928160
           13223:20140211:155621.446 rdx     =                1 =                    1 =                    1
           13223:20140211:155621.446 rax     =     7fcb34902fe0 =      140510736953312 =      140510736953312
           13223:20140211:155621.447 rcx     =                0 =                    0 =                    0
           13223:20140211:155621.447 rsp     =     7fffdcaa0588 =      140736895518088 =      140736895518088
           13223:20140211:155621.447 rip     =     7fcb34902fe0 =      140510736953312 =      140510736953312
           13223:20140211:155621.447 efl     =            10202 =                66050 =                66050
           13223:20140211:155621.447 csgsfs  =               33 =                   51 =                   51
           13223:20140211:155621.447 err     =                4 =                    4 =                    4
           13223:20140211:155621.447 trapno  =                e =                   14 =                   14
           13223:20140211:155621.447 oldmask =            14000 =                81920 =                81920
           13223:20140211:155621.447 cr2     =               18 =                   24 =                   24
           13223:20140211:155621.447 === Backtrace: ===
           13223:20140211:155621.449 15: /usr/sbin/zabbix_server_mysql(print_fatal_info+0x287) [0x44fec7]
           13223:20140211:155621.449 14: /usr/sbin/zabbix_server_mysql() [0x44ef26]
           13223:20140211:155621.449 13: /lib64/libc.so.6(+0x35cb0) [0x7fcb34b4bcb0]
           13223:20140211:155621.449 12: /lib64/libpthread.so.0(pthread_mutex_lock+0) [0x7fcb34902fe0]
           13223:20140211:155621.449 11: /lib64/libtspi.so.1(host_table_final+0x14) [0x7fcb30d9abc4]
           13223:20140211:155621.449 10: /lib64/ld-linux-x86-64.so.2(+0xf8ea) [0x7fcb374d68ea]
           13223:20140211:155621.449 9: /lib64/libc.so.6(+0x394c9) [0x7fcb34b4f4c9]
           13223:20140211:155621.449 8: /lib64/libc.so.6(+0x39515) [0x7fcb34b4f515]
           13223:20140211:155621.449 7: /usr/sbin/zabbix_server_mysql(zbx_on_exit+0x15f) [0x41470f]
           13223:20140211:155621.449 6: /lib64/libc.so.6(+0x35cb0) [0x7fcb34b4bcb0]
           13223:20140211:155621.449 5: /lib64/libc.so.6(socket+0x7) [0x7fcb34c0bf67]
           13223:20140211:155621.449 4: /usr/sbin/zabbix_server_mysql(zbx_tcp_listen+0x196) [0x45e0d6]
           13223:20140211:155621.449 3: /usr/sbin/zabbix_server_mysql(MAIN_ZABBIX_ENTRY+0x2b3) [0x4149d3]
           13223:20140211:155621.449 2: /usr/sbin/zabbix_server_mysql(daemon_start+0x23d) [0x44f48d]
           13223:20140211:155621.449 1: /lib64/libc.so.6(__libc_start_main+0xf5) [0x7fcb34b37d65]
           13223:20140211:155621.449 0: /usr/sbin/zabbix_server_mysql() [0x41066d]
          
          Show
          richlv added a comment - possibly useful contents from the logfile : 13223:20140211:155619.238 Starting Zabbix Server. Zabbix 2.0.10 (revision 40809). 13223:20140211:155619.434 End of DCsync_configuration() 13223:20140211:155619.435 Got signal [signal:15(SIGTERM),sender_pid:1,sender_uid:0,reason:0]. Exiting ... 13223:20140211:155621.445 Zabbix Server stopped. Zabbix 2.0.10 (revision 40809). 13223:20140211:155621.445 Got signal [signal:11(SIGSEGV),reason:1,refaddr:0x18]. Crashing ... 13223:20140211:155621.446 ====== Fatal information: ====== 13223:20140211:155621.446 Program counter: 0x7fcb34902fe0 13223:20140211:155621.446 === Registers: === 13223:20140211:155621.446 r8 = 0 = 0 = 0 13223:20140211:155621.446 r9 = 7fcb34ed1ea8 = 140510743043752 = 140510743043752 13223:20140211:155621.446 r10 = 7fffdcaa0350 = 140736895517520 = 140736895517520 13223:20140211:155621.446 r11 = 7fcb34c18640 = 140510740186688 = 140510740186688 13223:20140211:155621.446 r12 = 7fcb30fe4bc0 = 140510677060544 = 140510677060544 13223:20140211:155621.446 r13 = 20 = 32 = 32 13223:20140211:155621.446 r14 = 1 = 1 = 1 13223:20140211:155621.446 r15 = 7fffdcaa05b0 = 140736895518128 = 140736895518128 13223:20140211:155621.446 rdi = 8 = 8 = 8 13223:20140211:155621.446 rsi = 1 = 1 = 1 13223:20140211:155621.446 rbp = 7fffdcaa07f0 = 140736895518704 = 140736895518704 13223:20140211:155621.446 rbx = 7fcb376c39a0 = 140510784928160 = 140510784928160 13223:20140211:155621.446 rdx = 1 = 1 = 1 13223:20140211:155621.446 rax = 7fcb34902fe0 = 140510736953312 = 140510736953312 13223:20140211:155621.447 rcx = 0 = 0 = 0 13223:20140211:155621.447 rsp = 7fffdcaa0588 = 140736895518088 = 140736895518088 13223:20140211:155621.447 rip = 7fcb34902fe0 = 140510736953312 = 140510736953312 13223:20140211:155621.447 efl = 10202 = 66050 = 66050 13223:20140211:155621.447 csgsfs = 33 = 51 = 51 13223:20140211:155621.447 err = 4 = 4 = 4 13223:20140211:155621.447 trapno = e = 14 = 14 13223:20140211:155621.447 oldmask = 14000 = 81920 = 81920 13223:20140211:155621.447 cr2 = 18 = 24 = 24 13223:20140211:155621.447 === Backtrace: === 13223:20140211:155621.449 15: /usr/sbin/zabbix_server_mysql(print_fatal_info+0x287) [0x44fec7] 13223:20140211:155621.449 14: /usr/sbin/zabbix_server_mysql() [0x44ef26] 13223:20140211:155621.449 13: /lib64/libc.so.6(+0x35cb0) [0x7fcb34b4bcb0] 13223:20140211:155621.449 12: /lib64/libpthread.so.0(pthread_mutex_lock+0) [0x7fcb34902fe0] 13223:20140211:155621.449 11: /lib64/libtspi.so.1(host_table_final+0x14) [0x7fcb30d9abc4] 13223:20140211:155621.449 10: /lib64/ld-linux-x86-64.so.2(+0xf8ea) [0x7fcb374d68ea] 13223:20140211:155621.449 9: /lib64/libc.so.6(+0x394c9) [0x7fcb34b4f4c9] 13223:20140211:155621.449 8: /lib64/libc.so.6(+0x39515) [0x7fcb34b4f515] 13223:20140211:155621.449 7: /usr/sbin/zabbix_server_mysql(zbx_on_exit+0x15f) [0x41470f] 13223:20140211:155621.449 6: /lib64/libc.so.6(+0x35cb0) [0x7fcb34b4bcb0] 13223:20140211:155621.449 5: /lib64/libc.so.6(socket+0x7) [0x7fcb34c0bf67] 13223:20140211:155621.449 4: /usr/sbin/zabbix_server_mysql(zbx_tcp_listen+0x196) [0x45e0d6] 13223:20140211:155621.449 3: /usr/sbin/zabbix_server_mysql(MAIN_ZABBIX_ENTRY+0x2b3) [0x4149d3] 13223:20140211:155621.449 2: /usr/sbin/zabbix_server_mysql(daemon_start+0x23d) [0x44f48d] 13223:20140211:155621.449 1: /lib64/libc.so.6(__libc_start_main+0xf5) [0x7fcb34b37d65] 13223:20140211:155621.449 0: /usr/sbin/zabbix_server_mysql() [0x41066d]
          Hide
          emoziko added a comment -

          I have exactly same issue.

          Show
          emoziko added a comment - I have exactly same issue.
          Hide
          richlv added a comment -

          those who see this, how did you install zabbix ?

          Show
          richlv added a comment - those who see this, how did you install zabbix ?
          Hide
          emoziko added a comment -

          I installed it with yum.

          But after failure, i removed zabbix and compiled it from sources. Results are same..

          Show
          emoziko added a comment - I installed it with yum. But after failure, i removed zabbix and compiled it from sources. Results are same..
          Hide
          richlv added a comment -

          which package repository did you use ?
          when compiling from sources, how do you start the server ?

          Show
          richlv added a comment - which package repository did you use ? when compiling from sources, how do you start the server ?
          Hide
          emoziko added a comment -

          For RPMs i used Fedora default repos, just typed yum install zabbix zabbix-server etc.

          In compiled version, i started it with command: /usr/local/sbin/zabbix_server -c /usr/local/etc/zabbix_server.conf

          Show
          emoziko added a comment - For RPMs i used Fedora default repos, just typed yum install zabbix zabbix-server etc. In compiled version, i started it with command: /usr/local/sbin/zabbix_server -c /usr/local/etc/zabbix_server.conf
          Hide
          Christian Servin added a comment -

          The way I installed it was very similar to this:
          http://tecadmin.net/install-zabbix-network-monitoring-on-centos-rhel-and-fedora/#

          but i did not configured any special repositories

          it did not worked out of the box, I made the following changes

          mkdir /var/run/zabbix-server
          mkdir /var/run/zabbix-agent
          mkdir /var/log/zabbix

          usermod -a -G zabbix zabbix
          usermod -a -G zabbixsvr zabbix

          chown zabbixsvr:zabbix /var/run/zabbix-server
          chown zabbix:zabbix /var/run/zabbix-agent
          chown root:zabbix /var/log/zabbix

          had troubles with SELinux
          tail -f /var/log/audit/audit.log |grep -i avc

          setsebool httpd_can_network_connect on
          setsebool zabbix_can_network_connect on

          and another to the database connection... until audit.log was not showing trouble

          attaching my configuration files

          Show
          Christian Servin added a comment - The way I installed it was very similar to this: http://tecadmin.net/install-zabbix-network-monitoring-on-centos-rhel-and-fedora/# but i did not configured any special repositories it did not worked out of the box, I made the following changes mkdir /var/run/zabbix-server mkdir /var/run/zabbix-agent mkdir /var/log/zabbix usermod -a -G zabbix zabbix usermod -a -G zabbixsvr zabbix chown zabbixsvr:zabbix /var/run/zabbix-server chown zabbix:zabbix /var/run/zabbix-agent chown root:zabbix /var/log/zabbix had troubles with SELinux tail -f /var/log/audit/audit.log |grep -i avc setsebool httpd_can_network_connect on setsebool zabbix_can_network_connect on and another to the database connection... until audit.log was not showing trouble attaching my configuration files
          Christian Servin made changes -
          Attachment zabbix-configs.7z [ 26623 ]
          Hide
          Christian Servin added a comment -

          About the repositories im using basic fedora 20

          [root@fedora-virtualdomain cservin]# yum repolist
          Loaded plugins: langpacks, refresh-packagekit
          repo id repo name status
          fedora/20/x86_64 Fedora 20 - x86_64 38,597
          google-chrome google-chrome 3
          !updates/20/x86_64 Fedora 20 - x86_64 - Updates 12,620
          repolist: 51,220

          [root@fedora-virtualdomain cservin]# yum info zabbix-server
          Loaded plugins: langpacks, refresh-packagekit
          Installed Packages
          Name : zabbix-server
          Arch : x86_64
          Version : 2.0.10
          Release : 2.fc20
          Size : 14 k
          Repo : installed
          From repo : updates
          Summary : Zabbix server common files
          URL : http://www.zabbix.com
          License : GPLv2+
          Description : Zabbix server common files

          Show
          Christian Servin added a comment - About the repositories im using basic fedora 20 [root@fedora-virtualdomain cservin] # yum repolist Loaded plugins: langpacks, refresh-packagekit repo id repo name status fedora/20/x86_64 Fedora 20 - x86_64 38,597 google-chrome google-chrome 3 !updates/20/x86_64 Fedora 20 - x86_64 - Updates 12,620 repolist: 51,220 [root@fedora-virtualdomain cservin] # yum info zabbix-server Loaded plugins: langpacks, refresh-packagekit Installed Packages Name : zabbix-server Arch : x86_64 Version : 2.0.10 Release : 2.fc20 Size : 14 k Repo : installed From repo : updates Summary : Zabbix server common files URL : http://www.zabbix.com License : GPLv2+ Description : Zabbix server common files
          Oleksiy Zagorskyi made changes -
          Labels libpthread segfault crash segfault
          Oleksiy Zagorskyi made changes -
          Summary zabbix_server_m[13777]: segfault at 18 ip 00007fe6639a8fe0 sp 00007fffbb66e498 error 4 in libpthread-2.18 zabbix_server_m[13777]: segfault at 18 ip 00007fe6639a8fe0 sp 00007fffbb66e498 error 4 in libpthread-2.18 - crach in "main" process
          Oleksiy Zagorskyi made changes -
          Summary zabbix_server_m[13777]: segfault at 18 ip 00007fe6639a8fe0 sp 00007fffbb66e498 error 4 in libpthread-2.18 - crach in "main" process zabbix_server_m[13777]: segfault at 18 ip 00007fe6639a8fe0 sp 00007fffbb66e498 error 4 in libpthread-2.18 - crash in "main" process
          Christian Servin made changes -
          Attachment zabbix-configs.7z [ 26623 ]
          Hide
          richlv added a comment -

          btw, if possible, use xz/bzip2/gzip to compress the files (preferably, in that order )

          Show
          richlv added a comment - btw, if possible, use xz/bzip2/gzip to compress the files (preferably, in that order )
          Hide
          Christian Servin added a comment -

          It's all about flavor

          Show
          Christian Servin added a comment - It's all about flavor
          Christian Servin made changes -
          Attachment config.tar.bz2 [ 26633 ]
          Attachment config.7z [ 26634 ]
          Christian Servin made changes -
          Christian Servin made changes -
          Attachment ccpp-2014-02-10.7z [ 26606 ]
          Christian Servin made changes -
          Comment [ All about flavor ]
          Hide
          Pavel added a comment -

          I have the same issue after update to Fedora 20.

          (gdb) run
          Starting program: /usr/sbin/zabbix_server
          [Thread debugging using libthread_db enabled]
          Using host libthread_db library "/lib/libthread_db.so.1".
          [New Thread 0xb6b30b40 (LWP 5643)]
          [Thread 0xb6b30b40 (LWP 5643) exited]
          Detaching after fork from child process 5644.

          Program received signal SIGSEGV, Segmentation fault.
          _GI__pthread_mutex_lock (mutex=0x4) at ../nptl/pthread_mutex_lock.c:66
          66 unsigned int type = PTHREAD_MUTEX_TYPE_ELISION (mutex);
          (gdb) bt
          #0 _GI__pthread_mutex_lock (mutex=0x4) at ../nptl/pthread_mutex_lock.c:66
          #1 0xb761b594 in pthread_mutex_lock (mutex=0x4) at forward.c:192
          #2 0xb6da3084 in host_table_final () from /lib/libtspi.so.1
          #3 0xb6d979f5 in my_fini () from /lib/libtspi.so.1
          #4 0xb7feda84 in _dl_fini () at dl-fini.c:252
          #5 0xb7548011 in _run_exit_handlers (status=status@entry=0, listp=0xb76d03c4 <_exit_funcs>,
          run_list_atexit=run_list_atexit@entry=true) at exit.c:82
          #6 0xb754806d in __GI_exit (status=0) at exit.c:104
          #7 0x0809ec72 in daemon_start ()
          #8 0x08054d50 in main ()
          (gdb)

          Show
          Pavel added a comment - I have the same issue after update to Fedora 20. (gdb) run Starting program: /usr/sbin/zabbix_server [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/libthread_db.so.1". [New Thread 0xb6b30b40 (LWP 5643)] [Thread 0xb6b30b40 (LWP 5643) exited] Detaching after fork from child process 5644. Program received signal SIGSEGV, Segmentation fault. _ GI __pthread_mutex_lock (mutex=0x4) at ../nptl/pthread_mutex_lock.c:66 66 unsigned int type = PTHREAD_MUTEX_TYPE_ELISION (mutex); (gdb) bt #0 _ GI __pthread_mutex_lock (mutex=0x4) at ../nptl/pthread_mutex_lock.c:66 #1 0xb761b594 in pthread_mutex_lock (mutex=0x4) at forward.c:192 #2 0xb6da3084 in host_table_final () from /lib/libtspi.so.1 #3 0xb6d979f5 in my_fini () from /lib/libtspi.so.1 #4 0xb7feda84 in _dl_fini () at dl-fini.c:252 #5 0xb7548011 in _ run_exit_handlers (status=status@entry=0, listp=0xb76d03c4 < _exit_funcs>, run_list_atexit=run_list_atexit@entry=true) at exit.c:82 #6 0xb754806d in __GI_exit (status=0) at exit.c:104 #7 0x0809ec72 in daemon_start () #8 0x08054d50 in main () (gdb)
          Hide
          Pavel added a comment -
          Show
          Pavel added a comment - Bugreport in Fedora bugtracker: https://bugzilla.redhat.com/show_bug.cgi?id=1064300
          Hide
          Volker Fröhlich added a comment -

          https://bugzilla.redhat.com/show_bug.cgi?id=1064300#c18

          It seems to be an incompatibility with gnutls 3.1.20. It's not a silent ABI break though, as rebuilding doesn't help.

          Show
          Volker Fröhlich added a comment - https://bugzilla.redhat.com/show_bug.cgi?id=1064300#c18 It seems to be an incompatibility with gnutls 3.1.20. It's not a silent ABI break though, as rebuilding doesn't help.
          Show
          richlv added a comment - also see https://bugzilla.redhat.com/show_bug.cgi?id=1063089#c12
          Alexei Vladishev made changes -
          Workflow Zabbix workflow [ 33900 ] Zabbix workflow - new [ 40899 ]
          Hide
          Greg Cockburn added a comment - - edited

          Hi, this is affecting me also.

          I have tried the packages from EPEL and packages from repo.zabbix.com.

          Trousers is up to date (as per one of the bugzilla links).

          This host is brand new, fully patched Oracle Linux 7 host, and I am trying to install/run Zabbix 2.2.

          Is there any further movement on this bug? Can I provide anything further than what has already been provided?

          Thanks

          Show
          Greg Cockburn added a comment - - edited Hi, this is affecting me also. I have tried the packages from EPEL and packages from repo.zabbix.com. Trousers is up to date (as per one of the bugzilla links). This host is brand new, fully patched Oracle Linux 7 host, and I am trying to install/run Zabbix 2.2. Is there any further movement on this bug? Can I provide anything further than what has already been provided? Thanks
          Hide
          Christian Servin added a comment - - edited

          uninstall gnutls or downgrade, its legacy any ways, you will live without it.

          Show
          Christian Servin added a comment - - edited uninstall gnutls or downgrade, its legacy any ways, you will live without it.
          Hide
          Greg Cockburn added a comment -

          There are too many dependencies. Patching trousers seems to be the key.

          Show
          Greg Cockburn added a comment - There are too many dependencies. Patching trousers seems to be the key.
          richlv made changes -
          Link This issue is duplicated by ZBX-9449 [ ZBX-9449 ]
          Hide
          Jasper added a comment -

          After updating CentOS 7 to the second release 1503 we have the same issue:

          Apr 03 13:47:12 zabbix.tgho.nl systemd[1]: Starting Zabbix Server...
          – Subject: Unit zabbix-server.service has begun with start-up
          – Defined-By: systemd
          – Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel

          – Unit zabbix-server.service has begun starting up.
          Apr 03 13:47:12 zabbix kernel: zabbix_server[3476]: segfault at 18 ip 00007f7e0a419c00 sp 00007ffd32169c18 error 4 in libpthread-2.17.so[7f7e0a410000+16000]

          Show
          Jasper added a comment - After updating CentOS 7 to the second release 1503 we have the same issue: Apr 03 13:47:12 zabbix.tgho.nl systemd [1] : Starting Zabbix Server... – Subject: Unit zabbix-server.service has begun with start-up – Defined-By: systemd – Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel – – Unit zabbix-server.service has begun starting up. Apr 03 13:47:12 zabbix kernel: zabbix_server [3476] : segfault at 18 ip 00007f7e0a419c00 sp 00007ffd32169c18 error 4 in libpthread-2.17.so [7f7e0a410000+16000]
          Alexey Pustovalov made changes -
          Link This issue is duplicated by WIG-43 [ WIG-43 ]
          Hide
          Anton Samets added a comment -

          The same is here: upgraded to Centos 7.1. Fix: downgrade package gnutls from 3.3.8-12.el7 to 3.1.18-10.el7_0.

          Show
          Anton Samets added a comment - The same is here: upgraded to Centos 7.1. Fix: downgrade package gnutls from 3.3.8-12.el7 to 3.1.18-10.el7_0 .
          Hide
          Denis Pavlov added a comment - - edited

          Hi, same problem, CentOS Linux release 7.1.1503 (Core).

          cat /var/log/messages | grep zabbix:
          Apr 8 16:08:26 eye kernel: zabbix_server[2188]: segfault at 18 ip 00007ff010083c00 sp 00007fff96446708 error 4 in libpthread-2.17.so[7ff01007a000+16000]
          Apr 8 16:08:26 eye kernel: zabbix_server[2189]: segfault at 18 ip 00007ff010083c00 sp 00007fff96446708 error 4 in libpthread-2.17.so[7ff01007a000+16000]
          Apr 8 16:08:26 eye systemd: zabbix-server.service: control process exited, code=killed status=11
          Apr 8 16:08:26 eye systemd: Unit zabbix-server.service entered failed state.

          Show
          Denis Pavlov added a comment - - edited Hi, same problem, CentOS Linux release 7.1.1503 (Core). cat /var/log/messages | grep zabbix: Apr 8 16:08:26 eye kernel: zabbix_server [2188] : segfault at 18 ip 00007ff010083c00 sp 00007fff96446708 error 4 in libpthread-2.17.so [7ff01007a000+16000] Apr 8 16:08:26 eye kernel: zabbix_server [2189] : segfault at 18 ip 00007ff010083c00 sp 00007fff96446708 error 4 in libpthread-2.17.so [7ff01007a000+16000] Apr 8 16:08:26 eye systemd: zabbix-server.service: control process exited, code=killed status=11 Apr 8 16:08:26 eye systemd: Unit zabbix-server.service entered failed state.
          Hide
          chris heming added a comment -

          RHEL 7. Same as everyone else.
          segfault at 18 ip 00007ff010083c00 sp 00007fff96446708 error 4 in libpthread-2.17.so[7ff01007a000+16000]

          Anton Samets's fix worked for me:
          yum downgrade gnutls-3.1.18-10.el7_0 gnutls-devel-3.1.18-10.el7_0 gnutls-dane-3.1.18-10.el7_0 gnutls-c++-3.1.18-10.el7_0

          Show
          chris heming added a comment - RHEL 7. Same as everyone else. segfault at 18 ip 00007ff010083c00 sp 00007fff96446708 error 4 in libpthread-2.17.so[7ff01007a000+16000] Anton Samets 's fix worked for me: yum downgrade gnutls-3.1.18-10.el7_0 gnutls-devel-3.1.18-10.el7_0 gnutls-dane-3.1.18-10.el7_0 gnutls-c++-3.1.18-10.el7_0
          Hide
          Volker Fröhlich added a comment -

          As far as I can tell, it seems like there was a silent ABI breakage in gnutls, that is causing zabbix_server to crash through iksemel.

          gnutls had 3 breakages between 3.1.18 and 3.3.8: http://upstream.rosalinux.ru/versions/gnutls.html

          I would therefore think that a rebuild of iksemel should be enough. Please try these packages together with the latest version of gnutls supplied by EL: http://koji.fedoraproject.org/koji/taskinfo?taskID=9472122

          Show
          Volker Fröhlich added a comment - As far as I can tell, it seems like there was a silent ABI breakage in gnutls, that is causing zabbix_server to crash through iksemel. gnutls had 3 breakages between 3.1.18 and 3.3.8: http://upstream.rosalinux.ru/versions/gnutls.html I would therefore think that a rebuild of iksemel should be enough. Please try these packages together with the latest version of gnutls supplied by EL: http://koji.fedoraproject.org/koji/taskinfo?taskID=9472122
          Hide
          Tomas Brandysky added a comment - - edited

          Hello,

          I have the very same problem on CentOS Linux release 7.1.1503 (Core)
          The behaviour is the same with 2.2 and 2.4 releases:

          [root@server]# zabbix_server
          Segmentation fault

          Also tried to compile zabbix-server from SRPM but it didn't help

          I don't think it was caused by gnutls package as this package wasn't updated since I've installed the server.

          Show
          Tomas Brandysky added a comment - - edited Hello, I have the very same problem on CentOS Linux release 7.1.1503 (Core) The behaviour is the same with 2.2 and 2.4 releases: [root@server] # zabbix_server Segmentation fault Also tried to compile zabbix-server from SRPM but it didn't help I don't think it was caused by gnutls package as this package wasn't updated since I've installed the server.
          Hide
          Julien Savard added a comment -

          Same problem here with RHEL 7.1 and Zabbix 2.4...
          [root@SomeServer ~]# cat /etc/redhat-release
          Red Hat Enterprise Linux Server release 7.1 (Maipo)
          [root@SomeServer ~]# /usr/sbin/zabbix_server
          Segmentation fault
          [root@SomeServer ~]# rpm -qa | grep gnutls
          gnutls-3.3.8-12.el7.x86_64

          [root@SomeServer ~]# cat /var/log/zabbix/zabbix_server.log
          ...
          2645:20150414:111518.803 using configuration file: /etc/zabbix/zabbix_server.conf
          2645:20150414:111518.810 current database version (mandatory/optional): 02040000/02040000
          2645:20150414:111518.810 required mandatory version: 02040000
          2645:20150414:111518.820 listener failed: zbx_tcp_listen() fatal error: unable to serve on any address [[-]:10051]
          2645:20150414:111518.821 Got signal [signal:11(SIGSEGV),reason:1,refaddr:0x18]. Crashing ...
          ...

          According to some people it is related to the upgrade of gnutls from 3.1.8 to 3.3.8-12 ...

          Show
          Julien Savard added a comment - Same problem here with RHEL 7.1 and Zabbix 2.4... [root@SomeServer ~] # cat /etc/redhat-release Red Hat Enterprise Linux Server release 7.1 (Maipo) [root@SomeServer ~] # /usr/sbin/zabbix_server Segmentation fault [root@SomeServer ~] # rpm -qa | grep gnutls gnutls-3.3.8-12.el7.x86_64 [root@SomeServer ~] # cat /var/log/zabbix/zabbix_server.log ... 2645:20150414:111518.803 using configuration file: /etc/zabbix/zabbix_server.conf 2645:20150414:111518.810 current database version (mandatory/optional): 02040000/02040000 2645:20150414:111518.810 required mandatory version: 02040000 2645:20150414:111518.820 listener failed: zbx_tcp_listen() fatal error: unable to serve on any address [ [-] :10051] 2645:20150414:111518.821 Got signal [signal:11(SIGSEGV),reason:1,refaddr:0x18] . Crashing ... ... According to some people it is related to the upgrade of gnutls from 3.1.8 to 3.3.8-12 ...
          Hide
          Grzegorz Grabowski added a comment -

          The same as others.
          The solution is to downgrade gnutls to this one is included in Centos 7 install CD.
          I mounted Install CD, add CD repo and:

          1. yum downgrade gnutls

          gnutls.x86_64 0:3.3.8-12.el7 - was removed
          gnutls.x86_64 0:3.1.18-8.el7 - was installed
          And all work.

          I additionally tested it on a fresh Centos 7 + zabbix 2.4 (worked well) and after "yum update" zabbix_server crushed.

          Best
          Grzegorz

          Show
          Grzegorz Grabowski added a comment - The same as others. The solution is to downgrade gnutls to this one is included in Centos 7 install CD. I mounted Install CD, add CD repo and: yum downgrade gnutls gnutls.x86_64 0:3.3.8-12.el7 - was removed gnutls.x86_64 0:3.1.18-8.el7 - was installed And all work. I additionally tested it on a fresh Centos 7 + zabbix 2.4 (worked well) and after "yum update" zabbix_server crushed. Best Grzegorz
          Hide
          Volker Fröhlich added a comment -

          I investigated the issue quite a bit lately and it is definitely and only connected to the use of iksemel. If you build without-jabber, everything is fine. That also explains why the MySQL proxy has no issues, as it has no business with alerting. What is still a mystery to me, is, why the PostgreSQL server does fine. Also, rebuilding iksemel and even rebuilding Zabbix on top of that shows no effect whatsoever. It must be some really weird conflict with iksemel, gnutls and MySQL.

          Show
          Volker Fröhlich added a comment - I investigated the issue quite a bit lately and it is definitely and only connected to the use of iksemel. If you build without-jabber, everything is fine. That also explains why the MySQL proxy has no issues, as it has no business with alerting. What is still a mystery to me, is, why the PostgreSQL server does fine. Also, rebuilding iksemel and even rebuilding Zabbix on top of that shows no effect whatsoever. It must be some really weird conflict with iksemel, gnutls and MySQL.
          Hide
          Steve Huston added a comment -

          Confirming here; had tested a zabbix install on RHEL6, and was just updating my setup to 7 (not deployed yet) and found this exact issue. With zabbix-server-2.4.4-1.el7.x86_64, gnutls-3.3.8-12.el7.x86_64 and trousers-0.3.11.2-3.el7.x86_64 (which in the linked ZBX-9449 was mentioned as a possible culprit) the zabbix_server instance segfaults immediately.

          Show
          Steve Huston added a comment - Confirming here; had tested a zabbix install on RHEL6, and was just updating my setup to 7 (not deployed yet) and found this exact issue. With zabbix-server-2.4.4-1.el7.x86_64, gnutls-3.3.8-12.el7.x86_64 and trousers-0.3.11.2-3.el7.x86_64 (which in the linked ZBX-9449 was mentioned as a possible culprit) the zabbix_server instance segfaults immediately.
          Hide
          Volker Fröhlich added a comment - - edited

          Latest insights:

          • It's not a packaging artefact. A manual and stripped down build suffers from the same problems.
          • A rebuild of gnutls 3.1.18 with everything else left at 7.1 versions works
          • A hackish build of gnutls 3.1.28 works as well
          • A 3.3.8 rebuild works, if built --without-tpm (trousers)
          Show
          Volker Fröhlich added a comment - - edited Latest insights: It's not a packaging artefact. A manual and stripped down build suffers from the same problems. A rebuild of gnutls 3.1.18 with everything else left at 7.1 versions works A hackish build of gnutls 3.1.28 works as well A 3.3.8 rebuild works, if built --without-tpm (trousers)
          Show
          Volker Fröhlich added a comment - https://bugzilla.redhat.com/show_bug.cgi?id=1071171
          richlv made changes -
          Link This issue is duplicated by ZBX-9512 [ ZBX-9512 ]
          richlv made changes -
          Link This issue is duplicated by ZBX-9624 [ ZBX-9624 ]
          Hide
          Volker Fröhlich added a comment -

          This issue should be gone in any supported version of Fedora. I am not sure whether the corrected EPEL 7 package of trousers is available to users yet, but anyway, there is nothing Zabbix SIA can do any further about this issue. I would thus suggest to close this issue.

          Show
          Volker Fröhlich added a comment - This issue should be gone in any supported version of Fedora. I am not sure whether the corrected EPEL 7 package of trousers is available to users yet, but anyway, there is nothing Zabbix SIA can do any further about this issue. I would thus suggest to close this issue.
          Hide
          richlv added a comment -

          as per the previous comment, closing

          Show
          richlv added a comment - as per the previous comment, closing
          richlv made changes -
          Status Open [ 1 ] Closed [ 6 ]
          Resolution Won't Fix [ 2 ]
          Alexander Vladishev made changes -
          Workflow Zabbix workflow - new [ 40899 ] Copy of Zabbix workflow - new [ 68285 ]
          Alexander Vladishev made changes -
          Workflow Copy of Zabbix workflow - new [ 68285 ] Zabbix workflow - new [ 83507 ]
          Gatis Rumbens made changes -
          Issue Type Bug [ 1 ] Incident report [ 10110 ]
          Zabbix ID NA
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Closed Closed
          484d 11h 22m 1 richlv 2015 Jun 11 13:15

            People

            • Assignee:
              Unassigned
              Reporter:
              Christian Servin
            • Votes:
              12 Vote for this issue
              Watchers:
              21 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: