[Reconnoiter-devel] Memory leak in noitd

Michal Taborsky michal at taborsky.cz
Sun Apr 22 18:10:19 EDT 2012


The problem persists. I'll get to the repeat valgrind run tomorrow, but I
have narrowed it down a little. It happens when snmp checks are timing out.
I see you fixed something there, but probably there is more. I have an
unreachable target (firewall is dropping the packets), with several snmp
checks against it. If I disable it, the memory stabilizes. When I enable
it, the memory grows. So it's definitely somewhere in there. Hope this
helps.

--
Michal Taborsky
http://www.taborsky.cz



2012/4/22 Theo Schlossnagle <jesus at omniti.com>

> A whole bunch of fixes for mem-related stuff today... the two places that
> look bad in your leak diagnostics where the noit.gunzip lua wrapper and
> snmp in the event of timeouts.  However, both were quite small and would
> take a long time to accumulate to anything noticeable.  In otherwords,
> there's likely another more nefarious leak in there.
>
> Can you update everything to latest and run under valgrind again -- feel
> free to run it for an hour or so (as long as it performs well enough to
> stay current).
>
>
> On Sun, Apr 22, 2012 at 11:29 AM, Michal Taborsky <michal at taborsky.cz>wrote:
>
>> I am not familiar with valgrind, but the output is here
>> https://gist.github.com/2464624
>>
>> MT.
>>
>> 2012/4/22 Theo Schlossnagle <jesus at omniti.com>
>>
>>> Are you familiar with valgrind?  If you could, compile with debugging
>>> symbols (-g) and run:
>>>
>>> valgrind --log-file=noitd.leaks --leak-check=full ./noitd -D (other args
>>> like -M or -c if you use those)
>>>
>>> Let it run for about two minutes, telnet into noitd and run shutdown
>>>
>>> valgrind should spit a ton of junk out in noitd.leaks -- that should
>>> pinpoint the problem.
>>>
>>> 2012/4/22 Michal Taborsky <michal at taborsky.cz>
>>>
>>>> After upgrading to the latest code from master, I am experiencing some
>>>> memory leak in noitd. This noit runs about 200 checks per minute with
>>>> various modules and looses about a 500k per minute. It's CentOS 6.2 64bit.
>>>>
>>>> [root at server etc]# date;  ps uax | grep noitd
>>>> Sun Apr 22 13:18:21 CEST 2012
>>>> root     22405  0.0  0.1 146844  1088 ?        S    12:59   0:00
>>>> /usr/local/sbin/noitd -c /usr/local/etc/noit.conf
>>>> root     22406  1.1  3.9 940652 40108 ?        Sl   12:59   0:13
>>>> /usr/local/sbin/noitd -c /usr/local/etc/noit.conf
>>>> [root at server etc]# date;  ps uax | grep noitd
>>>> Sun Apr 22 13:42:30 CEST 2012
>>>> root     22405  0.0  0.1 146844  1088 ?        S    12:59   0:00
>>>> /usr/local/sbin/noitd -c /usr/local/etc/noit.conf
>>>> root     22406  1.1  5.0 951612 51560 ?        Sl   12:59   0:28
>>>> /usr/local/sbin/noitd -c /usr/local/etc/noit.conf
>>>>
>>>> I know this does not help much and I am ready to provide more info if
>>>> requested. For now, I did not have the time to try to disable the checks
>>>> one by one to see which one causes it. I'll do it later, if necessary.
>>>> Right now I am looking for some possible quick fix or advice how to
>>>> find it, if there is any.
>>>>
>>>> --
>>>> Michal Taborsky
>>>> http://www.taborsky.cz
>>>>
>>>>
>>>> _______________________________________________
>>>> Reconnoiter-devel mailing list
>>>> Reconnoiter-devel at lists.omniti.com
>>>> http://lists.omniti.com/mailman/listinfo/reconnoiter-devel
>>>>
>>>>
>>>
>>>
>>> --
>>>
>>> Theo Schlossnagle
>>>
>>> http://omniti.com/is/theo-schlossnagle
>>>
>>>
>>
>
>
> --
>
> Theo Schlossnagle
>
> http://omniti.com/is/theo-schlossnagle
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/reconnoiter-devel/attachments/20120423/315de864/attachment.html>


More information about the Reconnoiter-devel mailing list