From borising at gmail.com Thu May 1 22:08:20 2014 From: borising at gmail.com (Bo Agerskov Rising) Date: Fri, 02 May 2014 00:08:20 +0200 Subject: [Reconnoiter-devel] counter_dev In-Reply-To: References: <5307C1E3.5080300@gmail.com> <5309CC99.5000404@gmail.com> Message-ID: <5362C5D4.4000701@gmail.com> Just spinned up a postgresql 8.4.17 instance in my testlab, and tested with the same data coming into my prod 9.3 system, and now the counter_dev column get's updated properly. So it seems to me, that the issue is to be found in version 9.x. Regards, Bo Theo Schlossnagle wrote: > Any time the counter field varies over time you should see a deviation > over a containing time window) that is non-zero. > > e.g. if you looking in the 5m tables and see counters changing, you > should see a non-zero deviation in the 20m table. > > If you're seeing zeros everywhere, something is wrong -- maybe the code > is broken. > > > On Sun, Feb 23, 2014 at 2:25 AM, Bo Agerskov Rising > wrote: > > Hi Theo, > > Thanks for the explanation of the counter_dev field. > > In which situations, would I expect data in this field? > > Regards, > Bo > > Theo Schlossnagle wrote: > > IIRC, counter_dev should be the standard deviation of counter > calculations. > > > > There are already rollups in the database. If you were to write > custom > > aggregations funtions you'd need to write those yourself in > PL/pgsql to > > run over the arrays in those tables. (summing or whatever). > > > > > > On Fri, Feb 21, 2014 at 1:15 PM, Bo Agerskov Rising > > > >> wrote: > > > > Hi guys, > > > > I was wondering whats the idea behind the counter_dev field, > when you > > question the stratcon.fetch_dataset function? I can't really > find any > > uuid's that uses this field yet, neither any really good > explanations in > > the code that explains it in plain layman terms :) > > > > Also I am working on making a summarizing function in the UI, > which > > would sum the values of a metric hourly, weekly and so forth, > and was > > wondering which fields in the DB schema that I would use for that? > > > > Regards, > > Bo > > _______________________________________________ > > Reconnoiter-devel mailing list > > Reconnoiter-devel at lists.omniti.com > > > > > > http://lists.omniti.com/mailman/listinfo/reconnoiter-devel > > > > > > > > > > -- > > > > Theo Schlossnagle > > > > http://omniti.com/is/theo-schlossnagle > > > > > > > -- > > Theo Schlossnagle > > http://omniti.com/is/theo-schlossnagle > From borising at gmail.com Thu May 1 22:08:45 2014 From: borising at gmail.com (Bo Agerskov Rising) Date: Fri, 02 May 2014 00:08:45 +0200 Subject: [Reconnoiter-devel] Possible to subscribe 2 stratcons to 1 noit? In-Reply-To: References: <5360039D.1070908@gmail.com> <53615C20.9090809@gmail.com> <53616286.2060705@gmail.com> <536167AF.2080407@gmail.com> Message-ID: <5362C5ED.2050307@gmail.com> Thanks :) Theo Schlossnagle wrote: > Those are the IEP feeds and real-time playing feeds. They are fine. > > > On Wed, Apr 30, 2014 at 5:14 PM, Bo Agerskov Rising > wrote: > > I initiated a restart from the console, which apparently didn't > completey restart noitd. After completely restarting it it now doesn't > complain anymore :) > > > > > de01-stratcon > ~0000002 > ~0000001 > dk01-stratcon > > > > Are the 2 ~0000 subsribers supposed to be there? > > Regards, > Bo > > Theo Schlossnagle wrote: > > You restarted the noitd, right? > > > > > > On Wed, Apr 30, 2014 at 4:52 PM, Bo Agerskov Rising > > > >> wrote: > > > > Hmm, after changing that setting, I get the following: > > > > > > > > > > ~0000003 > > de01-stratcon > > ~0000001 > > > > > > > > And still getting the following in the test stratcon: > > > > [2014-04-30 22:49:07.853431] [error] [xxx.yyy.xxx.yyy:43191] > [de01] jlog > > reader[dk01-stratcon] error: JLOG_ERR_INVALID_SUBSCRIBER > > [2014-04-30 22:49:07.853541] [debug] Next jlog_streamer attempt in > > 15000ms > > [2014-04-30 22:49:08.096118] [debug] Pushing transient/iep > batch async > > [144.76.81.4:43191 > ] [de01]: [27553/2204] > > [2014-04-30 22:49:08.096244] [debug] Pushing transient/iep > checkpoint > > [144.76.81.4:43191 > ] [de01]: [27553/2204] > > [2014-04-30 22:49:08.151252] [debug] Pushing transient/iep > batch async > > [192.168.87.14:43191 > ] [dk01]: [2284/95] > > [2014-04-30 22:49:08.151372] [debug] Pushing transient/iep > checkpoint > > [192.168.87.14:43191 > ] [dk01]: [2284/95] > > > > I don't get anything logged in the DB, but see traffic in the IEP. > > > > Any ideas? :) > > > > Regards, > > Bo > > > > Theo Schlossnagle wrote: > > > You want just one of feed log that looks like: > > > > > > > > > > > > > > > On Wed, Apr 30, 2014 at 4:25 PM, Bo Agerskov Rising > > > > > > > > >>> wrote: > > > > > > Hi Theo, > > > > > > After connecting a second stratcon to my noit, I get the > > following error > > > in the test stratcon log (dk01-stratcon): > > > > > > [2014-04-30 22:12:54.039152] [error] [xxx.yyy.xxx.yyy:43191] > > [de01] jlog > > > reader[dk01-stratcon] error: JLOG_ERR_INVALID_SUBSCRIBER > > > > > > This is the config from the noit: > > > > > > > > path="/var/log/noitd.feed(de01-stratcon)"/> > > > > > path="/var/log/noitd.feed(dk01-stratcon)"/> > > > > > > > > > And the result when querying the API: > > > > > > > > > > > > de01-stratcon > > > ~0000006 > > > > > > > > > > > > I remember when I initially configured the noit, that I > had to > > change > > > the conf from using /var/log/noitd.feed(stratcon) to > > > /var/log/noitd.feed(de01-stratcon), before the noit > wanted to have > > > anything to do with the prod stratcon, which has the CN > name of > > > 'de01-stratcon'. > > > > > > Any ideas to whats wrong? > > > > > > Regards, > > > Bo > > > > > > Theo Schlossnagle wrote: > > > > Yes it is. Make sure they have different CNs (in > their SSL > > certs). A > > > > valid signed cert will connect and establish a new > subscriber. > > > Keep in > > > > mind that if it goes away, the noitd will continue to > queue all > > > message > > > > for that node until ti comes back.. If you don't want > this, > > you'll > > > want > > > > to use the noitd's "GET /feed" API and "DELETE > /feed/" > > > API to > > > > find and move the subscriber if you intend to not use > it and > > don't > > > want > > > > backlogged information. > > > > > > > > > > > > On Tue, Apr 29, 2014 at 9:55 PM, Bo Agerskov Rising > > > > > > > > >> > > > > > > > > > >>>> wrote: > > > > > > > > Hi, > > > > > > > > Would it be possible for 2 stratcon's to subscribe > to 1 > > noit? > > > > > > > > It would be nice getting the ability to feed a > > production and > > > a test > > > > environment, with the same live data. > > > > > > > > Regards, > > > > Bo > > > > _______________________________________________ > > > > Reconnoiter-devel mailing list > > > > Reconnoiter-devel at lists.omniti.com > > > > > > > > > >> > > > > > > > > > > > > >>> > > > > > http://lists.omniti.com/mailman/listinfo/reconnoiter-devel > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > Theo Schlossnagle > > > > > > > > http://omniti.com/is/theo-schlossnagle > > > > > > > > > > > > > > > > > > > -- > > > > > > Theo Schlossnagle > > > > > > http://omniti.com/is/theo-schlossnagle > > > > > > > > > > > > > -- > > > > Theo Schlossnagle > > > > http://omniti.com/is/theo-schlossnagle > > > > > > > -- > > Theo Schlossnagle > > http://omniti.com/is/theo-schlossnagle > From jesus at omniti.com Mon May 5 22:07:11 2014 From: jesus at omniti.com (Theo Schlossnagle) Date: Mon, 5 May 2014 18:07:11 -0400 Subject: [Reconnoiter-devel] SNMP Counter32 overflow In-Reply-To: <536789E0.4090904@gmail.com> References: <535F3E76.8050608@gmail.com> <53600184.6010207@gmail.com> <5360D6DC.3030705@gmail.com> <536789E0.4090904@gmail.com> Message-ID: Counter has nothing to do with deviation.. that's counter_dev. counter is the running discrete derivative ignoring negative changes. Basically "counter" is "derive for rolling metrics" On Mon, May 5, 2014 at 8:53 AM, Bo Agerskov Rising wrote: > Hi Theo, > > Have now moved some 100Mbit/s switchports over in my test environment, > which now runs on postgresql 8.4.17 (because of the counter_dev issue > with version 9). > > The polling period is set to 1000 for this switchport, as for your > recommendation. > > But the graph still shows large dropouts, when the counter32 overflows. > I have attached a screenshot(switchport_overflow_derived.png), where it > shows the drop. > > For fun I just tried to set the values to use counter > instead(switchport_overflow_counter.png), and then it shows the traffic > more correct. > > So now I am bit confused, as I thought that the counter was the computed > deviation? > > :) > > Regards, > Bo > > > Theo Schlossnagle wrote: > > No, counter should accommodate for this exact situation. We have > > counters roll ask the time here and things seem to work fine. > > > > On Apr 30, 2014 6:56 AM, "Bo Agerskov Rising" > > wrote: > > > > Hi, > > > > I am still getting the same amount of overflows, as I had before > > changing the period from 60000 to 1000. > > > > Do you think that my other issue with missing counter_dev values, be > > having any effect on this? > > > > Regards, > > Bo > > > > Bo Agerskov Rising wrote: > > > Hi Theo, > > > > > > Until now I had used the default 60s period, will try out setting > > it to > > > 1s for the interfaces that are problematic. > > > > > > Thanks :) > > > > > > Regards, > > > Bo > > > > > > Theo Schlossnagle wrote: > > >> Reconnoiter already should completely accommodate this (in the > > DB). The > > >> difference between the "derive" and "counter" types acknowledge > > >> rollover. "counter" is the rate of change assuming a > monotonically > > >> increasing source, so if the source sample a T1 is of lesser > > value than > > >> the sample at T0, the rate of change during that time interval is > > >> ignored for the purposes of calculating the rate. > > >> > > >> If your interface is rolling the Counter32 too quickly, just up > the > > >> frequency of the check for that interface to every second and you > > should > > >> get fairly accurate numbers. > > >> > > >> > > >> On Tue, Apr 29, 2014 at 7:53 AM, Bo Agerskov Rising > > > > >> >> wrote: > > >> > > >> Hi all, > > >> > > >> I have some HP 1810 switches with 1G interfaces, which I > > monitor with > > >> SNMP. The problem is that these switches only support > > Counter32 values, > > >> and as such the counters quickly overflow when they run at > > max speed. > > >> > > >> I was wondering if anything could be done inside reconnoiter > to > > >> accommodate for this? Or perhaps something smart source math > > inside the > > >> UI? :) > > >> > > >> Have any run into the same issue? > > >> > > >> Thanks in advance > > >> Bo > > >> _______________________________________________ > > >> Reconnoiter-devel mailing list > > >> Reconnoiter-devel at lists.omniti.com > > > > >> > > > > >> http://lists.omniti.com/mailman/listinfo/reconnoiter-devel > > >> > > >> > > >> > > >> > > >> -- > > >> > > >> Theo Schlossnagle > > >> > > >> http://omniti.com/is/theo-schlossnagle > > >> > > > -- Theo Schlossnagle http://omniti.com/is/theo-schlossnagle -------------- next part -------------- An HTML attachment was scrubbed... URL: From borising at gmail.com Tue May 6 04:46:41 2014 From: borising at gmail.com (Bo Agerskov Rising) Date: Tue, 06 May 2014 06:46:41 +0200 Subject: [Reconnoiter-devel] SNMP Counter32 overflow In-Reply-To: References: <535F3E76.8050608@gmail.com> Message-ID: <53686931.1050905@gmail.com> Hi Theo, My mistake .. I had somehow mixed the counter_dev and derive values together in my head :) Now when I look at your email below, I do see that you make references to the counter value. Sorry for the noise. Regards, Bo Theo Schlossnagle wrote: > Reconnoiter already should completely accommodate this (in the DB). The > difference between the "derive" and "counter" types acknowledge > rollover. "counter" is the rate of change assuming a monotonically > increasing source, so if the source sample a T1 is of lesser value than > the sample at T0, the rate of change during that time interval is > ignored for the purposes of calculating the rate. > > If your interface is rolling the Counter32 too quickly, just up the > frequency of the check for that interface to every second and you should > get fairly accurate numbers. > > > On Tue, Apr 29, 2014 at 7:53 AM, Bo Agerskov Rising > wrote: > > Hi all, > > I have some HP 1810 switches with 1G interfaces, which I monitor with > SNMP. The problem is that these switches only support Counter32 values, > and as such the counters quickly overflow when they run at max speed. > > I was wondering if anything could be done inside reconnoiter to > accommodate for this? Or perhaps something smart source math inside the > UI? :) > > Have any run into the same issue? > > Thanks in advance > Bo > _______________________________________________ > Reconnoiter-devel mailing list > Reconnoiter-devel at lists.omniti.com > > http://lists.omniti.com/mailman/listinfo/reconnoiter-devel > > > > > -- > > Theo Schlossnagle > > http://omniti.com/is/theo-schlossnagle > From borising at gmail.com Wed May 28 04:59:48 2014 From: borising at gmail.com (Bo Agerskov Rising) Date: Wed, 28 May 2014 06:59:48 +0200 Subject: [Reconnoiter-devel] counter_dev In-Reply-To: <5362C5D4.4000701@gmail.com> References: <5307C1E3.5080300@gmail.com> <5309CC99.5000404@gmail.com> <5362C5D4.4000701@gmail.com> Message-ID: <53856D44.3070608@gmail.com> Hi again, After experimenting with upgrading from 8.4 -> 9.0 -> 9.1 -> 9.2 -> 9.3, I experienced a breaking change when you upgrade to 9.2, where the counter_dev will not update anymore. Then I made a pg_dumpall of my prod system, running on 9.3.2, and imported it again into an 9.1.13 test system, and after running the rollup script it began to populate the counter_dev again. Any hints to where in the code, one should troubleshoot this issue? Regards, Bo Bo Agerskov Rising wrote: > Just spinned up a postgresql 8.4.17 instance in my testlab, and tested > with the same data coming into my prod 9.3 system, and now the > counter_dev column get's updated properly. > > So it seems to me, that the issue is to be found in version 9.x. > > Regards, > Bo > > Theo Schlossnagle wrote: >> Any time the counter field varies over time you should see a deviation >> over a containing time window) that is non-zero. >> >> e.g. if you looking in the 5m tables and see counters changing, you >> should see a non-zero deviation in the 20m table. >> >> If you're seeing zeros everywhere, something is wrong -- maybe the code >> is broken. >> >> >> On Sun, Feb 23, 2014 at 2:25 AM, Bo Agerskov Rising > > wrote: >> >> Hi Theo, >> >> Thanks for the explanation of the counter_dev field. >> >> In which situations, would I expect data in this field? >> >> Regards, >> Bo >> >> Theo Schlossnagle wrote: >> > IIRC, counter_dev should be the standard deviation of counter >> calculations. >> > >> > There are already rollups in the database. If you were to write >> custom >> > aggregations funtions you'd need to write those yourself in >> PL/pgsql to >> > run over the arrays in those tables. (summing or whatever). >> > >> > >> > On Fri, Feb 21, 2014 at 1:15 PM, Bo Agerskov Rising >> >> > >> wrote: >> > >> > Hi guys, >> > >> > I was wondering whats the idea behind the counter_dev field, >> when you >> > question the stratcon.fetch_dataset function? I can't really >> find any >> > uuid's that uses this field yet, neither any really good >> explanations in >> > the code that explains it in plain layman terms :) >> > >> > Also I am working on making a summarizing function in the UI, >> which >> > would sum the values of a metric hourly, weekly and so forth, >> and was >> > wondering which fields in the DB schema that I would use for that? >> > >> > Regards, >> > Bo >> > _______________________________________________ >> > Reconnoiter-devel mailing list >> > Reconnoiter-devel at lists.omniti.com >> >> > > > >> > http://lists.omniti.com/mailman/listinfo/reconnoiter-devel >> > >> > >> > >> > >> > -- >> > >> > Theo Schlossnagle >> > >> > http://omniti.com/is/theo-schlossnagle >> > >> >> >> >> >> -- >> >> Theo Schlossnagle >> >> http://omniti.com/is/theo-schlossnagle >> From jesus at omniti.com Fri May 30 19:28:47 2014 From: jesus at omniti.com (Theo Schlossnagle) Date: Fri, 30 May 2014 15:28:47 -0400 Subject: [Reconnoiter-devel] counter_dev In-Reply-To: <53856D44.3070608@gmail.com> References: <5307C1E3.5080300@gmail.com> <5309CC99.5000404@gmail.com> <5362C5D4.4000701@gmail.com> <53856D44.3070608@gmail.com> Message-ID: print statements in the routines that do those calculations. That's where I'd start. On Wed, May 28, 2014 at 12:59 AM, Bo Agerskov Rising wrote: > Hi again, > > After experimenting with upgrading from 8.4 -> 9.0 -> 9.1 -> 9.2 -> 9.3, > I experienced a breaking change when you upgrade to 9.2, where the > counter_dev will not update anymore. > > Then I made a pg_dumpall of my prod system, running on 9.3.2, and > imported it again into an 9.1.13 test system, and after running the > rollup script it began to populate the counter_dev again. > > Any hints to where in the code, one should troubleshoot this issue? > > Regards, > Bo > > Bo Agerskov Rising wrote: > > Just spinned up a postgresql 8.4.17 instance in my testlab, and tested > > with the same data coming into my prod 9.3 system, and now the > > counter_dev column get's updated properly. > > > > So it seems to me, that the issue is to be found in version 9.x. > > > > Regards, > > Bo > > > > Theo Schlossnagle wrote: > >> Any time the counter field varies over time you should see a deviation > >> over a containing time window) that is non-zero. > >> > >> e.g. if you looking in the 5m tables and see counters changing, you > >> should see a non-zero deviation in the 20m table. > >> > >> If you're seeing zeros everywhere, something is wrong -- maybe the code > >> is broken. > >> > >> > >> On Sun, Feb 23, 2014 at 2:25 AM, Bo Agerskov Rising >> > wrote: > >> > >> Hi Theo, > >> > >> Thanks for the explanation of the counter_dev field. > >> > >> In which situations, would I expect data in this field? > >> > >> Regards, > >> Bo > >> > >> Theo Schlossnagle wrote: > >> > IIRC, counter_dev should be the standard deviation of counter > >> calculations. > >> > > >> > There are already rollups in the database. If you were to write > >> custom > >> > aggregations funtions you'd need to write those yourself in > >> PL/pgsql to > >> > run over the arrays in those tables. (summing or whatever). > >> > > >> > > >> > On Fri, Feb 21, 2014 at 1:15 PM, Bo Agerskov Rising > >> > >> > >> wrote: > >> > > >> > Hi guys, > >> > > >> > I was wondering whats the idea behind the counter_dev field, > >> when you > >> > question the stratcon.fetch_dataset function? I can't really > >> find any > >> > uuid's that uses this field yet, neither any really good > >> explanations in > >> > the code that explains it in plain layman terms :) > >> > > >> > Also I am working on making a summarizing function in the UI, > >> which > >> > would sum the values of a metric hourly, weekly and so forth, > >> and was > >> > wondering which fields in the DB schema that I would use for > that? > >> > > >> > Regards, > >> > Bo > >> > _______________________________________________ > >> > Reconnoiter-devel mailing list > >> > Reconnoiter-devel at lists.omniti.com > >> > >> > >> > > >> > http://lists.omniti.com/mailman/listinfo/reconnoiter-devel > >> > > >> > > >> > > >> > > >> > -- > >> > > >> > Theo Schlossnagle > >> > > >> > http://omniti.com/is/theo-schlossnagle > >> > > >> > >> > >> > >> > >> -- > >> > >> Theo Schlossnagle > >> > >> http://omniti.com/is/theo-schlossnagle > >> > -- Theo Schlossnagle http://omniti.com/is/theo-schlossnagle -------------- next part -------------- An HTML attachment was scrubbed... URL: