[Reconnoiter-users] SSL read error: Input/output error from stratcon

Theo Schlossnagle jesus at omniti.com
Mon Jun 7 17:09:30 EDT 2010


Make sure all of the certs are signed by /opt/reconnoiter/etc/ca.crt and that it is the same on all the boxes.

It also looks like you are using the same key/cert for stratcond and noitd.  They should be different.  Also the CN used by stratcon's cert is the "subscriber name".  I suggest calling it "CN=stratcon" which makes the default configs work.  Otherwise, that jlog that ends with a (stratcon) in the noit config needs to be updated to reflect the actual CN of the stratcon node.

I think you're damn close. :-D

On Jun 7, 2010, at 4:56 PM, Toby DiPasquale wrote:

> Hi all,
> 
> I've been trying to setup Reconnoiter (svn checkout of Urskek release) all day. I've got noitd and stratcond compiled and running and I finally got the database schema loaded and whatnot but I keep getting the following errors from stratcond:
> 
>  [127.0.0.1:43191] SSL read error: Input/output error
>  Next jlog_streamer attempt in 4000ms
> 
> And the corresponding error from noitd:
> 
>  jlog reader[noit] error: JLOG_ERR_INVALID_SUBSCRIBER
> 
> I've created an OpenSSL CA to use for Reconnoiter, and I've created a cert/key pair for noitd (CN=noit) and another for stratcond (CN=stratcon), signed with my CA key and also installed that CA cert into reconnoiter/etc/ca.crt where both noit.conf and stratcon.conf are looking for it.
> 
> I've read on the mailing list that JLOG_ERR_INVALID_SUBSCRIBER is likely an issue having to do with a bad cert or something but I'm not sure how to proceed since there are no docs. I'm copying my noit.conf and stratcon.conf below so you can see how I have it configured. Both noitd and stratcond are running on localhost right now, as is a collectd instance from which noitd is supposed to be retreiving info. Thanks!
> 
> -=[ noit.conf ]=-
> 
> <?xml version="1.0" encoding="utf8" standalone="yes"?>
> <noit>
>  <eventer implementation="epoll">
>    <config>
>      <default_queue_threads>10</default_queue_threads>
>      <default_ca_chain>/opt/reconnoiter/etc/default-ca-chain.crt</default_ca_chain>
>    </config>
>  </eventer>
>  <logs>
>    <console_output>
>      <outlet name="stderr"/>
>      <log name="error"/>
>      <log name="debug" disabled="true"/>
>    </console_output>
>    <feeds>
>      <log name="feed" type="jlog" path="/var/log/noitd.feed(stratcon)"/>
>    </feeds>
>    <components>
>      <error>
>        <outlet name="error"/>
>        <log name="error/eventer"/>
>        <log name="error/ping_icmp"/>
>        <log name="error/serf"/>
>        <log name="error/snmp"/>
>      </error>
>      <debug>
>        <log name="debug/eventer" disabled="true"/>
>        <log name="debug/ping_icmp" disabled="true"/>
>        <log name="debug/serf" disabled="false"/>
>        <log name="debug/snmp" disabled="true"/>
>      </debug>
>    </components>
>    <feeds>
>      <outlet name="feed"/>
>      <log name="check">
>        <outlet name="error"/>
>      </log>
>      <log name="status"/>
>      <log name="metrics"/>
>      <log name="config"/>
>    </feeds>
>  </logs>
>  <modules directory="/usr/local/libexec/noit">
>    <loader image="lua" name="lua">
>      <config><directory>/usr/local/libexec/noit/?.lua</directory></config>
>    </loader>
>    <module image="selfcheck" name="selfcheck"/>
>    <module image="ssh2" name="ssh2"/>
>    <module image="postgres" name="postgres"/>
>    <module image="collectd" name="collectd"/>
>  </modules>
>  <listeners>
>    <sslconfig>
>      <certificate_file>/opt/reconnoiter/etc/noit.crt</certificate_file>
>      <key_file>/opt/reconnoiter/etc/noit.key</key_file>
>      <ca_chain>/opt/reconnoiter/etc/ca.crt</ca_chain>
>    </sslconfig>
>    <consoles type="noit_console">
>      <listener address="/tmp/noit">
>        <config>
>          <line_protocol>telnet</line_protocol>
>        </config>
>      </listener>
>      <listener address="*" port="32322">
>        <config>
>          <line_protocol>telnet</line_protocol>
>        </config>
>      </listener>
>      <listener address="*" port="32323" ssl="on"/>
>    </consoles>
>    <listener type="control_dispatch" address="*" port="43191" ssl="on">
>      <config>
>        <log_transit_feed_name>feed</log_transit_feed_name>
>      </config>
>    </listener>
>  </listeners>
>  <checks max_initial_stutter="30000" filterset="default">
>    <check uuid="f7cea020-f19d-11dd-85a6-cb6d3a2207dc" module="selfcheck" target="127.0.0.1" period="5000" timeout="4000"/>
>    <check uuid="1b4e28ba-2fa1-11d2-883f-e9b761bde3fb" module="collectd" target="127.0.0.1" period="60000" timeout="30000"/>
>    <check uuid="002d58ff-20ff-4db0-9420-782fc1748dc4" module="ssh2" target="127.0.0.1" period="60000" timeout="4000"/>
>    <databases>
>      <postgres module="postgres" period="300000">
>        <config>
>          <dsn>host=127.0.0.1 dbname=junk user=junk password=junk</dsn>
>          <sql>select datname, pg_database_size(datname) as size, xact_commit, xact_rollback from pg_stat_database</sql>
>        </config>
>        <check uuid="8c5ca46c-77d7-11dd-ab5b-53bc659517d6" target="127.0.0.1" timeout="4000"/>
>      </postgres>
>    </databases>
>  </checks>
>  <filtersets>
>    <filterset name="default">
>      <rule type="deny" module="^ping_icmp$" metric="^(?:minimum|maximum|count)$" />
>    </filterset>
>  </filtersets>
> </noit>
> 
> -=[ stratcon.conf ]=-
> 
> <?xml version="1.0" encoding="utf8" standalone="yes"?>
> <stratcon>
>  <eventer implementation="epoll"/>
> 
>  <logs>
>    <console_output>
>      <outlet name="stderr"/>
>      <log name="error"/>
>      <log name="debug"/>
>      <log name="error/iep"/>
>      <log name="error/eventer" disabled="true"/>
>      <log name="debug/eventer" disabled="true"/>
>    </console_output>
>  </logs>
> 
>  <noits>
>    <config>
>      <!--
>        If we have a connection failure, attempt to reconnect
>        immediately.  Upon failure wait 1000ms (1s) and
>        exponentially backoff up to 900000ms (900s or 15m)
>      -->
>      <reconnect_initial_interval>1000</reconnect_initial_interval>
>      <reconnect_maximum_interval>15000</reconnect_maximum_interval>
>    </config>
>    <sslconfig>
>      <key_file>/opt/reconnoiter/etc/noit.key</key_file>
>      <certificate_file>/opt/reconnoiter/etc/noit.crt</certificate_file>
>      <ca_chain>/opt/reconnoiter/etc/ca.crt</ca_chain>
>    </sslconfig>
>    <noit address="127.0.0.1" port="43191" />
>  </noits>
> 
>  <iep disabled="false"> <!-- false the default -->
>    <start directory="/opt/reconnoiter/var/db" command="/opt/reconnoiter/bin/run-iep.sh" />
>    <queries>
>      <statement id="6cc613a4-7f9c-11de-973f-db7e8ccb2e5c" provides="CheckDetails-ddl">
>        <epl>create window CheckDetails.std:unique(uuid).win:keepall() as NoitCheck</epl>
>      </statement>
>      <statement id="76598f5e-7f9c-11de-9f5b-ebb4dcb2494e" provides="CheckDetails">
>        <requires>CheckDetails-ddl</requires>
>        <epl>insert into CheckDetails select * from NoitCheck</epl>
>      </statement>
>      <statement id="ba189f08-7f99-11de-9013-733772d37479" provides="UnavailableStream">
>        <requires>CheckDetails</requires>
>        <epl>insert into UnavailableStream
>             select p.* as delta, cds.target as target, cds.module as module,
>                    cds.name as name, p.s.uuid as uuid
>             from pattern [ every
>                            s=NoitStatus(availability='A') ->
>                            ( n0 = NoitStatus(uuid=s.uuid, availability='U')
>                              and not NoitStatus(uuid=s.uuid, availability='A'))
>                          ].std:lastevent() as p
>             inner join CheckDetails as cds on cds.uuid = p.s.uuid
>        </epl>
>      </statement>
>      <query id="ce6bf8d2-3dd7-11de-a45c-a7df160cba9e" topic="status">
>        <epl>select * from NoitStatus</epl>
>      </query>
>    </queries>
>  </iep>
> 
>  <database>
>    <dbconfig>
>      <host>localhost</host>
>      <dbname>reconnoiter</dbname>
>      <user>stratcon</user>
>      <password>foobar</password>
>    </dbconfig>
>    <statements>
>      <allchecks><![CDATA[
>        SELECT remote_address, id, target, module, name
>          FROM stratcon.mv_loading_dock_check_s
>      ]]></allchecks>
>      <findcheck><![CDATA[
>        SELECT remote_address, id
>          FROM stratcon.mv_loading_dock_check_s
>         WHERE sid = $1
>      ]]></findcheck>
>      <check><![CDATA[
>        INSERT INTO stratcon.loading_dock_check_s
>                    (remote_address, whence, sid, id, target, module, name)
>             VALUES ($1, 'epoch'::timestamptz + ($2 || ' seconds')::interval,
>                     stratcon.generate_sid_from_id($3), $3, $4, $5, $6)
>      ]]></check>
>      <status><![CDATA[
>        INSERT INTO stratcon.loading_dock_status_archive_%Y%m
>                    ( whence,sid, state, availability,
>                     duration, status)
>             VALUES ('epoch'::timestamptz + ($1 || ' seconds')::interval,
>                     stratcon.generate_sid_from_id($2), $3, $4, $5, $6)
>      ]]></status>
>      <metric_numeric><![CDATA[
>        INSERT INTO stratcon.loading_dock_metric_numeric_archive_%Y%m
>                    (whence, sid, name, value)
>             VALUES ( 'epoch'::timestamptz + ($1 || ' seconds')::interval,
>                     stratcon.generate_sid_from_id($2), $3, $4)
>      ]]></metric_numeric>
>      <metric_text><![CDATA[
>        INSERT INTO stratcon.loading_dock_metric_text_archive_%Y%m
>                    ( whence, sid, name,value)
>             VALUES ('epoch'::timestamptz + ($1 || ' seconds')::interval,
>                     stratcon.generate_sid_from_id($2), $3, $4)
>      ]]></metric_text>
>      <config><![CDATA[
>        SELECT stratcon.update_config
>               ($1, $2, 
>                'epoch'::timestamptz + ($3 || ' seconds')::interval,
>                $4 )
>      ]]></config>
>    </statements>
>  </database>
> 
>  <listeners>
>    <sslconfig>
>      <key_file>/opt/reconnoiter/etc/stratcon.key</key_file>
>      <certificate_file>/opt/reconnoiter/etc/stratcon.crt</certificate_file>
>      <ca_chain>/opt/reconnoiter/etc/ca.crt</ca_chain>
>    </sslconfig>
>    <consoles type="noit_console">
>      <listener address="/tmp/stratcon">
>        <config><line_protocol>telnet</line_protocol></config>
>      </listener>
>    </consoles>
>    <realtime type="http_rest_api">
>      <listener address="*" port="8008">
>        <config>
>          <hostname>stratcon.localdomain</hostname>
>          <document_domain>localdomain</document_domain>
>        </config>
>      </listener>
>    </realtime>
>    <listener type="control_dispatch" address="*" port="43191" ssl="on" />
>  </listeners>
> 
> </stratcon>
> 
> _______________________________________________
> Reconnoiter-users mailing list
> Reconnoiter-users at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/reconnoiter-users

--
Theo Schlossnagle
http://omniti.com/is/theo-schlossnagle








More information about the Reconnoiter-users mailing list