root/docs/arch/stratcond.xml

Revision 23dea7e00df87480acb58bb3398283c2bb227949, 3.0 kB (checked in by Theo Schlossnagle <jesus@omniti.com>, 3 years ago)

pull docs into master

  • Property mode set to 100644
Line 
1 <?xml version='1.0' encoding='UTF-8' ?>
2 <!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.4//EN"
3   "http://www.oasis-open.org/docbook/xml/4.4/docbookx.dtd"
4 [
5   <!ENTITY % magic.fixup SYSTEM "http://labs.omniti.com/docbook/ent">
6   %magic.fixup;
7 ]>
8 <chapter id="arch.stratcond">
9   <title>stratcond</title>
10   <para>
11     <command>stratcond</command> is responsible for pulling collected metrics
12     from <command>noitd</command> processes running in the field.  It will
13     connect to <command>noitd</command> over an SSL-protected TCP transport on
14     port 34332 (or another port of your choosing), then
15     <command>noitd</command> will push a dataset to
16     <command>stratcond</command>.  <command>stratcond</command> will transform
17     this dataset and insert it into the PostgreSQL datastore.
18   </para>
19   <para>
20     Data collected by <command>noitd</command> is stored in a jlog journal
21     which is a segment-based, publish-subscribe journal.  Adhering to the
22     publish-subscribe model, <command>stratcond</command> must be a subscriber
23     to this log and the common name (CN) in the client-side SSL certificate
24     that <command>stratcond</command> presents to <command>noitd</command> upon
25     connecting is used as the jlog subscriber name.
26   </para>
27   <para>
28     See the
29     <ulink url="http://labs.omniti.com/trac/jlog"><citetitle>JLog</citetitle></ulink>
30     project page for more information about JLog.
31   </para>
32   <section id="arch.stratcond.design">
33     <title>stratcond Design</title>
34     <para>
35       <command>stratcond</command> is simple in both purpose and design.  All
36       connections to remote <command>noitd</command> deamons are managed in a
37       single thread in the event system.  As data is received from each
38       <command>noitd</command> daemon, it is pushed onto a datastore queue for
39       that daemon.  A database connection is established for each
40       <command>noitd</command> daemon (each in its own thread) and those
41       threads consume the datastore queue, transforming and inserting data into
42       PostgreSQL.
43     </para>
44     <para>
45       Inserts are performed in batches.  At the end of each batch,
46       <command>stratcond</command> will commit the database transaction and
47       advise the the remote <command>noitd</command> to checkpoint the
48       subscriber position.  This two-point commit is not XA-safe and there is
49       a possibility that data could be committed to the datastore and the
50       subscriber checkpoint not be updated.  This will result in an attempt to
51       repeat the data insertion, resulting in primary key violations which are
52       ignored.
53     </para>
54     <para>
55       <command>stratcond</command> uses the same configuration processing
56       system as <command>noitd</command>.  It is XML-based and provides
57       facilities for defining the SQL statements executed against PostgreSQL,
58       the connection parameters, and the various remote
59       <command>noitd</command> instances to which it should connect.
60     </para>
61   </section>
62 </chapter>
63 <!--
64 vim:ts=2:sw=2:et:
65 -->
Note: See TracBrowser for help on using the browser.