Chapter 3. stratcond

Table of Contents

3.1. stratcond Design

stratcond is responsible for pulling collected metrics from noitd processes running in the field. It will connect to noitd over an SSL-protected TCP transport on port 34332 (or another port of your choosing), then noitd will push a dataset to stratcond. stratcond will transform this dataset and insert it into the PostgreSQL datastore.

Data collected by noitd is stored in a jlog journal which is a segment-based, publish-subscribe journal. Adhering to the publish-subscribe model, stratcond must be a subscriber to this log and the common name (CN) in the client-side SSL certificate that stratcond presents to noitd upon connecting is used as the jlog subscriber name.

See the JLog project page for more information about JLog.

3.1. stratcond Design

stratcond is simple in both purpose and design. All connections to remote noitd deamons are managed in a single thread in the event system. As data is received from each noitd daemon, it is pushed onto a datastore queue for that daemon. A database connection is established for each noitd daemon (each in its own thread) and those threads consume the datastore queue, transforming and inserting data into PostgreSQL.

Inserts are performed in batches. At the end of each batch, stratcond will commit the database transaction and advise the the remote noitd to checkpoint the subscriber position. This two-point commit is not XA-safe and there is a possibility that data could be committed to the datastore and the subscriber checkpoint not be updated. This will result in an attempt to repeat the data insertion, resulting in primary key violations which are ignored.

stratcond uses the same configuration processing system as noitd. It is XML-based and provides facilities for defining the SQL statements executed against PostgreSQL, the connection parameters, and the various remote noitd instances to which it should connect.