Difference between revisions of "Writing advanced adapters"

From khika
Jump to navigation Jump to search
Line 6: Line 6:
 
**read the raw log messages one-by-one (from source such as files, queues, APIs, Databases etc),  
 
**read the raw log messages one-by-one (from source such as files, queues, APIs, Databases etc),  
 
**parse the log messages,  
 
**parse the log messages,  
**convert it in [[Khika Data Format|<span style="color:#0000ff"> Khika Data Format </span>]] </span>
+
**convert it in [[Khika Data Format|<span style="color:#0000ff"> Khika Data Format </span>]]
 
**Write the output to stdout
 
**Write the output to stdout
 +
*KHIKA Data Aggregator ''pipes'' the output of the Aggregator script and send it to KHIKA over a SSL connection

Revision as of 07:58, 31 May 2019

After understanding Khika Data Format and going through the initial exercise of Writing you own KHIKA Data Adapters , it is the time to create a production level KHIKA Adapter. A few points to note here before we begin writing our own Adapter:

  • Adapters are scripts that execute on KHIKA Data Aggregator
  • Adapters can be written in any programming language (our favorite is python 2.7)
  • Adapters are scheduled processes and KHIKA Data Aggregator is responsible for scheduling them to run at a periodic interval (typically 1 minute to 5 minutes)
  • The Adapter scripts
    • read the raw log messages one-by-one (from source such as files, queues, APIs, Databases etc),
    • parse the log messages,
    • convert it in Khika Data Format
    • Write the output to stdout
  • KHIKA Data Aggregator pipes the output of the Aggregator script and send it to KHIKA over a SSL connection