Writing advanced adapters

After understanding Khika Data Format and going through the initial exercise of Writing you own KHIKA Data Adapters , it is the time to create a production level KHIKA Adapter. A few points to note here before we begin writing our own Adapter:

• Adapters are scripts that execute on KHIKA Data Aggregator
• Adapters can be written in any programming language (our favorite is python 2.7)
• Adapters are scheduled processes and KHIKA Data Aggregator is responsible for scheduling them to run at a periodic interval (typically 1 minute to 5 minutes)
• The Adapter scripts
  • read the raw log messages one-by-one (from source such as files, queues, APIs, Databases etc),
  • parse the log messages,
  • convert it in Khika Data Format
  • Write the output to stdout
• KHIKA Data Aggregator pipes the output of the Aggregator script and send it to KHIKA over a SSL connection

With these concepts in mind, let proceed with an example of a production ready KHIKA Data Adapter. Login to your KHIKA Data Aggregator node (default username/password is khika/khika123). We will study a syslog based adapter that processes the messages received from a PaloAlto Firewall. Open file TLHook_Adaptor_PaloAlto.py from directory /opt/KHIKA/Apps/Adapters/PaloAltoFW.

Check first few lines of this file where we import some important python modules

     1 #!/bin/env python
     2 import os, sys
     3 import socket
     4 import csv,StringIO
     5 import logging
     6 import re
     7 import time
     8 from time import strptime
     9 from datetime import datetime
    10 import random
    11 import calendar
    12 from ipaddress import IPv4Network,IPv4Address
    13 import pdb

Go the bottom of this file and locate function "__main__". This is start of execution of the code.