KHIKA Visualizations

What is a KHIKA Visualization?

This is the feature that enables you to see your data in a user friendly, analytical fashion. Various options eg. Pie chart, Bar graph, Metrics, Heat maps, Line graphs etc are available to represent your data.

Visualizations, just like their name suggests are aimed at providing pictorial representation of your data. They enable quick insight into data trends, anomalies and hence behaviour of network components and traffic. It is a self-help feature. In the Visualize menu, as shall be explained later in detail, you can create a new one easily by selecting the pictorial options and placing your data / fields on it. Your data fields are brought up here for selection by ‘KHIKA’. You can create your own visualization like bar-graph, pie-charts etc, save it for later use into a dashboard or just for independent view.

When you Click “Visualize” on left panel, you land-up on the visualization page. It shows up the entire list of existing visualizations.

• You can click on any one from the list to see it independently.
• You can further edit this selected visualization.
• You can also build your own visualizations from here by clicking the “+” button.

viz1

The next screen has options of different types of visualizations to select from.

viz2

Say, if you click on Pie Chart, it shall open up the settings page to create a new pie chart. We shall see settings for each type of visualization, in more detail. You can view the pre-built visualisations shipped with KHIKA.

Creating a Visualization

The options to create new visualizations are discussed below.

Basic Charts

Line, Area and Bar charts	Compare different series in X/Y charts.
Heat maps	Shade cells within a matrix.
Pie chart	Display each source’s contribution to a total.

Data

Data table	Display the raw data of a composed aggregation.
Metric	Display a single number.
Goal and Gauge	Display a gauge.

There are two ways to feed the data to your visualization. You can either select an index and provide a query or you may select a saved search.

Specify a search query to retrieve the data for your visualization:

• To enter new search criteria, select the index pattern for the indices that contain the data you want to visualize. As explained before, the name of the index pattern for the raw logs is like this (say raw logs of linux workspace) business-linux_4-raw-linux-* and name of the index of the calculated data (summarized data derived from raw logs) is - business-linux_4-raw-linux_rpt-* . Refer to the section on selecting index patterns. This opens the visualization builder with a wildcard query that matches all of the documents in the selected indices.
• To build a visualization from a saved search, click the name of the saved search you want to use. This opens the visualization builder and loads the selected query.

Please Note : When you build a visualization from a saved search, any subsequent modifications to the saved search are automatically reflected in the visualization. To disable automatic updates, you can disconnect a visualization from the saved search.

In the visualization builder, choose a suitable metric aggregation for the Y axis. This decides how the data is counted.

Metric Aggregations:

• count
• average
• sum
• min
• max
• standard deviation
• unique count
• median (50th percentile)
• percentiles
• percentile ranks
• top hit

Parent Pipeline Aggregations:

• derivative
• cumulative sum
• moving average
• serial diff

Sibling Pipeline Aggregations:

• average bucket
• sum bucket
• min bucket
• max bucket

For X axis, select a bucket aggregation. This decides how the data is grouped and aggregated

• Date histogram
• Range
• Terms
• Filters
• Significant terms

For example, You could create a bar chart showing distribution of requests by real IP by specifying a Terms aggregation on “real_ip” field:

viz3

The y-axis shows the number of requests from each real IP, and the real IPs are displayed across the x-axis. Bar, line, or area chart visualizations use metrics for the y-axis and buckets for the x-axis. Buckets are analogous to SQL GROUP BY statements. Pie charts, use the metric for the slice size and the bucket for the number of slices. You can further break down the data by specifying sub aggregations. The first aggregation determines the data set for any subsequent aggregations. Sub aggregations are applied in order—you can drag the aggregations to change the order in which they’re applied. For example, you could add a terms sub aggregation on the url field to the real IP of Origin bar chart to see the URLs those requests were targeting.

viz4

Saving a filter on the Visualisation

viz5

Apart from the Search box at the top and the settings panel on the left, there is another option to add a filter by query. Click on “Add a Filter” A pop up opens. There are dropdowns to select fields, logic like AND / OR and create a filter query. When visualisation is saved, this gets saved with it and reflects on the visualisation till it is removed.

Line, Area, Bar charts, Pie charts and Data Table

Line, Area, Bar charts, Pie charts and Data table Line, Area, and Bar charts allow you to plot your data on X/Y axis. First, you should select your metrics which define Value axis. Metric Aggregations: Count The count aggregation returns a raw count of the elements in the selected index pattern. Average This aggregation returns the average of a numeric field. Select a field from the drop-down. Sum The sum aggregation returns the total sum of a numeric field. Select a field from the drop-down. Min The min aggregation returns the minimum value of a numeric field. Select a field from the drop-down. Max The max aggregation returns the maximum value of a numeric field. Select a field from the drop-down.

Unique Count The cardinality aggregation returns the number of unique values in a field. Select a field from the drop-down. Standard Deviation The extended stats aggregation returns the standard deviation of data in a numeric field. Select a field from the drop-down. Top Hit The top hits aggregation returns one or more of the top values from a specific field in your documents. Select a field from the drop-down, how you want to sort the documents and choose the top fields, and how many values should be returned. Percentiles The percentile aggregation divides the values in a numeric field into percentile bands that you specify. Select a field from the drop-down, then specify one or more ranges in the Percentiles fields. Click the X to remove a percentile field. Click + Add to add a percentile field. Percentile Rank The percentile ranks aggregation returns the percentile rankings for the values in the numeric field you specify. Select a numeric field from the drop-down, then specify one or more percentile rank values in the Values fields. Click the X to remove a values field. Click +Add to add a values field. Parent Pipeline Aggregations: For each of the parent pipeline aggregations you have to define the metric for which the aggregation is calculated. That could be one of your existing metrics or a new one. You can also nest this aggregations (for example to produce 3rd derivative) Derivative The derivative aggregation calculates the derivative of specific metrics. Cumulative Sum The cumulative sum aggregation calculates the cumulative sum of a specified metric in a parent histogram Moving Average The moving average aggregation will slide a window across the data and emit the average value of that window Serial Diff The serial differencing is a technique where values in a time series are subtracted from itself at different time lags or period Sibling Pipeline Aggregations: Just like with parent pipeline aggregations you need to provide a metric for which to calculate the sibling aggregation. On top of that you also need to provide a bucket aggregation which will define the buckets on which the sibling aggregation will run Average Bucket The avg bucket calculates the (mean) average value of a specified metric in a sibling aggregation Sum Bucket The sum bucket calculates the sum of values of a specified metric in a sibling aggregation Min Bucket The min bucket calculates the minimum value of a specified metric in a sibling aggregation Max Bucket The max bucket calculates the maximum value of a specified metric in a sibling aggregation You can add an aggregation by clicking the + Add Metrics button. Enter a string in the Custom Label field to change the display label. The buckets aggregations determine what information is being retrieved from your data set. Before you choose a buckets aggregation, specify if you are splitting slices within a single chart or splitting into multiple charts. A multiple chart split must run before any other aggregations. When you split a chart, you can change if the splits are displayed in a row or a column by clicking the Rows | Columns selector. The X axis of this chart is the buckets axis. You can define buckets for the X axis, for a split area on the chart, or for split charts. This chart’s X axis supports the following aggregations. Click the linked name of each aggregation to visit the main Elasticsearch documentation for that aggregation. Date Histogram A date histogram is built from a numeric field and organized by date. You can specify a time frame for the intervals in seconds, minutes, hours, days, weeks, months, or years. You can also specify a custom interval frame by selecting Custom as the interval and specifying a number and a time unit in the text field. Custom interval time units are s for seconds, m for minutes, h for hours, d for days, w for weeks, and y for years. Different units support different levels of precision, down to one second. Intervals are labelled at the start of the interval, using the date-key returned by ElasticSearch. For example, tooltip for a monthly interval will show the first day of the month. Histogram A standard histogram is built from a numeric field. Specify an integer interval for this field. Select the Show empty buckets checkbox to include empty intervals in the histogram. Range With a range aggregation, you can specify ranges of values for a numeric field. Click Add Range to add a set of range endpoints. Click the red (x) symbol to remove a range. Date Range A date range aggregation reports values that are within a range of dates that you specify. You can specify the ranges for the dates using date math expressions. Click Add Range to add a set of range endpoints. Click the red (x) symbol to remove a range. IPv4 Range The IPv4 range aggregation enables you to specify ranges of IPv4 addresses. Click Add Range to add a set of range endpoints. Click the red (x) symbol to remove a range. Terms A terms aggregation enables you to specify the top or bottom n elements of a given field to display, ordered by count or a custom metric. Filters You can specify a set of filters for the data. You can specify a filter as a query string or in JSON format, just as in the Discover search bar. Click Add Filter to add another filter. Click the label button to open the label field, where you can type in a name to display on the visualization. Significant Terms Displays the results of the experimental significant terms aggregation. Once you’ve specified an X axis aggregation, you can define sub-aggregations to refine the visualization. Click + Add Sub Aggregation to define a sub-aggregation, then choose Split Area or Split Chart, then select a sub-aggregation from the list of types. When multiple aggregations are defined on a chart’s axis, you can use the up or down arrows to the right of the aggregation’s type to change the aggregation’s priority. Enter a string in the Custom Label field to change the display label. You can customize the colors of your visualization by clicking the color dot next to each label to display the color picker.