KHIKA Visualizations

What is a KHIKA Visualization?

This is the feature that enables you to see your data in a user friendly, analytical fashion. Various options eg. Pie chart, Bar graph, Metrics, Heat maps, Line graphs etc are available to represent your data.

Visualizations, just like their name suggests are aimed at providing pictorial representation of your data. They enable quick insight into data trends, anomalies and hence behaviour of network components and traffic. It is a self-help feature. In the Visualize menu, as shall be explained later in detail, you can create a new one easily by selecting the pictorial options and placing your data / fields on it. Your data fields are brought up here for selection by ‘KHIKA’. You can create your own visualization like bar-graph, pie-charts etc, save it for later use into a dashboard or just for independent view.

When you Click “Visualize” on left panel, you land-up on the visualization page. It shows up the entire list of existing visualizations.

• You can click on any one from the list to see it independently.
• You can further edit this selected visualization.
• You can also build your own visualizations from here by clicking the “+” button.

viz1

The next screen has options of different types of visualizations to select from.

viz2

Say, if you click on Pie Chart, it shall open up the settings page to create a new pie chart. We shall see settings for each type of visualization, in more detail. You can view the pre-built visualisations shipped with KHIKA.

Creating a Visualization

The options to create new visualizations are discussed below.

Basic Charts

Line, Area and Bar charts	Compare different series in X/Y charts.
Heat maps	Shade cells within a matrix.
Pie chart	Display each source’s contribution to a total.

Data

Data table	Display the raw data of a composed aggregation.
Metric	Display a single number.
Goal and Gauge	Display a gauge.

There are two ways to feed the data to your visualization. You can either select an index and provide a query or you may select a saved search.

Specify a search query to retrieve the data for your visualization:

• To enter new search criteria, select the index pattern for the indices that contain the data you want to visualize. As explained before, the name of the index pattern for the raw logs is like this (say raw logs of linux workspace) business-linux_4-raw-linux-* and name of the index of the calculated data (summarized data derived from raw logs) is - business-linux_4-raw-linux_rpt-* . Refer to the section on selecting index patterns. This opens the visualization builder with a wildcard query that matches all of the documents in the selected indices.
• To build a visualization from a saved search, click the name of the saved search you want to use. This opens the visualization builder and loads the selected query.

Please Note : When you build a visualization from a saved search, any subsequent modifications to the saved search are automatically reflected in the visualization. To disable automatic updates, you can disconnect a visualization from the saved search.

In the visualization builder, choose a suitable metric aggregation for the Y axis. This decides how the data is counted.

Metric Aggregations:

• count
• average
• sum
• min
• max
• standard deviation
• unique count
• median (50th percentile)
• percentiles
• percentile ranks
• top hit

Parent Pipeline Aggregations:

• derivative
• cumulative sum
• moving average
• serial diff

Sibling Pipeline Aggregations:

• average bucket
• sum bucket
• min bucket
• max bucket

For X axis, select a bucket aggregation. This decides how the data is grouped and aggregated

• Date histogram
• Range
• Terms
• Filters
• Significant terms

For example, You could create a bar chart showing distribution of requests by real IP by specifying a Terms aggregation on “real_ip” field:

viz3

The y-axis shows the number of requests from each real IP, and the real IPs are displayed across the x-axis. Bar, line, or area chart visualizations use metrics for the y-axis and buckets for the x-axis. Buckets are analogous to SQL GROUP BY statements. Pie charts, use the metric for the slice size and the bucket for the number of slices. You can further break down the data by specifying sub aggregations. The first aggregation determines the data set for any subsequent aggregations. Sub aggregations are applied in order—you can drag the aggregations to change the order in which they’re applied.

For example, you could add a terms sub aggregation on the url field to the real IP of Origin bar chart to see the URLs those requests were targeting.

viz4

Saving a filter on the Visualisation

viz5

Apart from the Search box at the top and the settings panel on the left, there is another option to add a filter by query. Click on “Add a Filter” A pop up opens. There are dropdowns to select fields, logic like AND / OR and create a filter query. When visualisation is saved, this gets saved with it and reflects on the visualisation till it is removed.

Line, Area, Bar charts, Pie charts and Data Table

Line, Area, and Bar charts allow you to plot your data on X/Y axis. First, you should select your metrics which define Value axis.

Metric Aggregations:

Count The count aggregation returns a raw count of the elements in the selected index pattern. Average This aggregation returns the average of a numeric field. Select a field from the drop-down. Sum The sum aggregation returns the total sum of a numeric field. Select a field from the drop-down. Min The min aggregation returns the minimum value of a numeric field. Select a field from the drop-down. Max The max aggregation returns the maximum value of a numeric field. Select a field from the drop-down.

Unique Count The cardinality aggregation returns the number of unique values in a field. Select a field from the drop-down. Standard Deviation The extended stats aggregation returns the standard deviation of data in a numeric field. Select a field from the drop-down. Top Hit The top hits aggregation returns one or more of the top values from a specific field in your documents. Select a field from the drop-down, how you want to sort the documents and choose the top fields, and how many values should be returned. Percentiles The percentile aggregation divides the values in a numeric field into percentile bands that you specify. Select a field from the drop-down, then specify one or more ranges in the Percentiles fields. Click the X to remove a percentile field. Click + Add to add a percentile field. Percentile Rank The percentile ranks aggregation returns the percentile rankings for the values in the numeric field you specify. Select a numeric field from the drop-down, then specify one or more percentile rank values in the Values fields. Click the X to remove a values field. Click +Add to add a values field.

Parent Pipeline Aggregations:

For each of the parent pipeline aggregations you have to define the metric for which the aggregation is calculated. That could be one of your existing metrics or a new one. You can also nest this aggregations (for example to produce 3rd derivative) Derivative The derivative aggregation calculates the derivative of specific metrics. Cumulative Sum The cumulative sum aggregation calculates the cumulative sum of a specified metric in a parent histogram Moving Average The moving average aggregation will slide a window across the data and emit the average value of that window Serial Diff The serial differencing is a technique where values in a time series are subtracted from itself at different time lags or period

Sibling Pipeline Aggregations:

Just like with parent pipeline aggregations you need to provide a metric for which to calculate the sibling aggregation. On top of that you also need to provide a bucket aggregation which will define the buckets on which the sibling aggregation will run Average Bucket The avg bucket calculates the (mean) average value of a specified metric in a sibling aggregation Sum Bucket The sum bucket calculates the sum of values of a specified metric in a sibling aggregation Min Bucket The min bucket calculates the minimum value of a specified metric in a sibling aggregation Max Bucket The max bucket calculates the maximum value of a specified metric in a sibling aggregation You can add an aggregation by clicking the + Add Metrics button. Enter a string in the Custom Label field to change the display label.

The buckets aggregations determine what information is being retrieved from your data set. Before you choose a buckets aggregation, specify if you are splitting slices within a single chart or splitting into multiple charts. A multiple chart split must run before any other aggregations. When you split a chart, you can change if the splits are displayed in a row or a column by clicking the Rows | Columns selector.

The X axis of this chart is the buckets axis. You can define buckets for the X axis, for a split area on the chart, or for split charts.

This chart’s X axis supports the following aggregations.

Date Histogram A date histogram is built from a numeric field and organized by date. You can specify a time frame for the intervals in seconds, minutes, hours, days, weeks, months, or years. You can also specify a custom interval frame by selecting Custom as the interval and specifying a number and a time unit in the text field. Custom interval time units are s for seconds, m for minutes, h for hours, d for days, w for weeks, and y for years. Different units support different levels of precision, down to one second. Intervals are labelled at the start of the interval, using the date-key returned by ElasticSearch. For example, tooltip for a monthly interval will show the first day of the month.

Histogram A standard histogram is built from a numeric field. Specify an integer interval for this field. Select the Show empty buckets checkbox to include empty intervals in the histogram.

Range With a range aggregation, you can specify ranges of values for a numeric field. Click Add Range to add a set of range endpoints. Click the red (x) symbol to remove a range.

Date Range A date range aggregation reports values that are within a range of dates that you specify. You can specify the ranges for the dates using date math expressions. Click Add Range to add a set of range endpoints. Click the red (x) symbol to remove a range.

IPv4 Range The IPv4 range aggregation enables you to specify ranges of IPv4 addresses. Click Add Range to add a set of range endpoints. Click the red (x) symbol to remove a range.

Terms A terms aggregation enables you to specify the top or bottom n elements of a given field to display, ordered by count or a custom metric.

Filters You can specify a set of filters for the data. You can specify a filter as a query string or in JSON format, just as in the Discover search bar. Click Add Filter to add another filter. Click the label button to open the label field, where you can type in a name to display on the visualization.

Significant Terms Displays the results of the experimental significant terms aggregation.

Once you’ve specified an X axis aggregation, you can define sub-aggregations to refine the visualization. Click + Add Sub Aggregation to define a sub-aggregation, then choose Split Area or Split Chart, then select a sub-aggregation from the list of types.

When multiple aggregations are defined on a chart’s axis, you can use the up or down arrows to the right of the aggregation’s type to change the aggregation’s priority.

Enter a string in the Custom Label field to change the display label.

You can customize the colors of your visualization by clicking the color dot next to each label to display the color picker.

viz6

==== You can click the Advanced link to display more customization options for your metrics or bucket aggregation:

====

Exclude Pattern Specify a pattern in this field to exclude from the results. Include Pattern Specify a pattern in this field to include in the results.

Metrics & Axes

Select the Metrics & Axes tab to change the way each individual metric is shown on the chart. The data series are styled in the Metrics section, while the axes are styled in the X and Y axis sections. Metrics Modify how each metric from the Data panel is visualized on the chart. Chart type Choose between Area, Line, and Bar types. Mode stack the different metrics, or plot them next to each other Value Axis choose the axis you want to plot this data too (the properties of each are configured under Y-axes). Line mode should the outline of lines or bars appear smooth, straight, or stepped.

Y-axis Style all the Y-axes of the chart. Position position of the Y-axis (left or right for vertical charts, and top or bottom for horizontal charts). Scale type scaling of the values (linear, log, or square root)

Advanced Options Labels - Show Labels Allows you to hide axis labels Labels - Filter Labels If filter labels is enabled some labels will be hidden in case there is not enough space to display them Labels - Rotate You can enter the number in degrees for how much you want to rotate labels Labels - Truncate You can enter the size in pixels to which the label is truncated Scale to Data Bounds The default Y-axis bounds are zero and the maximum value returned in the data. Check this box to change both upper and lower bounds to match the values returned in the data. Custom Extents You can define custom minimum and maximum for each axis

X-Axis Position position of the X-Axis (left or right for horizontal charts, and top or bottom for vertical charts).

Advanced Options Labels - Show Labels Allows you to hide axis labels Labels - Filter Labels If filter labels is enabled some labels will be hidden in case there is not enough spave to display them Labels - Rotate You can enter the number in degrees for how much you want to rotate labels Labels - Truncate You can enter the size in pixels to which the label is truncated

Panel Settings These are options that apply to the entire chart and not just the individual data series. Common options edit Legend Position Move your legend to the left, right, top or bottom Show Tooltip Enables or disables the display of tooltip on hovering over chart objects Current Time Marker Show a line indicating the current time

Grid options You can enable grid on the chart. By default grid is displayed on the category axis only. X-axis You can disable the display of grid lines on category axis Y-axis You can choose on which (if any) of the value axes you want to display grid lines