Friday Night Wiki
TopicsAbout
Buy me a coffee
Kibana Tutorial

Discover: Exploring Your Data

Discover is Kibana's data exploration interface. It's where you search, filter, and examine documents from your Elasticsearch indices.

Opening Discover

  1. Click Discover in the navigation menu
  2. Select a data view from the dropdown (e.g., kibana_sample_data_ecommerce)
  3. You'll see a table of documents and a histogram showing document count over time

The Discover Interface

┌─────────────────────────────────────────────────────────────┐
│  Data View: [kibana_sample_data_ecommerce ▼]  [Time Picker] │
├─────────────────────────────────────────────────────────────┤
│  Search bar (KQL or Lucene)                                 │
│  [+Add filter]                                              │
├─────────────────────────────────────────────────────────────┤
│                    Histogram                                │
│     █                                                       │
│   █ █ █ █                                                   │
│ █ █ █ █ █ █                                                 │
├────────┬────────────────────────────────────────────────────┤
│Fields  │  Document Table                                    │
│List    │  ┌──────────────────────────────────────────┐    │
│        │  │ Time  Category  Product  Price  Customer │    │
│Selected│  │ 10:15 Clothing  Shirt   $45.00  John    │    │
│Fields: │  │ 10:14 Shoes     Boots   $89.00  Sarah   │    │
│• Time  │  │ 10:13 ...                                │    │
│• _source│  └──────────────────────────────────────────┘    │
└────────┴────────────────────────────────────────────────────┘

Key Components

  1. Data View Selector: Choose which indices to query
  2. Time Picker: Set the time range for your search
  3. Search Bar: Enter queries using KQL or Lucene syntax
  4. Filter Bar: Visual representation of active filters
  5. Histogram: Shows document distribution over time
  6. Fields List: Available fields from your documents
  7. Document Table: Actual documents matching your query

Adjusting the Time Range

The time picker controls what data you see.

Quick Ranges

Click the time picker to see common ranges:

Quick options:
- Last 15 minutes
- Last 30 minutes
- Last 1 hour
- Last 24 hours
- Last 7 days
- Last 30 days
- Last 90 days
- Last 1 year

Relative Time

Set ranges relative to now:

  1. Click "Relative" tab
  2. Enter: "Last 2 hours"
  3. Click "Apply"

Syntax examples:

now-15m     (15 minutes ago to now)
now-1h      (1 hour ago to now)
now-7d      (7 days ago to now)
now-1M      (1 month ago to now)

Absolute Time

Set exact start and end times:

  1. Click "Absolute" tab
  2. Select start date: January 1, 2024, 00:00
  3. Select end date: January 31, 2024, 23:59
  4. Click "Apply"

Recently Used

Kibana remembers your recent time selections for quick access.

Working with Fields

Field List (Left Sidebar)

The field list shows all available fields in your documents.

Field types indicated by icons:

  • # - Number
  • t - Text/String
  • 📅 - Date
  • 🔤 - Keyword
  • 🌐 - IP address
  • 📍 - Geo point

Most common fields appear at the top:

  • Time field (usually @timestamp)
  • Frequently queried fields

Examining a Field

Click any field name to see:

Field statistics:

  • Top 5 values
  • Document count per value
  • Percentage distribution

Example: Click category.keyword

Top 5 values:
Men's Clothing         1,234 (28.5%)
Women's Clothing       1,120 (25.8%)
Men's Shoes             945 (21.8%)
Women's Shoes           887 (20.5%)
Men's Accessories       134 (3.1%)

Field actions:

  • + Add to table
  • 👁️ Filter for value
  • ⛔ Filter out value
  • 📌 Pin to top

Adding Fields to Table

To customize which fields appear in the document table:

  1. Hover over a field in the field list
  2. Click the "+" button
  3. The field appears as a column in the table

Remove a field: Click the "X" next to its name in the "Selected fields" section.

Example columns for e-commerce:

  • order_date
  • customer_full_name
  • category.keyword
  • products.product_name
  • taxful_total_price

Basic Searching

Free Text Search

Simply type in the search bar:

shirt

This searches across all text fields for the word "shirt".

Search in specific fields:

category: "Men's Clothing"

customer_first_name: "Eddie"

Combining Terms

Use AND, OR, NOT:

category: "Men's Clothing" AND taxful_total_price > 50

category: "Clothing" OR category: "Shoes"

category: "Shoes" AND NOT manufacturer: "Nike"

Wildcards

Use * for any characters, ? for single character:

customer_first_name: Edd*
(Matches: Eddie, Eddy, Eduardo)

customer_first_name: Ed?ie
(Matches: Eddie, Edgie)

Range Queries

Search numeric or date ranges:

taxful_total_price >= 100

taxful_total_price: [100 TO 500]
(Between 100 and 500, inclusive)

order_date >= "2024-01-01"

Using Filters

Filters provide a visual way to narrow your data.

Adding a Filter Manually

  1. Click "+ Add filter"
  2. Select field: category.keyword
  3. Select operator: is
  4. Enter value: Men's Clothing
  5. Click "Save"

A filter pill appears: [category.keyword: "Men's Clothing"]

Filter Operators

OperatorPurposeExample
isExact matchcategory is "Shoes"
is notExclude valuestatus is not "cancelled"
is one ofMatch any valuecategory is one of "Shoes", "Clothing"
is not one ofExclude valuesstatus is not one of "failed", "error"
existsField has a valuecustomer_id exists
does not existField is missingdiscount does not exist
is betweenNumeric rangeprice is between 50 and 100

Filtering from Visualizations

Click values directly in the histogram or field stats:

  1. Click a bar in the histogram → Filters to that time bucket
  2. Click a value in field stats → Filters for that value
  3. Hold Cmd/Ctrl while clicking → Excludes that value

Filter Actions

Each filter pill has actions:

  • Toggle (click the pill): Enable/disable temporarily
  • Pin across apps: Make filter available in other Kibana apps
  • Edit: Modify filter criteria
  • Delete (X icon): Remove filter
  • Temporarily disable: Keep filter but don't apply it

Negating Filters

Make a filter exclude instead of include:

  1. Hover over filter pill
  2. Click the filter
  3. Toggle "Invert query"

Now category: "Shoes" becomes NOT category: "Shoes"

Combining Filters

Multiple filters use AND logic by default:

Filter 1: category: "Shoes"
Filter 2: taxful_total_price > 100

Result: Shoes AND price > 100

To use OR logic, use a custom filter with query DSL.

Examining Documents

Expanding a Document

Click the > arrow next to any document to see details:

Tabs available:

Table view:

Field                   Value
order_date             January 15, 2024 @ 10:45:32
customer_full_name     Eddie Underwood
category               ["Men's Clothing"]
taxful_total_price     98.45

JSON view (raw document):

{
  "_index": "kibana_sample_data_ecommerce",
  "_id": "abc123",
  "_source": {
    "order_date": "2024-01-15T10:45:32",
    "customer_full_name": "Eddie Underwood",
    "category": ["Men's Clothing"],
    "taxful_total_price": 98.45,
    ...
  }
}

Document Actions

With a document expanded:

  • View single document: Open in dedicated view
  • View surrounding documents: See documents before/after in time
  • Filter for value: Add field values as filters
  • Toggle column: Add/remove field from table

Viewing Document Context

To see documents around a specific one:

  1. Expand the document
  2. Click "View surrounding documents"
  3. See documents before and after in chronological order

Useful for:

  • Tracing transaction sequences
  • Following log sequences
  • Understanding event context

Save your query, filters, and field selection for later:

  1. Click "Save" in the top menu
  2. Enter a name: "Orders over $100"
  3. Optionally add description
  4. Click "Save"
  1. Click "Open" in the top menu
  2. Select your saved search from the list
  3. Or search by name

Share your search with others:

  1. Click "Share" in top menu
  2. Choose:
    • Permalink: Direct link (includes filters and time range)
    • Short URL: Shortened version
    • Copy as cURL: Get equivalent Elasticsearch query

Advanced Filtering Techniques

Nested Field Filtering

For nested objects like products in orders:

products:{ product_name: "shirt" AND quantity > 1 }

EXISTS Query

Find documents with/without a field:

discount: *
(Documents with discount field)

NOT discount: *
(Documents without discount field)

Field Name Patterns

Search across multiple fields:

customer_*: "John"
(Searches customer_first_name, customer_last_name, etc.)

Using the Histogram

The histogram shows document distribution over time.

Interacting with the Histogram

Zoom into a time range:

  1. Click and drag across histogram bars
  2. The time range adjusts automatically
  3. Click "Zoom Out" to reset

Brush selection:

  • Click and hold to select multiple bars
  • Filters to that exact time range

Interval adjustment: Kibana automatically chooses histogram interval (auto, hourly, daily, etc.)

You can manually adjust:

  1. Note the interval shown (e.g., "per 3 hours")
  2. Change time range to see different granularity

Understanding Gaps

Gaps in the histogram indicate:

  • No data in that time period
  • Data filtered out by your query
  • Index doesn't cover that time range

Sorting Results

Sort by Time (Default)

By default, documents are sorted by time field, newest first.

Sort by Field

Click any column header to sort by that field:

  1. Click taxful_total_price header
  2. Sorts ascending (lowest first)
  3. Click again for descending (highest first)

Multi-Field Sort

Hold Shift and click multiple headers to sort by multiple fields:

  1. Click category (primary sort)
  2. Hold Shift and click taxful_total_price (secondary sort)

Practical Examples

Example 1: Finding High-Value Orders

Goal: Find orders over $100 in the last 7 days

  1. Set time range: "Last 7 days"
  2. Add filter: taxful_total_price > 100
  3. Add field to table: customer_full_name
  4. Add field to table: taxful_total_price
  5. Sort by taxful_total_price descending

Example 2: Analyzing Customer Behavior

Goal: See what products a specific customer bought

  1. Search: customer_full_name: "Eddie Underwood"
  2. Add fields to table:
    • order_date
    • products.product_name
    • category
    • taxful_total_price
  3. Sort by order_date descending

Example 3: Finding Failed Transactions

Goal: Identify orders with issues (assuming you have a status field)

# If you had a status field
status: "failed" OR status: "cancelled"

For sample data, let's find returns:

  1. Add filter: products.quantity < 0
  2. This finds negative quantities (returns)

Example 4: Geographic Analysis

Goal: Find orders from specific regions

  1. Add filter: geoip.country_iso_code is "US"
  2. Add field: geoip.city_name
  3. Examine top values for city_name field
  4. Filter for specific cities

Tips and Best Practices

Performance Tips

Limit time range: Narrower ranges = faster queries

✅ Last 24 hours (fast)
❌ Last 1 year (slow)

Use keyword fields: Better for filtering

✅ category.keyword: "Shoes"
❌ category: "Shoes" (if category is text field)

Limit field count: Don't add too many columns

✅ 5-10 fields in table
❌ 50+ fields (slow rendering)

Search Tips

Be specific: Use field names when possible

✅ customer_first_name: "John"
❌ John (searches everywhere)

Use filters over queries: Filters are cached

✅ Filter: category.keyword is "Shoes"
✅ Query: category: "Shoes"
(Filter is slightly faster for repeated use)

Case sensitivity: Keyword fields are case-sensitive

category.keyword: "Shoes"     ✅
category.keyword: "shoes"     ❌ (no match)
category.keyword: "SHOES"     ❌ (no match)

Discovery Workflow

  1. Start broad: Begin with full time range, no filters
  2. Examine fields: Look at top values, identify interesting patterns
  3. Apply filters: Narrow down to interesting subset
  4. Add columns: Show relevant fields in table
  5. Inspect documents: Expand documents to see details
  6. Save search: Save for future reference or dashboard use

Common Issues

"No results found"

Possible causes:

  1. Time range doesn't match your data
    • Solution: Expand time range
  2. Filters are too restrictive
    • Solution: Remove filters one by one
  3. Search syntax error
    • Solution: Check for typos, correct field names

"No cached mapping for this field"

Cause: Field doesn't exist in the index pattern

Solution:

  1. Go to Stack Management → Data Views
  2. Refresh the data view
  3. Or verify the field name is correct

Slow query performance

Solutions:

  1. Narrow time range
  2. Use more specific filters
  3. Reduce number of documents to fetch (pagination)
  4. Check Elasticsearch cluster health

Summary

In this chapter, you learned:

  • ✅ How to navigate the Discover interface
  • ✅ Adjusting time ranges for your queries
  • ✅ Working with fields and field statistics
  • ✅ Searching data with queries and filters
  • ✅ Examining individual documents
  • ✅ Using the histogram for time-based analysis
  • ✅ Saving and sharing searches
  • ✅ Performance tips and best practices

Next up: Creating visualizations from your discovered data!