Elasticsearch Kibana authentication using SAML and Okta

As we’re running Elasticsearch inside a VPC, everyone at the office could access directly to the Kibana endpoint and do nasty things. For now we just wanted to limit the access to certain people/groups we analyzed several options as

  • Make a SSH tunnel – Nasty solution, doesn’t scale at all, hard to configure, need user knowledge on how SSH tunnels works
  • Authentication using just Cognito – Limited to Cognito user pools, cannot be linked to IAM users, need to maintain 3 user groups (IAM, LDAP/Okta and Cognito), doesn’t scale at all.
  • Authentication using SAML and Okta: the preferred way. We have Okta for most of our applications, it works well with SAML so we gave a try

Setup

For this application we’ll use a Cognito User Pool, a Cognito Identity Pool, a working (or newly created) AWS Elasticsearch cluster and an Okta implementation working with admin credentials (to create applications)

Cognito User Pool

  1. Create a Cognito User Pool with okta_dev_example name
  2. Disable the “Require Email” standard attribute – It may conflict with your current Okta implementation if it doesn’t forward the Email label.
  3. Go ahead on the follow steps and create the pool.
  4. Go to App Integration > Domain name and set a custom domain name here
  5. Take care about the User Pool ID and the App Client ID. We’ll use it later at the Cognito Identity Pool configuration

Cognito Identity Pool

  1. Switch to Cognito Identity Pool
  2. Create a new Identity Pool and give it a name as es_identity_pool_example
  3. Open the Authentication provider dropdown
    1. Set the User Pool ID (take it from the Cognito User Pool)
    2. Set the App Client ID (take it from the Cognito User Pool)
Set the cognito user and app client ID

Elasticsearch

  1. Configure your existing Elasticsearch cluster or your newly created cluster
  2. Enable the Cognito access
    1. Set the Cognito User Pool
    2. Set the Cognito Identity Pool
Cognito configuration on Elasticsearch
  1. Select the “Allow or deny access to one or more AWS accounts or IAM users” from the access policy dropdown select
  2.  Use the Cognito Identity Pool IAM role ARN on the account ID / ARN entry and save those changes

Okta

  1. Create a new application on Okta. Call it Kibana Backend
  2. Set the application visibility to “Do not display application icon to users”. Will explain this later
  3. Set the SAML 2.0 sign on method
  4. Click next to go on SAML settings
  5. For the Single Sign On URL grab the Domain name you wrote on the Cognito User Pool domain name and add /saml2/idpresponse to the URI. Example:
    1. https://kibana.auth.us-east-1.amazoncognito.com/saml2/idpresponse
  6. For the Audience URI, use your User Pool ID (the one you got from the Cognito User Pool configuration and prepend the URN format: urn:amazon:cognito:sp:
    1. Example: urn:amazon:cognito:sp:us-east-1_4f3boh5gg
  7. Go forward the process until end and download the metadata.xml file. Save it as we will use it next at the Cognito User Pool Federation configuration

Cognito User Pool (again)

  1. Go to Federation > Identity Providers
  2. Select SAML, upload the metadata.xml file and give it a name as “Login with Okta”
  3. Go to App Integration > App Client settings
  4. Setup the Okta identity provider
Set the OKTANOEMAIL identity provider

Okta (again)

This is a workaround to a non-working Okta button to login. That’s why I recommended you to don’t display the app icon to the user.

  1. Create a new application, find it called “Bookmark”
  2. Set the name to Kibana
  3. Set the URL to the Elasticsearch kibana endpoint
Use the Kibana URL endpoint

User control

Whenever an user requests Kibana access, assign both Kibana and Kibana Backend applications to the user. This way the user will see just the Kibana app, and in behind, Okta will do the login using SAML.

Debugging

I had to do some debugging when setting up some URLs, credentials, IDs and so on. I highly recommend the SAML-Tracer plugin for Firefox or Chrome. It’s really helpful when trying to understand what’s failing as the Cognito error messages are soooo cryptic.