Tutorials » Audit » Automatic Auditing

Automatic Auditing in Conjur Enterprise

In this tutorial, you will learn how to inspect audit records automatically recorded by Conjur Enterprise, and how to add custom events into the audit trail.

As an example, hypothetical Continuous Integration server will be considered, which performs build, acceptance and promotion of some cloud images. In order to perform such tasks, it needs access to cloud API keys, stored in Conjur variables.

Of course, the audit capabilities of Conjur are not limited only by secrets management domain. It is just used as an easy-to-understand example.

Objectives

  • Simulate access to credentials and inspect audit records.

Preparation

Make a working directory and cd to it:

$ mkdir -p ~/conjur/tutorials/audit
$ cd ~/conjur/tutorials/audit

Use a namespace prefix to keep your data separate from other users:

$ NAMESPACE=$USER-audit-tutorial

Install the Conjur CLI and Summon with the Conjur provider.

First, let's create a policy defining two secrets and a host that can execute (fetch) them:

policy.yml

- &variables
  - !variable cloud_access_key
  - !variable cloud_secret_key

- !host &ci_server ci_server

- !permit
  role: *ci_server
  privileges: [ execute ]
  resource: *variables

Load the policy with your namespace:

$ conjur policy load --as-group security_admin --namespace $NAMESPACE policy.yml
Create variable 'you-audit-tutorial/cloud_access_key'
Create variable 'you-audit-tutorial/cloud_secret_key'
Create host 'you-audit-tutorial/ci_server'
Permit host 'you-audit-tutorial/ci_server' to [execute] on variable 'you-audit-tutorial/cloud_access_key'
Permit host 'you-audit-tutorial/ci_server' to [execute] on variable 'you-audit-tutorial/cloud_secret_key'
{"docker:host:you-audit-tutorial/ci_server":"d39x8r1js3cqw1d7vqt739vmqn2p34ntz1wnyyk13tpg61ywmxyc8"}

The ci_server host's API key is written to console: d39x8r1js3cqw1d7vqt739vmqn2p34ntz1wnyyk13tpg61ywmxyc8.

Save the key as an environment variable; we'll use it to log in as the host later on:

$ APIKEY=<key from previous command's output>

We created the variables, but they have no value yet. Set a value for each variable:

$ conjur variable values add $NAMESPACE/cloud_access_key 9p81nd298dbp9
$ conjur variable values add $NAMESPACE/cloud_secret_key jnvalsbiuscca

To imitate server operation, create stub shell scripts responsible for each task:

build.sh

#!/bin/bash

if [ -z "\$ACCESS_KEY" ] || [ -z "\$SECRET_KEY" ]; then
  echo "Operation is impossible without cloud credentials!"
  exit 1
fi

echo "OK"

$ chmod +x build.sh
$ cp build.sh acceptance.sh
$ cp build.sh promote.sh

Finally, as in a real server environment, create a secrets.yml file that references the secrets you've just created.

secrets.yml

ACCESS_KEY: !var $namespace/cloud_access_key
SECRET_KEY: !var $namespace/cloud_secret_key

Automatically generated audit events

Now log in with the host identity and imitate operation of the server:

$ conjur authn login -u host/$NAMESPACE/ci_server -p $APIKEY
Logged in

$ summon -D namespace=$NAMESPACE ./build.sh
OK

$ summon -D namespace=$NAMESPACE ./acceptance.sh
OK

$ summon -D namespace=$NAMESPACE ./promote.sh
OK

You can inspect the audit trail with the Conjur UI, API or CLI. This tutorial covers the CLI.

Log in with your original account in order to inspect audit records:

$ conjur authn login -u you -p <password>

Audit per resource

Inspect history of access to cloud secret resource:

$ conjur audit resource -s variable:$NAMESPACE/cloud_secret_key
[2016-04-11 19:32:29 UTC] demo:user:admin (as demo:group:security_admin) created resource demo:variable:you-audit-tutorial/cloud_secret_key owned by demo:group:security_admin
[2016-04-11 19:32:29 UTC] demo:user:admin permitted demo:host:you-audit-tutorial/ci_server to execute demo:variable:you-audit-tutorial/cloud_secret_key (grant option: false)
[2016-04-11 19:47:24 UTC] demo:host:you-audit-tutorial/ci_server checked that they can execute demo:variable:you-audit-tutorial/cloud_secret_key (true)
[2016-04-11 19:49:35 UTC] demo:user:admin checked that they can update demo:variable:you-audit-tutorial/cloud_secret_key (true)
[2016-04-11 19:49:40 UTC] demo:user:admin checked that they can execute demo:variable:you-audit-tutorial/cloud_secret_key (true)
[2016-04-11 19:50:16 UTC] demo:host:you-audit-tutorial/ci_server checked that they can execute demo:variable:you-audit-tutorial/cloud_secret_key (true)

Audit per role

Inspect history of ci_server host actions

$ conjur audit role -s host:$NAMESPACE/ci_server
[2016-04-11 19:32:29 UTC] demo:user:admin (as demo:group:security_admin) created role demo:host:you-audit-tutorial/ci_server
[2016-04-11 19:32:29 UTC] demo:user:admin permitted demo:host:you-audit-tutorial/ci_server to read demo:host:you-audit-tutorial/ci_server (grant option: false)
[2016-04-11 19:32:29 UTC] demo:user:admin permitted demo:host:you-audit-tutorial/ci_server to execute demo:variable:you-audit-tutorial/cloud_access_key (grant option: false)
[2016-04-11 19:32:29 UTC] demo:user:admin permitted demo:host:you-audit-tutorial/ci_server to execute demo:variable:you-audit-tutorial/cloud_secret_key (grant option: false)
[2016-04-11 19:47:24 UTC] demo:host:you-audit-tutorial/ci_server checked that they can execute demo:variable:you-audit-tutorial/cloud_access_key (true)

All visible events

Inspect all audit events visible to your role:

$ conjur audit all -s
[2016-04-11 19:32:29 UTC] demo:user:admin (as demo:group:security_admin) created resource demo:variable:you-audit-tutorial/cloud_access_key owned by demo:group:security_admin
[2016-04-11 19:32:29 UTC] demo:user:admin (as demo:group:security_admin) created resource demo:variable:you-audit-tutorial/cloud_secret_key owned by demo:group:security_admin
[2016-04-11 19:32:29 UTC] demo:user:admin unknown event: role:all_roles!
[2016-04-11 19:32:29 UTC] demo:user:admin (as demo:group:security_admin) created role demo:host:you-audit-tutorial/ci_server
[2016-04-11 19:32:29 UTC] demo:user:admin (as demo:group:security_admin) created resource demo:host:you-audit-tutorial/ci_server owned by demo:group:security_admin
[2016-04-11 19:32:29 UTC] demo:user:admin permitted demo:host:you-audit-tutorial/ci_server to read demo:host:you-audit-tutorial/ci_server (grant option: false)
[2016-04-11 19:32:29 UTC] demo:user:admin permitted demo:host:you-audit-tutorial/ci_server to execute demo:variable:you-audit-tutorial/cloud_access_key (grant option: false)
[2016-04-11 19:32:29 UTC] demo:user:admin permitted demo:host:you-audit-tutorial/ci_server to execute demo:variable:you-audit-tutorial/cloud_secret_key (grant option: false)
[2016-04-11 19:47:24 UTC] demo:host:you-audit-tutorial/ci_server checked that they can execute demo:variable:you-audit-tutorial/cloud_access_key (true)      ```

Full audit format

In commands above, we used human-readable "short" audit format (this behaviour of audit command was triggered by -s switch). By default more comprehensive JSON output is provided.

Audit format is significantly extended since Conjur server version 4.3 in comparison with previous versions
$ conjur audit all
... lots of output snipped ...
{
  "resources": [
    "demo:variable:you-audit-tutorial/cloud_access_key"
  ],
  "roles": [
    "demo:host:you-audit-tutorial/ci_server"
  ],
  "resource": "demo:variable:you-audit-tutorial/cloud_access_key",
  "action": "check",
  "privilege": "execute",
  "allowed": true,
  "timestamp": "2016-04-11T19:54:01.695Z",
  "event_id": "b16861f1ba183300591236bf22e7d207",
  "id": 331,
  "user": "demo:host:you-audit-tutorial/ci_server",
  "acting_as": "demo:host:you-audit-tutorial/ci_server",
  "request": {
    "ip": "127.0.0.1",
    "url": "http://localhost:5100/demo/resources/variable/you-audit-tutorial/cloud_access_key?check=true&privilege=execute",
    "method": "GET",
    "params": {
      "check": "true",
      "privilege": "execute",
      "controller": "resources",
      "action": "check_permission",
      "account": "demo",
      "kind": "variable",
      "identifier": "you-audit-tutorial/cloud_access_key"
    },
    "uuid": "ec1b2ee2-9924-4764-8549-0d8c9aa0f245"
  },
  "conjur": {
    "domain": "authz",
    "env": "appliance",
    "user": "demo:host:you-audit-tutorial/ci_server",
    "role": "demo:host:you-audit-tutorial/ci_server",
    "account": "demo"
  },
  "kind": "resource"
}

This format is useful for automated data processing.

A good next step is the Custom Audit Records tutorial, where you will learn how to inject custom audit events into the stream.

References