When a client starts a new long-lived VM or server, Conjur needs to be instructed to create it as a new host and apply the appropriate policies. Host Factory is an easy, safe and secure way to do this. It runs as a service on the Conjur server and provides a management API for limited-time tokens, then acts on a token holder’s behalf to create new hosts in Conjur and add them to a layer.
What problems does Host Factory solve?
Conjur is flexible enough to manage host policies in many ways. The owner of the new hosts can use the Conjur CLI to manually set permissions when they are set up. But that strategy doesn’t scale to a large number of services with diverse policies and lifecycles. Removing the dependency on a human operator to correctly apply policies to each new host is an effective way to improve safety and compliance.
Host Factory makes it easy to control policies and maintain consistency. Hosts inherit the policy of the layer they belong to, and Host Factory allows authorized users and services to create a new host and add it to the appropriate layer.
Grouping hosts into layers allows the operator to maintain a consistent policy for every host on the layer. They can also allow specific users and automated processes to add new hosts to the layer without giving permission to see or tamper with other hosts on that layer. When a new service is created, its policy is applied according to its semantic meaning, represented by membership in a layer, without the client needing to remember to perform this step or have knowledge of the current policy.
Host Factory solves the problem of making this easy, safe, automatic, and scalable.
How does Host Factory work?
During preparation to start new servers and VMs, the operator creates a layer to which these new hosts will belong and applies a policy to that layer which includes a Host Factory. They use the factory to generate one or more host factory tokens, each of which is represented by a securely random string of characters. The operator communicates the tokens securely to the person or service that will be creating the new servers and VMs. The host factory token has an expiration date which a policy can specify, allowing the factory to create short-lived tokens for individual jobs or long-lived ones for repeated use.
When starting a new server or VM, a client must provide it with a valid token which it will in turn send to the Host Factory service. The Host Factory validates the token and responds with a new host identity for the server or VM. This host identity is automatically added to the layers that are defined on the Host Factory. Host Factory tokens can be provided to new hosts via automation tools such as Puppet, Chef, and CloudFormation.
To create a host using a Host Factory, a client does not have to authenticate with Conjur in the normal way (via login name + API key). Rather, the client presents a token associated with a single host factory. Because it has such a specific use, a host factory token is the most secure way to authorize scripts and code to create Conjur Hosts.
How is Host Factory secured?
In order to operate securely, the Host Factory uses unforgeable limited-time tokens and address (CIDR) restrictions.
The administrator of the Host Factory creates tokens which then must be presented in order to operate it. The tokens are unforgeable; only the operator can create them, and each one is new and unique. Expired tokens are of no use. Furthermore, tokens can be revoked at any time, and each token has an expiration date after which it automatically expires.
In order for this security system to work, the operator must communicate the tokens securely to clients using TLS. Tokens do not need to be retained on disk after use, and should not be transmitted or stored in plaintext.
Typically, clients do not need to be able to create new hosts from arbitrary machines. In this case, the operator can add an extra layer of security to the Host Factory token by adding a CIDR restriction. Only clients with matching addresses will be able to use the token to create new hosts.
Bootstrapping Machine Identity
For a host to automatically obtain a machine identity in Conjur, a Host Factory token must be made available to the host. Host Factory provides several features to ensure that the token cannot be misused. Depending on your organization’s security policy you may choose to use some or all of these features. Following is a guide to bootstrapping machines with Host Factory tokens, starting with the default security protections and followed by additional lock-down measures.
When a Host Factory token is created, an expiration date must be specified. However, this expiration date can be far in the future.
$ conjur hostfactory tokens create --duration-days 365 myfactory
Creating a long-lived Host Factory token gives you flexibility in how you distribute the token to hosts. Storing the token in a trusted location that your hosts can reach may already be an established distribution pattern for your organization. For example, you can distribute the token with config management tooling like Chef or Puppet. If you’re in AWS the token can be encrypted with KMS, stored in S3, and used by instances according to their IAM role and instance profile.
A Host Factory token always creates hosts in exactly the layers specified during its Host Factory’s creation. The layer(s) granted to new hosts by the Host Factory cannot be changed after creation. If a long-lived token is compromised, the effect of that compromise is limited to only the associated layers in your infrastructure.
Requires Conjur v4.6.0 and later
If you have known network subnets that hosts enter, you can limit use of Host Factory tokens by an IP range. Unless the request carrying the token originates from the specified IP range, Host Factory will return "401 Unauthorized" and will not create the new host or add it to a layer.
$ conjur hostfactory tokens create \ --duration-days 365 --cidr 10.0.1.0/24 myfactory
CIDR restriction is an enhancement to Host Factory that adds security at the network level. Note that the CIDR of a token cannot be changed after its creation.
To further prevent unauthorized use, Host Factory tokens can be set to expire soon after they are created. As illustrated in the following script, a token can be created with a short duration, rotated and pushed to a secure store automatically. Run every 12 hours, this script will refresh the token hosts use to bootstrap into layers.
#!/bin/bash $ conjur hostfactory tokens create --duration-hours 12 myfactory 2zj0g2279hekh1ag9z9h3tjeka7309v9v323kg9n04vwqvt1pbt2gn … # Push new token to a secure store
Tokens can also be revoked at any time, regardless of their expiration timestamp. Revoked tokens cannot be restored for further use.
$ conjur hostfactory tokens revoke <token>
Tokens can also be encrypted in storage and decrypted by the machine to be bootstrapped. Client data or a service like Amazon KMS can be used for encryption.
The KMS workflow, in brief:
- Create a Host Factory token.
- Encrypt the token with KMS.
- Store the encrypted token in S3.
- For each Conjur layer, create an IAM role that allows S3 bucket access and KMS decryption.
- Assign corresponding IAM roles to EC2 instances.
- When a new instance starts, it pulls the encrypted token from S3, decrypts it and obtains identity.
Known client data, reported by tools like Ohai or Facter, can also be used to encrypt Host Factory tokens. Identifying sets of client data that can map to privileged layers is the critical step in this workflow.
An alternative workflow is to establish machine identity and grant privilege with two separate operations. This simplifies establishing identity for new and existing machines by placing them in a layer that has very limited privileges. Once machines are identified, they can be added to privileged layers by manual or automatic processes that best fit your organization.
Host Factory tokens for unprivileged layers are not as sensitive because they are used only for machine enrollment and not to obtain privileges. A special-purpose group should be created and act as the initial owner of the unprivileged layer, Host Factory and hosts in said layer. Layer membership and host ownership can be modified later as required.
$ conjur group create --as-group security_admin hosts_admin $ conjur layer create --as-group host_admin hosts $ conjur hostfactory create --as-group host_admin --layer hosts myfactory $ conjur hostfactory tokens create --duration-days 365 myfactory 2zj0g2279hekh1ag9z9h3tjeka7309v9v323kg9n04vwqvt1pbt2gn
This token can now be used to bootstrap hosts with Conjur machine identities that hold no privileges. Moving these identities to a privileged layer will grant the layer's privileges to the hosts. In Conjur, a host can be a member of any number of layers at the same time.
Technical deep dive
conjur plugin install host-factory
Full documentation for the "hostfactory" CLI subcommand can be found in the following pages:
Each Host Factory is configured to construct new hosts and add them to a fixed set of Layers.
Each Host Factory acts as a distinct Conjur role, which is specified when the host factory is created. All Hosts created by the Host Factory will be owned by this designated role.
Once a host creation request is received and authorized, the host factory assumes the designated role and begins carrying out the host creation process.
First, the Host is created, using a unique host id chosen by the client. In Conjur 4.7 or later, the client can also specify an existing host id. In this case, the Host Factory attempts to rotate the API key of the host, and returns the new API key to the client.
Once the Host is found or created, it is added to a fixed set of Layers. These Layers are specified when the host factory is created, and the host factory role must be authorized to add hosts to these layers. The easiest way to ensure that the Host Factory is constructed correctly is to define it in a declarative Conjur Policy.
Permission to use the host factory (to view details, create tokens, and revoke tokens) can be granted through privilege authorization on the host factory. As always, the owner of the Host Factory has full control.
Host Factory tokens can be created at any time, by any role which has execute privilege on the Host Factory resource. Host Factory tokens can be revoked by any role which has update privilege on the Host Factory resource.
When a host factory is running on a follower, it delegates host creation to the master. Therefore, unlike most Conjur write operations, creation of new hosts via host factory token can be performed against the follower.
If authentication using the new Host API key fails, it may be due to replication delay between the master and follower. In Conjur 4.6 and later, replication status of any Conjur server can be obtained via the /health API route.