How to read Vault kv with java/spring boot

Question

I'm trying to figure out how to use Hashicorp's Vault with spring boot.

Initially, I have tried to follow the guide:

https://spring.io/guides/gs/vault-config/#scratch

But due to api changes I used following command in the vault CLI:

vault kv put secret/gs-vault-config example.username=demouser example.password=demopassword

which saved both and I'm able to retrieve it with the following command

vault kv get secret/gs-vault-config

Then I created the Application.java and MyConfiguration.java as described in the guide. At first, I ran the program without having the vault server running which resulted in a connection refused. Then I started the vault server and entered the username and password from the CLI. From the log I can see it actually enters the Application and writes out Here we goooo

@SpringBootApplication
public class Application implements CommandLineRunner {

@Autowired
private VaultTemplate vaultTemplate;

public static void main(String[] args) {
    SpringApplication.run(Application.class, args);
}

@Override
public void run(String... strings) throws Exception {

    // You usually would not print a secret to stdout
    System.out.println("Here we gooooo");
    VaultResponse response = vaultTemplate.read("secret/gs-vault-config");
    System.out.println("Value of username");
    System.out.println("-------------------------------");
    System.out.println(response.getData().get("example.username"));
    System.out.println("-------------------------------");
    System.out.println();

But im unable to retrieve any data from Vault - probably due to the V1 vs V2 issues

2018-08-30 17:10:07.375 ERROR 21582 --- [           main] o.s.boot.SpringApplication               : Application run failed

java.lang.IllegalStateException: Failed to execute CommandLineRunner
    at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:800) [spring-boot-2.0.3.RELEASE.jar!/:2.0.3.RELEASE]
    at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:781) [spring-boot-2.0.3.RELEASE.jar!/:2.0.3.RELEASE]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:335) [spring-boot-2.0.3.RELEASE.jar!/:2.0.3.RELEASE]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1255) [spring-boot-2.0.3.RELEASE.jar!/:2.0.3.RELEASE]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1243) [spring-boot-2.0.3.RELEASE.jar!/:2.0.3.RELEASE]
    at hello.Application.main(Application.java:23) [classes!/:na]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_181]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_181]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_181]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_181]
    at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) [gs-vault-config-0.1.0.jar:na]
    at org.springframework.boot.loader.Launcher.launch(Launcher.java:87) [gs-vault-config-0.1.0.jar:na]
    at org.springframework.boot.loader.Launcher.launch(Launcher.java:50) [gs-vault-config-0.1.0.jar:na]
    at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:51) [gs-vault-config-0.1.0.jar:na]
Caused by: java.lang.NullPointerException: null
    at hello.Application.run(Application.java:34) [classes!/:na]
    at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:797) [spring-boot-2.0.3.RELEASE.jar!/:2.0.3.RELEASE]
    ... 13 common frames omitted

Does anyone know if there is a similar guide to a spring-boot code snippet where data is retrieved from vault which has been entered with the kv engine?

Answer 1

I stumbled a single note in this page : https://cloud.spring.io/spring-cloud-vault/multi/multi_vault.config.backends.html

In which i says : Spring Cloud Vault adds the data/ context between the mount path and the actual context path

So i tried to change the code to :

VaultResponse response = vaultTemplate.read("/secret/data/gs-vault-config");

And then it worked.

Answer 2

Instead of start the server as dev, start the server using configuration file. To do that you can create a json file named vault.json and add following code.

ui = true

listener "tcp" {
  address          = "0.0.0.0:8200"
  cluster_address  = "192.168.56.1:8201"
  tls_disable      = "true"
}
storage "file" {
  path = "data"
}


api_addr = "http://192.168.56.1:8200"
cluster_addr = "https://192.168.56.1:8201"

To run the code you can use

$vault server -config=vault.json

Finally add the vault token in the bootstrap.yml file

spring:
    application.name: app-name
    cloud.vault:
        host: 127.0.0.1
        port: 8200
        authentication: TOKEN
        token:  your token
        scheme: http

Answer 3

I think it's due to the V1 vs V2 issue. I met the similar issue while trying the follow guide: https://spring.io/guides/gs/accessing-vault/

I used the Vault UI to create a V1 secret engine and added the secrets, and it worked. Following are the steps:

1. Login to http://127.0.0.1:8200/, use the token method to login, and enter the token in the guide(00000000-0000-0000-0000-000000000000)
2. On the right up corner, click Enable new engine
3. Select "KV", and click next
4. Make sure to select "1" in Version, then click "Enable Engine".(refer to Vault_Secret_Engine_V1.png )
5. Click "Create secret"
6. Input "Path", "Key", and "Value", then click save
7. The secrets will be saved with path "kv/github" (refer to Vault_Key.png)
8. Then change the code to:

VaultResponseresponse=vaultTemplate.read("kv/github");

If I change the Version to 2 in step 4, and leave all other steps the same. I will got the same exception as yours.

Vault_Secret_Engine_V1.png

Vault_Key.png

Answer 4

I had the same problem and solved it setting the key value store version to v1, as @johnathan-wan suggested.

The only thing I did different was setting the kv store version by command line, like this:

# first, check if you already have a v2 keystore for that path
vault secrets list -detailed
# if you already have a v2 of secret/gs-vault-config, then:
vault secrets disable secret/gs-vault-config

# create a new version 1 keystore for that path
vault secrets enable -path secret/gs-vault-config -version 1 kv

I found that after following the examples in: https://github.com/mp911de/spring-cloud-vault-config-samples

Answer 5

I confirm it's V1 vs V2 issue when Key-Value (kv) engine is used.

Here is code sample for both versions:

VaultEndpoint endpoint = VaultEndpoint.from(new URI(
    "https://my-cluster.smth.hashicorp.cloud:8200"));
VaultTemplate template = new VaultTemplate(endpoint, new TokenAuthentication(token));

// Path kv_version1 uses kv engine version 1
// It must be accessed directly with VaultTemplate
System.out.println(template.read("admin/kv_version1/my-name").getData().get("password"));

// Path kv_version2 uses kv engine version 1
// It must be accessed with VaultVersionedKeyValueTemplate
VaultVersionedKeyValueTemplate kvTemplate =
    new VaultVersionedKeyValueTemplate(template, "admin/kv_version2");
System.out.println(kvTemplate.get("my-name").getData().get("password"));