'How to avoid double-encoding of [] when using Spring RestTemplate?

I've a Spring Boot 2.2.5 application calling this REST endpoint

@RequestMapping(path = "/usable/search")
public List<Provider> findUsable(
      @RequestParam(name = "country-id", required = false) Integer countryId,
      @RequestParam(name = "network-ids[]", required = false) List<Integer> networkIds,
      @RequestParam(name = "usages[]") Set<Usage> usages)

My caller does

val url = clientUri.toString() + configuration.getUrl().getUseableProvidersForNetworks(); % 'providers/usable/search?'
val builder = UriComponentsBuilder.fromHttpUrl(url)
      .queryParam("usages[]", USAGES)
      .queryParam("network-ids[]", networkIds);
val response =
      restTemplate.exchange(
          builder.toUriString(),
          HttpMethod.GET,
          entity,
          new ParameterizedTypeReference<List<Provider>>() {});

where USAGES is a Set<Usage> and networkIds is a List<Integer>.

The RestTemplate is created by

RestTemplate eurekaRestTemplate() {
    val requestFactory = new HttpComponentsClientHttpRequestFactory();
    requestFactory.setReadTimeout(TIMEOUT);
    requestFactory.setConnectTimeout(TIMEOUT);
    return new RestTemplate(new BufferingClientHttpRequestFactory(requestFactory));
  }

Problem: The [] get double encoded to %255B%255D and I end up with a call to

providers/usable/search?usages%255B%255D=MESSAGE_DELIVERY&network-ids%255B%255D=2145118475&network-ids%255B%255D=-1536358371

Which obviously doesn't work; the server misses e.g. usages[]. It works with curl with [] instead of %255B%255D.

How can I avoid that?

I'm aware of this issue and the docs, but that didn't help yet.

javaspring-booturlurlencodequery-parameters


Solution 1:[1]

The solution was to call UriComponentsBuilder.build(false) to instruct the UriComponentsBuilder not to encode the url:

val response = restTemplate.exchange(
              builder.build(false).toUriString(),
              HttpMethod.GET,
              entity,
              new ParameterizedTypeReference<List<Provider>>() {});

This is tricky because a test with Mockito/JUnit of this will tell you that the url is not encoded while a test with MockRestServiceServer will show one level of encoding ([] become %5B%5D).

Solution 2:[2]

I think you shouldn't be dealing with using []. I mean, it's not necessary to name your parameters using "[]". Why would you do that?

val builder = UriComponentsBuilder.fromHttpUrl(url)
      .queryParam("usages", USAGES)
      .queryParam("network-ids", networkIds);

will perfectly work for:

public List<Provider> findUsable(
      @RequestParam(name = "country-id", required = false) Integer countryId,
      @RequestParam(name = "network-ids", required = false) List<Integer> networkIds,
      @RequestParam(name = "usages") Set<Usage> usages)

Hope it helps!

Hint: be careful when using array/collection parameters on URLs, there's a URL length limit that you must consider

Solution 3:[3]

If you wish to pass the encoded url directly to RestTemplate, you can instruct RestTemplate to not encode the url by passing the java.net.URI instead of a String. Here's an example using org.springframework.web.util.UriComponents

String url = "https://someurl/this%2Fis%2Fencoded/";
UriComponents uriComponents = UriComponentsBuilder.fromHttpUrl(url).build(true);
ResponseEntity<String[]> file = template.exchange(uriComponents.toUri(), 
HttpMethod.GET, entity,String[].class);

By passing UriComponentsBuilder.fromHttpUrl(url).build(true) You are essentially letting UriComponents know that your url is already encoded.

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 Martin Schröder
Solution 2 develo
Solution 3 Saideep Ullal

Related Questions