Spring cloud gateway is unable to decode when deployed in the staging environment

Question

I was recently tasked with upgrading a spring cloud app of ours due to the CVE-2022-22965 vulnerability.

As part of that process, it became clear that the then current zuul gateway was simply to outdated, so it was replaced with a spring cloud gateway instead.

Now, I've gotten it to work locally, but the staging environment won't play ball, even though it worked fine with the old gateway.

The error I get when attempting to talk to it is this:

WARN 1 --- [or-http-epoll-1] r.n.http.server.HttpServerOperations     : [72c0060d, L:/172.23.0.9:8080 - R:/10.151.3.198:57204] Decoding failed: DefaultFullHttpRequest(decodeResult: failure(java.lang.IllegalArgumentException: text is empty (possibly HTTP/0.9)), version: HTTP/1.0, content: UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeHeapByteBuf(ridx: 0, widx: 0, cap: 0))

GET /bad-request HTTP/1.0 : 

java.lang.IllegalArgumentException: text is empty (possibly HTTP/0.9)    
    at io.netty.handler.codec.http.HttpVersion.valueOf(HttpVersion.java:65) ~[netty-codec-http-4.1.75.Final.jar!/:4.1.75.Final]    
    at io.netty.handler.codec.http.HttpRequestDecoder.createMessage(HttpRequestDecoder.java:124) ~[netty-codec-http-4.1.75.Final.jar!/:4.1.75.Final]    
    at io.netty.handler.codec.http.HttpObjectDecoder.decode(HttpObjectDecoder.java:273) ~[netty-codec-http-4.1.75.Final.jar!/:4.1.75.Final]    
    at io.netty.handler.codec.http.HttpServerCodec$HttpServerRequestDecoder.decode(HttpServerCodec.java:140) ~[netty-codec-http-4.1.75.Final.jar!/:4.1.75.Final]    
    at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:510) ~[netty-codec-4.1.75.Final.jar!/:4.1.75.Final]    
    at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:449) ~[netty-codec-4.1.75.Final.jar!/:4.1.75.Final]    
    at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:279) ~[netty-codec-4.1.75.Final.jar!/:4.1.75.Final]    
    at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251) ~[netty-transport-4.1.75.Final.jar!/:4.1.75.Final]    
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.75.Final.jar!/:4.1.75.Final]    
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.75.Final.jar!/:4.1.75.Final]    
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.75.Final.jar!/:4.1.75.Final]    
    at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[netty-transport-4.1.75.Final.jar!/:4.1.75.Final]    
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.75.Final.jar!/:4.1.75.Final]    
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.75.Final.jar!/:4.1.75.Final]    
    at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[netty-transport-4.1.75.Final.jar!/:4.1.75.Final]    
    at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:800) ~[netty-transport-classes-epoll-4.1.75.Final.jar!/:4.1.75.Final]    
    at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) ~[netty-transport-classes-epoll-4.1.75.Final.jar!/:4.1.75.Final]
    at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) ~[netty-transport-classes-epoll-4.1.75.Final.jar!/:4.1.75.Final]
    at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) ~[netty-common-4.1.75.Final.jar!/:4.1.75.Final]
    at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[netty-common-4.1.75.Final.jar!/:4.1.75.Final]
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[netty-common-4.1.75.Final.jar!/:4.1.75.Final]
    at java.base/java.lang.Thread.run(Thread.java:866) ~[na:na]
Copy

As for the differences between testing locally, and the stage environment, the stage environment uses docker. So presumably the problem is related to that. However, docker was used before as well, without any issues. Suffice it to say, the old and new gateways are functionally very similar, as the new gateway is intended to be a dropin replacement.

Another possible wildcard here, is that I ran into a peculiar problem with our frontend midway through the development process. While I never found out exactly what went wrong, I did discover that that the Dockerfile was based on an image called tiangolo/node-frontend that itself was based on the offical node:10 image. The code was old, and local attempt to manually build said Dockerfile failed. So I simply elected to replace it completely with my own Dockerfile that based itself off the offical node:16 image over on dockerhub. This code appears to work fine, as the build server does not complain. All in all, I don't think this is the culprit.

I am at loss as to what's going on. Simple google searches suggests it's related to TLS, but only http is used here, irrespective of environment. Again, this only happens in the dockerized stage environment, local runs work fine.

I don't really know where to go from here.

Answer 1

I got this error and did not understand why. Than I did request by Postman and I received another error:

write EPROTO 49582920:error:100000f7:SSL routines:OPENSSL_internal:WRONG_VERSION_NUMBER:../../third_party/boringssl/src/ssl/tls_record.cc:242: So the rison of this problem - you do https request instead http.