Quarkus upgrade vertx dns resolver ignoring kubernetes dns

Question

I have upgraded the version of quarkus to 2.8.0.CR1

My quarkus apps are using a mix of camel, mutiny, resteasy, mongo, pubsub and it seems to have been reproduced on all.

It seems like there was an upgrade to vertx in the mean time that introduced an alternative dns resolver.

The issue however is that when I try to now deploy my application through my kubernetes cluster I am getting dns resolution issues.

If i try to launch it as is, I get this failure when trying to connect to my spring-cloud-config server (on init). The domain would not resolve since it' susing the google public dns.

May 06, 2022 7:42:53 AM io.netty.resolver.dns.DefaultDnsServerAddressStreamProvider
WARN: Default DNS servers: [/8.8.8.8:53, /8.8.4.4:53] (Google Public DNS as a fallback)
May 06, 2022 7:42:55 AM io.quarkus.runtime.ApplicationLifecycleManager run
ERROR: Failed to start application (with profile cloud)
java.net.UnknownHostException: Failed to resolve 'config-server.servers.svc.cluster.local'. Exceeded max queries per resolve 4 
    at io.netty.resolver.dns.DnsResolveContext.finishResolve(DnsResolveContext.java:1047)
    at io.netty.resolver.dns.DnsResolveContext.tryToFinishResolve(DnsResolveContext.java:1000)
    at io.netty.resolver.dns.DnsResolveContext.query(DnsResolveContext.java:418)
    at io.netty.resolver.dns.DnsResolveContext.onResponse(DnsResolveContext.java:629)
    at io.netty.resolver.dns.DnsResolveContext.access$400(DnsResolveContext.java:66)
    at io.netty.resolver.dns.DnsResolveContext$2.operationComplete(DnsResolveContext.java:462)
    at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:578)
    at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:571)
    at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:550)
    at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:491)
    at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:616)
    at io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:605)
    at io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:104)
    at io.netty.resolver.dns.DnsQueryContext.trySuccess(DnsQueryContext.java:216)
    at io.netty.resolver.dns.DnsQueryContext.finish(DnsQueryContext.java:208)
    at io.netty.resolver.dns.DnsNameResolver$DnsResponseHandler.channelRead(DnsNameResolver.java:1314)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
    at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
    at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
    at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
    at io.netty.channel.nio.AbstractNioMessageChannel$NioMessageUnsafe.read(AbstractNioMessageChannel.java:97)
    at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:722)
    at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:658)
    at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:584)
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:496)
    at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
    at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.base/java.lang.Thread.run(Thread.java:833)

If i set -Dvertx.disableDnsResolver=true however it seems like it doesn't even connect to my server instead (no request received on server side and no config being read)

2022-05-06 06:54:24,160 DEBUG [io.net.uti.ResourceLeakDetector] (main) -Dio.netty.leakDetection.level: simple
2022-05-06 06:54:24,161 DEBUG [io.net.uti.ResourceLeakDetector] (main) -Dio.netty.leakDetection.targetRecords: 4
2022-05-06 06:54:24,172 DEBUG [io.net.cha.MultithreadEventLoopGroup] (main) -Dio.netty.eventLoopThreads: 2
2022-05-06 06:54:24,181 DEBUG [io.net.uti.int.InternalThreadLocalMap] (main) -Dio.netty.threadLocalMap.stringBuilder.initialSize: 1024
2022-05-06 06:54:24,181 DEBUG [io.net.uti.int.InternalThreadLocalMap] (main) -Dio.netty.threadLocalMap.stringBuilder.maxSize: 4096
2022-05-06 06:54:24,184 DEBUG [io.net.cha.nio.NioEventLoop] (main) -Dio.netty.noKeySetOptimization: false
2022-05-06 06:54:24,184 DEBUG [io.net.cha.nio.NioEventLoop] (main) -Dio.netty.selectorAutoRebuildThreshold: 512
2022-05-06 06:54:24,188 DEBUG [io.net.uti.int.PlatformDependent] (main) org.jctools-core.MpscChunkedArrayQueue: available
2022-05-06 06:54:24,478 DEBUG [io.qua.spr.clo.con.cli.run.VertxSpringCloudConfigGateway] (main) Attempting to read configuration from 'http://config-server.servers.svc.cluster.local:8888/my-camel-service/cloud'.
2022-05-06 06:54:24,560 DEBUG [io.net.uti.NetUtil] (main) -Djava.net.preferIPv4Stack: false
2022-05-06 06:54:24,560 DEBUG [io.net.uti.NetUtil] (main) -Djava.net.preferIPv6Addresses: false
2022-05-06 06:54:24,561 DEBUG [io.net.uti.NetUtilInitializations] (main) Loopback interface: lo (lo, 127.0.0.1)
2022-05-06 06:54:24,562 DEBUG [io.net.uti.NetUtil] (main) /proc/sys/net/core/somaxconn: 1024
2022-05-06 06:54:24,779 DEBUG [io.net.buf.PooledByteBufAllocator] (vert.x-eventloop-thread-1) -Dio.netty.allocator.numHeapArenas: 2
2022-05-06 06:54:24,779 DEBUG [io.net.buf.PooledByteBufAllocator] (vert.x-eventloop-thread-1) -Dio.netty.allocator.numDirectArenas: 2
2022-05-06 06:54:24,779 DEBUG [io.net.buf.PooledByteBufAllocator] (vert.x-eventloop-thread-1) -Dio.netty.allocator.pageSize: 8192
2022-05-06 06:54:24,779 DEBUG [io.net.buf.PooledByteBufAllocator] (vert.x-eventloop-thread-1) -Dio.netty.allocator.maxOrder: 3
2022-05-06 06:54:24,779 DEBUG [io.net.buf.PooledByteBufAllocator] (vert.x-eventloop-thread-1) -Dio.netty.allocator.chunkSize: 65536
2022-05-06 06:54:24,779 DEBUG [io.net.buf.PooledByteBufAllocator] (vert.x-eventloop-thread-1) -Dio.netty.allocator.smallCacheSize: 256
2022-05-06 06:54:24,779 DEBUG [io.net.buf.PooledByteBufAllocator] (vert.x-eventloop-thread-1) -Dio.netty.allocator.normalCacheSize: 64
2022-05-06 06:54:24,779 DEBUG [io.net.buf.PooledByteBufAllocator] (vert.x-eventloop-thread-1) -Dio.netty.allocator.maxCachedBufferCapacity: 32768
2022-05-06 06:54:24,779 DEBUG [io.net.buf.PooledByteBufAllocator] (vert.x-eventloop-thread-1) -Dio.netty.allocator.cacheTrimInterval: 8192
2022-05-06 06:54:24,779 DEBUG [io.net.buf.PooledByteBufAllocator] (vert.x-eventloop-thread-1) -Dio.netty.allocator.cacheTrimIntervalMillis: 0
2022-05-06 06:54:24,779 DEBUG [io.net.buf.PooledByteBufAllocator] (vert.x-eventloop-thread-1) -Dio.netty.allocator.useCacheForAllThreads: true
2022-05-06 06:54:24,779 DEBUG [io.net.buf.PooledByteBufAllocator] (vert.x-eventloop-thread-1) -Dio.netty.allocator.maxCachedByteBuffersPerChunk: 1023
2022-05-06 06:54:24,859 DEBUG [io.net.buf.ByteBufUtil] (vert.x-eventloop-thread-1) -Dio.netty.allocator.type: pooled
2022-05-06 06:54:24,859 DEBUG [io.net.buf.ByteBufUtil] (vert.x-eventloop-thread-1) -Dio.netty.threadLocalDirectBufferSize: 0
2022-05-06 06:54:24,859 DEBUG [io.net.buf.ByteBufUtil] (vert.x-eventloop-thread-1) -Dio.netty.maxThreadLocalCharBufferSize: 16384
2022-05-06 06:54:25,056 DEBUG [io.net.buf.AbstractByteBuf] (vert.x-eventloop-thread-1) -Dio.netty.buffer.checkAccessible: true
2022-05-06 06:54:25,056 DEBUG [io.net.buf.AbstractByteBuf] (vert.x-eventloop-thread-1) -Dio.netty.buffer.checkBounds: true
2022-05-06 06:54:25,056 DEBUG [io.net.uti.ResourceLeakDetectorFactory] (vert.x-eventloop-thread-1) Loaded default ResourceLeakDetector: io.netty.util.ResourceLeakDetector@7e719269
2022-05-06 06:54:25,180 DEBUG [io.net.uti.Recycler] (vert.x-eventloop-thread-1) -Dio.netty.recycler.maxCapacityPerThread: 4096
2022-05-06 06:54:25,181 DEBUG [io.net.uti.Recycler] (vert.x-eventloop-thread-1) -Dio.netty.recycler.ratio: 8
2022-05-06 06:54:25,181 DEBUG [io.net.uti.Recycler] (vert.x-eventloop-thread-1) -Dio.netty.recycler.chunkSize: 32
2022-05-06 06:54:25,181 DEBUG [io.net.uti.Recycler] (vert.x-eventloop-thread-1) -Dio.netty.recycler.blocking: false
2022-05-06 06:54:26,563 DEBUG [io.net.buf.PoolThreadCache] (vert.x-eventloop-thread-1) Freed 2 thread-local buffer(s) from thread: vert.x-eventloop-thread-1

If i startup an application with the previous version of quarkus I get:

2022-05-06 07:21:16,937 DEBUG [io.net.uti.int.PlatformDependent] (main) org.jctools-core.MpscChunkedArrayQueue: available
2022-05-06 07:21:16,953 DEBUG [io.net.res.dns.DefaultDnsServerAddressStreamProvider] (main) Default DNS servers: [/10.24.0.10:53] (sun.net.dns.ResolverConfiguration)
2022-05-06 07:21:16,956 DEBUG [io.net.uti.NetUtil] (main) -Djava.net.preferIPv4Stack: false
2022-05-06 07:21:16,956 DEBUG [io.net.uti.NetUtil] (main) -Djava.net.preferIPv6Addresses: false
2022-05-06 07:21:16,957 DEBUG [io.net.uti.NetUtilInitializations] (main) Loopback interface: lo (lo, 127.0.0.1)
2022-05-06 07:21:16,957 DEBUG [io.net.uti.NetUtil] (main) /proc/sys/net/core/somaxconn: 1024
2022-05-06 07:21:17,248 DEBUG [io.qua.spr.clo.con.cli.run.VertxSpringCloudConfigGateway] (main) Attempting to read configuration from 'http://config-server.servers.svc.cluster.local:8888/my-camel-gateway/cloud'.
2022-05-06 07:21:17,541 DEBUG [io.net.buf.PooledByteBufAllocator] (vert.x-eventloop-thread-0) -Dio.netty.allocator.numHeapArenas: 2
2022-05-06 07:21:17,541 DEBUG [io.net.buf.PooledByteBufAllocator] (vert.x-eventloop-thread-0) -Dio.netty.allocator.numDirectArenas: 2

Is there a way to make it use my local dns configuration properly?

Answer 1

I set quarkus.naming.enableJndi=true and didn't use -Dvertx.disableDnsResolver and my app started using the intended dns again