Problem to read data from HBase on AWS EMR cluster using Java Spring boot client

Question

I'm trying to write a simple API application to read data from HBase on an AWS EMR cluster. But I get an UnknownHostException when I try to send the request.

However, in the console log you can see, connection to the master node is established and a zookeeper session is also created.

2022-05-13 22:16:00.106  INFO 24480 --- [  restartedMain] org.apache.zookeeper.ZooKeeper           : Initiating client connection, connectString=10-41-54-222:2181 sessionTimeout=90000 watcher=org.apache.hadoop.hbase.zookeeper.PendingWatcher@697477df
2022-05-13 22:16:00.129  INFO 24480 --- [49.71.222:2181)] org.apache.zookeeper.ClientCnxn          : Opening socket connection to server 10-41-54-222/10-41-54-222:2181. Will not attempt to authenticate using SASL (unknown error)
2022-05-13 22:16:00.422  INFO 24480 --- [49.71.222:2181)] org.apache.zookeeper.ClientCnxn          : Socket connection established to 10-41-54-222/10-41-54-222:2181, initiating session
2022-05-13 22:16:00.717  INFO 24480 --- [49.71.222:2181)] org.apache.zookeeper.ClientCnxn          : Session establishment complete on server 10-41-54-222/10-41-54-222:2181, sessionid = 0x1000006e6da0664, negotiated timeout = 40000

Connection bean:

    @Bean
    public Connection Connection() throws IOException {
        org.apache.hadoop.conf.Configuration conf = HBaseConfiguration.create();
        conf.set("hbase.zookeeper.quorum", "10-41-54-222");
        conf.set("hbase.zookeeper.property.clientPort", "2181");
        conf.set("hbase.master", "10-41-54-222:16000");
        conf.set("zookeeper.znode.parent", "/hbase");
        conf.set("hbase.zookeeper.property.dataDir", "s3://asp-emr-dev/hbase_new_2");
        return ConnectionFactory.createConnection(conf);
    }

Console output:

2022-05-13 22:15:53.751  INFO 24480 --- [  restartedMain] com.example.demo.DemoApplication         : Starting DemoApplication using Java 1.8.0-262 on ZSRIL-FD5K2B3 with PID 24480 (D:\My Projects\SpringBoot-HBase-Demo\target\classes started by pkulasoori in D:\My Projects\SpringBoot-HBase-Demo)
2022-05-13 22:15:53.754  INFO 24480 --- [  restartedMain] com.example.demo.DemoApplication         : No active profile set, falling back to 1 default profile: "default"
2022-05-13 22:15:53.934  INFO 24480 --- [  restartedMain] o.s.b.devtools.restart.ChangeableUrls    : The Class-Path manifest attribute in C:\Users\pkulasoori\.m2\repository\com\sun\xml\bind\jaxb-impl\2.2.3-1\jaxb-impl-2.2.3-1.jar referenced one or more files that do not exist: file:/C:/Users/pkulasoori/.m2/repository/com/sun/xml/bind/jaxb-impl/2.2.3-1/jaxb-api.jar,file:/C:/Users/pkulasoori/.m2/repository/com/sun/xml/bind/jaxb-impl/2.2.3-1/activation.jar,file:/C:/Users/pkulasoori/.m2/repository/com/sun/xml/bind/jaxb-impl/2.2.3-1/jsr173_1.0_api.jar,file:/C:/Users/pkulasoori/.m2/repository/com/sun/xml/bind/jaxb-impl/2.2.3-1/jaxb1-impl.jar
2022-05-13 22:15:53.935  INFO 24480 --- [  restartedMain] .e.DevToolsPropertyDefaultsPostProcessor : Devtools property defaults active! Set 'spring.devtools.add-properties' to 'false' to disable
2022-05-13 22:15:53.935  INFO 24480 --- [  restartedMain] .e.DevToolsPropertyDefaultsPostProcessor : For additional web related logging consider setting the 'logging.level.web' property to 'DEBUG'
2022-05-13 22:15:58.183  INFO 24480 --- [  restartedMain] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port(s): 8080 (http)
2022-05-13 22:15:58.195  INFO 24480 --- [  restartedMain] o.apache.catalina.core.StandardService   : Starting service [Tomcat]
2022-05-13 22:15:58.195  INFO 24480 --- [  restartedMain] org.apache.catalina.core.StandardEngine  : Starting Servlet engine: [Apache Tomcat/9.0.58]
2022-05-13 22:15:58.320  INFO 24480 --- [  restartedMain] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext
2022-05-13 22:15:58.320  INFO 24480 --- [  restartedMain] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 4384 ms
2022-05-13 22:15:58.686  WARN 24480 --- [  restartedMain] org.apache.hadoop.util.NativeCodeLoader  : Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2022-05-13 22:16:00.097  INFO 24480 --- [  restartedMain] o.a.h.h.zookeeper.RecoverableZooKeeper   : Process identifier=hconnection-0x639d0ae connecting to ZooKeeper ensemble=10-41-54-222:2181
2022-05-13 22:16:00.103  INFO 24480 --- [  restartedMain] org.apache.zookeeper.ZooKeeper           : Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT
2022-05-13 22:16:00.103  INFO 24480 --- [  restartedMain] org.apache.zookeeper.ZooKeeper           : Client environment:host.name=host.docker.internal
2022-05-13 22:16:00.103  INFO 24480 --- [  restartedMain] org.apache.zookeeper.ZooKeeper           : Client environment:java.version=1.8.0-262
2022-05-13 22:16:00.103  INFO 24480 --- [  restartedMain] org.apache.zookeeper.ZooKeeper           : Client environment:java.vendor=OpenLogic-OpenJDK
2022-05-13 22:16:00.104  INFO 24480 --- [  restartedMain] org.apache.zookeeper.ZooKeeper           : Client environment:java.home=C:\Program Files\OpenJDK\jdk-8.0.262.10-hotspot\jre
2022-05-13 22:16:00.105  INFO 24480 --- [  restartedMain] org.apache.zookeeper.ZooKeeper           : Client environment:java.io.tmpdir=C:\Users\PKULAS~1\AppData\Local\Temp\
2022-05-13 22:16:00.105  INFO 24480 --- [  restartedMain] org.apache.zookeeper.ZooKeeper           : Client environment:java.compiler=<NA>
2022-05-13 22:16:00.105  INFO 24480 --- [  restartedMain] org.apache.zookeeper.ZooKeeper           : Client environment:os.name=Windows 10
2022-05-13 22:16:00.105  INFO 24480 --- [  restartedMain] org.apache.zookeeper.ZooKeeper           : Client environment:os.arch=amd64
2022-05-13 22:16:00.105  INFO 24480 --- [  restartedMain] org.apache.zookeeper.ZooKeeper           : Client environment:os.version=10.0
2022-05-13 22:16:00.105  INFO 24480 --- [  restartedMain] org.apache.zookeeper.ZooKeeper           : Client environment:user.name=pkulasoori
2022-05-13 22:16:00.105  INFO 24480 --- [  restartedMain] org.apache.zookeeper.ZooKeeper           : Client environment:user.home=C:\Users\pkulasoori
2022-05-13 22:16:00.105  INFO 24480 --- [  restartedMain] org.apache.zookeeper.ZooKeeper           : Client environment:user.dir=D:\My Projects\SpringBoot-HBase-Demo
2022-05-13 22:16:00.106  INFO 24480 --- [  restartedMain] org.apache.zookeeper.ZooKeeper           : Initiating client connection, connectString=10-41-54-222:2181 sessionTimeout=90000 watcher=org.apache.hadoop.hbase.zookeeper.PendingWatcher@697477df
2022-05-13 22:16:00.129  INFO 24480 --- [49.71.222:2181)] org.apache.zookeeper.ClientCnxn          : Opening socket connection to server 10-41-54-222/10-41-54-222:2181. Will not attempt to authenticate using SASL (unknown error)
2022-05-13 22:16:00.422  INFO 24480 --- [49.71.222:2181)] org.apache.zookeeper.ClientCnxn          : Socket connection established to 10-41-54-222/10-41-54-222:2181, initiating session
2022-05-13 22:16:00.717  INFO 24480 --- [49.71.222:2181)] org.apache.zookeeper.ClientCnxn          : Session establishment complete on server 10-41-54-222/10-41-54-222:2181, sessionid = 0x1000006e6da0664, negotiated timeout = 40000
2022-05-13 22:16:02.733  INFO 24480 --- [  restartedMain] o.s.b.d.a.OptionalLiveReloadServer       : LiveReload server is running on port 35729
2022-05-13 22:16:02.775  INFO 24480 --- [  restartedMain] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 8080 (http) with context path ''
2022-05-13 22:16:02.785  INFO 24480 --- [  restartedMain] com.example.demo.DemoApplication         : Started DemoApplication in 9.855 seconds (JVM running for 11.511)
2022-05-13 22:17:34.275  INFO 24480 --- [nio-8080-exec-2] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring DispatcherServlet 'dispatcherServlet'
2022-05-13 22:17:34.275  INFO 24480 --- [nio-8080-exec-2] o.s.web.servlet.DispatcherServlet        : Initializing Servlet 'dispatcherServlet'
2022-05-13 22:17:34.277  INFO 24480 --- [nio-8080-exec-2] o.s.web.servlet.DispatcherServlet        : Completed initialization in 2 ms
2022-05-13 22:17:34.343  INFO 24480 --- [nio-8080-exec-2] o.a.h.h.zookeeper.RecoverableZooKeeper   : Process identifier=hconnection-0x68538d5e connecting to ZooKeeper ensemble=10-41-54-222:2181
2022-05-13 22:17:34.343  INFO 24480 --- [nio-8080-exec-2] org.apache.zookeeper.ZooKeeper           : Initiating client connection, connectString=10-41-54-222:2181 sessionTimeout=90000 watcher=org.apache.hadoop.hbase.zookeeper.PendingWatcher@15cabe52
2022-05-13 22:17:34.355  INFO 24480 --- [49.71.222:2181)] org.apache.zookeeper.ClientCnxn          : Opening socket connection to server 10-41-54-222/10-41-54-222:2181. Will not attempt to authenticate using SASL (unknown error)
2022-05-13 22:17:34.648  INFO 24480 --- [49.71.222:2181)] org.apache.zookeeper.ClientCnxn          : Socket connection established to 10-41-54-222/10-41-54-222:2181, initiating session
2022-05-13 22:17:34.941  INFO 24480 --- [49.71.222:2181)] org.apache.zookeeper.ClientCnxn          : Session establishment complete on server 10-41-54-222/10-41-54-222:2181, sessionid = 0x1000006e6da0665, negotiated timeout = 40000
2022-05-13 22:18:21.916  INFO 24480 --- [hared--pool2-t1] o.a.h.hbase.client.RpcRetryingCaller     : Call exception, tries=10, retries=35, started=45692 ms ago, cancelled=false, msg=ip-10-41-54-158.ec2.internal row 'emp,,99999999999999' on table 'hbase:meta' at region=hbase:meta,,1.1588230740, hostname=ip-10-41-54-158.ec2.internal,16020,1652454036400, seqNum=0
2022-05-13 22:18:32.812  INFO 24480 --- [hared--pool2-t1] o.a.h.hbase.client.RpcRetryingCaller     : Call exception, tries=11, retries=35, started=56595 ms ago, cancelled=false, msg=ip-10-41-54-158.ec2.internal row 'emp,,99999999999999' on table 'hbase:meta' at region=hbase:meta,,1.1588230740, hostname=ip-10-41-54-158.ec2.internal,16020,1652454036400, seqNum=0
java.net.SocketTimeoutException: callTimeout=60000, callDuration=70146: Call to address=ip-10-49-71-157.ec2.internal/10.49.71.157:16020 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=7,methodName=Scan], waitTime=843ms, rpcTimeout=788ms row 'emp,,99999999999999' on table 'hbase:meta' at region=hbase:meta,,1.1588230740, hostname=ip-10-49-71-157.ec2.internal,16020,1652454036400, seqNum=-1
    at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:160)
    at org.apache.hadoop.hbase.client.ResultBoundedCompletionService$QueueingFuture.run(ResultBoundedCompletionService.java:80)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call to address=ip-10-49-71-157.ec2.internal/10.49.71.157:16020 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=7,methodName=Scan], waitTime=843ms, rpcTimeout=788ms
    at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:217)
    at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:383)
    at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:89)
    at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:414)
    at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:410)
    at org.apache.hadoop.hbase.ipc.Call.setTimeout(Call.java:111)
    at org.apache.hadoop.hbase.ipc.RpcConnection$1.run(RpcConnection.java:136)
    at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:669)
    at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:744)
    at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:469)
    ... 1 more
Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=7,methodName=Scan], waitTime=843ms, rpcTimeout=788ms
    at org.apache.hadoop.hbase.ipc.RpcConnection$1.run(RpcConnection.java:137)
    ... 4 more

would appreciate some help here, Thanks