Could not successfully bind to port 2181

Question

I'm following https://github.com/PacktPublishing/Apache-Kafka-Series---Kafka-Connect-Hands-on-Learning and I've below docker-compose file and using Mac.

version: '2'

services:
  # this is our kafka cluster.
  kafka-cluster:
    image: landoop/fast-data-dev:cp3.3.0
    environment:
      ADV_HOST: localhost         # Change to 192.168.99.100 if using Docker Toolbox
      RUNTESTS: 0                 # Disable Running tests so the cluster starts faster
    ports:
      - 2181:2181                 # Zookeeper
      - 3030:3030                 # Landoop UI
      - 8081-8083:8081-8083       # REST Proxy, Schema Registry, Kafka Connect ports
      - 9581-9585:9581-9585       # JMX Ports
      - 9092:9092                 # Kafka Broker

and when I run

docker-compose up kafka-cluster
[+] Running 1/0
 ⠿ Container code-kafka-cluster-1  Created                                                       0.0s
Attaching to code-kafka-cluster-1
code-kafka-cluster-1  | Setting advertised host to 127.0.0.1.
code-kafka-cluster-1  | runtime: failed to create new OS thread (have 2 already; errno=22)
code-kafka-cluster-1  | fatal error: newosproc
code-kafka-cluster-1  | 
code-kafka-cluster-1  | runtime stack:
code-kafka-cluster-1  | runtime.throw(0x512269, 0x9)
code-kafka-cluster-1  |         /usr/lib/go/src/runtime/panic.go:566 +0x95
code-kafka-cluster-1  | runtime.newosproc(0xc420026000, 0xc420035fc0)
code-kafka-cluster-1  |         /usr/lib/go/src/runtime/os_linux.go:160 +0x194
code-kafka-cluster-1  | runtime.newm(0x5203a0, 0x0)
code-kafka-cluster-1  |         /usr/lib/go/src/runtime/proc.go:1572 +0x132
code-kafka-cluster-1  | runtime.main.func1()
code-kafka-cluster-1  |         /usr/lib/go/src/runtime/proc.go:126 +0x36
code-kafka-cluster-1  | runtime.systemstack(0x593600)
code-kafka-cluster-1  |         /usr/lib/go/src/runtime/asm_amd64.s:298 +0x79
code-kafka-cluster-1  | runtime.mstart()
code-kafka-cluster-1  |         /usr/lib/go/src/runtime/proc.go:1079
code-kafka-cluster-1  | 
code-kafka-cluster-1  | goroutine 1 [running]:
code-kafka-cluster-1  | runtime.systemstack_switch()
code-kafka-cluster-1  |         /usr/lib/go/src/runtime/asm_amd64.s:252 fp=0xc420020768 sp=0xc420020760
code-kafka-cluster-1  | runtime.main()
code-kafka-cluster-1  |         /usr/lib/go/src/runtime/proc.go:127 +0x6c fp=0xc4200207c0 sp=0xc420020768
code-kafka-cluster-1  | runtime.goexit()
code-kafka-cluster-1  |         /usr/lib/go/src/runtime/asm_amd64.s:2086 +0x1 fp=0xc4200207c8 sp=0xc4200207c0
code-kafka-cluster-1  | Could not successfully bind to port 2181. Maybe some other service
code-kafka-cluster-1  | in your system is using it? Please free the port and try again.
code-kafka-cluster-1  | Exiting.
code-kafka-cluster-1 exited with code 1

Note: % sudo lsof -i :2181 - this command shows no output.

Answer 1

the landoop/fast-data-dev library is not working on arm64 Apple M1 chip.

Here you can fix the problem by updating the Dockerfile. https://github.com/lensesio/fast-data-dev/issues/175#issuecomment-947001807

Answer 2

Error suggests you're running something else on port 2181 already. So either stop that, or remove the port mapping since you shouldn't be connecting to Zookeeper anyway for using Kafka. As of latest Kafka versions (which I doubt the linked course will be using), --zookeeper flags are removed from Kafka CLI tools

Other solution would be to not use the Landoop container. Plenty of other Docker Compose files exist on the web for Kafka

Overall, I'd suggest not using Docker at all for developing a Kafka Connector.