Forward traffic via libnetfilter_queue

Question

I have a need where i need to forward traffic with libnetfilter_queue to another server, after adding some informations in it.

Problem is that i'm stuck on the forward part , i can do what i want with the packet and add the needed informations, though even while requeing the packet (with NF_REPEAT) with another destination IP, my packet will still be processed by my machine and not sent to the remote host.

From what i understand, the problem lies in the iptables. My iptables config is

iptables -A INPUT -j NFQUEUE --queue-num 0 --queue-bypass
iptables -t nat -m mark --mark 1 -A PREROUTING -s $clientIP -p tcp -j DNAT --to-destination $serverIP
iptables -t nat -A POSTROUTING -j MASQUERADE

The configuration forwards the traffic when i stop my program (and thus the queue is bypassed), so maybe it has something to do with where i put my hook in the iptables flow? It would also help if i can masquerade the traffic but not mandatory (would be better for our logging tools to know where the packet comes from)

If it's usefull, here is the forward function

void reroute(struct iphdr *buffer){
        //TODO
        printf("The address before rerouting is : %" PRIu32 "\n", buffer->daddr);
        fflush(stdout);
        buffer->daddr = server_ip;//__bswap_32(server_ip);
        printf("The address is : %" PRIu32 "\n", buffer->daddr);
        return;

}

Here is the calling function

int ProcessPacket(struct iphdr* buffer)
{
        
        //Get the IP Header part of this packet
        struct iphdr *iph = buffer;
        printf("IP Protocol is: %d\n", iph->protocol);
        switch (iph->protocol) //Check the Protocol and do accordingly
        {
                case 1:  //ICMP Protocol
                        print_icmp_packet(buffer);
                        break;
                case IPPROTO_TCP:  //TCP Protocol
                        print_tcp_packet(buffer);
                        process(buffer);
                        reroute(buffer);
                        printf("The address after rerouting is: %" PRIu32 "\n", buffer->daddr);
                        return 2;

                        
                        break;
                default: 
                        return 1;
                        break;
        }
        return 0;
}

The libnetfilter interesting part is

 if(routing == 2 && !nfq_get_nfmark(nfad)){
                printf("destination address is, %x \n", ip->daddr);
                struct tcphdr *tcp_header = nfq_tcp_get_hdr(pkBuff);
                if(tcp_header != NULL )
                {
                        nfq_tcp_compute_checksum_ipv4(nfq_tcp_get_hdr(pkBuff),ip);
                }
                return nfq_set_verdict2(queue, ntohl(ph->packet_id), NF_REPEAT, 1, pktb_len(pkBuff), pktb_data(pkBuff)); //queued again so that once marked they can be forwarded
        }
        else if(routing ==1){
                return nfq_set_verdict(queue, ntohl(ph->packet_id), NF_ACCEPT,0, NULL);
        }
        else{
                return nfq_set_verdict(queue, ntohl(ph->packet_id), NF_ACCEPT, 0, NULL);
        }