How can I pass and access C data from Java?

Question

I've been doing some programming in Java and some in C but now I need to sort of use both together.

Here's the situation, I'm using Hadoop/Hbase to process and store a lot of data but I'm using C/Cuda to do number crunching on the data. Is there a stable/mature/common way to take data (it's basically a log file) in Java and pass it to a C program, which C processes the data it stores it as a linked list that is then accessible by the Java app?

I might not be searching for the right thing, but so far I found JavaCPP, which is good but seems to involve both programs together. Because Java handles the data flow and C handles the processing of the data, I thought it might be better to keep them as independent programs that can communicate to each other as opposed to a single program that may become confusing. But I'm totally flexible so any suggestions/solutions are welcomed.

Answer 1

You may find it easier to keep the programs testable and clear if you leave them separate and then use a client-server approach, or simply choose a common file format and have the latter steps poll the output directory for new files to process.

To make it easier to define file formats across different languages, consider a package like Apache Thrift or Google Protocol Buffers.

Answer 2

Here what I have on the top of my head 1. run C program using command line from java app. 2. Use JNI/JNA 3. Implement your own "client-server" architecture. It sounds complicated but in some cases it may be the best and the simplest solution. 4. Communicate using Web service, SOAP, REST, whatever.

I hope this is helpful for the beginning. You are welcome to ask more specific questions once you have.