linux fcntl file lock with timeout

Question

the standard linux fcntl call doesn't provide a timeout option. I'm considering implement a timeout lock with signal.

Here is the description of blocking lock:

---

F_SETLKW

This command shall be equivalent to F_SETLK except that if a shared or exclusive lock is blocked by other locks, the thread shall wait until the request can be satisfied. If a signal that is to be caught is received while fcntl() is waiting for a region, fcntl() shall be interrupted. Upon return from the signal handler, fcntl() shall return -1 with errno set to [EINTR], and the lock operation shall not be done.

---

So what kind of signal I need to use to indicate the lock to be interrupted? And since there're multiple threads running in my process, I only want to interrupt this IO thread who is blokcing for the file lock, other threads should not be affected, but signal is process-level, I'm not sure how to handle this situation.

Added:

I've written a simple implemnetation using signal.

int main(int argc, char **argv) {
  std::string lock_path = "a.lck";

  int fd = open(lock_path.c_str(), O_CREAT | O_RDWR, S_IRWXU | S_IRWXG | S_IRWXO);

  if (argc > 1) {
    signal(SIGALRM, [](int sig) {});
    std::thread([](pthread_t tid, unsigned int seconds) {
      sleep(seconds);
      pthread_kill(tid, SIGALRM);
    }, pthread_self(), 3).detach();
    int ret = file_rwlock(fd, F_SETLKW, F_WRLCK);

    if (ret == -1) std::cout << "FAIL to acquire lock after waiting 3s!" << std::endl;

  } else {
    file_rwlock(fd, F_SETLKW, F_WRLCK);
    while (1);
  }

  return 0;
}

by running ./main followed by ./main a, I expect the first process holding the lock forever, and second process try to get the lock and interrupted after 3s, but the second process just terminated.

Could anyone tell me what's wrong with my code?

Answer 1

So what kind of signal I need to use to indicate the lock to be interrupted?

The most obvious choice of signal would be SIGUSR1 or SIGUSR2. These are provided to serve user-defined purposes.

There is also SIGALRM, which would be natural if you're using a timer that produces such a signal to do your timekeeping, and which makes some sense even to generate programmatically, as long as you are not using it for other purposes.

And since there're multiple threads running in my process, I only want to interrupt this IO thread who is blokcing for the file lock, other threads should not be affected, but signal is process-level, I'm not sure how to handle this situation.

You can deliver a signal to a chosen thread in a multithreaded process via the pthread_kill() function. This also stands up well to the case where more than one thread is waiting on a lock at the same time.

With regular kill(), you also have the alternative of making all threads block the chosen signal (sigprocmask()), and then having the thread making the lock attempt unblock it immediately prior. When the chosen signal is delivered to the process, a thread that is not presently blocking it will receive it, if any such thread is available.

Example implementation

This supposes that a signal handler has already been set up to handle the chosen signal (it doesn't need to do anything), and that the signal number to use is available via the symbol LOCK_TIMER_SIGNAL. It provides the wanted timeout behavior as a wrapper function around fcntl(), with command F_SETLKW as described in the question.

#define _POSIX_C_SOURCE 200809L
#define _GNU_SOURCE

#include <unistd.h>
#include <signal.h>
#include <time.h>
#include <fcntl.h>
#include <sys/types.h>
#include <sys/syscall.h>

#if (__GLIBC__ < 2) || (__GLIBC__ == 2 && __GLIBC_MINOR__ < 30)
// glibc prior to 2.30 does not provide a wrapper 
// function for this syscall:    
static pid_t gettid(void) {
    return syscall(SYS_gettid);
}
#endif

/**
 * Attempt to acquire an fcntl() lock, with timeout
 *
 * fd: an open file descriptor identifying the file to lock
 * lock_info: a pointer to a struct flock describing the wanted lock operation
 * to_secs: a time_t representing the amount of time to wait before timing out
 */    
int try_lock(int fd, struct flock *lock_info, time_t to_secs) {
    int result;
    timer_t timer;

    result = timer_create(CLOCK_MONOTONIC,
            & (struct sigevent) {
                .sigev_notify = SIGEV_THREAD_ID,
                ._sigev_un = { ._tid = gettid() },
                // note: gettid() conceivably can fail
                .sigev_signo = LOCK_TIMER_SIGNAL },
            &timer);
    // detect and handle errors ...

    result = timer_settime(timer, 0,
            & (struct itimerspec) { .it_value = { .tv_sec = to_secs } },
            NULL);

    result = fcntl(fd, F_SETLKW, lock_info);
    // detect and handle errors (other than EINTR) ...
    // on EINTR, may want to check that the timer in fact expired

    result = timer_delete(timer);
    // detect and handle errors ...

    return result;
}

That works as expected for me.

Notes:

• signal dispositions are process-wide properties, not per-thread properties, so you need to coordinate your use of signals throughout the whole program. With that being the case, it is not useful (and it might be dangerous) for the try_lock function itself to modify the disposition of its chosen signal.
• The timer_* interfaces provide POSIX interval timers, but the provision for designating a specific thread to receive signals from such a timer is Linux-specific.
• On Linux, you'll need to link with -lrt for the timer_* functions.
• The above works around the fact that Glibc's struct sigevent does not conform to its own docs (at least in relatively old version 2.17). The docs claim that struct sigevent has a member sigev_notify_thread_id, but in fact it does not. Instead, it has an undocumented union containing a corresponding member, and it provides a macro to patch up the difference -- but that macro does not work as a member designator in a designated initializer.
• fcntl locks operate on a per-process basis. Thus, different threads of the same process cannot exclude each other via this kind of lock. Moreover, different threads of the same process can modify fcntl() locks obtained via other threads without any special effort or any notification to either thread.
• You could consider creating and maintaining a per-thread static timer for this purpose instead of creating and then destroying a new one on each call.
• Be aware that fcntl() will return EINTR if interrupted by any signal that does not terminate the thread. You might, therefore, want to use a signal handler that sets an affirmative per-thread flag by which you can verify that the actual timer signal was received, so as to retry the lock if it was interrupted by a different signal.
• It's up to you to ensure that the thread does not receive the chosen signal for some other reason, or else to confirm by some other means that time actually expired in the event that locking fails with EINTR.

Answer 2

A better solution might be to use select():

https://www.gnu.org/software/libc/manual/html_node/Waiting-for-I_002fO.html

#include <errno.h>
#include <stdio.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/time.h>

int
input_timeout (int filedes, unsigned int seconds)
{
  fd_set set;
  struct timeval timeout;

  /* Initialize the file descriptor set. */
  FD_ZERO (&set);
  FD_SET (filedes, &set);

  /* Initialize the timeout data structure. */
  timeout.tv_sec = seconds;
  timeout.tv_usec = 0;

  /* select returns 0 if timeout, 1 if input available, -1 if error. */
  return TEMP_FAILURE_RETRY (select (FD_SETSIZE,
                                     &set, NULL, NULL,
                                     &timeout));
}

int
main (void)
{
  fprintf (stderr, "select returned %d.\n",
           input_timeout (STDIN_FILENO, 5));
  return 0;
}

Answer 3

I've had some difficulty with this. Finally got it working.

// main1.cpp
#include <thread>
#include <chrono>
#include <iostream>

int main(int argc, char *argv[]) {
    int fd = open(argv[1],O_RDWR|O_CREAT,S_IRWXU | S_IRWXG | S_IRWXO);

    struct flock fd_lock;
    fd_lock.l_type = F_WRLCK;    /* read/write (exclusive) fd_lock_lock */
    fd_lock.l_whence = SEEK_SET; /* base for seek offsets */
    fd_lock.l_start = 0;         /* 1st byte in file */
    fd_lock.l_len = 0;           /* 0 here means 'until EOF' */
    fd_lock.l_pid = getpid();

    std::cout << "locked file\n";
    fcntl(fd, F_SETLKW, &fd_lock);

    std::cout << "file locked\n";
    std::this_thread::sleep_for(std::chrono::seconds(100));
}

// main2.cpp
#include <cstring>
#include <chrono>
#include <thread>
#include <iostream>

struct signal_trigger_thread_args { 
    int signum;
    pthread_t tid;
    unsigned int seconds;
};

void alarm_handler(int signum, siginfo_t *x, void *y) {
    // std::cout << "Alarm Handler!\n";
}

void *trigger_signal_after_time(void *arg) {
    struct signal_trigger_thread_args *_arg = (struct signal_trigger_thread_args*)arg; 

    std::this_thread::sleep_for(std::chrono::seconds(_arg->seconds));
    std::cout << "triggering signal!\n";
    pthread_kill(_arg->tid,_arg->signum);
    return NULL;
}

int fcntl_wait_for(int fd, int cmd, struct flock *_flock, int signum, unsigned int _seconds) {
    // Create a thread to trigger the signal.
    pthread_t signal_trigger_thread;

    struct signal_trigger_thread_args args;
    args.signum = signum;
    args.tid = pthread_self();
    args.seconds = _seconds;

    int return_value = pthread_create(&signal_trigger_thread, NULL, &trigger_signal_after_time,(void *)&args);

    if ( return_value ) {
        std::cout << "pthread creation failed\n";
        return -2;
    }

    return_value = fcntl(fd, cmd, _flock);

    if ( return_value == 0 ) { return 0; }

    if ( return_value = -1 && errno == EINTR ) {
        return 1;
    }
    return -1;
}

int main(int argc, char *argv[]) {
    // initialize_signal_handlers();
    static struct sigaction _sigact;

    memset(&_sigact,0,sizeof(_sigact));
    _sigact.sa_sigaction = alarm_handler;
    _sigact.sa_flags = SA_SIGINFO;

    sigaction(SIGUSR1,&_sigact,NULL);


    int fd = open(argv[1],O_RDWR|O_CREAT,S_IRWXU | S_IRWXG | S_IRWXO);

    struct flock fd_lock;
    fd_lock.l_type = F_WRLCK;    /* read/write (exclusive) fd_lock_lock */
    fd_lock.l_whence = SEEK_SET; /* base for seek offsets */
    fd_lock.l_start = 0;         /* 1st byte in file */
    fd_lock.l_len = 0;           /* 0 here means 'until EOF' */
    fd_lock.l_pid = getpid();

    std::cout << "waiting for file to be freed for 5 seconds\n";
    int return_value = fcntl_wait_for(fd, F_SETLKW, &fd_lock, SIGUSR1, 5);

    if ( return_value == 1 ) {
        std::cout << "fcntl was interrupted!\n";
    } else if ( return_value == 0 ) {
        std::cout << "fcntl obtained lock!\n";
    } else {
        std::cout << "fcntl failed!\n";
    }
}