While writing a Linux kernel module, I faced a problem with a kthread that I am unable to wake up while waiting for a semaphore to unlock. This causes the thread to be unstoppable and rmmod
to freeze when trying to unload the module.
Please note: This module runs on a 3.10 kernel and I have no way to update this to a newer version (customer demands running on stock CentOS 7, which features a 3.10 kernel).
Below are the interesting parts from the module source code. It represents a simple producer consumer problem, the list is not limited in size (thus no producer semaphore is required) and guarded by a mutex. The function to take something from the list is guarded by a semaphore that is upped by the producer and downed by the consumer. The producer function is called from an external event (in fact a char device) not shown in this code snippets to keep is as small as possible. The process works perfectly, except for the module unloading.
The parts that cause the freezing are marked with comments in the code snippets. The only way I know of to stop a kthread is to call kthread_stop
on it, which fails in this case since it apparently cannot wake up the sleeping thread. Because it waits for the thread to exit, the call will never return and the module will not unload.
How can I wake up and stop the kthread waiting for the semaphore to unload the module successfully?
List implementation:
#include <linux/mutex.h>
#include <linux/list.h>
#include <linux/semaphore.h>
static LIST_HEAD(list);
DEFINE_MUTEX(list_lock);
DEFINE_SEMAPHORE(sem_list_consumer);
void add_to_list(struct *some_struct) {
int rv = mutex_lock_interruptible(&list_lock);
if(rv != 0) {
return;
}
list_add(&some_struct->list, &list);
mutex_unlock(&list_lock);
up(&sem_list_consumer);
}
struct some_struct * take_from_list() {
int rv;
some_struct *entry;
/* this is where the kthread will freeze when module is unloaded */
rv = down_interruptible(&sem_list_consumer);
if(rv != 0) {
return NULL;
}
rv = mutex_lock_interruptible(&list_lock);
if(rv != 0) {
up(&sem_list_consumer);
return NULL;
}
if (list_empty(&list)) {
mutex_unlock(&list_lock);
return NULL;
} else {
entry = list_last_entry(&list, struct some_struct, list);
if (entry) {
list_del(&entry->list);
}
}
mutex_unlock(&list_lock);
return entry;
}
Consumer kthread implementation:
#include <linux/kthread.h>
#include <linux/sched.h>
int consumer_kthread(void *data) {
struct some_struct *entry;
set_current_state(TASK_INTERRUPTIBLE);
while (!kthread_should_stop()) {
/* Here the function including the semaphore is called */
entry = take_from_list();
if(entry != NULL) {
/* Do something with 'entry' here */
} else {
/* Some handling of returned NULL pointers */
}
set_current_state(TASK_INTERRUPTIBLE);
}
set_current_state(TASK_RUNNING);
return 0;
}
Module implementation:
#include <linux/init.h>
#include <linux/kthread.h>
#include <linux/module.h>
#include <linux/sched.h>
static struct task_struct *consumer_task;
static int __init initModule(void) {
consumer_task = kthread_run(consumer_kthread, NULL, "list-consumer");
return 0;
}
static void __exit exitModule(void) {
/* this call will cause rmmod to freeze forever */
kthread_stop(consumer_task);
}
module_init(initModule);
module_exit(exitModule);
MODULE_LICENSE("GPL v2");
MODULE_DESCRIPTION("My Module");
take_from_list
function that returnNULL
. In one of those three places it returns while still holding thesem_list_consumer
semaphore. I suspect this inconsistency has something to do with your problem. Otherwise, how could the caller tell whether it needs to release the semaphore or not?up(&sem_list_consumer);
between the linesmutex_unlock(&list_lock);
andreturn NULL;
in yourtake_from_list
function. That's unless you expect thesem_list_consumer
semaphore to be held even whentake_from_list
returnsNULL
, in which case you'd need to somehow deal withtake_from_list
sometimes returningNULL
without holding the semaphore.