How do you correctly cleanup and re-use SysV shared memory segments?

Question

I have been writing some gnarly test code for an API that uses the Linux SysV SHM interface. i.e. ftok(), shmget(), shmctl(), shmat()

Unfortunately I am not free to alter this interface (for example to use shm_open() instead or nuke it from orbit).

I discovered a bug via an integration test which spawns two unrelated processes using the shared memory in that it was not cleaning up shared memory segments using shmdt(). I further discovered that the convention is to use:

shmctl(segmentId, IPC_RMID, 0);

to tell the system that the segment can be deleted once all processes have detached. Adding this to the integration tests fixed the bug that the shared memory was not cleaned up.

Looking at segments using ipcs -m you can see they are now marked "dest".

I have also created lower-level 'unit' tests which work by forking a child process to create the shared memory. Attempting to apply the same correction to the test code I have some strange observations:

• If I do the right thing and add calls to shmctl(segmentId, IPC_RMID) the tests stop working.

• If I remove such calls the tests start working again. (for now my solution is simply not to use IPC_RMID in the unit test code)

• I'm not sure if this is the problem but, investigating further due to ftok() being collision prone the same Id is used for some tests after a previous test has deleted it. This is not the case for every failure though.

• If I change from O_CREAT to O_CREAT|O_EXCL opening re-using the same token fails with EEXIST. (suggesting 'dest' segments still exist in some sense and can be re-used)

• I also notice that from running many tests I have a fair number of shm segments listed as "dest" but with nattach=2 meaning two processes failed to call shmdt(). If I list the pids that supposedly created it and last used it using ipcs -mp they are not the processes that created or used it. Instead they belong to things like firefox which is just slightly confusing! I don't understand this. Presumably they will disappear on reboot but they don't seem directly related to this issue. I think these came killing the earlier integration tests before I fixed the missing shmdt() and shmctl(IPC_RMID) issues.

• Many segments listed by ipcs -m have a key of 0x00000000 which seems suspicious. Its not suspicious these are private segments.

Q Is there something special about how (sysV) SHM works in the context of parent and child processes which means the IPC_RMID behaves differently and or is not required?

Q Is there programmatic way to make it safe to re-use a segment Id after having 'removed' it using IPC_RMID a short while previously in a child process?

Q Is there a way to force removal (unattach / delete) the segments marked dest in "ipcs -m" without rebooting? (at the OS level)

Q Is "ipcs -pm" displaying actual pid's or something else? Is there another way to find out what the system thinks is really still attached? (I believe nothing is as I used kill -9 on everything using the SHM interface)

Q Is there a way to force detach a process as if shmdt() has been called?

Answer 1

I was misled into thinking it was proper form to call shmctl(segmentId, IPC_RMID) as soon as the process designated as the owner has attached to the shared memory.

In fact IPC_RMID should not be called until all processes have attached.

Part of the answer is here:

https://comp.unix.programmer.narkive.com/iLg3PhfZ/shmctl-ipc-rmid-oddity

It seems that IPC_RMID sets the segment to private so that no new processes can attach to it.

A way of guaranteeing a unique segment is deliberately using IPC_PRIVATE to start with:

id = shmget(IPC_PRIVATE, IPC_CREAT | mode);

This also avoids the need to use ftok() and risk colliding with another segment. Unfortunately I cannot use that here as the interface is predicated on identifying the segment with ftok(). At least I understand the issue here.

Someone wiser may be able to chip in with better ways of cleaning up before re-use.

See also https://www.linuxquestions.org/questions/programming-9/shmctl-ipc_rmid-precludes-further-attachments-574636/

Also consider this question and answer: Linux IPC: shared memory recovery