Moving a process running under strace to a screen session
Published: November 10, 2018
Today I was in a scenario where I started running a process under strace
with the hopes of capturing diagnostic information about an error. However, instead of hitting the error, the process began to run succesfully.
It would likely take at least an hour to complete, and I was nervous that I’d lose my SSH connection, causing the process to wind up failing. I was also nervous that if I stopped the process to restart in a screen session, it wouldn’t be safe to re-run as it would have partially imported some data at that point.
Here, I’ll walk through my findings about moving a process running under strace
to a screen
session.
The Standard Approach
The standard approach for moving a process to a screen
session is as follows…
- Use Ctrl-Z to suspend the process
- Run
bg
to resume the process in the background - Run
disown
to remove the process from the current shell’s job table - Start a
screen
session - Use
reptyr
to take over the process in thescreen
session.
I tried following these steps for the parent strace
process and got the following error…
[-] Process 1653 (php) shares 1650's process group. Unable to attach.
(This most commonly means that 1650 has sub-processes).
Unable to attach to pid 1650: Invalid argument
The -T
Flag
Googling the error brought me to the GitHub issue “Doesn’t work for processes which have spawned subprocesses” filed against the nelhage/reptyr repository. There I learned the the -T
flag was added to handle exactly this scenario.
I decided to give it a try…
Upgrading reptyr
I quickly learned, however, that the version of reptyr
installed on my system (CentOS 7) did not support the -T
flag. I would need to clone the nelhage/reptyr
repository and build from source.
NOTE: While I didn't personally test this, GitHub user Smattr provides instructions to build reptyr without sudo permissions here.
Permissions Issues
Once I had a version of reptyr
installed that supported the -T
flag I tried again. This time I ran into another issue.
Unable to attach to pid 1650: Operation not permitted
Not exactly sure the details on this, but strace
didn’t seem to want to allow me to use reptyr
.
Trying sudo
My next step was to try to reptyr
with sudo
. Immediately after running reptyr
with sudo
my terminal became unresponsive to any input.
However, when I closed the terminal and opened a new one I could see that both the parent strace
process and it’s child were still running. reptyr
-ing with sudo
was successful!
What About Without sudo
I wanted to take this a step further and see if this was possible without sudo
. For my use case what we can do is kill the parent strace
process and then reptyr
the child PHP process to the screen
session. This answer describes how to kill strace
without killing the child process. Putting it all together…
- Ctrl-Z to suspend the strace process
-
kill -9
the strace PID - Start a new screen session
-
reptyr
to take over the child process in the screen session