3

How to run the first process from a list of processes stored in a file and immediately delete the first line as if the file was a queue and I called "pop"?

I'd like to call the first command listed in a simple text file with \n as the separator in a pop-like fashion:

Figure 1:

cmdqueue.lst :

proc_C1
proc_C2
proc_C3
.
.

Figure 2:

Pop the first command via popcmd:

proc_A | proc_B | popcmd cmdqueue.lst | proc_D

Figure 3:

cmdqueue.lst :

proc_C2
proc_C3
proc_C4
.
.
1
  • I'm guessing that this is technically a queue as I doubt new commands are added to the top of the file as well. Implementing a stack in a file would make much more sense from the end of the file. Dec 14, 2008 at 18:05

7 Answers 7

5

Ooh, that's an amusing one-liner.

Okay, here's the deal. What you want is a program that, when called, prints the first line of the file to stdout, then delete that line from the file. Sounds like a job for sed(1).

Try

proc_A | proc_B | `(head -1 cmdstack.lst; sed -i -e '1d' cmdstack.lst)` | proc_D

I'm sure that someone who had already had their coffee could change the sed program to not need the head(1) call, but that works, and shows off using a subshell ("( foo )" runs in a sub-process.)

0
4

pop-cmd.py:

#!/usr/bin/env python
import os, shlex, sys
from subprocess import call
filename = sys.argv[1]
lines = open(filename).readlines()
if lines:
    command = lines[0].rstrip()
    open(filename, "w").writelines(lines[1:])
    if command:
        sys.exit(call(shlex.split(command) + sys.argv[2:]))

Example:

proc_A | proc_B | python pop-cmd.py cmdstack.lst | proc_D
4
  • Python is a bit overkill for this.
    – Svante
    Dec 14, 2008 at 17:00
  • @Harleqin: The script is cross-platform. It has no dependencies other than the python itself. What alternative could you suggest?
    – jfs
    Dec 14, 2008 at 18:38
  • How do you prevent overwriting an update that occurs between your read and write?
    – bruceatk
    Dec 21, 2008 at 16:43
  • @bruceatk: It doesn't prevent it. It is a good question how to do it in a simple portable way.
    – jfs
    Dec 22, 2008 at 6:57
2

I assume that you are constantly appending to the file also, so rewriting the file puts you in danger of overwriting data. For this type of task I think you would be better using individual files for each queue entry, using date/time to determine order, and then as you process each file you could append the data to a log file and then delete the trigger file.

Really need more information in order to suggest a good solution. It's important to know how the file is getting updated. Is it a lot of separate processes, just one process, etc.

1

I think you would need to rewrite the file - e.g. run a command to list all lines but the first, write that to a temporary file and rename it to the original. That could be done using tail or awk or perl depending on the commands you have available.

1

If you want to treat a file like a stack, then a better approach would be to have the top of the stack at the end of the file.

Thus you can easily cut off the file at the beginning of the last line (= pop), and simply append to the file as you push.

1

You can use a little bash script; name it "popcmd":

#!/bin/bash
cmd=`head -n 1 $1`
tail -n +2 $1 > ~tmp~
mv -f ~tmp~ $1
$cmd

edit: Using sed for the middle two lines, like Charlie Martin showed, is much more elegant, of course:

#!/bin/bash
cmd=`head -n 1 $1`
sed -i -e '1d' $1
$cmd

edit: You can use this exactly as in your example usage code:

proc_A | proc_B | popcmd cmdstack.lst | proc_D
0

You can't write to the beginning of a file, so cutting out line 1 would be a lot of work (rewrite the rest of the file (which isn't actually that much work for the programmer (it's what every other answer post has written for you :) ) ) ).

I'd recommend keeping the whole thing in memory and using a classic stack rather than a file.

1
  • How do you keep the whole thing in memory and still get the pipeline described in the use case? For a small file, rewriting it really isn't all that much work anyway. Dec 14, 2008 at 17:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.