6

I need to update a text file whenever my IP address changes, and then run a few commands from the shell afterwards.

  1. Create variable LASTKNOWN = "212.171.135.53" This is the ip address we have while writing this script.

  2. Get the current IP address. It will change on a daily basis.

  3. Create variable CURRENT for the new IP.

  4. Compare (as strings) CURRENT to LASTKNOWN

  5. If they are the same, exit()

  6. If they differ,

    A. "Copy" the old config file (/etc/ipf.conf) containing LASTKNOWN IP address into /tmp B. Replace LASTKNOWN with CURRENT in the /tmp/ipf.conf file.
    C. Using subprocess "mv /tmp/ipf.conf /etc/ipf.conf"
    D. Using subprocess execute, "ipf -Fa -f /etc/ipf.conf"
    E. Using subprocess execute, "ipnat -CF -f /etc/ipnat.conf"

  7. exit()

I know how to do steps 1 through 6. I fall down on the "file editing" part, A -> C. I can't tell what module to use or whether I should be editing the file in place. There are so many ways to do this, I can't decide on the best approach. I guess I want the most conservative one.

I know how to use subprocess, so you don't need to comment on that.

I don't want to replace entire lines; just a specific dotted quad.

Thanks!

3
  • "There are so many ways to do this, I can't decide on the best approach." Since you're looking for a best way, and not just any way, perhaps it would be better to say a little more about the constraints: Does the answer have to be very efficient (memory or running time) etc?
    – Edan Maor
    Commented Oct 17, 2009 at 17:39
  • No, it need not be super efficient. The file I'm editing is small. But perhaps in the future I'll need to do this type of operation on a 20MB text file with 15,000 lines. I'm guessing you can tell me a more efficient way for such.
    – mr.zog
    Commented Oct 17, 2009 at 20:02
  • mv command (and its python equivalent shutil.move()) is not guaranteed to be atomic. Better place new version in the same directory (the simplest way to place on the same logical drive) and then use os.rename() call to do atomic rename. Commented Oct 18, 2009 at 5:51

6 Answers 6

27

Another way to simply edit files in place is to use the fileinput module:

import fileinput, sys
for line in fileinput.input(["test.txt"], inplace=True):
    line = line.replace("car", "truck")
    # sys.stdout is redirected to the file
    sys.stdout.write(line)
1
  • 3
    The square brackets [ & ] around "text.txt" aren't required.
    – martineau
    Commented May 4, 2011 at 17:59
8

Replace LASTKNOWN by CURRENT in /etc/ipf.conf

Replace all at once

filename = "/etc/ipf.conf"
text = open(filename).read()
open(filename, "w").write(text.replace(LASTKNOWN, CURRENT))

Replace line by line

from __future__ import with_statement
from contextlib import nested

in_filename, outfilename = "/etc/ipf.conf", "/tmp/ipf.conf"
with nested(open(in_filename), open(outfilename, "w")) as in_, out:
     for line in in_:
         out.write(line.replace(LASTKNOWN, CURRENT))
os.rename(outfilename, in_filename)

Note: "/tmp/ipf.conf" should be replaced by tempfile.NamedTemporaryFile() or similar
Note: the code is not tested.

3
  • Yeah. Thanks. ./pyifchk-0.py:23: Warning: 'with' will become a reserved keyword in Python 2.6 File "./pyifchk-0.py", line 23 with nested(open(in_filename), open(outfilename, "w")) as in_, out: ^ SyntaxError: invalid syntax
    – mr.zog
    Commented Oct 18, 2009 at 15:05
  • The ^ points at where "nested" and "(in_filename)" are joined. I tried it with a space between them with nested (in_filename), open(outfilename, "w")) as in_, out: and got the same error.
    – mr.zog
    Commented Oct 18, 2009 at 15:09
  • @javadong: on Python 2.5 you need from __future__ import with_statement.
    – jfs
    Commented Oct 19, 2009 at 16:50
4

fileinput module has very ugly API, I find beautiful module for this task - in_place, example for Python 3:

import in_place

with in_place.InPlace('data.txt') as file:
    for line in file:
        line = line.replace('test', 'testZ')
        file.write(line)

main difference from fileinput:

  • Instead of hijacking sys.stdout, a new filehandle is returned for writing.
  • The filehandle supports all of the standard I/O methods, not just readline().

for example - fileinput can line by line editing only, in_pace allow read whole file to memory (if it not big) and modify it.

2
  • Does the in_place read the file only from the current directory? I tried using to read a file in a server (using telenetlib) but I'm getting an error that the particular file is not present in the current(python) directory.
    – Ganesh
    Commented Mar 29, 2020 at 7:14
  • Not, it can read file from path, for example with in_place.InPlace('/home/worldmind/back.svg') as file: Commented Apr 9, 2020 at 12:57
2

You're trying to "atomically" update the contents of a file, and there have been many delightful flame wars on the subject. But the general pattern is:

1) Write the new file to a temp file, and make sure you flush and close.

2) Use your operating system's facilities to atomically rename the temp file to the old file.

Now, you simply can't atomically rename a file in Windows, but it sounds like you're on a unix-like system anyway. You atomically rename using os.rename().

8
  • 1
    +1: "update in place" is a crazy thing to try. It's simpler to create new and rename.
    – S.Lott
    Commented Oct 17, 2009 at 17:59
  • Yes, this script will run on BSD. Yeah, I have never been able to figure out how to "edit files in place" using Python. It may be possible but it doesn't seem easy.
    – mr.zog
    Commented Oct 17, 2009 at 18:39
  • So... does my answer provide what you needed? Commented Oct 17, 2009 at 19:03
  • Nope. "Write the new file to a temp file" using what? Does the new file already have the new IP address in it or not? Isn't it more like, "Copy the existing file into /tmp and perform the string substitution on it there then move it into /etc"?
    – mr.zog
    Commented Oct 17, 2009 at 21:37
  • There are three steps at least involved ;)
    – mr.zog
    Commented Oct 17, 2009 at 21:38
1

Probably the simplest way would be to open a file using f=open(filename, mode). Then, read all the lines using f.readlines() (this will return a list of strings representing the lines of the program).

You can then search these strings to find the address and replace it with the new one (using standard string replacing, regular expressions, or whatever you want).

At the end, you can write the lines back to the file using f.writelines(lines), which conveniently takes back a list of lines.

NOTE: This is not an efficient way to do this, it's just the easiest. Please

Example code:

f = open(filename, "r")
lines = f.readlines()

# Assume that change_ip is a function that takes a string and returns a new one with the ip changed): example below
ret_lines = [change_ip(lines) for line in lines]
new_file = open(new_filename, "w")
new_file.writelines(lines)

def change_ip(str):
   ''' Gets a string, returns a new string where the ip is changed '''
   # Add implementation, something like: return str.replace(old_ip, new_ip) or something similair.
10
  • Thanks. I'll try this. It's one of the paths I was starting down.
    – mr.zog
    Commented Oct 17, 2009 at 18:40
  • If the file is one that is read and monitored by other processes, this approach may result in the other processes seeing an inconsistent state, perhaps with nonsensical contents. If that's a concern, write to a temp, sync that file, then move it into the right place.
    – divegeek
    Commented Oct 17, 2009 at 18:46
  • Not sure what you mean by "sync" in this context. Sync to what from what?
    – mr.zog
    Commented Oct 17, 2009 at 20:01
  • Also, "change_ip" looks like a "verb" but I think you intend for me to put the string for the NEW IP address there. Remember, I already have the "known" and "current" IP addresses stored as variables of type str()
    – mr.zog
    Commented Oct 17, 2009 at 20:34
  • LASTKNOWN = '175.48.234.168' import socket import fileinput import subprocess import string import re CURRENT = socket.getaddrinfo(socket.gethostname(), None)[0][4][0] # the above get my actual current IP if CURRENT == LASTKNOWN: print 'Nevermind.' subprocess.sys.exit() else:
    – mr.zog
    Commented Oct 17, 2009 at 20:35
0

After checking the comments and the bit of code that you put on pastebin, here is a working solution. To begin with, the file /tmp/iiiipf.conf contains:

Simply a test file 175.48.204.168

And two times 175.48.204.168 on this line 175.48.204.168

Done.

After running the code, the file /tmp/iiiipf.conf contains:

Simply a test file 10.73.144.112

And two times 10.73.144.112 on this line 10.73.144.112

Done.

And here is the tested working code with my stuff merged into your pastebin code:

import socket
import fileinput
import subprocess
import string
import re

CURRENT = socket.getaddrinfo(socket.gethostname(), None)[0][4][0]
LASTKNOWN = '175.48.204.168'

if CURRENT == LASTKNOWN:
    print 'Nevermind.'
    subprocess.sys.exit()

else:

    cf = open("/tmp/iiiipf.conf", "r")
    lns = cf.readlines()
    # close it so that we can open for writing later
    cf.close()

    # assumes LASTKNOWN and CURRENT are strings with dotted notation IP addresses
    lns = "".join(lns)
    lns = re.sub(LASTKNOWN, CURRENT, lns)  # This replaces all occurences of LASTKNOWN with CURRENT

    cf = open("/tmp/iiiipf.conf", "w")
    cf.write(lns)
    cf.close()

This bit of code will do what you need even if the IP address is used several times within the configuration file. It will also change it in comment lines.

This method does not require copying to /tmp and uses one less subprocess call when restarting the firewall and NAT.

16
  • Thanks, this looks like it will work. However ipf and ipnat will still need to be "restarted." Ipfilter can't see changes to its config file by itself. It needs to have the old rules flushed and the new rules installed. It also makes sense to "refresh" ipnat as well. I have used ipf/ipnat since 1999 and this is the only reliable way to do it.
    – mr.zog
    Commented Oct 17, 2009 at 20:20
  • When I tried to use the above I got: root@spork# ./pyifchk.py Traceback (most recent call last): File "./pyifchk.py", line 22, in <module> lns = re.sub(LASTKNOWN,CURRENT,lns) File "/usr/local/lib/python2.5/re.py", line 150, in sub return _compile(pattern, 0).sub(repl, string, count) TypeError: expected string or buffer
    – mr.zog
    Commented Oct 17, 2009 at 20:29
  • That's why I said "code fragment". You would need to insert this into your script which does the other stuff, including assigning a string value to CURRENT and LASTKNOWN. Or if you have different variable names for the strings holding the IP address, then change the names in my example. I'm relying on you to know where and when to call subprocess to stop and start the firewall and NAT because I'm not that familiar with Solaris details. Commented Oct 17, 2009 at 20:51
  • Again, not asking for help with BSD or Solaris or ipfilter. lns = cf.readlines() returns a LIST, not a string.
    – mr.zog
    Commented Oct 17, 2009 at 21:50
  • No, should be: lns = (re.sub(LASTKNOWN, CURRENT, l) for l in lns). Calling readline() just gets the first line in the file; that generator expression does the substitution on every line and returns the result as an iterable, which write will iterate through. Commented Oct 18, 2009 at 0:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.