Python Problem: String Methods, Functions, Slashes, Backslashes

By Xah Lee. Date: . Last updated: .

Python problem.

just spend 2 hours and still going.

here's the short story.

so recently i switched from Cygwin to a Windows version of python. Now, Windows version takes path using win backslash, instead of Cygwin slash. This freaking broke my find/replace scripts that takes a dir level as input. Because i was counting slashes.

Ok no problem. My sloppiness. After all, my implementation wasn't portable. So, let's fix it. After a while, discovered there's the os.sep. Ok, replace "/" to os.sep, done. Then, bang, all hell went lose. Because, the backslash is used as escape in string, so any regex that manipulate path got screwed majorly. So, now you need to find a quoting mechanism. Then, screw python doc incomprehensible scattered comp-sci-R-us BNF shit. Then, screw python for “os.path” and “os” modules then string object and string functions inconsistent ball. And SCREW Guido who wants to screw change python for his idiotic OOP concept of “elegance” so that some of these are deprecated.

So after several exploration of repr(), format(), str.count(), os.path.normpath(), re.split(), len(re.search().group()) etc, after a long time, let's use re.escape(). 2 hours has passed. Also, discovered that “os.path.walk” is now deprecated, and one is supposed to use the sparkling “os.walk”. In the process of refreshing my python, the “os.path.walk” semantics is really one screwed up screw. Meanwhile, the “os.walk” went into incomprehensible OOP objects and iterators and generators and enumerators screw.

now, it's close to 3 hours. This fix is supposed to be done in 10 min. I'd have written the entire script from scratch emacs lisp in just 20 minutes if not for my waywardness.

This is Before

def process_file(dummy, current_dir, file_list):
   current_dir_level = len(re.split("/", current_dir)) - len(re.split("/", input_dir))
   cur_file_level = current_dir_level+1
   if min_level <= cur_file_level <= max_level:
      for a_file in file_list:
         if re.search(r"\.html$", a_file, re.U) and os.path.isfile(current_dir + "/" + a_file):
            replace_string_in_file(current_dir + "/" + a_file)

This is After

def process_file(dummy, current_dir, file_list):
   current_dir = os.path.normpath(current_dir)
   cur_dir_level = re.sub( "^" + re.escape(input_dir), "", current_dir).count( os.sep)
   cur_file_level = cur_dir_level + 1
   if min_level <= cur_file_level <= max_level:
      for a_file in file_list:
         if re.search(r"\.html$", a_file, re.U) and os.path.isfile(current_dir + re.escape(os.sep) + a_file):
            replace_string_in_file(current_dir + os.sep + a_file)
            # print "%d %s" % (cur_file_level, (current_dir + os.sep + a_file))

Complete File

# -*- coding: utf-8 -*-
# Python

# find & replace strings in a dir

import os, sys, shutil, re

# if this is not empty, then only these files will be processed
my_files  = []

input_dir = "c:/Users/h3/web/xahlee_org/lojban/hrefgram2/"
input_dir = "/cygdrive/c/Users/h3/web/zz"
input_dir = "c:/Users/h3/web/xahlee_org/"

min_level = 2; # files and dirs inside input_dir are level 1.
max_level = 2; # inclusive

print_no_change = False

find_replace_list = [

(
u"""<iframe style="width:100%;border:none" src="http://xahlee.org/footer.html"></iframe>""",
u"""<iframe style="width:100%;border:none" src="../footer.html"></iframe>""",
),

]

def replace_string_in_file(file_path):
   "Replaces all findStr by repStr in file file_path"
   temp_fname = file_path + "~lc~"
   backup_fname = file_path + "~bk~"

   # print "reading:", file_path
   input_file = open(file_path, "rb")
   file_content = unicode(input_file.read(), "utf-8")
   input_file.close()

   num_replaced = 0
   for a_pair in find_replace_list:
      num_replaced += file_content.count(a_pair[0])
      output_text = file_content.replace(a_pair[0], a_pair[1])
      file_content = output_text

   if num_replaced > 0:
      print "◆ ", num_replaced, " ", file_path.replace("\\", "/")
      shutil.copy2(file_path, backup_fname)
      output_file = open(file_path, "r+b")
      output_file.read() # we do this way instead of “os.rename” to preserve file creation date
      output_file.seek(0)
      output_file.write(output_text.encode("utf-8"))
      output_file.truncate()
      output_file.close()
   else:
      if print_no_change == True:
         print "no change:", file_path

def process_file(dummy, current_dir, file_list):
   current_dir = os.path.normpath(current_dir)
   cur_dir_level = re.sub( "^" + re.escape(input_dir), "", current_dir).count( os.sep)
   cur_file_level = cur_dir_level + 1
   if min_level <= cur_file_level <= max_level:
      for a_file in file_list:
         if re.search(r"\.html$", a_file, re.U) and os.path.isfile(current_dir + re.escape(os.sep) + a_file):
            replace_string_in_file(current_dir + os.sep + a_file)
            # print "%d %s" % (cur_file_level, (current_dir + os.sep + a_file))

input_dir = os.path.normpath(input_dir)

if (len(my_files) != 0):
   for my_file in my_files: replace_string_in_file(os.path.normpath(my_file) )
else:
   os.path.walk(input_dir, process_file, "dummy")

print "Done."

Note: many have written to me that Windows accept slash as path separator as well, since DOS era. What i should've done is just to consistently use slash. That would make the code simpler than dealing with backslash.

For tutorial of how to write find and replace script in python, see: