F��� Python: String Methods, Functions, Slashes and Backslashes

Xah Lee, 2012-04-08, 2012-04-11

f�ck Python.

just f���ing spend 2 hours and still going.

here's the short story.

so recently i switched to a Windows version of python. Now, Windows version takes path using win backslash, instead of cygwin slash. This f�cking broke my find/replace scripts that takes a dir level as input. Because i was counting slashes.

Ok no problem. My sloppiness. After all, my implementation wasn't portable. So, let's fix it. After a while, discovered there's the os.sep. Ok, replace "/" to os.sep, done. Then, bang, all hell went lose. Because, the backslash is used as escape in string, so any regex that manipulate path got f���ed majorly. So, now you need to find a quoting mechanism. Then, f��� python doc incomprehensible scattered comp-sci-R-us BNF shit. Then, f��� python for “os.path” and “os” modules then string object and string functions inconsistent ball. And F�CK Guido who wants to f��� change python for his idiotic OOP concept of “elegance” so that some of these are deprecated.

So after several exploration of repr(), format(), ‹str›.count(), os.path.normpath(), re.split(), len(re.search().group()) etc, after a long time, let's use re.escape(). 2 hours has passed. Also, discovered that “os.path.walk” is now deprecated, and one is supposed to use the sparkling “os.walk”. In the process of refreshing my python, the “os.path.walk” semantics is really one f�cked up f���. Meanwhile, the “os.walk” went into incomprehensible OOP objects ＆ iterators ＆ generators ＆ enumerators f���.

now, it's close to 3 hours. This fix is supposed to be done in 10 min. I'd have done it in elisp in just 10 minutes if not for my waywardness.

This is Before

def process_file(dummy, current_dir, file_list):
   current_dir_level = len(re.split("/", current_dir)) - len(re.split("/", input_dir))
   cur_file_level = current_dir_level+1
   if min_level <= cur_file_level <= max_level:
      for a_file in file_list:
         if re.search(r"\.html$", a_file, re.U) and os.path.isfile(current_dir + "/" + a_file):
            replace_string_in_file(current_dir + "/" + a_file)

This is After

def process_file(dummy, current_dir, file_list):
   current_dir = os.path.normpath(current_dir)
   cur_dir_level = re.sub( "^" + re.escape(input_dir), "", current_dir).count( os.sep)
   cur_file_level = cur_dir_level + 1
   if min_level <= cur_file_level <= max_level:
      for a_file in file_list:
         if re.search(r"\.html$", a_file, re.U) and os.path.isfile(current_dir + re.escape(os.sep) + a_file):
            replace_string_in_file(current_dir + os.sep + a_file)
            # print "%d %s" % (cur_file_level, (current_dir + os.sep + a_file))

Complete File

# -*- coding: utf-8 -*-
# Python

# find & replace strings in a dir

import os, sys, shutil, re

# if this is not empty, then only these files will be processed
my_files  = []

input_dir = "c:/Users/h3/web/xahlee_org/lojban/hrefgram2/"
input_dir = "/cygdrive/c/Users/h3/web/zz"
input_dir = "c:/Users/h3/web/xahlee_org/"

min_level = 2; # files and dirs inside input_dir are level 1.
max_level = 2; # inclusive

print_no_change = False

find_replace_list = [

(
u"""<iframe style="width:100%;border:none" src="http://xahlee.org/footer.html"></iframe>""", 
u"""<iframe style="width:100%;border:none" src="../footer.html"></iframe>""", 
),

]

def replace_string_in_file(file_path):
   "Replaces all findStr by repStr in file file_path"
   temp_fname = file_path + "~lc~"
   backup_fname = file_path + "~bk~"

   # print "reading:", file_path
   input_file = open(file_path, "rb")
   file_content = unicode(input_file.read(), "utf-8")
   input_file.close()

   num_replaced = 0
   for a_pair in find_replace_list:
      num_replaced += file_content.count(a_pair[0])
      output_text = file_content.replace(a_pair[0], a_pair[1])
      file_content = output_text

   if num_replaced > 0:
      print "◆ ", num_replaced, " ", file_path.replace("\\", "/")
      shutil.copy2(file_path, backup_fname)
      output_file = open(file_path, "r+b")
      output_file.read() # we do this way instead of “os.rename” to preserve file creation date
      output_file.seek(0)
      output_file.write(output_text.encode("utf-8"))
      output_file.truncate()
      output_file.close()
   else:
      if print_no_change == True:
         print "no change:", file_path

def process_file(dummy, current_dir, file_list):
   current_dir = os.path.normpath(current_dir)
   cur_dir_level = re.sub( "^" + re.escape(input_dir), "", current_dir).count( os.sep)
   cur_file_level = cur_dir_level + 1
   if min_level <= cur_file_level <= max_level:
      for a_file in file_list:
         if re.search(r"\.html$", a_file, re.U) and os.path.isfile(current_dir + re.escape(os.sep) + a_file):
            replace_string_in_file(current_dir + os.sep + a_file)
            # print "%d %s" % (cur_file_level, (current_dir + os.sep + a_file))

input_dir = os.path.normpath(input_dir)

if (len(my_files) != 0):
   for my_file in my_files: replace_string_in_file(os.path.normpath(my_file) )
else:
   os.path.walk(input_dir, process_file, "dummy")

print "Done."

Note: many have written to me that Windows accept slash as path separator as well, since DOS era. What i should've done is just to consistently use slash. That would make the code simpler than dealing with backslash.

For tutorial of how to write find ＆ replace script in python, see:

• Python: Find/Replace by Regex Text Pattern
• Perl: Find/Replace on Multiple Files