2005-04-22-2024Z

A couple of little gems for any programmers reading this drivel... I needed a way, in Python, to merge a series of arrays down to what they had in common; in other words, "intersection" of sets, that we learned sometime in high school. I couldn't find anything built-in, though there may well be something. I didn't bother looking in numpy, though I'm fairly sure it's there, because it's not on the server on which my customer is hosted, and it would be a pain to install without shell access.

So anyway, the idea is to delete elements from the "merged" lists, like so:

 results = []
 for token in tokens:
  found = findToken(token)
  if not len(results): results = found
  elif not len(found): continue
  else:
   for screen in results:
    if screen not in found: results.remove(screen)

Seems to work fine... but in reality you're losing a bunch of results. I didn't notice until I entered a token that had exactly one match, and it was not in common with any of the other search results. I got one result back, when I should have had none. What happens, when you're deleting from the same list that you're iterating over, is that the deletion shifts the whole array, while the iterator is already pointing at the next array element, which has been shifted out from under it.

So, to keep a short story short, here's the quick fix: duplicate the list before the iteration: for screen in list(results):. Now your iterator has a clean copy to do its thing with, while you delete from the master copy. I remember Larry Wall warned about this same thing in the original camel book, but I guess I was hoping for some magic in Python that wasn't in Perl.

OK, here's another. You want to just upload what you've modified in your source tree. Of course you could set up rsync, but I've had so many problems with that... anyway, I added a neat little rule to my Makefile:

upload: $(wildcard html/*) $(wildcard html_additional/*) search.cgi tokens.dat
        for file in $?; do \
         ncftpput -c $(SERVER) \
          www/$$(basename $$file) < $$file && \
         touch upload; \
        done

The $? is makefile magic for "all dependencies newer than the target", not bash's errorlevel. I'm using ncftpput with the -c option because the target site is flat, so whatever changed in my sources goes to just one directory on the server. The && makes sure it doesn't update the timestamp on 'upload' (which is just an empty file) if a transfer fails, but I think the 'for' loop might keep running anyway... haven't had a problem yet so we'll see.

Back to blog or home page

last updated 2013-01-10 20:36:31. served from tektonic.jcomeau.com