Makes slicing and dicing Excel spreadsheets a breeze (without COM). It works on Windows, Solaris, Linux and Mac OSX.

Speaking of spreadsheets, that Joel guy’s post was interesting. Maybe I should take back my comment about “identical statues” insanity.

As far as linking the words list and the parts of speech list, yeah. It’s a good idea. Originally, I was thinking of making an object to represent vocabulary items and giving it “word”, “part of speech”, “bookworm level”, “lesson taught” and “chinese” components. I may still do that at some point. This time, though I really just needed to get a few hundred more words added to my list.

FYI, the list will almost certainly never surpass a few thousand entries, so run-time concerns are irrelevant compared to Mark’s-time concerns.

from itertools import izip
l2set = set(l2words)
b2dict = [(word, pos) for (word, pos) in izip(b2words, b2pos) if word not in l2set]

If you’re trying to remove duplicates in Excel, I think there’s a way to put the whole thing into a Pivot table, and remove duplicates that way. There’s a bit of a learning curve with pivot tables, but they can be very powerful if you want to keep it all in excel only.

Concerning the code example, this algorithm looks simple, but I think there is a problem if the number of words get large. The algorithm operates in at best O(n^2) run time — that is, the run time varies with the square of the number of words in the list (or more precisely, the product of the number of elements in the two lists). It may even run in O(n^3) time if the del operator takes O(n) time (see more below).

What I would recommend instead is to put all the elements of l2words into a hash table ( O(n) operation) so that the ‘in’ operator executes in (roughly) constant time instead of O(n) time. I think this is what you’d want, roughly:

<pre>
l2dict = dict([(word, 1) for word in l2words])
for i in range(len(b2words)):
<t />    if b2words[i] in l2dict:
<t /><t />        del b2words[i]
<t /><t />        del b2pos[i]
</pre>

(I’m a Python hack, so there is undoubtedly a more elegant way to create a dictionary (i.e. hash table) from a list.)

‘del’ is probably an O(n) operation on b2words (making it O(n^2) overall). It depends on whether b2words is implemented internally as a linked list or a fixed-position array. In either case, either ‘del’ is slow, or ‘[i]‘ is slow. To be on the safe side, it’s probably better to create a new list instead of deleting from the existing list:

<pre>
l2dict = dict([(word, 42) for word in l2words])
b2pos = [b2pos[i] for i, word in enumerate(b2words) if word not in l2dict];
b2words = [word for word in b2words if word not in l2dict];
</pre>

I suspect that you may have meant for b2pos and b2words to be more closely linked. Without seeing the rest of the code, you may have wanted this to be a dictionary of pairs. Maybe this code is the right approach:

<pre>
l2dict = dict([(word, 91) for word in l2words])
b2dict = dict([(word, b2pos[i]) for i, word in enumerate(b2words) if word not in l2dict])
</pre>

I haven’t run any of this code, so I’m expecting at least a couple errors.

A C programmer will typically tackle these types of problems using lots of ‘for (i=0; i<n; i++)’ loops. In using more modern languages, I’ve decided that it’s almost always some kind of mistake when you see this. Usually, there’s some other construct based on lists, arrays, or hash tables that executes faster and more gracefully. Python has a cool construct called the ‘list comprehension’ that makes it easy to do list manipulations.

if b2words[i] in l2words:

will just traverse l2words to see if b2words[i] matches anything in l2words? Thanks for the tip!