This is probably the last part in series of post dealing with Beautiful Soup. At least for a while. Good news is, I found the bug that caused observed weird behavior and it was partly self-inflicted.

My problem appeared, because I was using variables that resembled html tags (e.g. imgs, ul…) and didn’t put space after < in statements as good practice dictates. So I had statements like:

if (i<imgs.length)


Beautiful Soup relies on sgmllib.py for processing of documents and it’s sgmllib that causes havoc by processing CDATA (found in script and style blocks), when it shouldn’t. But the problem is not really with sgmllib, which has a literal switch to turn processing off. It’s just that Beautiful Soup doesn’t use it.

So fix is really adding one line

self.literal = 1

after if statement on line 835. Or downloading fixed version .

Reading source I also realized another potential pitfall. If you use javascript to inject javascript with document.write method, you’ll probably confuse sgmllib (and hence Beautiful Soup). When processing CDATA, it doesn’t try to understand it, it just copies data until it finds an endtag (</script> in javascript’s case). But if you’re doing that, you have yourself to blame.

BeautifulSoup 2.1.1 is out , which fixes my bug and several others. So you better fetch that version.

It seems I praised Beautiful Soup too quickly. I was playing with it yesterday and discovered it mangles Javascript included in file. I can’t say it does this every time, but it’s fairly obvious if you have few kilobytes of code stuck in file. There you have it, another argument why Javascript and CSS should be stored in external files.

It’s not an option for me though, so I’ll have to work around it.

It’s still a great module, as long as you can work within its limitations, but it’s not panacea. At least not yet.

If I find time, I’ll try to fix the problem, since I really dislike idea of processing files using usual methods (regular expressions & co.).

We’ve been working with a talented group of designers, whose work I admired for a long time, on a design for our new service. Initial concept was exactly what we were looking for and the only changes that needed to be made, were consequences of our bad choices earlier on. In fact, everything was great except one thing.

We didn’t like proposed color schemes.

Every time I work with a designer, I’m afraid I’ll have to say that. It’s probably the most common complaint they hear and I really hate to join this particular crowd. Sure, they’ve always been graceful (at least the good designers), but I’m certain they wouldn’t mind spiking a voodoo doll of me, given a chance.

That got me thinking, is there a specific, but very common complaint for every trade out there? I couldn’t recall one for programmers, but that doesn’t mean there isn’t one. I certainly would love to hear your experiences.