Harvestman

"Welcome to the project page of HarvestMan, a web-crawler cum web-search utility, written in the python programming language. HarvestMan is a full-featured, multithreaded web-crawler written in Python.
HarvestMan supports as much as 45 customization options as of the current stable version. "

Patchable??