August 1st, 2009 Need Website Crawler Project2i need a crawler for one website.The website require authentication, the search is java based as far as i can see.The script is necessary because their search engine is sub par and i need better control over the search on that website.the site has around 50000 items, with 4 info..title , price , category and type.i need to be able to retrieve those items, store them in a mysql database and do searches on them. but price, category type using keywor