Using Visual Web Ripper To Release Data Behind Captcha

For a web page harvester, CAPTCHA technology creates an obstacle that slows your progress

. CAPTCHA is the human detection software that requires web site visitors to enter a correct string of words, letters, or numbers stretched, squished, and often obscured by dots. While this makes web page harvesting rather difficult or impossible, for Visual Web Ripper, it simply adds an extra step. There are two types of CAPTCHA protection: semi-automatic and full-automatic. The semi-automatic version requires the manual decoding of CAPTCHA images while running a project. Full-automatic data scraping is costly and requires an account with a third-party CAPTCHA recognition service, but there's no manual decoding.

CAPTCHA Data Extraction

The following is required for a successful CAPTCHA processing. The text in bold is required only for the Full-Automatic Data Extraction:

1.Add a content element that selects the CAPTCHA image. Then use theMisc options tab to uncheck the Save content option.

2.Add aFormField element that selects the CAPTCHA input field. Then use theAdvanced Options tab to select the image element as a CAPTCHA element.

3.Use the Advanced Options tab to add a Decode CAPTCHA script to the FormField element that selects the CAPTCHA input field.

4.Add a FormSubmit template that submits the CAPTCHA form. Youmay need to set the Misc option Optional template if the CAPTCHA form is not always displayed.

Upon encountering a Semi-Automatic CAPTCHA element, Visual Web Ripper displays the image and requests the code.

Full-Automatic Data Extraction

Register for an account with a third party recognition service (http://www.decaptcher.com uses a compatible API and a default DLL library, so it's highly recommended). This particular service charges $2 per 1000 decodes. No fees will be assessed by Sequentum.

Add a decode CAPTCHA script to a FormField element by clicking theDecode CAPTCHA scriptoptionbutton in Advanced Options.This is used to call the recognition service. It will receive the image as an input parameter and return the decoded CAPTCHA as a string

The script editor should open with the default code.

If you used the http://www.decaptcher.com service, input your login name and password.

Note: The default decode CAPTCHA script uses a DLL file provided by http://www.decaptcher.com. Some virus programs will say this is a bad file and is therefore no longer natively included in the Visual Web Ripperinstaller package. Download the file separately and place it in the Visual Web Ripper installation folder. You can download the file from: www.visualwebripper.com/download/decaptcher.zip

If you are using a service other than Decaptcher, you may manually write your own script in the editor. A working knowledge of OOP, especially and including C# and VB.NET is EXTREMELY useful.

Note: When writing the code, it is important not to change the name of the method DecodeCaptcha.

Overcoming a CAPTCHA dialog box is tricky for any web page harvesting project. In many cases, these sites are an insurmountable or unprofitable challenge. Using Visual Web Ripper with the above instructions allows harvesting of the data you seek.

by: Tracy Morgan

POST COMMENT

Smart Tricks To Increase Weblog Web Traffic What Makes Good Web Design Things To Keep In Mind When You Opt For Cheap Web Hosting The Many Ways To Use A Web Conferencing Recording Tips When Looking For A Web Design Firm In Phoenix Web Design Sydney Web Development Content Writing Tips: Understanding 6 Types Of Web Content Readers Benefit Of Doing Php Live Project Training With Expert Web Development Company Off Page Optimization - Reaching Clients Through Appropriate Web Content How To Find A Reputed Professional Web Design Company In India? How To Find Out A Cheap Web Hosting Services Provider? Floortrus Bespoke Open Web Floor Solution Manchester Web Design