Customer validates at 5 - 8.5 million records per hour

Support information relating to the CASS Certification process of AccuZIP6. More information can be found in the NCOALink forum.

Moderators: Chrissy, Steve, Terry, Donna, Dave, Eric, MartyH, Kristen, Geoff, Vince

Post Reply
User avatar
Eric
Moderator
Posts: 1909
Joined: Tue Sep 19, 2006 3:29 pm
Contact:

Customer validates at 5 - 8.5 million records per hour

Post by Eric » Fri Nov 12, 2010 8:48 am

From a customer's email:
I moved Accuzip into a virtual machine so that I could move the VM (and thus Accuzip) from server to server as required. That works very well, in fact I built a new server with tons of cores and RAM and moved the VM to that and it (Accuzip) ran significantly slower, due to lower core clock AFAICT. So I just created a little server specifically for the VM and Accuzip and it is zipping along now.

I am consistently getting well over 5 million records / hour and sometimes getting around 8.5 million records / hour by the end of the validating records process and I thought I'd share what I found when working to get my speeds up.

You guys told me long ago to sort by Zip5/Zip4 (Optimize Database - Order Database for Optimum Performance) which got my speeds up from around 2 million to something like 3.5 million records / hour.

The next thing I did was to give the VM a 30G Solid State Disk as a work disk. I placed all of the USPS database files out there. I also made the SSD the disk that held all of the input / output directories (where you build your db processing files). Just doing that allowed me to get about 5.3 million records / hour.

Finally I pre-filtered all of the records sent to Accuzip to get rid of the "Move - no forwarding address", ANK and otherwise invalid addresses. Essentially after the first Accuzip pass (I Accuzip monthly) I only send Accuzip addresses that you told me were valid addresses on the last pass.
Doing that bumped my speeds up, sometimes though not always dramatically.
Chat with an AccuZIP6 technician weekdays from 7-4 PST with AccuZIP6 Live Online Support

Post Reply