Filling in missing postcodes/zip codes
Below is a recipe for filling in missing postcodes - useful for sites who have started inputting postcodes themselves and now have access to a postcode xml file that can be loaded by the dataload utility. [Note that I have a Pentaho Kettle transform that will take a data from a csv file and generate the xml file and have use this to generate xml files for South Africa and Hong Kong.] The recipe is as follows: (this uses the Australian postcode file in the release package but will work with any xml file for another country)
First edit postcodesAU.xml and save it as say postcodesX.xml – for the lines
Add ‘X’ (must be uppercase because the codes have to be uppercase) prior to each of the codes (NOT the ids) and names - so you will end up creating a new country Xaustralia with states Xvictoria, Xnew South Wales, etc
Now run dataload to load postcodesX.xml – it will pour out errors – one for every existing postcode (with lots of lines for each error). However, it will generate the missing ones – albeit for the X states.
You should be able to use Administration|Lookups|State to examine the X states and see that they just have the missing postcodes
We now need to use the lookup replace facility. First replace Xaustralia by Australia using Administrator|Lookups|Country (and tick the box to delete Xaustralia at the same time)
Now use Administration|Lookups|State to replace Xnew South Wales by New South Wales (and tick the box to delete Xnew South Wales at the same time). Repeat for the other X states.
Regards, Tim G
Re: Filling in missing postcodes/zip codes
Hi,
We tried this, but all we got was postcodes loaded without a target. And no new Xstates. So I was unable to follow the rest of the instructions and do the lookup replacement.
We replaced code="NSW" with code="XNSW" and name-"New South Wales" with "Name="XNew South Wales".
Is this an issue with propercasing (i.e. would we have a different result if we had done Xnsw and Xnew South Wales?
Thanks,
Adrian
Re: Filling in missing postcodes/zip codes
So I am not sure but I think Tim G might have missed or confused a step
Please note I have not followed these instructions myself but reading his howto..it might need more explanation.
What your doing is modifying the PostCodesAU file to create a new virtual country called XAustralia
PLEASE NOTE THIS WILL NOT WORK IF YOU HAVE ALREADY LOADED THE PostCodesAU.xml at some point in the past
The Loader uses the "id" field to work out if it has ever loaded these items -
If an ID already exists in the database ETLlog then it is skipped - these are the errors TimG expected. Because you have already loaded the PostCodesAU it will see the IDs for the new XStates and XCountry match and wont load them. In this case the ID: CTY1 probably exists and thus the new XAustralia country wont be loaded.
That being said it should link the missing postcodes in the AU file to the existing country and state anyway and you shouldnt need the XState stuff.
The point of creating the XStuff was to avoid a conflict with the states already in the system during the file loading. I wouldnt bother manually creating them as they still wont exist in the ETL Log to allow relationship matching.
Re: Filling in missing postcodes/zip codes
Adrian - did you also adjust the country so that the new x states link to xaustralia ?
Regards, Tim G
Re: Filling in missing postcodes/zip codes
Can I add that if you have manually added Postcodes its is likely that you will get double ups which will need manual cleaning anyway.
Re: Filling in missing postcodes/zip codes
Hi,
I modified only the file only as shown below.
- I do have double ups now of some manually entered post codes.
- I also have 801 pages of postcodes without a link to a state or country.
I am not clear how to proceed from here to achieve our aim.
thanks,
adrian
Re: Filling in missing postcodes/zip codes
Adrian - I just logged into your system.
1) there is no Country record - and your states are thus not linked to any country.
2) after running the dataload I would have thought that you should have ended up with the xAustralia country and x states linked to it - did you?
3) "801 pages of postcodes" - I would have expected 801 pages of error output from the dataload, but not 801 pages of postcodes.
4) because of 1 above, I would have thought that there was a problem when you attempted to replace xAustralia with Australia - do you have one?
Regards, Tim G
Re: Filling in missing postcodes/zip codes
Adrian - I just logged into your system.
1) there is no Country record - and your states are thus not linked to any country.
2) after running the dataload I would have thought that you should have ended up with the xAustralia country and x states linked to it - did you?
3) "801 pages of postcodes" - I would have expected 801 pages of error output from the dataload, but not 801 pages of postcodes.
4) because of 1 above, I would have thought that there was a problem when you attempted to replace xAustralia with Australia - do you have one?
Regards, Tim G
Re: Filling in missing postcodes/zip codes
Hi Tim,
1) Yes - we noticed after we did the dataload that we did not have a country.
2) We did not not end up with and XAustralia or XStates.
3) We ended up with 810 pages of postcodes!
4) Not sure what the question is. We didn't started with no entries in the country records and we ended with nothing in the country records.
PS> TimG - Not an urgency so please don't feel pressure to respond!