<abstract>
<para>A plpgsql based geocoder written to work with the <ulink url="http://www.census.gov/geo/www/tiger/">TIGER (Topologically Integrated Geographic Encoding and Referencing system ) / Line and Master Address database export</ulink> released by the US Census Bureau. In prior versions the TIGER files were
released in ASCII format. The older geocoder used to work with that format is in <varname>extras/tiger_geocoder/tiger_2006andbefore</varname>. </para>
- <para>There are four components to the geocoder: the data loader functions, the address normalizer, the address geocoder, and the reverse geocoder. The latest version updated to use the TIGER 2010 census data is located in the <varname>extras/tiger_geocoder/tiger_2010</varname> folder.</para>
+ <para>There are four components to the geocoder: the data loader functions, the address normalizer, the address geocoder, and the reverse geocoder. The latest version updated to use the TIGER 2011 census data is located in the <varname>extras/tiger_geocoder/tiger_2011</varname> folder.</para>
<para>Although it is designed specifically for the US, a lot of the concepts and functions are applicable and can be adapted to work with other country address and road networks.</para>
<para>The script builds a schema called <varname>tiger</varname> to house all the tiger related functions, reusable lookup data such as road type prefixes, suffixes, states, various control tables for managing data load, and skeleton base tables from which all the tiger loaded tables inherit from.</para>
<para>Another schema called <varname>tiger_data</varname> is also created which houses all the census data for each state that the loader downloads from Census site and loads into the database. In the current model, each set of state tables is
- prefixed with the state code e.g ma_addr, ca_edges etc with constraints to enforce only that state data. Each of these tables inherits from the base addr, faces, edges, etc located in the tiger schema. </para>
+ prefixed with the state code e.g <varname>ma_addr</varname>, <varname>ma_edges</varname> etc with constraints to enforce only that state data. Each of these tables inherits from the tables <varname>addr</varname>, <varname>faces</varname>, <varname>edges</varname>, etc located in the <varname>tiger schema</varname>. </para>
<para>All the geocode functions only reference the base tables, so there is no requirement that the data schema be called <varname>tiger_data</varname> or that data can't be further partitioned into other schemas -- e.g a different schema
for each state, as long as all the tables inherit from the tables in the <varname>tiger</varname> schema.</para>
<para>
<note><para>
-If you are using a prerelease version of PostGIS 2.0.0 tiger geocoder,
+If you are using tiger geocoder (tiger_2010),
you can upgrade the scripts using the accompanying upgrade_geocoder.bat
-/ .sh scripts in tiger_2010. We'll be refining the upgrade scripts
-until release.
+/ .sh scripts in tiger_2011. One major change between <varname>tiger_2010</varname> and <varname>tiger_2011</varname> is that the county and <varname>county</varname> and <varname>state</varname> tables are no longer broken out by state. We'll be refining the upgrade scripts
+until release. If you have data from tiger_2010 and want replace with tiger_2011 refer to <xref linkend="upgrade_tiger_geocoder" />
</para></note>
</para>
<para>Design:</para>
</refsection>
</refentry>
+ <refentry id="Drop_Nation_Tables_Generate_Script">
+ <refnamediv>
+ <refname>Drop_Nation_Tables_Generate_Script</refname>
+
+ <refpurpose>Generates a script that drops all tables in the specified schema that start with <varname>county_all</varname>, <varname>state_all</varname> or stae code followed by <varname>county</varname> or <varname>state</varname>.</refpurpose>
+ </refnamediv>
+
+ <refsynopsisdiv>
+ <funcsynopsis>
+ <funcprototype>
+ <funcdef>text <function>Drop_State_Tables_Generate_Script</function></funcdef>
+ <paramdef><type>text </type> <parameter>param_state</parameter></paramdef>
+ <paramdef choice='opt'><type>text </type> <parameter>param_schema=tiger_data</parameter></paramdef>
+ </funcprototype>
+ </funcsynopsis>
+ </refsynopsisdiv>
+
+ <refsection>
+ <title>Description</title>
+
+ <para>Generates a script that drops all tables in the specified schema that start with <varname>county_all</varname>, <varname>state_all</varname> or stae code followed by <varname>county</varname> or <varname>state</varname>. This is needed if you are upgrading from <varname>tiger_2010</varname> to <varname>tiger_2011</varname> data.</para>
+
+
+ <para>Availability: 2.1.0</para>
+
+
+ </refsection>
+
+
+ <refsection>
+ <title>Examples</title>
+ <programlisting>SELECT drop_nation_tables_generate_script();
+DROP TABLE tiger_data.county_all;
+DROP TABLE tiger_data.county_all_lookup;
+DROP TABLE tiger_data.state_all;
+DROP TABLE tiger_data.ma_county;
+DROP TABLE tiger_data.ma_state;</programlisting>
+
+ </refsection>
+
+ <!-- Optionally add a "See Also" section -->
+ <refsection>
+ <title>See Also</title>
+ <para><xref linkend="Loader_Generate_Nation_Script"/></para>
+ </refsection>
+ </refentry>
+
<refentry id="Drop_State_Tables_Generate_Script">
<refnamediv>
<refname>Drop_State_Tables_Generate_Script</refname>
</refsection>
+ <!-- Optionally add a "See Also" section -->
+ <refsection>
+ <title>See Also</title>
+ <para><xref linkend="Loader_Generate_Script" /></para>
+ </refsection>
+ </refentry>
+
+ <refentry id="Loader_Generate_Script">
+ <refnamediv>
+ <refname>Loader_Generate_Script</refname>
+
+ <refpurpose>Generates a shell script for the specified platform for the specified states that will download Tiger data, stage and load into <varname>tiger_data</varname> schema. Each state script is returned as a separate record. Latest version supports Tiger 2010 structural changes and also loads census tract, block groups, and blocks tables.</refpurpose>
+ </refnamediv>
+
+ <refsynopsisdiv>
+ <funcsynopsis>
+ <funcprototype>
+ <funcdef>setof text <function>loader_generate_script</function></funcdef>
+ <paramdef><type>text[]</type> <parameter>param_states</parameter></paramdef>
+ <paramdef><type>text</type> <parameter>os</parameter></paramdef>
+ </funcprototype>
+ </funcsynopsis>
+ </refsynopsisdiv>
+
+ <refsection>
+ <title>Description</title>
+
+ <para>Generates a shell script for the specified platform for the specified states that will download Tiger data, stage and load into <varname>tiger_data</varname> schema. Each state script is returned as a separate record.</para>
+ <para>It uses unzip on Linux (7-zip on Windows by default) and wget to do the downloading. It uses <xref linkend="shp2pgsql_usage" /> to load in the data. Note the smallest unit it does is a whole state, but you can overwrite this by downloading the files yourself. It will only
+ process the files in the staging and temp folders.</para>
+ <para>It uses the following control tables to control the process and different OS shell syntax variations.</para>
+
+ <orderedlist>
+ <listitem>
+ <para><varname>loader_variables</varname> keeps track of various variables such as census site, year, data and staging schemas</para>
+ </listitem>
+ <listitem>
+ <para><varname>loader_platform</varname> profiles of various platforms and where the various executables are located. Comes with windows and linux. More can be added.</para>
+ </listitem>
+ <listitem>
+ <para><varname>loader_lookuptables</varname> each record defines a kind of table (state, county), whether to process records in it and how to load them in. Defines the steps to import data, stage data, add, removes columns, indexes, and constraints for each. Each table is prefixed with the state and inherits from a table in the tiger schema. e.g. creates <varname>tiger_data.ma_faces</varname> which inherits from <varname>tiger.faces</varname></para>
+ </listitem>
+ </orderedlist>
+ <para>Availability: 2.0.0 to support Tiger 2010 structured data and load census tract (tract), block groups (bg), and blocks (tabblocks) tables .</para>
+
+
+ </refsection>
+
+
+ <refsection>
+ <title>Examples</title>
+ <para>Generate script to load up data for 2 states in Windows shell script format.</para>
+ <programlisting>SELECT loader_generate_script(ARRAY['MA','RI'], 'windows') AS result;
+-- result --
+set STATEDIR="\gisdata\www2.census.gov\geo\pvs\tiger2010st\44_Rhode_Island"
+set TMPDIR=\gisdata\temp\
+set UNZIPTOOL="C:\Program Files\7-Zip\7z.exe"
+set WGETTOOL="C:\wget\wget.exe"
+set PGBIN=C:\Program Files\PostgreSQL\8.4\bin\
+set PGPORT=5432
+set PGHOST=localhost
+set PGUSER=postgres
+set PGPASSWORD=yourpasswordhere
+set PGDATABASE=geocoder
+set PSQL="%PGBIN%psql"
+set SHP2PGSQL="%PGBIN%shp2pgsql"
+
+%WGETTOOL% http://www2.census.gov/geo/pvs/tiger2010st/44_Rhode_Island/ --no-parent --relative --recursive --level=2 --accept=zip,txt --mirror --reject=html
+:
+:</programlisting>
+ <para>Generate sh script</para>
+ <programlisting>SELECT loader_generate_script(ARRAY['MA','RI'], 'sh') AS result;
+-- result --
+STATEDIR="/gisdata/www2.census.gov/geo/pvs/tiger2010st/44_Rhode_Island"
+TMPDIR="/gisdata/temp/"
+UNZIPTOOL=unzip
+PGPORT=5432
+PGHOST=localhost
+PGUSER=postgres
+PGPASSWORD=yourpasswordhere
+PGDATABASE=geocoder
+PSQL=psql
+SHP2PGSQ=shp2pgsql
+
+wget http://www2.census.gov/geo/pvs/tiger2010st/44_Rhode_Island/ --no-parent --relative --recursive --level=2 --accept=zip,txt --mirror --reject=html
+:
+:</programlisting>
+
+ </refsection>
+
+ <!-- Optionally add a "See Also" section -->
+ <refsection>
+ <title>See Also</title>
+ <para></para>
+ </refsection>
+ </refentry>
+
+ <refentry id="Loader_Generate_Nation_Script">
+ <refnamediv>
+ <refname>Loader_Generate_Nation_Script</refname>
+
+ <refpurpose>Generates a shell script for the specified platform that loads in the county and state lookup tables.</refpurpose>
+ </refnamediv>
+
+ <refsynopsisdiv>
+ <funcsynopsis>
+ <funcprototype>
+ <funcdef>text <function>loader_generate_nation_script</function></funcdef>
+ <paramdef><type>text</type> <parameter>os</parameter></paramdef>
+ </funcprototype>
+ </funcsynopsis>
+ </refsynopsisdiv>
+
+ <refsection>
+ <title>Description</title>
+
+ <para>Generates a shell script for the specified platform that loads in the <varname>county_all</varname>, <varname>county_all_lookup</varname>, <varname>state_all</varname> tables into <varname>tiger_data</varname> schema. These inherit respectively from the <varname>county</varname>, <varname>county_lookup</varname>, <varname>state</varname> tables in <varname>tiger</varname> schema.</para>
+ <para>It uses unzip on Linux (7-zip on Windows by default) and wget to do the downloading. It uses <xref linkend="shp2pgsql_usage" /> to load in the data. </para>
+ <para>It uses the following control tables <varname>tiger.loader_platform</varname>, <varname>tiger.loader_variables</varname>, and <varname>tiger.loader_lookuptables</varname> to control the process and different OS shell syntax variations.</para>
+
+ <orderedlist>
+ <listitem>
+ <para><varname>loader_variables</varname> keeps track of various variables such as census site, year, data and staging schemas</para>
+ </listitem>
+ <listitem>
+ <para><varname>loader_platform</varname> profiles of various platforms and where the various executables are located. Comes with windows and linux/unix. More can be added.</para>
+ </listitem>
+ <listitem>
+ <para><varname>loader_lookuptables</varname> each record defines a kind of table (state, county), whether to process records in it and how to load them in. Defines the steps to import data, stage data, add, removes columns, indexes, and constraints for each. Each table is prefixed with the state and inherits from a table in the tiger schema. e.g. creates <varname>tiger_data.ma_faces</varname> which inherits from <varname>tiger.faces</varname></para>
+ </listitem>
+ </orderedlist>
+ <para>Availability: 2.1.0 </para>
+ <note><para>If you were running <varname>tiger_2010</varname> version and you want to reload as state with <varname>tiger_2011</varname>, you'll need to for the very first load generate and run drop statements <xref linkend="Drop_Nation_Tables_Generate_Script" /> before you run this script.</para></note>
+ </refsection>
+
+
+ <refsection>
+ <title>Examples</title>
+ <para>Generate script script to load nation data Windows.</para>
+ <programlisting>SELECT loader_generate_nation_script('windows'); </programlisting>
+ <para>Generate script to load up data for Linux/Unix systems.</para>
+ <programlisting>SELECT loader_generate_nation_script('sh'); </programlisting>
+ </refsection>
+
<!-- Optionally add a "See Also" section -->
<refsection>
<title>See Also</title>
projects / postgis / commitdiff