LISAsoft Leverages GeoKettle ETL Tool

LISAsoft has added ETL expertise to its repertoire of Spatial Data Management competencies. LISAsoft and its sister company, Terrapages, broker and value-add data from a variety of data sources with different formats and purposes. ETL ( extract, transform and load ) is a process in database management practice which involves extracting data from outside sources, transforming it to fit operational needs and loading it into a target database, whether an operational data store or data warehouse.

It is a key process to bring all the data together in a standard, homogeneous environment. ETL processes can involve considerable complexity, and significant operational problems can occur with improperly designed ETL systems. Added to this was the complexity and specific considerations of the geospatial data which we deal with a lot. LISAsoft spent a considerable effort sourcing the most suitable ETL tools.

After an initial survey we decided on comprehensive evaluations of three: GeoKettle, an extension of the popular Pentaho framework ETL tool Kettle; Talend, another popular Open Source ETL program has a Geospatial extension which made it a candidate for evaluation; Feature Manipulation Engine (FME), a spatial specific ETL suite produced and marketed by Safe Software.  LISAsoft at last decided that GeoKettle would be the most suitable for our needs.

Forethought in our design was scalability across the lifetime of the usage of our ETL tools. This required understanding the volumes of data that will have to be processed. Initial work focused on our high-value PSMA datasets.

Several LISAsoft and Terrapages data solutions have benefited from our adoption of ETL. Terrapages AddressHelper, for example is an ideal solution for organisations which require online address validation, but cannot justify the investment of creating or maintaining their own address database. Terrapages also offers geocoding and reverse-geocoding, real estate information and other useful location-based services all kept up to date through our new ETL regime.

Talk to LISAsoft today to find out how we can help you on your next ETL project or discuss how we can help you with your data management problems.