We have a task to sqoop-import tables from a postgreSQL db. With "--hive-overwrite" the data correctly gets refreshed. However, if a schema change occurs on the original table, it appears that the schema does not get refreshed.
Is there a way to get sqoop to re-create the table if the schema has changed? Or will I need to delete the table beforehand?
There are two ways you can handle, 1. Simple way is to add columns to the hive table and sqoop data into it. In that case always the db columns need to be added to the end of the table. (there is a way to handle if datatype of a column changed or column added in the middle) 2. If you have to handle schema changes on many tables, then you have to write a process to automatically find the schema changes and reflect that on the hive table. This is a multi step process. On Tue, Jan 28, 2014 at 8:04 AM, Krishna Rao <[EMAIL PROTECTED]> wrote:
Yes, that is the easy way if table is not too big. On Tue, Jan 28, 2014 at 9:29 AM, Krishna Rao <[EMAIL PROTECTED]> wrote:
NEW: Monitor These Apps!
Apache Lucene, Apache Solr and all other Apache Software Foundation project and their respective logos are trademarks of the Apache Software Foundation.
Elasticsearch, Kibana, Logstash, and Beats are trademarks of Elasticsearch BV, registered in the U.S. and in other countries. This site and Sematext Group is in no way affiliated with Elasticsearch BV.
Service operated by Sematext