欢迎您访问程序员文章站本站旨在为大家提供分享程序员计算机编程知识!
您现在的位置是: 首页  >  科技

Sqoop抽取数据到MySQL时遇到的问题

程序员文章站 2022-12-03 13:35:15
背景 对数据进行标准的ETL流程的操作 Sqoop ==data==> Hive ETL==sqoop==> MySQL xxx.properties文件 mysql_...

背景

对数据进行标准的ETL流程的操作
Sqoop ==data==> Hive ETL==sqoop==> MySQL

xxx.properties文件

mysql_driver="jdbc:mysql://localhost:3306/ruozedata_bi"
mysql_username="root"
mysql_password="root"

driver_rep="jdbc:mysql://localhost:3306/bireport?useUnicode=true&characterEncoding=utf8"
username_rep="root"
password_rep="root"

HOSTNAME="localhost" 
PORT="3306"
USERNAME="root"
PASSWORD="root"
DBNAME="bireport" 

tables=ruozedata_operatetrendandcity_rep,ruozedata_dealcompany_rep

第一次报错

The driver has not received any packets from the server.

大概意思是mysql连接超时,解决步骤:

修改driver_rep:
jdbc:mysql://localhost:3306//bireport?useUnicode=true&characterEncoding=utf8&autoReconnect=true&failOverReadOnly=false

修改/etc/my.cnf
wait_timeout=31536000
interactive_timeout=31536000

重启mysql

没有效果,再度报错:

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
        at com.mysql.jdbc.Util.handleNewInstance(Util.java:404)
        at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:988)
        at com.mysql.jdbc.MysqlIO.(MysqlIO.java:341)
        at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2251)
        at com.mysql.jdbc.ConnectionImpl.connectWithRetries(ConnectionImpl.java:2104)
        ... 31 more
Caused by: java.net.UnknownHostException: localshot
        at java.net.InetAddress.getAllByName0(InetAddress.java:1259)
        at java.net.InetAddress.getAllByName(InetAddress.java:1171)
        at java.net.InetAddress.getAllByName(InetAddress.java:1105)
        at com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:188)
        at com.mysql.jdbc.MysqlIO.(MysqlIO.java:300)

定位到原因:Caused by: java.net.UnknownHostException: localshot
修改localhost为机器的ip即可