backend/app/datax/log/2022-08-08/2hive1659950348_json-17_19_09.002.log

2022-08-08 17:19:09.160 [main] INFO  VMInfo - VMInfo# operatingSystem class => com.sun.management.internal.OperatingSystemImpl
2022-08-08 17:19:09.164 [main] INFO  Engine - the machine info  => 

	osInfo:	Ubuntu 11 11.0.16+8-post-Ubuntu-0ubuntu120.04
	jvmInfo:	Linux amd64 5.13.0-52-generic
	cpu num:	8

	totalPhysicalMemory:	-0.00G
	freePhysicalMemory:	-0.00G
	maxFileDescriptorCount:	-1
	currentOpenFileDescriptorCount:	-1

	GC Names	[G1 Young Generation, G1 Old Generation]

	MEMORY_NAME                    | allocation_size                | init_size                      
	CodeHeap 'profiled nmethods'   | 117.21MB                       | 2.44MB                         
	G1 Old Gen                     | 1,024.00MB                     | 970.00MB                       
	G1 Survivor Space              | -0.00MB                        | 0.00MB                         
	CodeHeap 'non-profiled nmethods' | 117.22MB                       | 2.44MB                         
	Compressed Class Space         | 1,024.00MB                     | 0.00MB                         
	Metaspace                      | -0.00MB                        | 0.00MB                         
	G1 Eden Space                  | -0.00MB                        | 54.00MB                        
	CodeHeap 'non-nmethods'        | 5.57MB                         | 2.44MB                         


2022-08-08 17:19:09.175 [main] INFO  Engine - 
{
	"content":[
		{
			"reader":{
				"name":"mysqlreader",
				"parameter":{
					"connection":[
						{
							"jdbcUrl":[
								"jdbc:mysql://192.168.2.2:3306/gene"
							],
							"querySql":[
								"select * from gene.user_info_transfer"
							]
						}
					],
					"password":"*********",
					"username":"root"
				}
			},
			"writer":{
				"name":"hdfswriter",
				"parameter":{
					"column":[
						{
							"name":"user_id",
							"type":"string"
						},
						{
							"name":"area_id",
							"type":"string"
						},
						{
							"name":"age",
							"type":"int"
						},
						{
							"name":"occupation",
							"type":"string"
						}
					],
					"compress":"",
					"defaultFS":"hdfs://hadoop03:8020/",
					"fieldDelimiter":"\t",
					"fileName":"user_info_data_3.txt",
					"fileType":"text",
					"path":"/user/hive/warehouse/user_info/",
					"writeMode":"append"
				}
			}
		}
	],
	"setting":{
		"speed":{
			"channel":"2"
		}
	}
}

2022-08-08 17:19:09.183 [main] WARN  Engine - prioriy set to 0, because NumberFormatException, the value is: null
2022-08-08 17:19:09.184 [main] INFO  PerfTrace - PerfTrace traceId=job_-1, isEnable=false, priority=0
2022-08-08 17:19:09.184 [main] INFO  JobContainer - DataX jobContainer starts job.
2022-08-08 17:19:09.185 [main] INFO  JobContainer - Set jobId = 0
2022-08-08 17:19:09.371 [job-0] INFO  OriginalConfPretreatmentUtil - Available jdbcUrl:jdbc:mysql://192.168.2.2:3306/gene?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true.
2022-08-08 17:19:09.899 [job-0] INFO  JobContainer - jobContainer starts to do prepare ...
2022-08-08 17:19:09.899 [job-0] INFO  JobContainer - DataX Reader.Job [mysqlreader] do prepare work .
2022-08-08 17:19:09.900 [job-0] INFO  JobContainer - DataX Writer.Job [hdfswriter] do prepare work .
2022-08-08 17:19:09.960 [job-0] INFO  HdfsWriter$Job - 由于您配置了writeMode append, 写入前不做清理工作, [/user/hive/warehouse/user_info/] 目录下写入相应文件名前缀  [user_info_data_3.txt] 的文件
2022-08-08 17:19:09.960 [job-0] INFO  JobContainer - jobContainer starts to do split ...
2022-08-08 17:19:09.960 [job-0] INFO  JobContainer - Job set Channel-Number to 2 channels.
2022-08-08 17:19:09.961 [job-0] INFO  JobContainer - DataX Reader.Job [mysqlreader] splits to [1] tasks.
2022-08-08 17:19:09.962 [job-0] INFO  HdfsWriter$Job - begin do split...
2022-08-08 17:19:09.965 [job-0] INFO  HdfsWriter$Job - splited write file name:[hdfs://hadoop03:8020//user/hive/warehouse/user_info__d08e04a5_91f2_4610_ad2f_fec59a7bc14d/user_info_data_3.txt__b7ee6106_5144_4078_be54_6aed202d6de3]
2022-08-08 17:19:09.965 [job-0] INFO  HdfsWriter$Job - end do split.
2022-08-08 17:19:09.965 [job-0] INFO  JobContainer - DataX Writer.Job [hdfswriter] splits to [1] tasks.
2022-08-08 17:19:09.973 [job-0] INFO  JobContainer - jobContainer starts to do schedule ...
2022-08-08 17:19:09.975 [job-0] INFO  JobContainer - Scheduler starts [1] taskGroups.
2022-08-08 17:19:09.976 [job-0] INFO  JobContainer - Running by standalone Mode.
2022-08-08 17:19:09.979 [taskGroup-0] INFO  TaskGroupContainer - taskGroupId=[0] start [1] channels for [1] tasks.
2022-08-08 17:19:09.983 [taskGroup-0] INFO  Channel - Channel set byte_speed_limit to -1, No bps activated.
2022-08-08 17:19:09.983 [taskGroup-0] INFO  Channel - Channel set record_speed_limit to -1, No tps activated.
2022-08-08 17:19:09.989 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] taskId[0] attemptCount[1] is started
2022-08-08 17:19:09.991 [0-0-0-reader] INFO  CommonRdbmsReader$Task - Begin to read record by Sql: [select * from gene.user_info_transfer
] jdbcUrl:[jdbc:mysql://192.168.2.2:3306/gene?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2022-08-08 17:19:10.003 [0-0-0-writer] INFO  HdfsWriter$Task - begin do write...
2022-08-08 17:19:10.003 [0-0-0-writer] INFO  HdfsWriter$Task - write to file : [hdfs://hadoop03:8020//user/hive/warehouse/user_info__d08e04a5_91f2_4610_ad2f_fec59a7bc14d/user_info_data_3.txt__b7ee6106_5144_4078_be54_6aed202d6de3]
2022-08-08 17:19:10.006 [0-0-0-reader] INFO  CommonRdbmsReader$Task - Finished read record by Sql: [select * from gene.user_info_transfer
] jdbcUrl:[jdbc:mysql://192.168.2.2:3306/gene?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2022-08-08 17:19:10.139 [0-0-0-writer] INFO  HdfsWriter$Task - end do write
2022-08-08 17:19:10.189 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] taskId[0] is successed, used[201]ms
2022-08-08 17:19:10.190 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] completed it's tasks.
2022-08-08 17:19:19.989 [job-0] INFO  StandAloneJobContainerCommunicator - Total 8 records, 158 bytes | Speed 15B/s, 0 records/s | Error 0 records, 0 bytes |  All Task WaitWriterTime 0.000s |  All Task WaitReaderTime 0.000s | Percentage 100.00%
2022-08-08 17:19:19.990 [job-0] INFO  AbstractScheduler - Scheduler accomplished all tasks.
2022-08-08 17:19:19.991 [job-0] INFO  JobContainer - DataX Writer.Job [hdfswriter] do post work.
2022-08-08 17:19:19.992 [job-0] INFO  HdfsWriter$Job - start rename file [hdfs://hadoop03:8020//user/hive/warehouse/user_info__d08e04a5_91f2_4610_ad2f_fec59a7bc14d/user_info_data_3.txt__b7ee6106_5144_4078_be54_6aed202d6de3] to file [hdfs://hadoop03:8020//user/hive/warehouse/user_info/user_info_data_3.txt__b7ee6106_5144_4078_be54_6aed202d6de3].
2022-08-08 17:19:20.006 [job-0] INFO  HdfsWriter$Job - finish rename file [hdfs://hadoop03:8020//user/hive/warehouse/user_info__d08e04a5_91f2_4610_ad2f_fec59a7bc14d/user_info_data_3.txt__b7ee6106_5144_4078_be54_6aed202d6de3] to file [hdfs://hadoop03:8020//user/hive/warehouse/user_info/user_info_data_3.txt__b7ee6106_5144_4078_be54_6aed202d6de3].
2022-08-08 17:19:20.006 [job-0] INFO  HdfsWriter$Job - start delete tmp dir [hdfs://hadoop03:8020/user/hive/warehouse/user_info__d08e04a5_91f2_4610_ad2f_fec59a7bc14d] .
2022-08-08 17:19:20.018 [job-0] INFO  HdfsWriter$Job - finish delete tmp dir [hdfs://hadoop03:8020/user/hive/warehouse/user_info__d08e04a5_91f2_4610_ad2f_fec59a7bc14d] .
2022-08-08 17:19:20.019 [job-0] INFO  JobContainer - DataX Reader.Job [mysqlreader] do post work.
2022-08-08 17:19:20.019 [job-0] INFO  JobContainer - DataX jobId [0] completed successfully.
2022-08-08 17:19:20.021 [job-0] INFO  HookInvoker - No hook invoked, because base dir not exists or is a file: /home/zhouze/PycharmProjects/yili/yili-portal/app/datax/hook
2022-08-08 17:19:20.128 [job-0] INFO  JobContainer - 
	 [total cpu info] => 
		averageCpu                     | maxDeltaCpu                    | minDeltaCpu                    
		-1.00%                         | -1.00%                         | -1.00%
                        

	 [total gc info] => 
		 NAME                 | totalGCCount       | maxDeltaGCCount    | minDeltaGCCount    | totalGCTime        | maxDeltaGCTime     | minDeltaGCTime     
		 G1 Young Generation  | 2                  | 2                  | 2                  | 0.014s             | 0.014s             | 0.014s             
		 G1 Old Generation    | 0                  | 0                  | 0                  | 0.000s             | 0.000s             | 0.000s             

2022-08-08 17:19:20.129 [job-0] INFO  JobContainer - PerfTrace not enable!
2022-08-08 17:19:20.130 [job-0] INFO  StandAloneJobContainerCommunicator - Total 8 records, 158 bytes | Speed 15B/s, 0 records/s | Error 0 records, 0 bytes |  All Task WaitWriterTime 0.000s |  All Task WaitReaderTime 0.000s | Percentage 100.00%
2022-08-08 17:19:20.132 [job-0] INFO  JobContainer - 
任务启动时刻                    : 2022-08-08 17:19:09
任务结束时刻                    : 2022-08-08 17:19:20
任务总计耗时                    :                 10s
任务平均流量                    :               15B/s
记录写入速度                    :              0rec/s
读出记录总数                    :                   8
读写失败总数                    :                   0