Kettle_ES2GP
0
/
-
2020/09/09 17:25:31.380
-
2020/09/14 14:08:10.848
ID_JOB
Y
ID_JOB
CHANNEL_ID
Y
CHANNEL_ID
JOBNAME
Y
JOBNAME
STATUS
Y
STATUS
LINES_READ
Y
LINES_READ
LINES_WRITTEN
Y
LINES_WRITTEN
LINES_UPDATED
Y
LINES_UPDATED
LINES_INPUT
Y
LINES_INPUT
LINES_OUTPUT
Y
LINES_OUTPUT
LINES_REJECTED
Y
LINES_REJECTED
ERRORS
Y
ERRORS
STARTDATE
Y
STARTDATE
ENDDATE
Y
ENDDATE
LOGDATE
Y
LOGDATE
DEPDATE
Y
DEPDATE
REPLAYDATE
Y
REPLAYDATE
LOG_FIELD
Y
LOG_FIELD
EXECUTING_SERVER
N
EXECUTING_SERVER
EXECUTING_USER
N
EXECUTING_USER
START_JOB_ENTRY
N
START_JOB_ENTRY
CLIENT
N
CLIENT
ID_BATCH
Y
ID_BATCH
CHANNEL_ID
Y
CHANNEL_ID
LOG_DATE
Y
LOG_DATE
JOBNAME
Y
TRANSNAME
JOBENTRYNAME
Y
STEPNAME
LINES_READ
Y
LINES_READ
LINES_WRITTEN
Y
LINES_WRITTEN
LINES_UPDATED
Y
LINES_UPDATED
LINES_INPUT
Y
LINES_INPUT
LINES_OUTPUT
Y
LINES_OUTPUT
LINES_REJECTED
Y
LINES_REJECTED
ERRORS
Y
ERRORS
RESULT
Y
RESULT
NR_RESULT_ROWS
Y
NR_RESULT_ROWS
NR_RESULT_FILES
Y
NR_RESULT_FILES
LOG_FIELD
N
LOG_FIELD
COPY_NR
N
COPY_NR
ID_BATCH
Y
ID_BATCH
CHANNEL_ID
Y
CHANNEL_ID
LOG_DATE
Y
LOG_DATE
LOGGING_OBJECT_TYPE
Y
LOGGING_OBJECT_TYPE
OBJECT_NAME
Y
OBJECT_NAME
OBJECT_COPY
Y
OBJECT_COPY
REPOSITORY_DIRECTORY
Y
REPOSITORY_DIRECTORY
FILENAME
Y
FILENAME
OBJECT_ID
Y
OBJECT_ID
OBJECT_REVISION
Y
OBJECT_REVISION
PARENT_CHANNEL_ID
Y
PARENT_CHANNEL_ID
ROOT_CHANNEL_ID
Y
ROOT_CHANNEL_ID
N
Start
SPECIAL
Y
N
N
0
0
60
12
0
1
1
N
Y
0
112
80
转换【查询ES、加载数据】
TRANS
filename
${Internal.Entry.Current.Directory}/Kettle_ES_One.ktr
N
N
N
N
N
N
N
N
Basic
N
N
Y
N
N
N
Pentaho local
${es_method2}
${es_sql}
${es_table}
Y
N
Y
0
688
80
设置变量
SET_VARIABLES
Y
JVM
es_method
POST
JVM
es_sql
{ "query": "\n SELECT * FROM user_student2 \n ", "fetch_size": 10000 }
JVM
es_table
user_student3
JVM
es_sql2
{ "query": "\n SELECT * FROM user_student2 \n ", "fetch_size": 1}
JVM
es_method2
application/json
JVM
N
Y
0
240
80
检验字段的值【10000】
SIMPLE_EVAL
variable
es_total
es_total
number
10000
equal
inlist
false
N
N
Y
0
864
80
转换【分页查询ES、加载数据】
TRANS
filename
${Internal.Entry.Current.Directory}/Kettle_ES_Two.ktr
N
N
N
N
N
N
N
N
Basic
N
N
Y
N
N
N
Pentaho local
${es_method2}
${es_sql}
${es_cursor}
${es_table}
${es_columns}
Y
N
Y
0
784
192
Dummy
SPECIAL
N
Y
N
0
0
60
12
0
1
1
N
Y
0
944
192
成功
SUCCESS
N
Y
0
1024
80
转换【数据表检查、建表】
TRANS
filename
${Internal.Entry.Current.Directory}/Kettle_ES_Zero.ktr
N
N
N
N
N
N
N
N
Basic
N
N
Y
N
N
N
Pentaho local
${es_table}
${es_sql2}
${es_method}
Y
N
Y
0
496
80
创建一个目录
CREATE_FOLDER
/D:/KettleWork/SQL_File/${es_table}
N
N
Y
0
368
80
Start
设置变量
0
0
Y
Y
Y
转换【查询ES、加载数据】
检验字段的值【10000】
0
0
Y
Y
N
检验字段的值【10000】
转换【分页查询ES、加载数据】
0
0
Y
Y
N
转换【分页查询ES、加载数据】
Dummy
0
0
Y
Y
N
Dummy
检验字段的值【10000】
0
0
Y
Y
Y
转换【数据表检查、建表】
转换【查询ES、加载数据】
0
0
Y
Y
N
设置变量
创建一个目录
0
0
Y
Y
N
创建一个目录
转换【数据表检查、建表】
0
0
Y
Y
N
检验字段的值【10000】
成功
0
0
Y
N
N
METASTORE.NamedClusters
NamedCluster
{"namespace":"NamedClusters","id":"NamedCluster","name":"NamedCluster","description":"A NamedCluster","metaStoreName":null}
METASTORE.pentaho
Default Run Configuration
{"namespace":"pentaho","id":"Default Run Configuration","name":"Default Run Configuration","description":"Defines a default run configuration","metaStoreName":null}
{"_":"Embedded MetaStore Elements","namespace":"NamedClusters","type":"NamedCluster"}
10.10.14.111
{"children":[{"children":[],"id":"hdfsPassword","value":"ZHNpZGVhbA=="},{"children":[],"id":"oozieUrl","value":"http://localhost:8080/oozie"},{"children":[],"id":"mapr","value":"N"},{"children":[],"id":"useGateway","value":"N"},{"children":[],"id":"lastModifiedDate","value":"1600308712444"},{"children":[],"id":"jobTrackerHost","value":"10.10.14.111"},{"children":[],"id":"zooKeeperHost","value":"10.10.14.111"},{"children":[],"id":"shimIdentifier","value":null},{"children":[],"id":"gatewayUrl","value":null},{"children":[],"id":"jobTrackerPort","value":"9001"},{"children":[],"id":"zooKeeperPort","value":"2181"},{"children":[],"id":"name","value":"10.10.14.111"},{"children":[],"id":"hdfsPort","value":"9000"},{"children":[],"id":"hdfsUsername","value":"root"},{"children":[],"id":"gatewayPassword","value":null},{"children":[],"id":"kafkaBootstrapServers","value":null},{"children":[],"id":"storageScheme","value":"hdfs"},{"children":[],"id":"hdfsHost","value":"10.10.14.111"},{"children":[],"id":"gatewayUsername","value":null}],"id":"10.10.14.111","value":null,"name":"10.10.14.111","owner":null,"ownerPermissionsList":[]}
{"_":"Embedded MetaStore Elements","namespace":"pentaho","type":"Default Run Configuration"}
Pentaho local
{"children":[{"children":[],"id":"server","value":null},{"children":[],"id":"clustered","value":"N"},{"children":[],"id":"name","value":"Pentaho local"},{"children":[],"id":"description","value":null},{"children":[],"id":"pentaho","value":"N"},{"children":[],"id":"readOnly","value":"Y"},{"children":[],"id":"sendResources","value":"N"},{"children":[],"id":"logRemoteExecutionLocally","value":"N"},{"children":[],"id":"remote","value":"N"},{"children":[],"id":"local","value":"Y"},{"children":[],"id":"showTransformations","value":"N"}],"id":"Pentaho local","value":null,"name":"Pentaho local","owner":null,"ownerPermissionsList":[]}