Skip to content
项目
群组
代码片段
帮助
当前项目
正在载入...
登录 / 注册
切换导航面板
S
seatunnel-web
项目
项目
详情
活动
周期分析
仓库
仓库
文件
提交
分支
标签
贡献者
图表
比较
统计图
议题
0
议题
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
CI / CD
CI / CD
流水线
作业
日程
统计图
Wiki
Wiki
代码片段
代码片段
成员
成员
折叠边栏
关闭边栏
活动
图像
聊天
创建新问题
作业
提交
问题看板
Open sidebar
宋勇
seatunnel-web
Commits
a573a68d
提交
a573a68d
authored
10月 21, 2023
作者:
宋勇
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
修改添加静态实例
上级
75753937
隐藏空白字符变更
内嵌
并排
正在显示
46 个修改的文件
包含
380 行增加
和
36 行删除
+380
-36
ElasticSearchDataSourceChannel.java
.../plugin/elasticsearch/ElasticSearchDataSourceChannel.java
+9
-0
ElasticSearchDataSourceFactory.java
.../plugin/elasticsearch/ElasticSearchDataSourceFactory.java
+1
-1
InfluxdbDataSourceChannel.java
...datasource/plugin/influxdb/InfluxdbDataSourceChannel.java
+8
-0
InfluxdbDataSourceFactory.java
...datasource/plugin/influxdb/InfluxdbDataSourceFactory.java
+1
-1
ClickhouseJdbcDataSourceChannel.java
...ugin/clickhouse/jdbc/ClickhouseJdbcDataSourceChannel.java
+8
-0
ClickhouseJdbcDataSourceFactory.java
...ugin/clickhouse/jdbc/ClickhouseJdbcDataSourceFactory.java
+1
-1
HiveJdbcDataSourceChannel.java
...atasource/plugin/hive/jdbc/HiveJdbcDataSourceChannel.java
+7
-0
HiveJdbcDataSourceFactory.java
...atasource/plugin/hive/jdbc/HiveJdbcDataSourceFactory.java
+1
-1
MysqlJdbcDataSourceChannel.java
...asource/plugin/mysql/jdbc/MysqlJdbcDataSourceChannel.java
+8
-0
MysqlJdbcDataSourceFactory.java
...asource/plugin/mysql/jdbc/MysqlJdbcDataSourceFactory.java
+1
-1
OracleDataSourceChannel.java
...atasource/plugin/oracle/jdbc/OracleDataSourceChannel.java
+8
-0
OracleJdbcDataSourceFactory.java
...ource/plugin/oracle/jdbc/OracleJdbcDataSourceFactory.java
+1
-1
PostgresqlDataSourceChannel.java
...e/plugin/postgresql/jdbc/PostgresqlDataSourceChannel.java
+9
-0
PostgresqlDataSourceFactory.java
...e/plugin/postgresql/jdbc/PostgresqlDataSourceFactory.java
+1
-1
RedshiftDataSourceChannel.java
...ource/plugin/redshift/jdbc/RedshiftDataSourceChannel.java
+7
-0
RedshiftDataSourceFactory.java
...ource/plugin/redshift/jdbc/RedshiftDataSourceFactory.java
+1
-1
SqlServerDataSourceChannel.java
...rce/plugin/sqlserver/jdbc/SqlServerDataSourceChannel.java
+8
-0
SqlServerDataSourceFactory.java
...rce/plugin/sqlserver/jdbc/SqlServerDataSourceFactory.java
+1
-1
StarRocksJdbcDataSourceChannel.java
...plugin/starrocks/jdbc/StarRocksJdbcDataSourceChannel.java
+8
-0
StarRocksJdbcDataSourceFactory.java
...plugin/starrocks/jdbc/StarRocksJdbcDataSourceFactory.java
+1
-1
TidbJdbcDataSourceChannel.java
...atasource/plugin/tidb/jdbc/TidbJdbcDataSourceChannel.java
+8
-0
TidbJdbcDataSourceFactory.java
...atasource/plugin/tidb/jdbc/TidbJdbcDataSourceFactory.java
+1
-1
KafkaDataSourceChannel.java
...unnel/datasource/plugin/kafka/KafkaDataSourceChannel.java
+7
-0
KafkaDataSourceFactory.java
...unnel/datasource/plugin/kafka/KafkaDataSourceFactory.java
+1
-1
MqttDataSourceChannel.java
...atunnel/datasource/plugin/mqtt/MqttDataSourceChannel.java
+14
-1
MqttDataSourceFactory.java
...atunnel/datasource/plugin/mqtt/MqttDataSourceFactory.java
+1
-1
MqttOptionRule.java
...ache/seatunnel/datasource/plugin/mqtt/MqttOptionRule.java
+1
-1
MqttClientServiceTest.java
...atunnel/datasource/plugin/mqtt/MqttClientServiceTest.java
+14
-0
MysqlCDCDataSourceChannel.java
...atasource/plugin/cdc/mysql/MysqlCDCDataSourceChannel.java
+8
-0
MysqlCDCDataSourceFactory.java
...atasource/plugin/cdc/mysql/MysqlCDCDataSourceFactory.java
+1
-1
OpcUaClientService.java
...seatunnel/datasource/plugin/opcua/OpcUaClientService.java
+47
-5
OpcuaDataSourceChannel.java
...unnel/datasource/plugin/opcua/OpcuaDataSourceChannel.java
+9
-3
OpcuaDataSourceFactory.java
...unnel/datasource/plugin/opcua/OpcuaDataSourceFactory.java
+1
-1
OpcuaOptionRule.java
...he/seatunnel/datasource/plugin/opcua/OpcuaOptionRule.java
+2
-2
pom.xml
...source/seatunnel-datasource-plugins/datasource-s3/pom.xml
+56
-0
HadoopS3AConfiguration.java
...eatunnel/datasource/plugin/s3/HadoopS3AConfiguration.java
+1
-3
S3DataSourceFactory.java
...e/seatunnel/datasource/plugin/s3/S3DataSourceFactory.java
+1
-1
S3DatasourceChannel.java
...e/seatunnel/datasource/plugin/s3/S3DatasourceChannel.java
+8
-0
HadoopS3AConfigurationTest.java
...nnel/datasource/plugin/s3/HadoopS3AConfigurationTest.java
+30
-0
pom.xml
...eatunnel-datasource-plugins/datasource-s3redshift/pom.xml
+53
-2
S3RedshiftDataSourceChannel.java
...ource/plugin/redshift/s3/S3RedshiftDataSourceChannel.java
+8
-0
S3RedshiftDataSourceFactory.java
...ource/plugin/redshift/s3/S3RedshiftDataSourceFactory.java
+1
-1
SqlServerCDCDataSourceChannel.java
...e/plugin/cdc/sqlserver/SqlServerCDCDataSourceChannel.java
+8
-0
SqlServerCDCDataSourceFactory.java
...e/plugin/cdc/sqlserver/SqlServerCDCDataSourceFactory.java
+1
-1
StarRocksDataSourceChannel.java
...tasource/plugin/starrocks/StarRocksDataSourceChannel.java
+7
-0
StarRocksDataSourceFactory.java
...tasource/plugin/starrocks/StarRocksDataSourceFactory.java
+1
-1
没有找到文件。
seatunnel-datasource/seatunnel-datasource-plugins/datasource-elasticsearch/src/main/java/org/apache/seatunnel/datasource/plugin/elasticsearch/ElasticSearchDataSourceChannel.java
浏览文件 @
a573a68d
...
...
@@ -38,6 +38,15 @@ public class ElasticSearchDataSourceChannel implements DataSourceChannel {
private
static
final
String
DATABASE
=
"default"
;
public
static
class
Holder
{
private
static
final
ElasticSearchDataSourceChannel
INSTANCE
=
new
ElasticSearchDataSourceChannel
();
}
public
static
ElasticSearchDataSourceChannel
getInstance
()
{
return
Holder
.
INSTANCE
;
}
@Override
public
boolean
canAbleGetSchema
()
{
return
true
;
...
...
seatunnel-datasource/seatunnel-datasource-plugins/datasource-elasticsearch/src/main/java/org/apache/seatunnel/datasource/plugin/elasticsearch/ElasticSearchDataSourceFactory.java
浏览文件 @
a573a68d
...
...
@@ -43,6 +43,6 @@ public class ElasticSearchDataSourceFactory implements DataSourceFactory {
@Override
public
DataSourceChannel
createChannel
()
{
return
new
ElasticSearchDataSourceChannel
();
return
ElasticSearchDataSourceChannel
.
getInstance
();
}
}
seatunnel-datasource/seatunnel-datasource-plugins/datasource-influxdb/src/main/java/org/apache/seatunnel/datasource/plugin/influxdb/InfluxdbDataSourceChannel.java
浏览文件 @
a573a68d
...
...
@@ -39,6 +39,14 @@ public class InfluxdbDataSourceChannel implements DataSourceChannel {
private
static
final
String
DATABASE
=
"default"
;
public
static
class
Holder
{
private
static
final
InfluxdbDataSourceChannel
INSTANCE
=
new
InfluxdbDataSourceChannel
();
}
public
static
InfluxdbDataSourceChannel
getInstance
()
{
return
Holder
.
INSTANCE
;
}
@Override
public
boolean
canAbleGetSchema
()
{
return
true
;
...
...
seatunnel-datasource/seatunnel-datasource-plugins/datasource-influxdb/src/main/java/org/apache/seatunnel/datasource/plugin/influxdb/InfluxdbDataSourceFactory.java
浏览文件 @
a573a68d
...
...
@@ -43,6 +43,6 @@ public class InfluxdbDataSourceFactory implements DataSourceFactory {
@Override
public
DataSourceChannel
createChannel
()
{
return
new
InfluxdbDataSourceChannel
();
return
InfluxdbDataSourceChannel
.
getInstance
();
}
}
seatunnel-datasource/seatunnel-datasource-plugins/datasource-jdbc-clickhouse/src/main/java/org/apache/seatunnel/datasource/plugin/clickhouse/jdbc/ClickhouseJdbcDataSourceChannel.java
浏览文件 @
a573a68d
...
...
@@ -42,6 +42,14 @@ import static com.google.common.base.Preconditions.checkNotNull;
@Slf4j
public
class
ClickhouseJdbcDataSourceChannel
implements
DataSourceChannel
{
public
static
class
Holder
{
private
static
final
ClickhouseJdbcDataSourceChannel
INSTANCE
=
new
ClickhouseJdbcDataSourceChannel
();
}
public
static
ClickhouseJdbcDataSourceChannel
getInstance
()
{
return
Holder
.
INSTANCE
;
}
@Override
public
OptionRule
getDataSourceOptions
(
@NonNull
String
pluginName
)
{
...
...
seatunnel-datasource/seatunnel-datasource-plugins/datasource-jdbc-clickhouse/src/main/java/org/apache/seatunnel/datasource/plugin/clickhouse/jdbc/ClickhouseJdbcDataSourceFactory.java
浏览文件 @
a573a68d
...
...
@@ -43,6 +43,6 @@ public class ClickhouseJdbcDataSourceFactory implements DataSourceFactory {
@Override
public
DataSourceChannel
createChannel
()
{
return
new
ClickhouseJdbcDataSourceChannel
();
return
ClickhouseJdbcDataSourceChannel
.
getInstance
();
}
}
seatunnel-datasource/seatunnel-datasource-plugins/datasource-jdbc-hive/src/main/java/org/apache/seatunnel/datasource/plugin/hive/jdbc/HiveJdbcDataSourceChannel.java
浏览文件 @
a573a68d
...
...
@@ -44,6 +44,13 @@ import java.util.Map;
@Slf4j
public
class
HiveJdbcDataSourceChannel
implements
DataSourceChannel
{
public
static
class
Holder
{
private
static
final
HiveJdbcDataSourceChannel
INSTANCE
=
new
HiveJdbcDataSourceChannel
();
}
public
static
HiveJdbcDataSourceChannel
getInstance
()
{
return
Holder
.
INSTANCE
;
}
@Override
public
OptionRule
getDataSourceOptions
(
@NonNull
String
pluginName
)
{
...
...
seatunnel-datasource/seatunnel-datasource-plugins/datasource-jdbc-hive/src/main/java/org/apache/seatunnel/datasource/plugin/hive/jdbc/HiveJdbcDataSourceFactory.java
浏览文件 @
a573a68d
...
...
@@ -48,6 +48,6 @@ public class HiveJdbcDataSourceFactory implements DataSourceFactory {
@Override
public
DataSourceChannel
createChannel
()
{
return
new
HiveJdbcDataSourceChannel
();
return
HiveJdbcDataSourceChannel
.
getInstance
();
}
}
seatunnel-datasource/seatunnel-datasource-plugins/datasource-jdbc-mysql/src/main/java/org/apache/seatunnel/datasource/plugin/mysql/jdbc/MysqlJdbcDataSourceChannel.java
浏览文件 @
a573a68d
...
...
@@ -38,6 +38,14 @@ import static com.google.common.base.Preconditions.checkNotNull;
public
class
MysqlJdbcDataSourceChannel
implements
DataSourceChannelExt
{
List
<
String
>
typeList
=
Arrays
.
asList
(
"varchar"
,
"char"
,
"json"
);
public
static
class
Holder
{
private
static
final
MysqlJdbcDataSourceChannel
INSTANCE
=
new
MysqlJdbcDataSourceChannel
();
}
public
static
MysqlJdbcDataSourceChannel
getInstance
()
{
return
Holder
.
INSTANCE
;
}
@Override
public
OptionRule
getDataSourceOptions
(
@NonNull
String
pluginName
)
{
return
MysqlDataSourceConfig
.
OPTION_RULE
;
...
...
seatunnel-datasource/seatunnel-datasource-plugins/datasource-jdbc-mysql/src/main/java/org/apache/seatunnel/datasource/plugin/mysql/jdbc/MysqlJdbcDataSourceFactory.java
浏览文件 @
a573a68d
...
...
@@ -43,6 +43,6 @@ public class MysqlJdbcDataSourceFactory implements DataSourceFactory {
@Override
public
DataSourceChannel
createChannel
()
{
return
new
MysqlJdbcDataSourceChannel
();
return
MysqlJdbcDataSourceChannel
.
getInstance
();
}
}
seatunnel-datasource/seatunnel-datasource-plugins/datasource-jdbc-oracle/src/main/java/org/apache/seatunnel/datasource/plugin/oracle/jdbc/OracleDataSourceChannel.java
浏览文件 @
a573a68d
...
...
@@ -36,6 +36,14 @@ import static com.google.common.base.Preconditions.checkNotNull;
public
class
OracleDataSourceChannel
implements
DataSourceChannelExt
{
List
<
String
>
typeList
=
Arrays
.
asList
(
"varchar"
,
"varchar2"
,
"char"
,
"json"
);
public
static
class
Holder
{
private
static
final
OracleDataSourceChannel
INSTANCE
=
new
OracleDataSourceChannel
();
}
public
static
OracleDataSourceChannel
getInstance
()
{
return
Holder
.
INSTANCE
;
}
@Override
public
OptionRule
getDataSourceOptions
(
@NonNull
String
pluginName
)
{
return
OracleDataSourceConfig
.
OPTION_RULE
;
...
...
seatunnel-datasource/seatunnel-datasource-plugins/datasource-jdbc-oracle/src/main/java/org/apache/seatunnel/datasource/plugin/oracle/jdbc/OracleJdbcDataSourceFactory.java
浏览文件 @
a573a68d
...
...
@@ -42,6 +42,6 @@ public class OracleJdbcDataSourceFactory implements DataSourceFactory {
@Override
public
DataSourceChannel
createChannel
()
{
return
new
OracleDataSourceChannel
();
return
OracleDataSourceChannel
.
getInstance
();
}
}
seatunnel-datasource/seatunnel-datasource-plugins/datasource-jdbc-postgresql/src/main/java/org/apache/seatunnel/datasource/plugin/postgresql/jdbc/PostgresqlDataSourceChannel.java
浏览文件 @
a573a68d
...
...
@@ -36,6 +36,15 @@ import static com.google.common.base.Preconditions.checkNotNull;
public
class
PostgresqlDataSourceChannel
implements
DataSourceChannelExt
{
List
<
String
>
typeList
=
Arrays
.
asList
(
"varchar"
,
"char"
,
"json"
);
public
static
class
Holder
{
private
static
final
PostgresqlDataSourceChannel
INSTANCE
=
new
PostgresqlDataSourceChannel
();
}
public
static
PostgresqlDataSourceChannel
getInstance
()
{
return
Holder
.
INSTANCE
;
}
@Override
public
OptionRule
getDataSourceOptions
(
@NonNull
String
pluginName
)
{
return
PostgresqlDataSourceConfig
.
OPTION_RULE
;
...
...
seatunnel-datasource/seatunnel-datasource-plugins/datasource-jdbc-postgresql/src/main/java/org/apache/seatunnel/datasource/plugin/postgresql/jdbc/PostgresqlDataSourceFactory.java
浏览文件 @
a573a68d
...
...
@@ -41,6 +41,6 @@ public class PostgresqlDataSourceFactory implements DataSourceFactory {
@Override
public
DataSourceChannel
createChannel
()
{
return
new
PostgresqlDataSourceChannel
();
return
PostgresqlDataSourceChannel
.
getInstance
();
}
}
seatunnel-datasource/seatunnel-datasource-plugins/datasource-jdbc-redshift/src/main/java/org/apache/seatunnel/datasource/plugin/redshift/jdbc/RedshiftDataSourceChannel.java
浏览文件 @
a573a68d
...
...
@@ -40,6 +40,13 @@ import java.util.Map;
import
static
com
.
google
.
common
.
base
.
Preconditions
.
checkNotNull
;
public
class
RedshiftDataSourceChannel
implements
DataSourceChannel
{
public
static
class
Holder
{
private
static
final
RedshiftDataSourceChannel
INSTANCE
=
new
RedshiftDataSourceChannel
();
}
public
static
RedshiftDataSourceChannel
getInstance
()
{
return
Holder
.
INSTANCE
;
}
@Override
public
OptionRule
getDataSourceOptions
(
@NonNull
String
pluginName
)
{
...
...
seatunnel-datasource/seatunnel-datasource-plugins/datasource-jdbc-redshift/src/main/java/org/apache/seatunnel/datasource/plugin/redshift/jdbc/RedshiftDataSourceFactory.java
浏览文件 @
a573a68d
...
...
@@ -43,6 +43,6 @@ public class RedshiftDataSourceFactory implements DataSourceFactory {
@Override
public
DataSourceChannel
createChannel
()
{
return
new
RedshiftDataSourceChannel
();
return
RedshiftDataSourceChannel
.
getInstance
();
}
}
seatunnel-datasource/seatunnel-datasource-plugins/datasource-jdbc-sqlserver/src/main/java/org/apache/seatunnel/datasource/plugin/sqlserver/jdbc/SqlServerDataSourceChannel.java
浏览文件 @
a573a68d
...
...
@@ -36,6 +36,14 @@ import static com.google.common.base.Preconditions.checkNotNull;
@Slf4j
public
class
SqlServerDataSourceChannel
implements
DataSourceChannelExt
{
public
static
class
Holder
{
private
static
final
SqlServerDataSourceChannel
INSTANCE
=
new
SqlServerDataSourceChannel
();
}
public
static
SqlServerDataSourceChannel
getInstance
()
{
return
Holder
.
INSTANCE
;
}
@Override
public
OptionRule
getDataSourceOptions
(
@NonNull
String
pluginName
)
{
return
SqlServerDataSourceConfig
.
OPTION_RULE
;
...
...
seatunnel-datasource/seatunnel-datasource-plugins/datasource-jdbc-sqlserver/src/main/java/org/apache/seatunnel/datasource/plugin/sqlserver/jdbc/SqlServerDataSourceFactory.java
浏览文件 @
a573a68d
...
...
@@ -43,6 +43,6 @@ public class SqlServerDataSourceFactory implements DataSourceFactory {
@Override
public
DataSourceChannel
createChannel
()
{
return
new
SqlServerDataSourceChannel
();
return
SqlServerDataSourceChannel
.
getInstance
();
}
}
seatunnel-datasource/seatunnel-datasource-plugins/datasource-jdbc-starrocks/src/main/java/org/apache/seatunnel/datasource/plugin/starrocks/jdbc/StarRocksJdbcDataSourceChannel.java
浏览文件 @
a573a68d
...
...
@@ -40,6 +40,14 @@ import java.util.Map;
import
static
com
.
google
.
common
.
base
.
Preconditions
.
checkNotNull
;
public
class
StarRocksJdbcDataSourceChannel
implements
DataSourceChannel
{
public
static
class
Holder
{
private
static
final
StarRocksJdbcDataSourceChannel
INSTANCE
=
new
StarRocksJdbcDataSourceChannel
();
}
public
static
StarRocksJdbcDataSourceChannel
getInstance
()
{
return
Holder
.
INSTANCE
;
}
@Override
public
OptionRule
getDataSourceOptions
(
@NonNull
String
pluginName
)
{
...
...
seatunnel-datasource/seatunnel-datasource-plugins/datasource-jdbc-starrocks/src/main/java/org/apache/seatunnel/datasource/plugin/starrocks/jdbc/StarRocksJdbcDataSourceFactory.java
浏览文件 @
a573a68d
...
...
@@ -41,6 +41,6 @@ public class StarRocksJdbcDataSourceFactory implements DataSourceFactory {
@Override
public
DataSourceChannel
createChannel
()
{
return
new
StarRocksJdbcDataSourceChannel
();
return
StarRocksJdbcDataSourceChannel
.
getInstance
();
}
}
seatunnel-datasource/seatunnel-datasource-plugins/datasource-jdbc-tidb/src/main/java/org/apache/seatunnel/datasource/plugin/tidb/jdbc/TidbJdbcDataSourceChannel.java
浏览文件 @
a573a68d
...
...
@@ -38,6 +38,14 @@ import static com.google.common.base.Preconditions.checkNotNull;
public
class
TidbJdbcDataSourceChannel
implements
DataSourceChannelExt
{
List
<
String
>
typeList
=
Arrays
.
asList
(
"varchar"
,
"char"
,
"json"
);
public
static
class
Holder
{
private
static
final
TidbJdbcDataSourceChannel
INSTANCE
=
new
TidbJdbcDataSourceChannel
();
}
public
static
TidbJdbcDataSourceChannel
getInstance
()
{
return
Holder
.
INSTANCE
;
}
@Override
public
OptionRule
getDataSourceOptions
(
@NonNull
String
pluginName
)
{
return
TidbDataSourceConfig
.
OPTION_RULE
;
...
...
seatunnel-datasource/seatunnel-datasource-plugins/datasource-jdbc-tidb/src/main/java/org/apache/seatunnel/datasource/plugin/tidb/jdbc/TidbJdbcDataSourceFactory.java
浏览文件 @
a573a68d
...
...
@@ -43,6 +43,6 @@ public class TidbJdbcDataSourceFactory implements DataSourceFactory {
@Override
public
DataSourceChannel
createChannel
()
{
return
new
TidbJdbcDataSourceChannel
();
return
TidbJdbcDataSourceChannel
.
getInstance
();
}
}
seatunnel-datasource/seatunnel-datasource-plugins/datasource-kafka/src/main/java/org/apache/seatunnel/datasource/plugin/kafka/KafkaDataSourceChannel.java
浏览文件 @
a573a68d
...
...
@@ -41,6 +41,13 @@ import static com.google.common.base.Preconditions.checkArgument;
@Slf4j
public
class
KafkaDataSourceChannel
implements
DataSourceChannel
{
public
static
class
Holder
{
private
static
final
KafkaDataSourceChannel
INSTANCE
=
new
KafkaDataSourceChannel
();
}
public
static
KafkaDataSourceChannel
getInstance
()
{
return
Holder
.
INSTANCE
;
}
private
static
final
String
DATABASE
=
"default"
;
private
static
final
DescribeClusterOptions
DEFAULT_TIMEOUT_OPTIONS
=
...
...
seatunnel-datasource/seatunnel-datasource-plugins/datasource-kafka/src/main/java/org/apache/seatunnel/datasource/plugin/kafka/KafkaDataSourceFactory.java
浏览文件 @
a573a68d
...
...
@@ -53,6 +53,6 @@ public class KafkaDataSourceFactory implements DataSourceFactory {
@Override
public
DataSourceChannel
createChannel
()
{
return
new
KafkaDataSourceChannel
();
return
KafkaDataSourceChannel
.
getInstance
();
}
}
seatunnel-datasource/seatunnel-datasource-plugins/datasource-mqtt/src/main/java/org/apache/seatunnel/datasource/plugin/mqtt/MqttDataSourceChannel.java
浏览文件 @
a573a68d
...
...
@@ -26,6 +26,7 @@ import org.apache.seatunnel.datasource.plugin.api.model.TableField;
import
org.apache.commons.collections4.CollectionUtils
;
import
org.apache.commons.lang3.StringUtils
;
import
org.eclipse.paho.client.mqttv3.MqttClient
;
import
org.eclipse.paho.client.mqttv3.MqttMessage
;
import
com.alibaba.fastjson2.JSONObject
;
...
...
@@ -41,6 +42,14 @@ public class MqttDataSourceChannel implements DataSourceChannel {
private
InitCallback
initCallback
;
public
static
class
Holder
{
private
static
final
MqttDataSourceChannel
INSTANCE
=
new
MqttDataSourceChannel
();
}
public
static
MqttDataSourceChannel
getInstance
()
{
return
Holder
.
INSTANCE
;
}
@Override
public
OptionRule
getDataSourceOptions
(
@NonNull
String
pluginName
)
{
return
MqttOptionRule
.
optionRule
();
...
...
@@ -88,7 +97,11 @@ public class MqttDataSourceChannel implements DataSourceChannel {
try
{
MqttClientService
mqttClient
=
createMqttClient
(
requestParams
);
// just test the connection
mqttClient
.
getMqttClient
().
connect
();
MqttClient
mqttClient1
=
mqttClient
.
getMqttClient
();
if
(
mqttClient1
.
isConnected
())
{
return
true
;
}
mqttClient1
.
connect
();
return
StringUtils
.
isNotEmpty
(
mqttClient
.
getTopic
());
}
catch
(
Exception
ex
)
{
...
...
seatunnel-datasource/seatunnel-datasource-plugins/datasource-mqtt/src/main/java/org/apache/seatunnel/datasource/plugin/mqtt/MqttDataSourceFactory.java
浏览文件 @
a573a68d
...
...
@@ -53,6 +53,6 @@ public class MqttDataSourceFactory implements DataSourceFactory {
@Override
public
DataSourceChannel
createChannel
()
{
return
new
MqttDataSourceChannel
();
return
MqttDataSourceChannel
.
getInstance
();
}
}
seatunnel-datasource/seatunnel-datasource-plugins/datasource-mqtt/src/main/java/org/apache/seatunnel/datasource/plugin/mqtt/MqttOptionRule.java
浏览文件 @
a573a68d
...
...
@@ -53,7 +53,7 @@ public class MqttOptionRule {
.
withDescription
(
"mqtt server password"
);
public
static
OptionRule
optionRule
()
{
return
OptionRule
.
builder
().
required
(
HOST
,
PORT
).
optional
(
USERNAME
,
PASSWORD
).
build
();
return
OptionRule
.
builder
().
required
(
HOST
,
PORT
,
TOPIC
,
USERNAME
,
PASSWORD
).
build
();
}
public
static
OptionRule
metadataRule
()
{
...
...
seatunnel-datasource/seatunnel-datasource-plugins/datasource-mqtt/src/test/java/org/apache/seatunnel/datasource/plugin/mqtt/MqttClientServiceTest.java
0 → 100644
浏览文件 @
a573a68d
package
org
.
apache
.
seatunnel
.
datasource
.
plugin
.
mqtt
;
import
static
org
.
junit
.
jupiter
.
api
.
Assertions
.*;
class
MqttClientServiceTest
{
public
static
void
main
(
String
[]
args
)
throws
Exception
{
InitCallback
initCallback
=
new
InitCallback
();
MqttClientService
mqttClientService
=
new
MqttClientService
(
"172.32.1.83"
,
1833
,
""
,
""
,
"dd"
,
initCallback
);
System
.
out
.
println
(
mqttClientService
);
}
}
seatunnel-datasource/seatunnel-datasource-plugins/datasource-mysql-cdc/src/main/java/org/apache/seatunnel/datasource/plugin/cdc/mysql/MysqlCDCDataSourceChannel.java
浏览文件 @
a573a68d
...
...
@@ -36,6 +36,14 @@ public class MysqlCDCDataSourceChannel implements DataSourceChannelExt {
public
static
final
Set
<
String
>
MYSQL_SYSTEM_DATABASES
=
Sets
.
newHashSet
(
"information_schema"
,
"mysql"
,
"performance_schema"
,
"sys"
);
public
static
class
Holder
{
private
static
final
MysqlCDCDataSourceChannel
INSTANCE
=
new
MysqlCDCDataSourceChannel
();
}
public
static
MysqlCDCDataSourceChannel
getInstance
()
{
return
Holder
.
INSTANCE
;
}
@Override
public
boolean
canAbleGetSchema
()
{
return
true
;
...
...
seatunnel-datasource/seatunnel-datasource-plugins/datasource-mysql-cdc/src/main/java/org/apache/seatunnel/datasource/plugin/cdc/mysql/MysqlCDCDataSourceFactory.java
浏览文件 @
a573a68d
...
...
@@ -41,6 +41,6 @@ public class MysqlCDCDataSourceFactory implements DataSourceFactory {
@Override
public
DataSourceChannel
createChannel
()
{
return
new
MysqlCDCDataSourceChannel
();
return
MysqlCDCDataSourceChannel
.
getInstance
();
}
}
seatunnel-datasource/seatunnel-datasource-plugins/datasource-opcua/src/main/java/org/apache/seatunnel/datasource/plugin/opcua/OpcUaClientService.java
浏览文件 @
a573a68d
...
...
@@ -26,6 +26,8 @@ import lombok.Data;
import
java.nio.file.Files
;
import
java.nio.file.Path
;
import
java.nio.file.Paths
;
import
java.time.LocalDateTime
;
import
java.time.ZoneOffset
;
import
java.util.ArrayList
;
import
java.util.List
;
import
java.util.Objects
;
...
...
@@ -153,7 +155,21 @@ public class OpcUaClientService {
// 标识符
String
id
=
String
.
valueOf
(
nodeId
.
getIdentifier
());
System
.
out
.
println
(
id
+
": "
+
value
.
getValue
().
getValue
());
return
(
Object
[])
value
.
getValue
().
getValue
();
Object
ob
=
value
.
getValue
().
getValue
();
Object
[]
objects
=
new
Object
[]
{
null
};
if
(
ob
!=
null
)
{
if
(
ob
instanceof
DateTime
)
{
LocalDateTime
localDateTime
=
((
DateTime
)
ob
)
.
getJavaInstant
()
.
atOffset
(
ZoneOffset
.
ofHours
(
8
))
.
toLocalDateTime
();
objects
=
new
Object
[]
{
localDateTime
+
""
};
}
else
{
objects
=
new
Object
[]
{
ob
+
""
};
}
}
return
objects
;
}
public
Object
[]
readNodeValue
(
int
namespaceIndex
,
Integer
identifier
)
throws
Exception
{
...
...
@@ -169,9 +185,21 @@ public class OpcUaClientService {
// 标识符
String
id
=
String
.
valueOf
(
nodeId
.
getIdentifier
());
System
.
out
.
println
(
id
+
": "
+
value
.
getValue
().
getValue
());
Object
[]
value1
=
(
Object
[])
value
.
getValue
().
getValue
();
return
value1
;
Object
ob
=
value
.
getValue
().
getValue
();
Object
[]
objects
=
new
Object
[]
{
null
};
if
(
ob
!=
null
)
{
if
(
ob
instanceof
DateTime
)
{
LocalDateTime
localDateTime
=
((
DateTime
)
ob
)
.
getJavaInstant
()
.
atOffset
(
ZoneOffset
.
ofHours
(
8
))
.
toLocalDateTime
();
objects
=
new
Object
[]
{
localDateTime
+
""
};
}
else
{
objects
=
new
Object
[]
{
ob
+
""
};
}
}
return
objects
;
}
/**
...
...
@@ -191,7 +219,21 @@ public class OpcUaClientService {
String
id
=
String
.
valueOf
(
nodeId
.
getIdentifier
());
System
.
out
.
println
(
nodeId
);
System
.
out
.
println
(
id
+
": "
+
value
.
getValue
().
getValue
());
return
(
Object
[])
value
.
getValue
().
getValue
();
Object
ob
=
value
.
getValue
().
getValue
();
Object
[]
objects
=
new
Object
[]
{
null
};
if
(
ob
!=
null
)
{
if
(
ob
instanceof
DateTime
)
{
LocalDateTime
localDateTime
=
((
DateTime
)
ob
)
.
getJavaInstant
()
.
atOffset
(
ZoneOffset
.
ofHours
(
8
))
.
toLocalDateTime
();
objects
=
new
Object
[]
{
localDateTime
+
""
};
}
else
{
objects
=
new
Object
[]
{
ob
+
""
};
}
}
return
objects
;
}
/**
...
...
seatunnel-datasource/seatunnel-datasource-plugins/datasource-opcua/src/main/java/org/apache/seatunnel/datasource/plugin/opcua/OpcuaDataSourceChannel.java
浏览文件 @
a573a68d
...
...
@@ -40,6 +40,14 @@ public class OpcuaDataSourceChannel implements DataSourceChannel {
private
static
final
String
DATABASE
=
"default"
;
public
static
class
Holder
{
private
static
final
OpcuaDataSourceChannel
INSTANCE
=
new
OpcuaDataSourceChannel
();
}
public
static
OpcuaDataSourceChannel
getInstance
()
{
return
Holder
.
INSTANCE
;
}
@Override
public
OptionRule
getDataSourceOptions
(
@NonNull
String
pluginName
)
{
return
OpcuaOptionRule
.
optionRule
();
...
...
@@ -178,7 +186,7 @@ public class OpcuaDataSourceChannel implements DataSourceChannel {
}
return
tableFields
;
}
catch
(
Exception
e
)
{
//不是json
//
不是json
TableField
tableField
=
new
TableField
();
tableField
.
setName
(
nodeId
.
getIdentifier
()
+
""
);
tableField
.
setType
(
"String"
);
...
...
@@ -192,8 +200,6 @@ public class OpcuaDataSourceChannel implements DataSourceChannel {
}
}
@Override
public
Map
<
String
,
List
<
TableField
>>
getTableFields
(
@NonNull
String
pluginName
,
...
...
seatunnel-datasource/seatunnel-datasource-plugins/datasource-opcua/src/main/java/org/apache/seatunnel/datasource/plugin/opcua/OpcuaDataSourceFactory.java
浏览文件 @
a573a68d
...
...
@@ -53,6 +53,6 @@ public class OpcuaDataSourceFactory implements DataSourceFactory {
@Override
public
DataSourceChannel
createChannel
()
{
return
new
OpcuaDataSourceChannel
();
return
OpcuaDataSourceChannel
.
getInstance
();
}
}
seatunnel-datasource/seatunnel-datasource-plugins/datasource-opcua/src/main/java/org/apache/seatunnel/datasource/plugin/opcua/OpcuaOptionRule.java
浏览文件 @
a573a68d
...
...
@@ -42,10 +42,10 @@ public class OpcuaOptionRule {
Options
.
key
(
"type"
).
stringType
().
defaultValue
(
"int"
).
withDescription
(
"type"
);
public
static
OptionRule
optionRule
()
{
return
OptionRule
.
builder
().
required
(
HOST
,
PORT
,
SUFFIX
).
optional
(
TYPE
).
build
();
return
OptionRule
.
builder
().
required
(
HOST
,
PORT
,
SUFFIX
,
NS
,
ID
,
TYPE
).
build
();
}
public
static
OptionRule
metadataRule
()
{
return
OptionRule
.
builder
().
required
(
NS
,
ID
).
optional
(
TYPE
).
build
();
return
OptionRule
.
builder
().
required
(
NS
,
ID
,
TYPE
).
build
();
}
}
seatunnel-datasource/seatunnel-datasource-plugins/datasource-s3/pom.xml
浏览文件 @
a573a68d
...
...
@@ -67,11 +67,67 @@
<groupId>
org.apache.logging.log4j
</groupId>
<artifactId>
log4j-1.2-api
</artifactId>
</exclusion>
<exclusion>
<groupId>
org.apache.hadoop
</groupId>
<artifactId>
hadoop-common
</artifactId>
</exclusion>
<exclusion>
<groupId>
org.apache.hadoop
</groupId>
<artifactId>
hadoop-client
</artifactId>
</exclusion>
<exclusion>
<groupId>
org.slf4j
</groupId>
<artifactId>
log4j-over-slf4j
</artifactId>
</exclusion>
<exclusion>
<groupId>
org.slf4j
</groupId>
<artifactId>
slf4j-reload4j
</artifactId>
</exclusion>
</exclusions>
</dependency>
<!-- <dependency>-->
<!-- <groupId>org.slf4j</groupId>-->
<!-- <artifactId>slf4j-reload4j</artifactId>-->
<!-- <version>1.7.35</version>-->
<!-- <scope>test</scope>-->
<!-- </dependency>-->
<dependency>
<groupId>
org.apache.hadoop
</groupId>
<artifactId>
hadoop-common
</artifactId>
<version>
3.3.5
</version>
<exclusions>
<exclusion>
<groupId>
org.slf4j
</groupId>
<artifactId>
slf4j-reload4j
</artifactId>
</exclusion>
<exclusion>
<groupId>
org.slf4j
</groupId>
<artifactId>
log4j-over-slf4j
</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>
org.apache.hadoop
</groupId>
<artifactId>
hadoop-client
</artifactId>
<version>
3.3.5
</version>
<exclusions>
<exclusion>
<groupId>
org.slf4j
</groupId>
<artifactId>
slf4j-reload4j
</artifactId>
</exclusion>
<exclusion>
<groupId>
org.slf4j
</groupId>
<artifactId>
log4j-over-slf4j
</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>
org.apache.hadoop
</groupId>
<artifactId>
hadoop-aws
</artifactId>
<version>
3.3.5
</version>
</dependency>
<dependency>
<groupId>
com.amazonaws
</groupId>
...
...
seatunnel-datasource/seatunnel-datasource-plugins/datasource-s3/src/main/java/org/apache/seatunnel/datasource/plugin/s3/HadoopS3AConfiguration.java
浏览文件 @
a573a68d
...
...
@@ -26,8 +26,6 @@ import lombok.extern.slf4j.Slf4j;
import
java.util.Map
;
import
static
org
.
apache
.
hadoop
.
fs
.
FileSystem
.
FS_DEFAULT_NAME_KEY
;
@Slf4j
public
class
HadoopS3AConfiguration
{
...
...
@@ -58,7 +56,7 @@ public class HadoopS3AConfiguration {
}
String
fsImpl
=
protocol
.
equals
(
S3A_PROTOCOL
)
?
HDFS_S3A_IMPL
:
HDFS_S3N_IMPL
;
Configuration
hadoopConf
=
new
Configuration
();
hadoopConf
.
set
(
FS_DEFAULT_NAME_KEY
,
bucket
);
hadoopConf
.
set
(
"fs.defaut.name"
,
bucket
);
hadoopConf
.
set
(
S3OptionRule
.
FS_S3A_ENDPOINT
.
key
(),
s3Options
.
get
(
S3OptionRule
.
FS_S3A_ENDPOINT
.
key
()));
...
...
seatunnel-datasource/seatunnel-datasource-plugins/datasource-s3/src/main/java/org/apache/seatunnel/datasource/plugin/s3/S3DataSourceFactory.java
浏览文件 @
a573a68d
...
...
@@ -53,6 +53,6 @@ public class S3DataSourceFactory implements DataSourceFactory {
@Override
public
DataSourceChannel
createChannel
()
{
return
new
S3DatasourceChannel
();
return
S3DatasourceChannel
.
getInstance
();
}
}
seatunnel-datasource/seatunnel-datasource-plugins/datasource-s3/src/main/java/org/apache/seatunnel/datasource/plugin/s3/S3DatasourceChannel.java
浏览文件 @
a573a68d
...
...
@@ -33,6 +33,14 @@ import java.util.List;
import
java.util.Map
;
public
class
S3DatasourceChannel
implements
DataSourceChannel
{
public
static
class
Holder
{
private
static
final
S3DatasourceChannel
INSTANCE
=
new
S3DatasourceChannel
();
}
public
static
S3DatasourceChannel
getInstance
()
{
return
Holder
.
INSTANCE
;
}
@Override
public
OptionRule
getDataSourceOptions
(
@NonNull
String
pluginName
)
{
return
S3OptionRule
.
optionRule
();
...
...
seatunnel-datasource/seatunnel-datasource-plugins/datasource-s3/src/test/java/org/apache/seatunnel/datasource/plugin/s3/HadoopS3AConfigurationTest.java
0 → 100644
浏览文件 @
a573a68d
package
org
.
apache
.
seatunnel
.
datasource
.
plugin
.
s3
;
import
org.apache.hadoop.conf.Configuration
;
import
java.util.HashMap
;
import
java.util.Map
;
import
static
org
.
junit
.
jupiter
.
api
.
Assertions
.*;
class
HadoopS3AConfigurationTest
{
public
static
void
main
(
String
[]
args
)
{
Map
<
String
,
String
>
map
=
new
HashMap
<>();
map
.
put
(
"groupName"
,
"测试s3适配器1"
);
map
.
put
(
"adapterId"
,
"1714559639070314496"
);
map
.
put
(
"bucket"
,
"hdfs://192.168.1.174:9001/s3atests3"
);
map
.
put
(
"fs.s3a.endpoint"
,
"http://192.168.1.174:9001"
);
map
.
put
(
"fs.s3a.connection.ssl.enabled"
,
"false"
);
map
.
put
(
"fs.s3a.aws.credentials.provider"
,
"org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider"
);
map
.
put
(
"hadoop_s3_properties"
,
" "
);
map
.
put
(
"access_key"
,
"zyminio"
);
map
.
put
(
"secret_key"
,
"zysoft123"
);
Configuration
configuration
=
HadoopS3AConfiguration
.
getConfiguration
(
map
);
System
.
out
.
println
(
configuration
);
}
}
seatunnel-datasource/seatunnel-datasource-plugins/datasource-s3redshift/pom.xml
浏览文件 @
a573a68d
...
...
@@ -34,14 +34,65 @@
<dependency>
<groupId>
org.apache.seatunnel
</groupId>
<artifactId>
seatunnel-hadoop3-3.1.4-uber
</artifactId>
<version>
2.3.
2
</version>
<version>
2.3.
3
</version>
<exclusions>
<exclusion>
<groupId>
org.apache.avro
</groupId>
<artifactId>
avro
</artifactId>
</exclusion>
<exclusion>
<groupId>
org.apache.hadoop
</groupId>
<artifactId>
hadoop-common
</artifactId>
</exclusion>
<exclusion>
<groupId>
org.slf4j
</groupId>
<artifactId>
slf4j-reload4j
</artifactId>
</exclusion>
<exclusion>
<groupId>
org.apache.hadoop
</groupId>
<artifactId>
hadoop-client
</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>
org.slf4j
</groupId>
<artifactId>
slf4j-reload4j
</artifactId>
<version>
1.7.35
</version>
<scope>
test
</scope>
</dependency>
<dependency>
<groupId>
org.apache.hadoop
</groupId>
<artifactId>
hadoop-client
</artifactId>
<version>
3.3.5
</version>
<exclusions>
<exclusion>
<groupId>
org.slf4j
</groupId>
<artifactId>
slf4j-reload4j
</artifactId>
</exclusion>
<exclusion>
<groupId>
org.slf4j
</groupId>
<artifactId>
log4j-over-slf4j
</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>
org.apache.hadoop
</groupId>
<artifactId>
hadoop-common
</artifactId>
<version>
3.3.5
</version>
<exclusions>
<exclusion>
<groupId>
org.slf4j
</groupId>
<artifactId>
slf4j-reload4j
</artifactId>
</exclusion>
<exclusion>
<groupId>
org.slf4j
</groupId>
<artifactId>
log4j-over-slf4j
</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>
com.amazon.redshift
</groupId>
<artifactId>
redshift-jdbc42
</artifactId>
...
...
@@ -50,7 +101,7 @@
<dependency>
<groupId>
org.apache.hadoop
</groupId>
<artifactId>
hadoop-aws
</artifactId>
<version>
${hadoop-aws.version}
</version>
<version>
3.3.5
</version>
<exclusions>
<exclusion>
<groupId>
jdk.tools
</groupId>
...
...
seatunnel-datasource/seatunnel-datasource-plugins/datasource-s3redshift/src/main/java/org/apache/seatunnel/datasource/plugin/redshift/s3/S3RedshiftDataSourceChannel.java
浏览文件 @
a573a68d
...
...
@@ -47,6 +47,14 @@ import java.util.Set;
@Slf4j
public
class
S3RedshiftDataSourceChannel
implements
DataSourceChannel
{
public
static
class
Holder
{
private
static
final
S3RedshiftDataSourceChannel
INSTANCE
=
new
S3RedshiftDataSourceChannel
();
}
public
static
S3RedshiftDataSourceChannel
getInstance
()
{
return
Holder
.
INSTANCE
;
}
@Override
public
OptionRule
getDataSourceOptions
(
@NonNull
String
pluginName
)
{
...
...
seatunnel-datasource/seatunnel-datasource-plugins/datasource-s3redshift/src/main/java/org/apache/seatunnel/datasource/plugin/redshift/s3/S3RedshiftDataSourceFactory.java
浏览文件 @
a573a68d
...
...
@@ -51,6 +51,6 @@ public class S3RedshiftDataSourceFactory implements DataSourceFactory {
@Override
public
DataSourceChannel
createChannel
()
{
return
new
S3RedshiftDataSourceChannel
();
return
S3RedshiftDataSourceChannel
.
getInstance
();
}
}
seatunnel-datasource/seatunnel-datasource-plugins/datasource-sqlserver-cdc/src/main/java/org/apache/seatunnel/datasource/plugin/cdc/sqlserver/SqlServerCDCDataSourceChannel.java
浏览文件 @
a573a68d
...
...
@@ -35,6 +35,14 @@ import java.util.*;
@Slf4j
public
class
SqlServerCDCDataSourceChannel
implements
DataSourceChannelExt
{
public
static
class
Holder
{
private
static
final
SqlServerCDCDataSourceChannel
INSTANCE
=
new
SqlServerCDCDataSourceChannel
();
}
public
static
SqlServerCDCDataSourceChannel
getInstance
()
{
return
Holder
.
INSTANCE
;
}
public
static
final
Set
<
String
>
MYSQL_SYSTEM_DATABASES
=
Sets
.
newHashSet
(
"master"
,
"tempdb"
,
"model"
,
"msdb"
);
...
...
seatunnel-datasource/seatunnel-datasource-plugins/datasource-sqlserver-cdc/src/main/java/org/apache/seatunnel/datasource/plugin/cdc/sqlserver/SqlServerCDCDataSourceFactory.java
浏览文件 @
a573a68d
...
...
@@ -42,6 +42,6 @@ public class SqlServerCDCDataSourceFactory implements DataSourceFactory {
@Override
public
DataSourceChannel
createChannel
()
{
return
new
SqlServerCDCDataSourceChannel
();
return
SqlServerCDCDataSourceChannel
.
getInstance
();
}
}
seatunnel-datasource/seatunnel-datasource-plugins/datasource-starrocks/src/main/java/org/apache/seatunnel/datasource/plugin/starrocks/StarRocksDataSourceChannel.java
浏览文件 @
a573a68d
...
...
@@ -38,6 +38,13 @@ import java.util.List;
import
java.util.Map
;
public
class
StarRocksDataSourceChannel
implements
DataSourceChannel
{
public
static
class
Holder
{
private
static
final
StarRocksDataSourceChannel
INSTANCE
=
new
StarRocksDataSourceChannel
();
}
public
static
StarRocksDataSourceChannel
getInstance
()
{
return
Holder
.
INSTANCE
;
}
private
static
final
Logger
LOGGER
=
LoggerFactory
.
getLogger
(
StarRocksDataSourceChannel
.
class
);
...
...
seatunnel-datasource/seatunnel-datasource-plugins/datasource-starrocks/src/main/java/org/apache/seatunnel/datasource/plugin/starrocks/StarRocksDataSourceFactory.java
浏览文件 @
a573a68d
...
...
@@ -40,6 +40,6 @@ public class StarRocksDataSourceFactory implements DataSourceFactory {
@Override
public
DataSourceChannel
createChannel
()
{
return
new
StarRocksDataSourceChannel
();
return
StarRocksDataSourceChannel
.
getInstance
();
}
}
编写
预览
Markdown
格式
0%
重试
或
添加新文件
添加附件
取消
您添加了
0
人
到此讨论。请谨慎行事。
请先完成此评论的编辑!
取消
请
注册
或者
登录
后发表评论