Skip to content
项目
群组
代码片段
帮助
当前项目
正在载入...
登录 / 注册
切换导航面板
S
seatunnel-web
项目
项目
详情
活动
周期分析
仓库
仓库
文件
提交
分支
标签
贡献者
图表
比较
统计图
议题
0
议题
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
CI / CD
CI / CD
流水线
作业
日程
统计图
Wiki
Wiki
代码片段
代码片段
成员
成员
折叠边栏
关闭边栏
活动
图像
聊天
创建新问题
作业
提交
问题看板
Open sidebar
宋勇
seatunnel-web
Commits
4ed9e844
提交
4ed9e844
authored
12月 07, 2023
作者:
李纤
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
修改minio连接
上级
da025db5
隐藏空白字符变更
内嵌
并排
正在显示
4 个修改的文件
包含
33 行增加
和
41 行删除
+33
-41
HadoopS3AConfiguration.java
...eatunnel/datasource/plugin/s3/HadoopS3AConfiguration.java
+4
-19
S3ClientService.java
...pache/seatunnel/datasource/plugin/s3/S3ClientService.java
+13
-10
S3DataSourceFactory.java
...e/seatunnel/datasource/plugin/s3/S3DataSourceFactory.java
+3
-2
S3DatasourceChannel.java
...e/seatunnel/datasource/plugin/s3/S3DatasourceChannel.java
+13
-10
没有找到文件。
seatunnel-datasource/seatunnel-datasource-plugins/datasource-s3/src/main/java/org/apache/seatunnel/datasource/plugin/s3/HadoopS3AConfiguration.java
浏览文件 @
4ed9e844
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package
org
.
apache
.
seatunnel
.
datasource
.
plugin
.
s3
;
import
lombok.extern.slf4j.Slf4j
;
import
org.apache.hadoop.conf.Configuration
;
import
org.apache.seatunnel.shade.com.typesafe.config.Config
;
import
org.apache.seatunnel.shade.com.typesafe.config.ConfigFactory
;
import
org.apache.hadoop.conf.Configuration
;
import
lombok.extern.slf4j.Slf4j
;
import
java.util.Map
;
@Slf4j
...
...
seatunnel-datasource/seatunnel-datasource-plugins/datasource-s3/src/main/java/org/apache/seatunnel/datasource/plugin/s3/S3ClientService.java
浏览文件 @
4ed9e844
...
...
@@ -21,7 +21,8 @@ public class S3ClientService {
String
username
,
String
password
,
String
bucket
,
Integer
port
)
throws
MinioException
{
Integer
port
)
throws
MinioException
{
this
.
ENDPOINT
=
endpoint
;
this
.
PROVIDER
=
provider
;
this
.
USERNAME
=
username
;
...
...
@@ -32,7 +33,6 @@ public class S3ClientService {
}
/**
*
* @param endpoint
* @param provider
* @param username
...
...
@@ -41,15 +41,18 @@ public class S3ClientService {
* @param port
* @throws MinioException
*/
public
void
setMinioClient
(
String
endpoint
,
String
provider
,
String
username
,
String
password
,
String
bucket
,
Integer
port
)
String
endpoint
,
String
provider
,
String
username
,
String
password
,
String
bucket
,
Integer
port
)
throws
MinioException
{
minioClient
=
new
MinioClient
.
Builder
()
.
endpoint
(
endpoint
,
port
,
true
)
.
credentials
(
username
,
password
)
.
build
();
minioClient
=
new
MinioClient
.
Builder
(
)
.
endpoint
(
endpoint
,
port
,
true
)
.
credentials
(
username
,
password
)
.
build
();
}
}
seatunnel-datasource/seatunnel-datasource-plugins/datasource-s3/src/main/java/org/apache/seatunnel/datasource/plugin/s3/S3DataSourceFactory.java
浏览文件 @
4ed9e844
...
...
@@ -17,13 +17,14 @@
package
org
.
apache
.
seatunnel
.
datasource
.
plugin
.
s3
;
import
com.google.auto.service.AutoService
;
import
com.google.common.collect.Sets
;
import
org.apache.seatunnel.datasource.plugin.api.DataSourceChannel
;
import
org.apache.seatunnel.datasource.plugin.api.DataSourceFactory
;
import
org.apache.seatunnel.datasource.plugin.api.DataSourcePluginInfo
;
import
org.apache.seatunnel.datasource.plugin.api.DatasourcePluginTypeEnum
;
import
com.google.auto.service.AutoService
;
import
com.google.common.collect.Sets
;
import
java.util.Set
;
@AutoService
(
DataSourceFactory
.
class
)
...
...
seatunnel-datasource/seatunnel-datasource-plugins/datasource-s3/src/main/java/org/apache/seatunnel/datasource/plugin/s3/S3DatasourceChannel.java
浏览文件 @
4ed9e844
...
...
@@ -17,13 +17,14 @@
package
org
.
apache
.
seatunnel
.
datasource
.
plugin
.
s3
;
import
lombok.NonNull
;
import
org.apache.seatunnel.api.configuration.util.OptionRule
;
import
org.apache.seatunnel.common.utils.SeaTunnelException
;
import
org.apache.seatunnel.datasource.plugin.api.DataSourceChannel
;
import
org.apache.seatunnel.datasource.plugin.api.DataSourcePluginException
;
import
org.apache.seatunnel.datasource.plugin.api.model.TableField
;
import
lombok.NonNull
;
import
java.util.List
;
import
java.util.Map
;
...
...
@@ -74,14 +75,15 @@ public class S3DatasourceChannel implements DataSourceChannel {
throw
new
DataSourcePluginException
(
"check s3 connectivity failed, "
+
ex
.
getMessage
(),
ex
);
}
// try (FileSystem fs = FileSystem.get(conf)) {
//// fs.listStatus(new Path("/"));
//
// return true;
// } catch (IOException e) {
// throw new DataSourcePluginException(
// String.format("check s3 connectivity failed, config is: %s", requestParams), e);
// }
// try (FileSystem fs = FileSystem.get(conf)) {
//// fs.listStatus(new Path("/"));
//
// return true;
// } catch (IOException e) {
// throw new DataSourcePluginException(
// String.format("check s3 connectivity failed, config is: %s",
// requestParams), e);
// }
}
@Override
...
...
@@ -112,7 +114,8 @@ public class S3DatasourceChannel implements DataSourceChannel {
String
password
=
requestParams
.
get
(
"secret_key"
)
+
""
;
String
bucket
=
requestParams
.
get
(
"bucket"
)
+
""
;
try
{
S3ClientService
s3ClientService
=
new
S3ClientService
(
endpoint
,
provider
,
username
,
password
,
bucket
,
port
);
S3ClientService
s3ClientService
=
new
S3ClientService
(
endpoint
,
provider
,
username
,
password
,
bucket
,
port
);
return
s3ClientService
;
}
catch
(
Exception
e
)
{
...
...
编写
预览
Markdown
格式
0%
重试
或
添加新文件
添加附件
取消
您添加了
0
人
到此讨论。请谨慎行事。
请先完成此评论的编辑!
取消
请
注册
或者
登录
后发表评论