【异地备份】如何将MinIO的S3服务挂载为本地磁盘路径(s3fs-fuse、rclone)

教育   2024-11-23 06:32   甘肃  

简介

要将S3挂载为本地路径,可以使用一些工具,例如 s3fs-fuserclone。这两个工具都可以将 Amazon S3(或兼容的 S3 服务)存储桶挂载为 Linux 文件系统,从而像本地文件一样访问。

Linux方法 1:使用 s3fs-fuse

https://github.com/s3fs-fuse/s3fs-fuse

s3fs-fuse 可以让你将 S3 存储桶挂载为文件系统。

 1sudo yum install s3fs-fuse -y
2
3echo "12345678:12345678" > ~/.passwd-s3fs
4chmod 600 ~/.passwd-s3fs
5
6
7mkdir -p /s3fs_dbbk
8s3fs dbbk /s3fs_dbbk -o passwd_file=~/.passwd-s3fs -o url=http://192.16.7.162:9001 -o use_path_request_style  -o allow_other
9
10
11fusermount -uz /s3fs_dbbk
12pkill -9 s3fs
13
14
15
16[gpadmin@mdw s3fs]$ df -T | grep s3
17s3fs                    fuse.s3fs 70368744161280         0 70368744161280   0% /s3fs
18[gpadmin@mdw s3fs]$ ps -ef|grep s3
19root     2440517       1  0 19:54 ?        00:00:00 s3fs dbbk /s3fs -o passwd_file=/root/.passwd-s3fs -o url=http://192.16.7.162:9001 -o use_path_request_style -o allow_other
20gpadmin  2446635 2440693  0 19:56 pts/10   00:00:00 grep --color=auto s3
21[gpadmin@mdw s3fs]$ 

Linux方法 2:使用 rclone

https://rclone.org/

https://downloads.rclone.org/v1.68.1/rclone-v1.68.1-windows-amd64.zip

兼容性:https://go.dev/wiki/MinimumRequirements#linuxlinux

Kernel version 2.6.32 or later. [This depends on architecture though, we need to have specific builder for this.] Linux/ARMv5 requires much newer kernels, at least v3.1 (for __kuser_cmpxchg64).

We don’t support CentOS 5. The kernel is too old (2.6.18).

For little-endian MIPS64, kernel version 4.1 is known to fail, and 4.8 works.

If you are using tinyconfig (e.g. make tinyconfig) for embedded  systems, you will also almost certainly enable printk in the kernel as  well as a console; we will not include those generic options here. For  Go, you must also enable CONFIG_FUTEX and CONFIG_EPOLL.

On arm64, an out of date (lower than version 2.33) ld.gold may cause shared library tests to fail (see https://github.com/golang/go/issues/28334).

rclone 是一个强大的命令行工具,支持多种云存储服务,包括 S3。可以用它将 S3 存储桶挂载为本地文件系统。

 1wget https://downloads.rclone.org/v1.68.1/rclone-v1.68.1-linux-amd64.zip 
2unzip rclone-v1.68.1-linux-amd64.zip  -d /usr/local/
3ln -s /usr/local/rclone-v1.68.1-linux-amd64 /usr/local/rclone
4ln -s /usr/local/rclone/rclone /usr/local/bin
5
6# /usr/local/rclone/rclone config
7
8cat > ~/.config/rclone/rclone.conf <<"EOF"
9[mys3]
10type = s3
11provider = Minio
12access_key_id = minioadmin
13secret_access_key = minioadmin
14region = us-east-1
15endpoint = http://192.16.7.162:9001
16acl = private
17EOF
18
19
20rclone lsd mys3: -vv
21
22
23
24mkdir ~/s3bucket
25ln -s /usr/bin/fusermount /usr/bin/fusermount3
26
27/usr/local/rclone/rclone mount  mys3:pgbk  ~/s3bucket --allow-other &
28
29
30rclone config show
31
32pkill -9 rclone
33umount ~/s3bucket

Windows挂载S3服务端为本地盘--使用rclone

https://github.com/winfsp/winfsp  ,Windows File System Proxy - FUSE for Windows

https://downloads.rclone.org/

https://downloads.rclone.org/v1.68.1/rclone-v1.68.1-windows-amd64.zip

兼容性:https://go.dev/wiki/MinimumRequirements#linuxlinux

For Go 1.10: Windows XP (w/ Service Pack 3) or higher.

For Go 1.11 and later: Windows 7 and higher or Windows Server 2008 R2 and higher. We test on Windows Server 2008 R2, 2012 R2, and 2016, which are roughly Windows 7, Windows 8.1, and Windows 10.

For Go 1.21 and later: Windows 10 and higher or Windows Server 2016 and higher.

安装winfsp-2.0.23075.msi,解压rclone-current-windows-amd64.zip即可。

 1https://github.com/winfsp/winfsp/releases/download/v2.0/winfsp-2.0.23075.msi
2https://downloads.rclone.org/rclone-current-windows-amd64.zip
3
4rclone.exe config
5
6-- 默认配置文件: %AppData%\rclone\rclone.conf ,也可以自定义文件位置,然后启动时通过--config "D:\rclone\rclone.conf" 指定
7[mys3]
8type = s3
9provider = Minio
10access_key_id = 12345678
11secret_access_key = 12345678
12region = ""
13endpoint = http://192.16.7.162:9001
14acl = private
15
16
17rclone.exe  mount  mys3:gpbk  h:\  --vfs-cache-mode writes
18rclone.exe  mount  mys3:gpbk  h:\  --vfs-cache-mode writes --config "D:\rclone\rclone.conf"
19
20# 后台运行
21start /b rclone.exe  mount  mys3:gpbk  h:\  --vfs-cache-mode writes --config "D:\rclone\rclone.conf"
22
23gpbk为buket名称,
24就可以看到本地会挂载一个H盘,名称为mys3_gpbk
25
26
27# bat脚本放入开机启动 C:\ProgramData\Microsoft\Windows\Start Menu\Programs\StartUp
28shell:Common Startup

总结

1、使用s3fs-fuse要比rclone简单的多,但是s3fs-fuse不支持windows系统



AiDBA
【PostgreSQL培训认证】【Oracle OCP、OCM、高可用(RAC+DG+OGG)培训认证】【MySQL OCP培训认证】【GreenPlum培训】【SQL Server培训】官网:www.dbaup.com,学习不止数据库
 最新文章