astelm/minio
1
0
Fork 0
mirror of https://github.com/pgsty/minio.git synced 2026-07-21 05:00:22 +03:00
Code Issues Packages Projects Releases Wiki Activity

cleanup markdown docs across multiple files (#14296)

Browse Source 
enable markdown-linter
This commit is contained in:
Harshavardhana
2022-02-11 16:51:25 -08:00
committed by GitHub
parent 2c0f121550
commit e3e0532613
71 changed files with 1023 additions and 595 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/shared-backend/DESIGN.md
+32 -30
View File
@@ -1,15 +1,12 @@
Introduction [![Slack](https://slack.min.io/slack?type=svg)](https://slack.min.io)
------------
# Introduction [![Slack](https://slack.min.io/slack?type=svg)](https://slack.min.io)
This feature allows MinIO to serve a shared NAS drive across multiple MinIO instances. There are no special configuration changes required to enable this feature. Access to files stored on NAS volume are locked and synchronized by default.
Motivation
----------
## Motivation
Since MinIO instances serve the purpose of a single tenant there is an increasing requirement where users want to run multiple MinIO instances on a same backend which is managed by an existing NAS (NFS, GlusterFS, Other distributed filesystems) rather than a local disk. This feature is implemented also with minimal disruption in mind for the user and overall UI.
Restrictions
------------
## Restrictions
* A PutObject() is blocked and waits if another GetObject() is in progress.
* A CompleteMultipartUpload() is blocked and waits if another PutObject() or GetObject() is in progress.
@@ -24,11 +21,13 @@ Running MinIO instances on shared backend is no different than running on a stan
Example 1: Start MinIO instance on a shared backend mounted and available at `/path/to/nfs-volume`.
On linux server1
```shell
minio gateway nas /path/to/nfs-volume
```
On linux server2
```shell
minio gateway nas /path/to/nfs-volume
```
@@ -38,11 +37,13 @@ minio gateway nas /path/to/nfs-volume
Example 1: Start MinIO instance on a shared backend mounted and available at `\\remote-server\cifs`.
On windows server1
```cmd
minio.exe gateway nas \\remote-server\cifs\data
```
On windows server2
```cmd
minio.exe gateway nas \\remote-server\cifs\data
```
@@ -50,47 +51,48 @@ minio.exe gateway nas \\remote-server\cifs\data
Alternatively if `\\remote-server\cifs` is mounted as `D:\` drive.
On windows server1
```cmd
minio.exe gateway nas D:\data
```
On windows server2
```cmd
minio.exe gateway nas D:\data
```
Architecture
------------------
## Architecture
## POSIX/Win32 Locks
### POSIX/Win32 Locks
### Lock process
#### Lock process
With in the same MinIO instance locking is handled by existing in-memory namespace locks (**sync.RWMutex** et. al). To synchronize locks between many MinIO instances we leverage POSIX `fcntl()` locks on Unixes and on Windows `LockFileEx()` Win32 API. Requesting write lock block if there are any read locks held by neighboring MinIO instance on the same path. So does the read lock if there are any active write locks in-progress.
### Unlock process
#### Unlock process
Unlocking happens on filesystems locks by just closing the file descriptor (fd) which was initially requested for lock operation. Closing the fd tells the kernel to relinquish all the locks held on the path by the current process. This gets trickier when there are many readers on the same path by the same process, it would mean that closing an fd relinquishes locks for all concurrent readers as well. To properly manage this situation a simple fd reference count is implemented, the same fd is shared between many readers. When readers start closing on the fd we start reducing the reference count, once reference count has reached zero we can be sure that there are no more readers active. So we proceed and close the underlying file descriptor which would relinquish the read lock held on the path.
This doesn't apply for the writes because there is always one writer and many readers for any unique object.
## Handling Concurrency.
### Handling Concurrency
An example here shows how the contention is handled with GetObject().
GetObject() holds a read lock on `fs.json`.
```go
fsMetaPath := pathJoin(fs.fsPath, minioMetaBucket, bucketMetaPrefix, bucket, object, fsMetaJSONFile)
rlk, err := fs.rwPool.Open(fsMetaPath)
if err != nil {
return toObjectErr(err, bucket, object)
}
defer rlk.Close()
fsMetaPath := pathJoin(fs.fsPath, minioMetaBucket, bucketMetaPrefix, bucket, object, fsMetaJSONFile)
rlk, err := fs.rwPool.Open(fsMetaPath)
if err != nil {
return toObjectErr(err, bucket, object)
}
defer rlk.Close()
... you can perform other operations here ...
_, err = io.Copy(writer, reader)
_, err = io.Copy(writer, reader)
... after successful copy operation unlocks the read lock ...
```
@@ -98,19 +100,19 @@ GetObject() holds a read lock on `fs.json`.
A concurrent PutObject is requested on the same object, PutObject() attempts a write lock on `fs.json`.
```go
fsMetaPath := pathJoin(fs.fsPath, minioMetaBucket, bucketMetaPrefix, bucket, object, fsMetaJSONFile)
wlk, err := fs.rwPool.Create(fsMetaPath)
if err != nil {
return ObjectInfo{}, toObjectErr(err, bucket, object)
}
// This close will allow for locks to be synchronized on `fs.json`.
defer wlk.Close()
fsMetaPath := pathJoin(fs.fsPath, minioMetaBucket, bucketMetaPrefix, bucket, object, fsMetaJSONFile)
wlk, err := fs.rwPool.Create(fsMetaPath)
if err != nil {
return ObjectInfo{}, toObjectErr(err, bucket, object)
}
// This close will allow for locks to be synchronized on `fs.json`.
defer wlk.Close()
```
Now from the above snippet the following code one can notice that until the GetObject() returns writing to the client. Following portion of the code will block.
```go
wlk, err := fs.rwPool.Create(fsMetaPath)
wlk, err := fs.rwPool.Create(fsMetaPath)
```
This restriction is needed so that corrupted data is not returned to the client in between I/O. The logic works vice-versa as well an on-going PutObject(), GetObject() would wait for the PutObject() to complete.
@@ -121,15 +123,15 @@ Consider for example 3 servers sharing the same backend
On minio1
- DeleteObject(object1) --> lock acquired on `fs.json` while object1 is being deleted.
* DeleteObject(object1) --> lock acquired on `fs.json` while object1 is being deleted.
On minio2
- PutObject(object1) --> lock waiting until DeleteObject finishes.
* PutObject(object1) --> lock waiting until DeleteObject finishes.
On minio3
- PutObject(object1) --> (concurrent request during PutObject minio2 checking if `fs.json` exists)
* PutObject(object1) --> (concurrent request during PutObject minio2 checking if `fs.json` exists)
Once lock is acquired the minio2 validates if the file really exists to avoid obtaining lock on an fd which is already deleted. But this situation calls for a race with a third server which is also attempting to write the same file before the minio2 can validate if the file exists. It might be potentially possible `fs.json` is created so the lock acquired by minio2 might be invalid and can lead to a potential inconsistency.
docs/shared-backend/README.md
+10 -11
View File
@@ -1,20 +1,20 @@
# Shared Backend MinIO Quickstart Guide [![Slack](https://slack.min.io/slack?type=svg)](https://slack.min.io) [![Docker Pulls](https://img.shields.io/docker/pulls/minio/minio.svg?maxAge=604800)](https://hub.docker.com/r/minio/minio/)
MinIO shared mode lets you use single [NAS](https://en.wikipedia.org/wiki/Network-attached_storage) (like NFS, GlusterFS, and other
distributed filesystems) as the storage backend for multiple MinIO servers. Synchronization among MinIO servers is taken care by design.
MinIO shared mode lets you use single [NAS](https://en.wikipedia.org/wiki/Network-attached_storage) (like NFS, GlusterFS, and other
distributed filesystems) as the storage backend for multiple MinIO servers. Synchronization among MinIO servers is taken care by design.
Read more about the MinIO shared mode design [here](https://github.com/minio/minio/blob/master/docs/shared-backend/DESIGN.md).
MinIO shared mode is developed to solve several real world use cases, without any special configuration changes. Some of these are
- You have already invested in NAS and would like to use MinIO to add S3 compatibility to your storage tier.
- You need to use NAS with an S3 interface due to your application architecture requirements.
- You expect huge traffic and need a load balanced S3 compatible server, serving files from a single NAS backend.
- You expect huge traffic and need a load balanced S3 compatible server, serving files from a single NAS backend.
With a proxy running in front of multiple, shared mode MinIO servers, it is very easy to create a Highly Available, load balanced, AWS S3 compatible storage system.
With a proxy running in front of multiple, shared mode MinIO servers, it is very easy to create a Highly Available, load balanced, AWS S3 compatible storage system.
# Get started
If you're aware of stand-alone MinIO set up, the installation and running remains the same.
If you're aware of stand-alone MinIO set up, the installation and running remains the same.
## 1. Prerequisites
@@ -24,12 +24,10 @@ Install MinIO - [MinIO Quickstart Guide](https://docs.min.io/docs/minio-quicksta
To run MinIO shared backend instances, you need to start multiple MinIO servers pointing to the same backend storage. We'll see examples on how to do this in the following sections.
*Note*
- All the nodes running shared MinIO need to have same access key and secret key. To achieve this, we export access key and secret key as environment variables on all the nodes before executing MinIO server command.
- The drive paths below are for demonstration purposes only, you need to replace these with the actual drive paths/folders.
#### MinIO shared mode on Ubuntu 16.04 LTS.
### MinIO shared mode on Ubuntu 16.04 LTS
You'll need the path to the shared volume, e.g. `/path/to/nfs-volume`. Then run the following commands on all the nodes you'd like to launch MinIO.
@@ -39,7 +37,7 @@ export MINIO_ROOT_PASSWORD=<SECRET_KEY>
minio gateway nas /path/to/nfs-volume
```
#### MinIO shared mode on Windows 2012 Server
### MinIO shared mode on Windows 2012 Server
You'll need the path to the shared volume, e.g. `\\remote-server\smb`. Then run the following commands on all the nodes you'd like to launch MinIO.
@@ -49,9 +47,9 @@ set MINIO_ROOT_PASSWORD=my-password
minio.exe gateway nas \\remote-server\smb\export
```
*Windows Tip*
### Windows Tip
If a remote volume, e.g. `\\remote-server\smb` is mounted as a drive, e.g. `M:\`. You can use [`net use`](https://technet.microsoft.com/en-us/library/bb490717.aspx) command to map the drive to a folder.
If a remote volume, e.g. `\\remote-server\smb` is mounted as a drive, e.g. `M:\`. You can use [`net use`](https://technet.microsoft.com/en-us/library/bb490717.aspx) command to map the drive to a folder.
```cmd
set MINIO_ROOT_USER=my-username
@@ -65,6 +63,7 @@ minio.exe gateway nas M:\export
To test this setup, access the MinIO server via browser or [`mc`](https://docs.min.io/docs/minio-client-quickstart-guide). You’ll see the uploaded files are accessible from the all the MinIO shared backend endpoints.
## Explore Further
- [Use `mc` with MinIO Server](https://docs.min.io/docs/minio-client-quickstart-guide)
- [Use `aws-cli` with MinIO Server](https://docs.min.io/docs/aws-cli-with-minio)
- [Use `s3cmd` with MinIO Server](https://docs.min.io/docs/s3cmd-with-minio)