poornas
5006e28367
Add disk-cache and worm to user-agent ( #5729 )
2018-03-28 16:26:27 -07:00
poornas
a3e806ed61
Add disk based edge caching support. ( #5182 )
...
This PR adds disk based edge caching support for minio server.
Cache settings can be configured in config.json to take list of disk drives,
cache expiry in days and file patterns to exclude from cache or via environment
variables MINIO_CACHE_DRIVES, MINIO_CACHE_EXCLUDE and MINIO_CACHE_EXPIRY
Design assumes that Atime support is enabled and the list of cache drives is
fixed.
- Objects are cached on both GET and PUT/POST operations.
- Expiry is used as hint to evict older entries from cache, or if 80% of cache
capacity is filled.
- When object storage backend is down, GET, LIST and HEAD operations fetch
object seamlessly from cache.
Current Limitations
- Bucket policies are not cached, so anonymous operations are not supported in
offline mode.
- Objects are distributed using deterministic hashing among list of cache
drives specified.If one or more drives go offline, or cache drive
configuration is altered - performance could degrade to linear lookup.
Fixes #4026
2018-03-28 14:14:06 -07:00
Harshavardhana
2f9354b17e
Use uuid.New() for uploadID in azure gateway ( #5721 )
2018-03-28 11:17:25 -07:00
Harshavardhana
6e9c853312
After healing re-load disks with the new format ( #5718 )
...
This PR also fixes correct calculation of drive states
before and after healing of objects.
Fixes #5700
Fixes #5708
2018-03-28 06:41:39 +05:30
poornas
76d1e8bbcd
change fs.json format to include checksum fields ( #5685 )
2018-03-27 17:23:10 -07:00
Bala FA
3ebe61abdf
Quick support to server level WORM ( #5602 )
...
This is a trival fix to support server level WORM. The feature comes
with an environment variable `MINIO_WORM`.
Usage:
```
$ export MINIO_WORM=on
$ minio server endpoint
```
2018-03-27 16:44:45 -07:00
Takeshi Watanabe
2182c1a4f7
Use paho configuration API instead of setting it directly with struct initializer. ( #5707 )
2018-03-26 16:45:21 -07:00
Nitish Tiwari
58291acb8d
Fix object deletion where quorum is not available ( #5709 )
...
Object deletion should not be possible if quorum is not
available. This PR updates deleteObject() to check for
quorum errors before proceeding with object deletion.
Fixes #5535
2018-03-26 16:39:28 -07:00
Harshavardhana
de44be86d0
Use readQuorum instead of writeQuorum to check bucket exists ( #5715 )
...
Fixes #5708
Fixes #5700
2018-03-26 16:36:57 -07:00
Kanagaraj M
19451e374a
revert browser newux changes ( #5714 )
2018-03-26 12:49:12 -07:00
Harshavardhana
35e64573fa
Add missing env doc for MINIO_DOMAIN ( #5705 )
2018-03-26 09:11:39 -07:00
kannappanr
7c747a9643
Return complete Location URL in CompleteMultipartUpload ( #5692 )
...
Remove getLocation function.
Fixes #5687
2018-03-23 13:46:57 -07:00
Kanagaraj M
1c91c7b54d
added eslint prettier check to travis ( #5701 )
...
- "yarn eslint" will throw the error if there are any formatting issues
- "yarn format" will format the js file based on the project's
prettier configuration.
2018-03-23 11:41:10 -07:00
Harshavardhana
020cc8b669
Remove old unmaintained docs ( #5699 )
...
- removes backend/{fs,xl} files.
- removes FreeBSD doc since we already have FreeNAS doc.
- removes sets/ directory which is redundant with
large-bucket directory.
- removes browser/ directory which attempts
to explain about browser RPC calls, since
its an internal API we do not need to document it.
2018-03-24 00:07:04 +05:30
Nitish Tiwari
850caaa1be
Add iptables/ufw command example to open relevant ports for Minio ( #5604 )
...
fixes #5022
2018-03-24 00:00:47 +05:30
Kanagaraj M
4ca0ec4586
added yarn test to travis.yml ( #5696 )
2018-03-23 14:36:20 +05:30
Kanagaraj M
85154508b0
updated favicon.ico and ui-assets.go ( #5698 )
2018-03-23 14:06:18 +05:30
Rushan
de84f41bec
Fix the styling of upload abort button ( #5697 )
2018-03-23 13:04:39 +05:30
Kaan Kabalak
3805a055ab
Fix New UX related bugs for Alerts, BucketList and Abort Upload modal ( #5694 )
...
- Center text on Alert component appearing on top
- Change "Upload" to "Continue" on New UX Abort Upload modal
- Fix issue where long bucket names were overflowing instead of
continuing on the next line
- Fix word wrap issue similar to the last one for the alerts that appear
on top
- Help fix the cluttering issue on #5690 as the right margin has been
increased in alerts
Fixes #5689 , #5691 and #5695
2018-03-23 12:19:27 +05:30
Harshavardhana
17409517ec
Fix docs to expand on new limits on a distributed setup ( #5677 )
2018-03-22 16:24:14 -07:00
Nitish Tiwari
cf67f1838e
Add unit tests for endpoint-ellipses.go ( #5681 )
2018-03-22 16:23:54 -07:00
Harshavardhana
787f158f92
update browser assets for newux
2018-03-22 13:30:39 -07:00
Kanagaraj M
c0e45f9098
Fix failing unit tests in browser ( #5688 )
...
* format js files using prettier
Used the following command to format the files
prettier --write "browser/app/js/**/*.js"
* fix failing unit tests in browser
2018-03-22 12:25:56 -07:00
Kanagaraj M
cb3818be27
adding local pagination to bucket list ( #5684 )
...
* adding local pagination to bucket list
When there are more than 5000 buckets, browser ui
becomes unresponsive since react needs to create
5000 elements which takes browser resources.
So we show only 100 buckets for the first time,
and load more buckets when the user is scrolling down.
* move inline styles to less file
2018-03-22 23:28:48 +05:30
Rushan
1459c4be1e
Browser: Update UI with new components and elements ( #5671 )
2018-03-21 11:38:57 -07:00
Kaan Kabalak
384b4fdf28
Tweak fetchBuckets to comply with the changes made to the routes ( #5592 )
...
This commit removes the argument in the fetchBuckets function that was
introduced in #5580 . Due to the adjustment made in #5584 , we no longer
need to differentiate between where the function has been called from.
2018-03-21 11:38:56 -07:00
Kanagaraj M
6a15b89b9a
refactor Dropzone component and added tests ( #5578 )
2018-03-21 11:38:56 -07:00
Kanagaraj M
54e5ee6535
removed duplicate route components from browser ( #5584 )
...
All routes '/', '/:bucket/', '/:bucket/*' render the same
component. Instead we could just have a single route like following which
combines all the above routes
'/:bucket?/*'
bucket is optional here, so it can cover '/'
2018-03-21 11:38:56 -07:00
Kaan Kabalak
a6adef0bdf
Refactor bucket delete and bucket policy ( #5580 )
...
This commit adds the bucket delete and bucket policy functionalities
to the browser.
Part of rewriting the browser code to follow best practices and
guidelines of React (issues #5409 and #5410 )
The backend code has been modified by @krishnasrinivas to prevent
issue #4498 from occuring. The relevant changes have been made to the
code according to the latest commit and the unit tests in the backend.
This commit also addresses issue #5449 .
2018-03-21 11:38:56 -07:00
Kanagaraj M
416841869a
allow non-loggedin users to access public bucket ( #5570 )
...
* conditionally render main action buttons
- Make bucket action will be available only for loggedIn users
- File upload button will be avaialble for loggedIn users
and non-loggedIn users if the prefix is writable
* select the bucket and prefix from the url
When the url contains bucket and prefix, it will be selected
by default instead of the first bucket from the list.
* show BucketSearch only for LoggedIn users
* allow non-LoggedIn users to access public bucket
* removed unused Router imports
* fix test case failures in BucketList.test.js
* remove dupicate minioBrowserPrefix from url
since history is already initialized with minioBrowserPrefix,
no need to use it in push or replace
* remove unused match from App component
* remove unused minioBrowserPrefix imports
2018-03-21 11:30:50 -07:00
Kanagaraj M
bb0adea494
Refactor browser dropdown links and components ( #5562 )
...
* refactor browser links and about modal
Moved about modal to separate component and added unit tests.
* refactor change password modal component
* added unit tests for ChangePasswordModal
* fix logout function in browser dropdown
* remove older unused BrowserDropdown component
* remove unused variables from BrowserDropdown component
* show BrowserDropdown and StorageInfo only for LoggedIn users
Non-loggedIn users will see a 'Login' button
2018-03-21 11:30:50 -07:00
Kanagaraj M
6a42727e00
Refactor download object and bulk action components ( #5546 )
2018-03-21 11:30:50 -07:00
Kanagaraj M
da4558a8f7
Refactor delete object and share object components ( #5537 )
2018-03-21 11:30:50 -07:00
Kaan Kabalak
566ac78b8b
Add missing unit tests for PR #5499 ( #5527 )
...
Part of #5410
2018-03-21 11:30:50 -07:00
Kanagaraj M
44f8f7059c
Refactor make bucket and upload components ( #5521 )
2018-03-21 11:30:50 -07:00
Kanagaraj M
9bfa07ecf5
re-organize components and actions by feature ( #5518 )
...
Now the files grouped based on the features instead of
the previous approach of grouping by type.
2018-03-21 11:30:50 -07:00
Kaan Kabalak
ead6337eab
Add missing unit tests to JavaScript and React components ( #5505 )
...
Add the missing unit tests login, bucket listing, and bucket searching functionalities.
2018-03-21 11:30:50 -07:00
Kanagaraj M
feb726dd98
Refactor of components ( #5499 )
2018-03-21 11:30:50 -07:00
Kanagaraj M
9ab6a8035f
upgrading react and refactoring components ( #5409 ) ( #5444 )
...
- upgraded react from v16.2.0
- upgraded react-router to v4.2.0 and re-writen the routes
- using prettier to format the code
- added jest to unit test components/reducers/selectors
This provides a skeleton to start of with. Only basic unit test
cases are added, remaining needs to be added.
In terms of functionality, it provides login, listing and searching
buckets. Remaining functionalities will be added in upcoming patches.
2018-03-21 11:30:50 -07:00
Kanagaraj M
76be54551f
support for sourcemaps in browser code ( #5451 )
...
changed from 'eval' to 'cheap-module-eval-source-map' to
generate source map of the original code instead of transpiled code.
2018-03-21 11:30:50 -07:00
Kaan Kabalak
0da377f3e9
Upgrade webpack from v1 to v3 ( #5448 )
...
- Changed webpack.config.js and webpack.production.config.js
according to the migration guidelines
- Updated packages that had unmet peer dependencies after the
webpack update
Fixes #5398
2018-03-21 11:30:50 -07:00
Kaan Kabalak
9cc77e51f5
Change name of 'Settings' dropdown link to 'Change Password' ( #5394 )
2018-03-21 11:30:50 -07:00
Kaan Kabalak
bbbbb8d3e2
Center dropdown toggle correctly across browsers ( #5395 )
...
This commit also fixes the shifting issue for Firefox when the dropdown
toggle is clicked.
2018-03-21 11:30:50 -07:00
Harshavardhana
565dc8b283
Do not check for GOPATH env for builds ( #5628 )
...
Since go1.8 GOPATH is not required to set prior, as
it defaults to "${HOME}/go" we only need to check if
go tool detected GOPATH correctly. If yes then we
proceed if not we fail.
2018-03-20 12:05:49 -07:00
Harshavardhana
c726145baf
Fix regression in removing notification ( #5673 )
...
fixes a regression introduced in 0e4431725c
when removing a previously applied notification configuration.
event.ParseConfig() was stricter in terms of handling notification
configuration, we need to allow when notification configuration is
sent empty, this is the way to remove notification configuration.
2018-03-20 12:02:56 -07:00
Harshavardhana
b4ae2bd2f5
Remove snap reference from the repo ( #5676 )
...
snap builds are not well tested, since
some releases have stopped working. Remove
their reference to keep it simple.
Fixes #5674
Fixes #5472
2018-03-20 09:33:02 +05:30
Minio Trusted
fb3159277c
Update yaml files to latest version RELEASE.2018-03-19T19-22-06Z
2018-03-19 19:25:23 +00:00
Andreas Auernhammer
a4d2e2b428
[refactor] simplify en/decrypted size computation ( #5658 )
...
This commit replaces the en/decrypted size computation
with functions from the `sio` package.
Fixes #5657
2018-03-19 22:18:12 +05:30
Harshavardhana
2938e332ba
Fix format migration regression ( #5668 )
...
Migration regression got introduced in 9083bc152e
adding more unit tests to catch this scenario, we need to fix this by
re-writing the formats after the migration to 'V3'.
This bug only happens when a user is migrating directly from V1 to V3,
not from V1 to V2 and V2 to V3.
Added additional unit tests to cover these situations as well.
Fixes #5667
2018-03-19 21:43:00 +05:30
Minio Trusted
049090126e
Update yaml files to latest version RELEASE.2018-03-16T22-52-12Z
2018-03-16 22:55:04 +00:00
Harshavardhana
f23944aed7
Fix heal bucket deadlock after replacing disks ( #5661 )
...
Fixes #5659
2018-03-16 15:09:31 -07:00
Harshavardhana
3145462ad2
Return InvalidDigest when md5 sent by client is invalid ( #5654 )
...
This is to ensure proper compatibility with AWS S3, handle
special cases where
- Content-Md5 is set to empty
- Content-Md5 is set to invalid
2018-03-16 11:22:34 -07:00
Krishna Srinivas
9ede179a21
Use context.Background() instead of nil
...
Rename Context[Get|Set] -> [Get|Set]Context
2018-03-15 16:28:25 -07:00
Ersan
33fe42df8a
Adds context support for nas, oss and s3
2018-03-15 16:28:25 -07:00
kannappanr
bdb1a90720
Add context support for gateway b2, manta, gcs
2018-03-15 16:28:25 -07:00
Krishna Srinivas
e452377b24
Add context to the object-interface methods.
...
Make necessary changes to xl fs azure sia
2018-03-15 16:28:25 -07:00
Krishna Srinivas
9083bc152e
Flat multipart backend implementation for Erasure backend ( #5447 )
2018-03-15 13:55:23 -07:00
Bala FA
0e4431725c
make notification as separate package ( #5294 )
...
* Remove old notification files
* Add net package
* Add event package
* Modify minio to take new notification system
2018-03-15 13:03:41 -07:00
Nitish Tiwari
abffa00b76
Update healthcheck related examples and add head support ( #5650 )
...
- Add head method for healthcheck endpoint. Some platforms/users
may use the HTTP Head method to check for health status.
- Add liveness and readiness probe examples in Kubernetes yaml
example docs. Note that readiness probe not added to StatefulSet
example due to https://github.com/kubernetes/kubernetes/issues/27114
2018-03-15 09:55:02 +05:30
Nitish Tiwari
1c0c3369c9
Cleanup large bucket support documents ( #5641 )
2018-03-14 15:22:17 -07:00
Harshavardhana
4af89543cf
Update minio-go dependencies to latest 5.0.0 release ( #5640 )
...
With following changes
- Add SSE and refactor encryption API (#942 ) <Andreas Auernhammer>
- add copyObject test changing metadata and preserving etag (#944 ) <Harshavardhana>
- Add SSE-C tests for multipart, copy, get range operations (#941 ) <Harshavardhana>
- Removing conditional check for notificationInfoCh in api-notication (#940 ) <Matthew Magaldi>
- Honor prefix parameter in ListBucketPolicies API (#929 ) <kannappanr>
- test for empty objects uploaded with SSE-C headers (#927 ) <kannappanr>
- Encryption headers should also be set during initMultipart (#930 ) <Harshavardhana>
- Add support for Content-Language metadata header (#928 ) <kannappanr>
- Fix check for duplicate notification configuration entries (#917 ) <kannappanr>
- allow OS to cleanup sockets in TIME_WAIT (#925 ) <Harshavardhana>
- Sign V2: Fix signature calculation in virtual host style (#921 ) <A. Elleuch>
- bucket policy: Support json string in Principal field (#919 ) <A. Elleuch>
- Fix copyobject failure for empty files (#918 ) <kannappanr>
- Add new constructor NewWithOptions to SDK (#915 ) <poornas>
- Support redirect headers to sign again with new Host header. (#829 ) <Harshavardhana>
- Fail in PutObject if invalid user metadata is passed <Harshavadhana>
- PutObjectOptions Header: Don't include invalid header <Isaac Hess>
- increase max retry count to 10 (#913 ) <poornas>
- Add new regions for Paris and China west. (#905 ) <Harshavardhana>
- fix s3signer to use req.Host header (#899 ) <Bartłomiej Nogaś>
2018-03-14 19:38:29 +05:30
Harshavardhana
5dc5e4928c
Support content-language header ( #5648 )
2018-03-14 15:27:32 +05:30
tparker00
bbd7f8a4af
Updated healthcheck file ( #5647 )
...
Healthcheck is no longer checking for a file so the -I in the curl check will fail.
2018-03-14 13:21:56 +05:30
Nitish Tiwari
6882467db8
Update volume in Dockerfiles ( #5646 )
...
All sample commands in Minio docs use the volume path as /data.
Updated the dockerfiles to follow the same pattern.
Fixes #5635
2018-03-14 06:59:27 +05:30
Yueh-Ting Chen
e8d8360d00
fix spelling ( #5643 )
2018-03-13 19:23:37 +05:30
Alexander Stuart-Kregor
5053f96274
Swap max_hw_sectors_kb for max_sectors_kb. Fixes #5633 . ( #5638 )
2018-03-13 18:47:49 +05:30
Minio Trusted
16000255f8
Update yaml files to latest version RELEASE.2018-03-12T21-25-28Z
2018-03-12 21:28:21 +00:00
Nitish Tiwari
4a4d1d1b82
Add Minio TLS configuration doc for Kubernetes deployment ( #5617 )
2018-03-12 14:22:23 -07:00
Harshavardhana
29ef7d29e4
Fix deadlock in in-place CopyObject decryption/encryption ( #5637 )
...
In-place decryption/encryption already holds write
locks on them, attempting to acquire a read lock would
fail.
2018-03-12 13:52:38 -07:00
Nitish Tiwari
574b667c56
Remove madmin docs from top level docs directory ( #5636 )
...
madmin package is well documented in its source directory here
https://github.com/minio/minio/tree/master/pkg/madmin .
Hence, keeping another copy is not required as it makes it difficult
to maintain.
2018-03-12 11:51:58 -07:00
Nitish Tiwari
10b01ac836
Add healthcheck endpoints ( #5543 )
...
This PR adds readiness and liveness endpoints to probe Minio server
instance health. Endpoints can only be accessed without authentication
and the paths are /minio/health/live and /minio/health/ready for
liveness and readiness respectively.
The new healthcheck liveness endpoint is used for Docker healthcheck
now.
Fixes #5357
Fixes #5514
2018-03-12 11:46:53 +05:30
Harshavardhana
d90985b6d8
Return authHeaderMalformed for an incorrect region in signature ( #5618 )
2018-03-09 18:18:57 -08:00
Harshavardhana
7aaf01eb74
Save ETag when updating metadata ( #5626 )
...
Fixes #5622
2018-03-09 10:50:39 -08:00
Nitish Tiwari
ba0c7544ea
Cleanup orchestration documents ( #5623 )
...
- Remove hostPort from Kubernetes deployment example docs. Initially
hostPort was added to ensure Minio pods are allocated to separate
machines, but as per latest Kubernetes documents this is not
recommended approach (ref: https://kubernetes.io/docs/concepts/
configuration/overview/#services). To define pod allocations,
Affinity and Anti-Affinity concepts are the recommended approach.
(ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node )
- Add Minio release tag to Docker-Compose example file.
2018-03-09 15:21:41 +05:30
kannappanr
380e0ddb57
Remove unwanted errorIf calls ( #5621 )
...
Remove errorIf call with a errSignatureMismatch error
2018-03-09 00:51:05 -08:00
Andreas Auernhammer
889dd387f1
[doc] fix openssl command for ECDSA key generation ( #5616 )
...
This change fixes the command for generating ECDSA private keys.
The current command produces private key files which cannot be parsed
by the server.
Fixes #5614
2018-03-08 15:06:42 -08:00
Dee Koder
973ff2fabd
Fix mqtt example py code which was not working ( #5619 )
2018-03-08 16:46:52 +05:30
poornas
247c1bb5ef
Pass location in MakeBucketWithLocation call ( #5605 )
...
fixes regression for gateways
2018-03-08 12:56:20 +05:30
Harshavardhana
27258b9c54
Ensure to load only regular files for CAs ( #5612 )
...
In kubernetes statefulset like environments when secrets
are mounted to pods they have sub-directories, we should
ideally be only looking for regular files here and skip
all others.
2018-03-07 22:16:28 +05:30
Harshavardhana
b325593b47
SSE-C CopyObject key-rotation doesn't need metadata REPLACE value ( #5611 )
...
Fix a compatibility issue with AWS S3 where to do key rotation
we need to replace an existing object's metadata. In such a
scenario "REPLACE" metadata directive is not necessary.
2018-03-06 16:04:48 -08:00
Dee Koder
a34901af77
Add missing wget command for easy cut copy paste. ( #5609 )
...
We should keep this for easy one click copy paste on remote servers
rather than 2 things for users to deal with.
2018-03-06 13:36:48 +05:30
Anis Elleuch
cac10bcbf7
SSE-C: Add support in Bucket Post Policy ( #5607 )
...
* SSE-C: Add support in Bucket Post Policy
* Rename isSSECustomerRequest & isSSECopyCustomerRequest to hasSSECustomerHeader hasSSECopyCustomerHeader
2018-03-05 08:02:56 -08:00
Aditya Manthramurthy
ea8973b7d7
Return bit-rot verified data instead of re-reading from disk ( #5568 )
...
- Data from disk was being read after bitrot verification to return
data for GetObject. Strictly speaking this does not guarantee bitrot
protection, as disks may return bad data even temporarily.
- This fix reads data from disk, verifies data for bitrot and then
returns data to the client directly.
2018-03-04 14:16:45 -08:00
Harshavardhana
52eea7b9c1
Support SSE-C multipart source objects in CopyObject ( #5603 )
...
Current code didn't implement the logic to support
decrypting encrypted multiple parts, this PR fixes
by supporting copying encrypted multipart objects.
2018-03-02 17:24:02 -08:00
Harshavardhana
e4f6877c8b
Handle incoming proxy requests ip, scheme ( #5591 )
...
This PR implements functions to get the right ip, scheme
from the incoming proxied requests.
2018-03-02 15:23:04 -08:00
Harshavardhana
d71b1d25f8
Make sure to filter out internal metadata ( #5601 )
...
Currently we reply back `X-Minio-Internal` values
back to the client for an encrypted object, we should
filter these out and only reply AWS compatible headers.
2018-03-01 16:15:53 -08:00
Anis Elleuch
5f37988db5
Use toAPIErrorCode in HeadObject handler when decrypting request fails ( #5600 )
2018-03-01 16:01:56 -08:00
Andreas Auernhammer
4e5237b02a
vendor update github.com/minio/sio ( #5599 )
...
This change updates the sio library and adds DARE 2.0 support
to the server.
2018-03-01 15:28:34 -08:00
Harshavardhana
1b7b8f14c9
Set appropriate encryption headers in HEAD object response ( #5596 )
...
Currently we don't set two SSE-C specific headers fix it
for AWS S3 compatibility.
2018-03-01 14:16:40 -08:00
Anis Elleuch
120b061966
Add multipart support in SSE-C encryption ( #5576 )
...
*) Add Put/Get support of multipart in encryption
*) Add GET Range support for encryption
*) Add CopyPart encrypted support
*) Support decrypting of large single PUT object
2018-03-01 11:37:57 -08:00
kannappanr
d32f90fe95
Honor global flags irrespective of the position. ( #5486 )
...
Flags like `json, config-dir, quiet` are now honored even if they are
between minio and gateway in the cli, like, `minio --json gateway s3`.
Fixes #5403
2018-02-28 20:13:33 -08:00
Harshavardhana
6faa1ef11a
Fix shadowing issue reported by go vet ( #5590 )
2018-02-28 14:30:00 -08:00
Harshavardhana
9af254a82f
Remove stable sort usage when not needed ( #5586 )
...
Stable sort is needed when we are sorting based on two or more
distinct elements. When equal elements are indistinguishable,
such as with integers, or more generally, any data where the
entire element is the key like `PartNumber`, stability is not
an issue.
2018-02-28 08:33:00 +05:30
Harshavardhana
6b3db7556a
Fix gofmt issues reported for simplification ( #5581 )
...
added travis to catch this issue
2018-02-26 23:52:03 +05:30
Harshavardhana
5204a754db
Move rpc version to 2.0.0 to align with backend migration ( #5575 )
...
Fixes #5574
2018-02-23 16:58:37 -08:00
Harshavardhana
7cc678c653
Support encryption for CopyObject, GET-Range requests ( #5544 )
...
- Implement CopyObject encryption support
- Handle Range GETs for encrypted objects
Fixes #5193
2018-02-23 15:07:21 -08:00
poornas
b7536570f8
update docs for NAS gateway ( #5569 )
2018-02-22 09:15:19 +05:30
Harshavardhana
e09d97abaf
Fix docs in admin API ( #5559 )
2018-02-21 12:00:46 -08:00
Krishnan Parthasarathi
e5e3d17216
Do not close *lock.LockedFile on failure ( #5565 )
2018-02-21 11:28:24 -08:00
Harshavardhana
0ea54c9858
Change CopyObject{Part} to single srcInfo argument ( #5553 )
...
Refactor such that metadata and etag are
combined to a single argument `srcInfo`.
This is a precursor change for #5544 making
it easier for us to provide encryption/decryption
functions.
2018-02-21 14:18:47 +05:30
Krishna Srinivas
a00e052606
Provide more descriptive error during erasure init ( #5282 )
...
fixes #5239
2018-02-20 18:42:09 -08:00
Anis Elleuch
d2d49f6c6c
xl: Avoid removing directory content in Delete API ( #5548 )
...
Delete & Multi Delete API should not try to remove the directory content.
The only permitted case is with zero size object with a trailing slash
in its name.
2018-02-20 15:33:26 -08:00
Harshavardhana
db9e83de62
Avoid significant connections in TIME_WAIT ( #5555 )
...
MaxIdleConns limits the total number of connections
kept in the pool for re-use. In addition, MaxIdleConnsPerHost
limits the number for a single host. Since minio gateways
usually connect to the same host, setting `MaxIdleConns = 100`
won't really have much of an impact since the idle connection
pool is limited to 2 anyway.
Now, with the pool set to a limit of 2, and when using
the client heavily from 2+ goroutines, the `http.Transport`
will open a connection, use it, then try to return it to
the idle-pool which often fails since there's a limit of 2.
So it's going to close the connection and new ones will be
opened on demand again, many of which get closed soon after
being used. Since those connections/sockets don't disappear
from the OS immediately, use `MaxIdleConnsPerHost = 100`
which fixes this problem.
2018-02-20 12:23:37 -08:00
poornas
25107c2e11
Add NAS gateway support ( #5516 )
2018-02-20 12:21:12 -08:00
Anis Elleuch
926e480156
posix.RenameFile(): Allow overwriting an empty directory ( #5551 )
...
Overwriting files is allowed, but since the introduction of
the object directory, we will aslo need to allow overwriting
an empty directory. Putting twice the same object directory
won't fail with 403 error anymore.
2018-02-20 12:20:18 -08:00
Harshavardhana
b2b5056163
gateway/gcs: Remove unused storageEndpoint ( #5556 )
2018-02-20 15:07:31 +05:30
A. Elleuch
1e7e41fada
tests: Fix failed notify webhook test ( #5528 )
...
TestNewWebHookNotify wasn't passing in my local machine. The reason is
that the test expects the POST handler (as a webhook endpoint) is always
running on port 80, which is not always the case.
2018-02-17 19:06:43 -08:00
Harshavardhana
03923947c4
Fix delete bucket policies properly ( #5540 )
...
There was bug in previous PR where deleteBucketMetadata()
was never called, fix it correctly.
2018-02-16 20:16:48 -08:00
Harshavardhana
d12bdd50ee
Rename minio-limitations.md to minio-limits.md ( #5541 )
2018-02-16 09:35:02 +05:30
Harshavardhana
fb96779a8a
Add large bucket support for erasure coded backend ( #5160 )
...
This PR implements an object layer which
combines input erasure sets of XL layers
into a unified namespace.
This object layer extends the existing
erasure coded implementation, it is assumed
in this design that providing > 16 disks is
a static configuration as well i.e if you started
the setup with 32 disks with 4 sets 8 disks per
pack then you would need to provide 4 sets always.
Some design details and restrictions:
- Objects are distributed using consistent ordering
to a unique erasure coded layer.
- Each pack has its own dsync so locks are synchronized
properly at pack (erasure layer).
- Each pack still has a maximum of 16 disks
requirement, you can start with multiple
such sets statically.
- Static sets set of disks and cannot be
changed, there is no elastic expansion allowed.
- Static sets set of disks and cannot be
changed, there is no elastic removal allowed.
- ListObjects() across sets can be noticeably
slower since List happens on all servers,
and is merged at this sets layer.
Fixes #5465
Fixes #5464
Fixes #5461
Fixes #5460
Fixes #5459
Fixes #5458
Fixes #5460
Fixes #5488
Fixes #5489
Fixes #5497
Fixes #5496
2018-02-15 17:45:57 -08:00
Harshavardhana
dd80256151
Directory HEADs with encryption headers shouldn't return errors ( #5539 )
...
Since we do not encrypt directories we don't need to send
errors with encryption headers when the directory doesn't
have encryption metadata.
Continuation PR from 4ca10479b5
2018-02-15 14:18:28 -08:00
Harshavardhana
22897de4c7
fail when endpoints point to same path locally ( #5523 )
2018-02-15 14:38:17 +05:30
Harshavardhana
e22438c8cd
Cleanup banners and remove fossa/snap ( #5530 )
2018-02-15 09:45:42 +05:30
Harshavardhana
c59f1e3a80
revamp minio build messages ( #5519 )
2018-02-14 10:29:19 +05:30
Harshavardhana
994fe53669
Avoid shadowing ignored errors listAllBuckets() ( #5524 )
...
It can happen such that one of the disks that was down would
return 'errDiskNotFound' but the err is preserved due to
loop shadowing which leads to issues when healing the bucket.
2018-02-13 17:03:50 -08:00
Andreas Auernhammer
4ca10479b5
[SSE-C]: avoid encrypting empty objects. ( #5525 )
...
This change adds an object size check such that the server does not
encrypt empty objects (typically folders) for SSE-C. The server still
returns SSE-C headers but the object is not encrypted since there is no
point to encrypt such objects.
Fixes #5493
2018-02-13 15:43:46 -08:00
Harshavardhana
91101b11bb
Converge repeated code to common deleteBucketMetadata() ( #5508 )
2018-02-12 18:34:30 -08:00
Harshavardhana
8de6cf4124
update dsync implementation to fix a regression ( #5513 )
...
Currently minio master requires 4 servers, we
have decided to run on a minimum of 2 servers
instead - fixes a regression from previous
releases where 3 server setups were supported.
2018-02-12 15:16:12 +05:30
poornas
4f73fd9487
Unify gateway and object layer. ( #5487 )
...
* Unify gateway and object layer. Bring bucket policies into
object layer.
2018-02-09 15:19:30 -08:00
Minio Trusted
a7f6e14370
Update yaml files to latest version RELEASE.2018-02-09T22-40-05Z
2018-02-09 22:43:57 +00:00
dingjs
289457568c
Translate #5365 to Chinese. ( #5439 )
2018-02-08 17:00:03 -08:00
Harshavardhana
fd3897d0c3
Move to go1.9.4 with recent security release ( #5502 )
2018-02-08 14:33:22 +05:30
Krishna Srinivas
047b7aff0c
Seek to offset 0 after Truncate() ( #5375 )
2018-02-06 15:37:48 -08:00
Harshavardhana
1164fc60f3
Bring semantic versioning to provide for rolling upgrades ( #5495 )
...
This PR brings semver capabilities in our RPC layer to
ensure that we can upgrade the servers in rolling fashion
while keeping I/O in progress. This is only a framework change
the functionality remains the same as such and we do not
have any special API changes for now. But in future when
we bring in API changes we will be able to upgrade servers
without a downtime.
Additional change in this PR is to not abort when serverVersions
mismatch in a distributed cluster, instead wait for the quorum
treat the situation as if the server is down. This allows
for administrator to properly upgrade all the servers in the cluster.
Fixes #5393
2018-02-06 15:07:17 -08:00
kannappanr
48218272cc
Document object name limitations on Windows ( #5491 )
...
Fixes #5161
2018-02-03 19:57:40 +05:30
Harshavardhana
0c880bb852
Deprecate and remove in-memory object caching ( #5481 )
...
in-memory caching cannot be cleanly implemented
without the access to GC which Go doesn't naturally
provide. At times we have seen that object caching
is more of an hindrance rather than a boon for
our use cases.
Removing it completely from our implementation
related to #5160 and #5182
2018-02-02 10:17:13 -08:00
Harshavardhana
1ebbc2ce88
Make sure to convert the disk errors to object errors ( #5480 )
...
Fixes a bug introduced in the directory support PR, with
this fix s3fs works properly.
2018-02-02 14:04:15 +05:30
A. Elleuch
da2faa19a1
Reduce Minio access key minimum length to 3 ( #5478 )
...
This is a generic minimum value. The current reason is to support
Azure blob storage accounts name whose length is less than 5. 3 is the
minimum length for Azure.
2018-02-02 09:13:30 +05:30
Krishna Srinivas
2afd196c83
Quorum based listing for XL ( #5475 )
...
fixes #5380
2018-02-01 10:47:49 -08:00
Krishna Srinivas
b606ba3f81
fs.json file should be closed in CompleteMultipartUpload ( #5482 )
2018-02-01 15:27:12 +05:30
Harshavardhana
3316dbc037
simplify storage class validation ( #5470 )
...
Check if the storage class is set in an
non XL setup instead of relying on `globalEndpoints`
value. Also converge the checks for both SS
and RRS parity configuration.
This PR also removes redundant `tt.name` in all
test cases, since each testcase doesn't need to
be numbered explicitly they are numbered implicitly.
2018-02-01 13:00:07 +05:30
Harshavardhana
033cfb5cef
Remove stale code from minio server ( #5479 )
2018-01-31 18:28:28 -08:00
Krishna Srinivas
3b2486ebaf
Lock free multipart backend implementation for FS ( #5401 )
2018-01-31 13:17:24 -08:00
Aditya Manthramurthy
018813b98f
Fix configuration handling bugs: ( #5473 )
...
* Update the GetConfig admin API to use the latest version of
configuration, along with fixes to the corresponding RPCs.
* Remove mutex inside the configuration struct, and inside
notification struct.
* Use global config mutex where needed.
* Add `serverConfig.ConfigDiff()` that provides a more granular diff
of what is different between two configurations.
2018-01-31 08:15:54 -08:00
ebozduman
e608e05cda
Removes capitalization of error causes ( #5468 )
2018-01-30 21:42:15 -08:00
Harshavardhana
3ea28e9771
Support creating directories on erasure coded backend ( #5443 )
...
This PR continues from #5049 where we started supporting
directories for erasure coded backend
2018-01-30 08:13:13 +05:30
Krishna Srinivas
45c35b3544
Autocorrect user provided Azure endpoint ( #5417 )
...
fixes #5373
2018-01-29 10:30:08 -08:00
Andreas Auernhammer
09a9002f12
add documentation about PKCS-8 encrypted RSA keys ( #5454 )
...
This change adds documentation about PKCS-8 vs PKCS-1 pitfalls. It
also provides a command to convert encrypted PKCS-8 RSA keys to
encrypted PKCS-1 RSA keys.
Fixes #5453
2018-01-27 09:30:02 +05:30
Aditya Manthramurthy
5cdcc73bd5
Admin API auth and heal related fixes ( #5445 )
...
- Fetch region for auth from global state
- Fix SHA256 handling for empty body in heal API
2018-01-25 19:24:00 +05:30
poornas
2dd117f647
fix testcases to init nslock properly ( #5429 )
2018-01-24 09:04:09 +05:30
Harshavardhana
2d19663fef
Update dockerfile go version to 1.9.2 ( #5441 )
2018-01-23 17:19:19 +05:30
Aditya Manthramurthy
254b05e314
Fix locking in some admin APIs: ( #5438 )
...
- read lock for get config
- write lock for update creds
- write lock for format file
2018-01-22 18:09:12 -08:00
Aditya Manthramurthy
a003de72bf
Update madmin doc ( fixes #5432 ) ( #5433 )
2018-01-22 16:10:43 -08:00
Aditya Manthramurthy
a337ea4d11
Move admin APIs to new path and add redesigned heal APIs ( #5351 )
...
- Changes related to moving admin APIs
- admin APIs now have an endpoint under /minio/admin
- admin APIs are now versioned - a new API to server the version is
added at "GET /minio/admin/version" and all API operations have the
path prefix /minio/admin/v1/<operation>
- new service stop API added
- credentials change API is moved to /minio/admin/v1/config/credential
- credentials change API and configuration get/set API now require TLS
so that credentials are protected
- all API requests now receive JSON
- heal APIs are disabled as they will be changed substantially
- Heal API changes
Heal API is now provided at a single endpoint with the ability for a
client to start a heal sequence on all the data in the server, a
single bucket, or under a prefix within a bucket.
When a heal sequence is started, the server returns a unique token
that needs to be used for subsequent 'status' requests to fetch heal
results.
On each status request from the client, the server returns heal result
records that it has accumulated since the previous status request. The
server accumulates upto 1000 records and pauses healing further
objects until the client requests for status. If the client does not
request any further records for a long time, the server aborts the
heal sequence automatically.
A heal result record is returned for each entity healed on the server,
such as system metadata, object metadata, buckets and objects, and has
information about the before and after states on each disk.
A client may request to force restart a heal sequence - this causes
the running heal sequence to be aborted at the next safe spot and
starts a new heal sequence.
2018-01-22 14:54:55 -08:00
Harshavardhana
f3f09ed14e
Fix a bug in dsync initialization and communication ( #5428 )
...
In current implementation we used as many dsync clients
as per number of endpoints(along with path) which is not
the expected implementation. The implementation of Dsync
was expected to be just for the endpoint Host alone such
that if you have 4 servers and each with 4 disks we need
to only have 4 dsync clients and 4 dsync servers. But
we currently had 8 clients, servers which in-fact is
unexpected and should be avoided.
This PR brings the implementation back to its original
intention. This issue was found #5160
2018-01-22 10:25:10 -08:00
Harshavardhana
bb73c84b10
Add notification structure link ( #5426 )
...
Fixes #4545
2018-01-20 09:23:09 +05:30
Harshavardhana
e19eddd759
Remove requirement for custom RPCClient ( #5405 )
...
This change is a simplification over existing
code since it is not required to have a separate
RPCClient structure instead keep authRPCClient can
do the same job.
There is no code which directly uses netRPCClient(),
keeping authRPCClient is better and simpler. This
simplication also allows for removal of multiple
levels of locking code per object.
Observed in #5160
2018-01-19 16:38:47 -08:00
Andreas Auernhammer
7f99cc9768
add HighwayHash256 support ( #5359 )
...
This change adds the HighwayHash256 PRF as bitrot protection / detection
algorithm. Since HighwayHash256 requires a 256 bit we generate a random
key from the first 100 decimals of π - See nothing-up-my-sleeve-numbers.
This key is fixed forever and tied to the HighwayHash256 bitrot algorithm.
Fixes #5358
2018-01-19 10:18:21 -08:00
Aditya Manthramurthy
2760409656
Remove dead code and associated dead code warning ( #5424 )
2018-01-19 10:16:21 -08:00
fossabot
1f13235cbd
Add license scan report and status ( #5430 )
2018-01-19 13:16:59 +05:30
poornas
dd5a3289dd
fix: listobjects return empty response for invalid prefix/marker ( #5425 )
...
Currently minio server returns a NotImplemented error when marker
is not common with prefix. Instead, return an empty ListObjectsResponse
2018-01-18 14:39:39 -08:00
Harshavardhana
b6e4f053a3
Fix lock rpc server maintenance loop go-routine leak ( #5423 )
...
The problem was after the globalServiceDoneCh receives a
message, we cleanly stop the ticker as expected. But the
go-routine where the `select` loop is running is never
returned from. The stage at which point this may occur
i.e server is being restarted, doesn't seriously affect
servers usage. But any build up like this on server has
consequences as the new functionality would come in future.
2018-01-18 14:39:24 -08:00
Minio Trusted
1c3f55ff64
Update yaml files to latest version RELEASE.2018-01-18T20-33-21Z
2018-01-18 20:41:53 +00:00
Nitish Tiwari
e2d5a87b26
Fix free and total space reported in startup banner ( #5419 )
...
With storage class support, the free and total space
reported in Minio XL startup banner should be based on
totalDisks - standardClassParityDisks, instead of totalDisks/2.
fixes #5416
2018-01-17 11:25:51 -08:00
Andreas Auernhammer
d0a43af616
replace all "crypto/sha256" with "github.com/minio/sha256-simd" ( #5391 )
...
This change replaces all imports of "crypto/sha256" with
"github.com/minio/sha256-simd". The sha256-simd package
is faster on ARM64 (NEON instructions) and can take advantage
of AVX-512 in certain scenarios.
Fixes #5374
2018-01-17 10:54:31 -08:00
Paul Stack
a020a70484
gateway/manta: Bump manta dependencies ( #5414 )
...
Internally, triton-go, what manta minio is built on, changed it's internal
error handling. This means we no longer need to unwrap specific error types
This doesn't change any manta minio functionality - it just changes how errors are
handled internally and adds a wrapper for a 404 error
2018-01-17 10:38:39 -08:00
Andreas Auernhammer
3f09c17bfe
fix authentication bypass against Admin-API ( #5412 )
...
This change fixes an authentication bypass attack against the
minio Admin-API. Therefore the Admin-API rejects now all types of
requests except valid signature V2 and signature V4 requests - this
includes signature V2/V4 pre-signed requests.
Fixes #5411
2018-01-17 10:36:25 -08:00
ebozduman
24d9d7e5fa
Removes logrus package and refactors logging messages ( #5293 )
...
This fix removes logrus package dependency and refactors the console
logging as the only logging mechanism by removing file logging support.
It rearranges the log message format and adds stack trace information
whenever trace information is not available in the error structure.
It also adds `--json` flag support for server logging.
When minio server is started with `--json` flag, all log messages are
displayed in json format, with no start-up and informational log
messages.
Fixes #5265 #5220 #5197
2018-01-17 07:24:46 -08:00
kannappanr
85580fe0d6
Document pre-existing data in fs mode ( #5365 )
...
Document that pre-existing data in fs mode is safe and
accessible
Fixes #5050
2018-01-17 20:44:31 +05:30
Nitish Tiwari
8a1dc10c60
Update storage class related documents ( #5399 )
...
- Add storage usage info in storage class doc
- Update distributed & erasure code doc with info on storage class
2018-01-17 14:52:42 +05:30
Krishnan Parthasarathi
17301fe45d
Don't delete lock ops entry during state change ( #5388 )
...
lock ops entry is removed in deleteLockEntryForOps, it shouldn't be removed
in status*To* functions.
2018-01-16 12:00:12 -08:00
Aditya Manthramurthy
aa7e5c71e9
Remove upload healing related dead code ( #5404 )
2018-01-15 18:20:39 -08:00
Kaan Kabalak
78a641fc6a
Fix multi-file dropzone upload issue causing bucket listing duplication ( #5392 )
...
This commit fixes the order of the functions inside the selectPrefix
function due to the fact that, as multiple files were being uploaded,
the resetObjects function (that clears the object list) ran repeatedly
for each of these objects, right before the appendObjects function (that
appends the objects being uploaded to the current list of objects) also
ran for all of these objects. This caused all the objects in the bucket
to be repeated in the list for the number of objects that were dragged
into the dropzone.
2018-01-13 22:45:20 +05:30
Harshavardhana
12f67d47f1
Fix a possible race during PutObject() ( #5376 )
...
Under any concurrent removeObjects in progress
might have removed the parents of the same prefix
for which there is an ongoing putObject request.
An inconsistent situation may arise as explained
below even under sufficient locking.
PutObject is almost successful at the last stage when
a temporary file is renamed to its actual namespace
at `a/b/c/object1`. Concurrently a RemoveObject is
also in progress at the same prefix for an `a/b/c/object2`.
To create the object1 at location `a/b/c` PutObject has
to create all the parents recursively.
```
a/b/c - os.MkdirAll loops through has now created
'a/' and 'b/' about to create 'c/'
a/b/c/object2 - at this point 'c/' and 'object2'
are deleted about to delete b/
```
Now for os.MkdirAll loop the expected situation is
that top level parent 'a/b/' exists which it created
, such that it can create 'c/' - since removeObject
and putObject do not compete for lock due to holding
locks at different resources. removeObject proceeds
to delete parent 'b/' since 'c/' is not yet present,
once deleted 'os.MkdirAll' would receive an error as
syscall.ENOENT which would fail the putObject request.
This PR tries to address this issue by implementing
a safer/guarded approach where we would retry an operation
such as `os.MkdirAll` and `os.Rename` if both operations
observe syscall.ENOENT.
Fixes #5254
2018-01-13 22:43:02 +05:30
poornas
0bb6247056
Move nslocking from s3 layer to object layer ( #5382 )
...
Fixes #5350
2018-01-13 10:04:52 +05:30
Andreas Auernhammer
dd202a1a5f
restrict TLS cipher suites of the server ( #5245 )
...
This change restircts the supported cipher suites of the minio server.
The server only supports AEAD ciphers (Chacha20Poly1305 and
AES-GCM)
The supported cipher suites are:
- tls.TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305
- tls.TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305
- tls.TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
- tls.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
- tls.TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
- tls.TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
Fixes #5244 and #5291
2018-01-13 09:12:11 +05:30
Nitish Tiwari
ede504400f
Add validation of xlMeta ErasureInfo field ( #5389 )
2018-01-12 18:16:30 +05:30
Harshavardhana
4b2d04c86f
Add chroot environment doc for minio ( #5366 )
...
Fixes #4659
2018-01-12 07:55:40 +05:30
Nitish Tiwari
42633748db
Update madmin package to return storage class parity ( #5387 )
...
After the addition of Storage Class support, readQuorum
and writeQuorum are decided on a per object basis, instead
of deployment wide static quorums.
This PR updates madmin api to remove readQuorum/writeQuorum
and add Standard storage class and reduced redundancy storage
class parity as return values. Since these parity values are
used to decide the quorum for each object.
Fixes #5378
2018-01-12 07:52:52 +05:30
Aditya Manthramurthy
cd22feecf8
Remove healing of incomplete multipart uploads ( #5390 )
...
Since the server performs automatic clean-up of multipart uploads that
have not been resumed for more than a couple of weeks, it was decided
to remove functionality to heal multipart uploads.
2018-01-11 15:07:43 -08:00
kannappanr
20584dc08f
Remove unnecessary errors printed on the console ( #5386 )
...
Some of the errors printed on server console can be
removed as those error message is unnecessary.
Fixes #5385
2018-01-11 11:42:05 -08:00
Aditya Manthramurthy
8e4eb591c1
Update error response when heal is not implemented ( #5383 )
2018-01-11 10:21:41 -08:00
Nitish Tiwari
1b721d76b1
Assume standard storage class if not set in metadata ( #5370 )
...
If STANDARD storage class is set before starting up Minio server,
but x-amz-storage-class metadata field is not set in a PutObject
request, Minio server defaults to N/2 data and N/2 parity disks.
This PR changes the behaviour to use data and parity disks set in
STANDARD storage class, even if x-amz-storage-class metadata
field is not present in PutObject requests.
2018-01-11 14:58:12 +05:30
Aditya Manthramurthy
f413224b24
Fix config set handler ( #5384 )
...
- Return error when the config JSON has duplicate keys (fixes #5286 )
- Limit size of configuration file provided to 256KiB - this prevents
another form of DoS
2018-01-11 12:36:36 +05:30
Harshavardhana
b526cd7e55
Remove requirement for issued at JWT claims ( #5364 )
...
Remove the requirement for IssuedAt claims from JWT
for now, since we do not currently have a way to provide
a leeway window for validating the claims. Expiry does
the same checks as IssuedAt with an expiry window.
We do not need it right now since we have clock skew check
in our RPC layer to handle this correctly.
rpc-common.go
```
func isRequestTimeAllowed(requestTime time.Time) bool {
// Check whether request time is within acceptable skew time.
utcNow := UTCNow()
return !(requestTime.Sub(utcNow) > rpcSkewTimeAllowed ||
utcNow.Sub(requestTime) > rpcSkewTimeAllowed)
}
```
Once the PR upstream is merged https://github.com/dgrijalva/jwt-go/pull/139
We can bring in support for leeway later.
Fixes #5237
2018-01-10 10:34:00 -08:00
Aditya Manthramurthy
3f8379d07d
Update Elasticsearch documentation with authentication information ( #5381 )
...
- Add documentation to show how to supply credential to access a
secured elasticsearch server.
Fixes #5329
2018-01-10 09:20:42 +05:30
Harshavardhana
7350543f24
Allow x-amz-content-sha256 to be optional for PutObject() ( #5340 )
...
x-amz-content-sha256 can be optional for any AWS signature v4
requests, make sure to skip sha256 calculation when payload
checksum is not set.
Here is the overall expected behavior
** Signed request **
- X-Amz-Content-Sha256 is set to 'empty' or some 'value' or its
not 'UNSIGNED-PAYLOAD'- use it to validate the incoming payload.
- X-Amz-Content-Sha256 is set to 'UNSIGNED-PAYLOAD' - skip checksum verification
- X-Amz-Content-Sha256 is not set we use emptySHA256
** Presigned request **
- X-Amz-Content-Sha256 is set to 'empty' or some 'value' or its
not 'UNSIGNED-PAYLOAD'- use it to validate the incoming payload
- X-Amz-Content-Sha256 is set to 'UNSIGNED-PAYLOAD' - skip checksum verification
- X-Amz-Content-Sha256 is not set we use 'UNSIGNED-PAYLOAD'
Fixes #5339
2018-01-09 12:49:50 +05:30
Nitish Tiwari
56bde5df31
Refactor storage class parsing for Gateway mode ( #5331 )
...
This PR updates the behaviour to print relevant error message
if storage class is set in config.json for gateway
This PR also fixes the case where storage class set via
environment variables is not parsed properly into config.json.
2018-01-08 22:26:13 -08:00
Paul Trunk
bd9cdcf379
Add custom secret names for Docker ( #5355 )
2018-01-09 10:46:25 +05:30
Krishna Srinivas
7c72d14027
Separate the codebase for XL and FS format.json related code ( #5317 )
2018-01-08 14:30:55 -08:00
Harshavardhana
ccd9767b7a
Remove incomplete older chinese translation doc ( #5368 )
2018-01-08 12:48:59 +05:30
Harshavardhana
dae8193bd4
Remove duplicate http constants ( #5367 )
2018-01-08 10:17:48 +05:30
kannappanr
1de3bd6911
Save http trace to a file ( #5300 )
...
Save http trace to a file instead of displaying it onto the console.
the environment variable MINIO_HTTP_TRACE will be a filepath instead
of a boolean.
This to handle the scenario where both json and http tracing are
turned on. In that case, both http trace and json output are displayed
on the screen making the json not parsable. Loging this trace onto
a file helps us avoid that scenario.
Fixes #5263
2018-01-05 11:24:31 -08:00
Kaan Kabalak
de2ce5acb4
Correct color code of Excel file icon ( #5361 )
...
Fixes #5352
2018-01-05 16:19:41 +05:30
Paul Stack
a1a98617ca
gateway/manta: Add support for RBAC ( #5332 )
...
Manta has the ability to allow users to authenticate with a
username other than the main account. We want to expose
this functionality to minio manta gateway.
2018-01-05 13:30:29 +05:30
Andreas Auernhammer
b85c75996d
add support for encrypted TLS private keys ( #5308 )
...
This change adds support for password-protected private keys.
If the private key is encrypted the server tries to decrypt
the key with the password provided by the env variable
MINIO_CERT_PASSWD.
Fixes #5302
2018-01-05 13:18:08 +05:30
Harshavardhana
cc2497f52f
Exitcode with '1' when update is available ( #5354 )
...
--quiet should simply update the binary without any prompt.
Fixes #5347
2018-01-04 21:26:43 +05:30
Nitish Tiwari
1e5fb4b79a
Fix storage class related issues ( #5338 )
...
- Update startup banner to print storage class in capitals. This
makes it easier to identify different storage classes available.
- Update response metadata to not send STANDARD storage class.
This is in accordance with AWS S3 behaviour.
- Update minio-go library to bring in storage class related
changes. This is needed to make transparent translation of
storage class headers for Minio S3 Gateway.
2018-01-04 11:44:45 +05:30
kannappanr
6f7c6fc560
Honor browser enabled config value in startup message ( #5313 )
...
Currently, browser access information is displayed without checking
if browser enabled flag is turned off in config.json. Fixing it to
hide the information if the flag is turned off.
Fixes #5312
2018-01-04 11:00:52 +05:30
Harshavardhana
c0721164be
Automatically set goroutines based on shardSize ( #5346 )
...
Update reedsolomon library to enable feature to automatically
set number of go-routines based on the input shard size,
since shard size is sort of a constant in Minio for
objects > 10MiB (default blocksize)
klauspost reported around 15-20% improvement in performance
numbers on older systems such as AVX and SSE3
```
name old speed new speed delta
Encode10x2x10000-8 5.45GB/s ± 1% 6.22GB/s ± 1% +14.20% (p=0.000 n=9+9)
Encode100x20x10000-8 1.44GB/s ± 1% 1.64GB/s ± 1% +13.77% (p=0.000 n=10+10)
Encode17x3x1M-8 10.0GB/s ± 5% 12.0GB/s ± 1% +19.88% (p=0.000 n=10+10)
Encode10x4x16M-8 7.81GB/s ± 5% 8.56GB/s ± 5% +9.58% (p=0.000 n=10+9)
Encode5x2x1M-8 15.3GB/s ± 2% 19.6GB/s ± 2% +28.57% (p=0.000 n=9+10)
Encode10x2x1M-8 12.2GB/s ± 5% 15.0GB/s ± 5% +22.45% (p=0.000 n=10+10)
Encode10x4x1M-8 7.84GB/s ± 1% 9.03GB/s ± 1% +15.19% (p=0.000 n=9+9)
Encode50x20x1M-8 1.73GB/s ± 4% 2.09GB/s ± 4% +20.59% (p=0.000 n=10+9)
Encode17x3x16M-8 10.6GB/s ± 1% 11.7GB/s ± 4% +10.12% (p=0.000 n=8+10)
```
2018-01-03 13:47:22 -08:00
Minio Trusted
b1fb550d5c
Update yaml files to latest version RELEASE.2018-01-02T23-07-00Z
2018-01-02 23:11:15 +00:00
Andreas Auernhammer
a6318dbdaf
fix timing oracle attack against signature V2/V4 verification ( #5335 )
...
This change replaces the non-constant time comparison of
request signatures with a constant time implementation. This
prevents a timing attack which can be used to learn a valid
signature for a request without knowing the secret key.
Fixes #5334
2018-01-02 12:00:02 +05:30
Harshavardhana
e39d7ddb0f
Fix PostPolicy form tests without hardcoded dates ( #5337 )
...
Fixes #5336
2018-01-01 07:28:10 +05:30
Kaan Kabalak
659f724f4c
Integrate existing remove bucket functionality from newux to current UI ( #5289 )
...
This commit takes the existing remove bucket functionality written by
brendanashworth, integrates it to the current UI with a dropdown for
each bucket, and fixes small issues that were present, like the dropdown
not disappearing after the user clicks on 'Delete' for certain buckets.
This feature only deletes a bucket that is empty (that has no objects).
Fixes #4166
2017-12-29 18:45:44 +05:30
Nitish Tiwari
baaf67d82e
Update config.json guide with details of version 22 ( #5328 )
...
Fixes #5296
2017-12-28 23:04:44 +05:30
A. Elleuch
2244adff07
fix: Better printing of XL config init error ( #5284 )
2017-12-28 23:02:48 +05:30
Nitish Tiwari
e3d841ffd1
Fix config.json parsing to fetch correct storage class ( #5327 )
2017-12-28 14:19:45 +05:30
Minio Trusted
751632d79e
Update yaml files to lastest version RELEASE.2017-12-28T01-21-00Z
2017-12-28 01:26:09 +00:00
Nitish Tiwari
545a9e4a82
Fix storage class related issues ( #5322 )
...
- Add storage class metadata validation for request header
- Change storage class header values to be consistent with AWS S3
- Refactor internal method to take only the reqd argument
2017-12-27 10:06:16 +05:30
Harshavardhana
f25ec31565
Set maxResources appropriately for gateway like server ( #5321 )
2017-12-24 20:09:30 +05:30
Matthieu Paret
374feda237
add HTTPStats to madmin ( #5299 )
2017-12-22 17:47:30 -08:00
A. Elleuch
6ef0161835
fix: Restore empty files when healing ( #5257 )
...
HealFile() does not process the case when an empty file is lost in
some disks. Since, Reedsolomon erasure doesn't handle restoring empty
data, HealFile will create empty files similarly to CreateFile().
2017-12-22 14:57:57 -08:00
Nitish Tiwari
1a3dbbc9dd
Add x-amz-storage-class support ( #5295 )
...
This adds configurable data and parity options on a per object
basis. To use variable parity
- Users can set environment variables to cofigure variable
parity
- Then add header x-amz-storage-class to putobject requests
with relevant storage class values
Fixes #4997
2017-12-22 16:58:13 +05:30
Aditya Manthramurthy
f1355da72e
Add base64 encoded MD5 output for Hash Reader ( #5315 )
...
- Use it to send the Content-MD5 header correctly encoded to S3
Gateway
- Fixes a bug in PutObject (including anonymous PutObject) and
PutObjectPart with S3 Gateway found when testing with Mint.
2017-12-21 17:27:33 -08:00
Krishnan Parthasarathi
bbe521ffec
ReInitDisk RPC handler should use retryStorage ( #5310 )
2017-12-21 12:28:01 +05:30
Paul Stack
7d75d61621
Add Support for Manta Object Storage as a Gateway ( #5025 )
...
Manta is an Object Storage by [Joyent](https://www.joyent.com/ )
This PR adds initial support for Manta. It is intended as non-production
ready so that feedback can be obtained.
2017-12-20 13:37:56 +05:30
Harshavardhana
1f77708a30
Limit number of connections upto system maxlimit ( #5109 )
2017-12-20 13:30:14 +05:30
Timon Wong
84fc78d60f
Implement Alibaba Cloud OSS gateway support ( #5103 )
2017-12-19 13:55:17 +05:30
poornas
a182fe8c15
update steps to make changes to config.json ( #5292 )
2017-12-17 21:00:12 -08:00
Harshavardhana
819d1e80c6
Add more delays on distributed startup for slow network ( #5240 )
...
Refer #5237
2017-12-16 08:25:29 -08:00
Kaan Kabalak
ffdf115bf2
Fix mobile issue where checkbox was seen over sidebar on list item hover ( #5288 )
...
Fixes #5287
2017-12-15 12:36:05 -08:00
Harshavardhana
eb7c690ea9
Support in-place upgrades of new minio binary and releases. ( #4961 )
...
This PR allows 'minio update' to not only shows update banner
but also allows for in-place upgrades.
Updates are done safely by validating the downloaded
sha256 of the binary.
Fixes #4781
2017-12-15 12:33:42 -08:00
Nitish Tiwari
8c08571cd9
Update Kubernetes example yaml files ( #5278 )
...
Removed the non production ready Kubernetes constructs that are not needed
for standard Minio deployment. General cleanup of the documents.
2017-12-12 10:29:00 +05:30
kannappanr
2853fa1882
Remove logger field info from docs ( #5281 )
...
Logger field is removed from the docs,
as it has been removed from the config file.
2017-12-11 13:20:05 -08:00
Harshavardhana
1672c73988
Fix snap docs with instructions to run ( #5266 )
2017-12-07 08:38:58 +05:30
Harshavardhana
5476415016
Remove unused deps for cloudresourcemanager ( #5274 )
2017-12-06 16:41:56 -08:00
Kaan Kabalak
67ac74471d
Fix error message on browser window resize when user has no buckets ( #5275 )
...
This commit handles the case where the list of buckets is empty on the
listObjects function of actions.js
Fixes #5267
2017-12-06 15:52:23 -08:00
kannappanr
a1c1a18dc5
Remove "logger" field from config.json ( #5268 )
...
File logging removed as part of improvement to server logging.
config.json format updated to version 21.
Fixes #5176
2017-12-06 12:48:29 +05:30
Harshavardhana
eb2894233c
Convert gateways into respective packages ( #5200 )
...
- Make azure gateway a package
- Make b2 gateway a package
- Make gcs gateway a package
- Make s3 gateway a package
- Make sia gateway a package
2017-12-05 17:58:09 -08:00
Nitish Tiwari
52e382b697
Update go version in snap file ( #5256 )
2017-12-04 15:49:31 -08:00
techknowlogick
0d435e11b1
Change container name in b2 docs ( #5259 )
2017-12-04 22:34:00 +05:30
Harshavardhana
2755a0b763
Check if SSL is configured to validate input arguments ( #5252 )
...
This PR handles following situations
- secure endpoints provided, server should fail to start
if TLS is not configured
- insecure endpoints provided, server starts ignoring
if TLS is configured or not.
Fixes #5251
2017-12-04 12:17:12 +05:30
Aditya Manthramurthy
043e030a4a
Add CopyObjectPart support to gateway S3 ( #5213 )
...
- Adds a metadata argument to the CopyObjectPart API to facilitate
implementing encryption for copying APIs too.
- Update vendored minio-go - this version implements the
CopyObjectPart client API for use with the S3 gateway.
Fixes #4885
2017-12-02 08:33:59 +05:30
Harshavardhana
490c30f853
erasure: Support cleaning up of stale multipart objects ( #5250 )
...
Just like our single directory/disk setup, this PR brings
the functionality to cleanup stale multipart objects
older > 2 weeks.
2017-11-30 18:11:42 -08:00
Harshavardhana
59749a2b85
erasure: Remove prefix based listing support on ListMultipartUploads ( #5248 )
...
Previously we have removed this support under FS on #4996 ,
deprecate this on erasure coded backend as well to simplify
our multipart support.
2017-11-30 15:58:46 -08:00
Michael Lynch
fc3cf97b81
Removing isValidObjectName from Sia gateway ( #5243 )
...
This check incorrectly rejects most valid filenames. The only filenames Sia
forbids are leading forward slashes and path traversal characters, but it's
better to simply allow Sia to reject invalid names on its own rather than try
to anticipate errors from Sia:
https://github.com/NebulousLabs/Sia/blob/master/doc/api/Renter.md#path-parameters-4
2017-11-30 14:43:21 -08:00
Harshavardhana
d45a8784fc
Fix hash order to generate more even distribution ( #5247 )
...
The problem in existing code was the following line
```
start := int(keyCrc%uint32(cardinality)) | 1
```
A given a value of N cardinality the ending result
because of the the bitwise '|' would lead to always
higher affinity to odd sequences.
As can be seen from the test cases that this can
lead to many objects being allocated the same set
of disks or atleast the first disk is an odd disk
always. This introduces a performance problem
for majority of the objects under concurrent load.
Remove `| 1` to provide a more cleaner distribution
and the new code will be.
```
start := int(keyCrc % uint32(cardinality))
```
Thanks to Krishna Srinivas for pointing out the bitwise
situation here.
2017-11-30 12:57:03 -08:00
Nitish Tiwari
6d7319380c
Add Transparent Hugepage information ( #5246 )
...
Fixes #5242
2017-11-30 12:18:00 -08:00
Krishna Srinivas
14e6c5ec08
Simplify the steps to make changes to config.json ( #5186 )
...
This change introduces following simplified steps to follow
during config migration.
```
// Steps to move from version N to version N+1
// 1. Add new struct serverConfigVN+1 in config-versions.go
// 2. Set configCurrentVersion to "N+1"
// 3. Set serverConfigCurrent to serverConfigVN+1
// 4. Add new migration function (ex. func migrateVNToVN+1()) in config-migrate.go
// 5. Call migrateVNToVN+1() from migrateConfig() in config-migrate.go
// 6. Make changes in config-current_test.go for any test change
```
2017-11-29 13:12:47 -08:00
A. Elleuch
98d07210e7
fix: Ignore logging some tcp routine errors ( #5097 )
2017-11-28 13:51:17 -08:00
Nitish Tiwari
6923630389
Update bucket notification docs to mention events supported ( #5235 )
...
Fixes #4898
2017-11-28 15:41:26 +05:30
Nitish Tiwari
0c73c81919
Cleanup TLS setup document ( #5231 )
...
Fixes #4959
2017-11-28 15:15:50 +05:30
Harshavardhana
a46b640da3
gateway/sia: Support proper {make,get}Bucket operations ( #5229 )
...
Current implementation we faked the makeBucket operations
to allow for s3 clients to behave properly. But instead
we can create a placeholder zero byte file instead, which
is a hexadecimal representation of the bucket name itself.
2017-11-28 13:40:44 +05:30
Krishna Srinivas
71f9d2beff
Increase maximum size of PUT request to 5TB ( #5241 )
...
fixes #5148
2017-11-28 12:59:02 +05:30
Michael Lynch
cf414a6053
Fixing Sia file uploads ( #5233 )
...
The Sia gateway had a bug with uploading that prevented the user's uploads
from reaching the Sia backend. The PutObject function called fsRemoveFile at
the end of the function, which didn't give the Sia backend enough time to
upload the file to the Sia network.
This adds a goroutine that watches the file upload progress and doesn't delete
the file until the upload reaches 100% complete.
Note that this solution has the limitation where if the minio process dies in
the middle of upload, it will leave orphaned files in the SIA_TEMP directory
that the user will need to remove manually.
2017-11-28 12:25:15 +05:30
Harshavardhana
05b395e81d
Add more unit tests for azure/gcs/b2 gateway ( #5236 )
...
Also adds a blazer SDK update exposing
error response headers.
2017-11-27 18:29:22 -08:00
Paul Nicholls
6a2d7ae808
gateway/azure: ListParts return an empty list if no parts uploaded yet ( #5230 )
...
This makes azure ListParts implementation behave more like S3
by returning an empty list rather than an error when no parts have
been uploaded yet.
2017-11-27 17:42:27 -08:00
Eduardo Reyes
685afb6749
Fixes last item in the list menu being blocked. ( #5234 )
...
This commit fixes an issue where the last item's menu on a list of files
that scrolls gets blocked by the floating add button.
The fix is simply add the same padding that we use for the responsive
view, since it works just fine in responsive.
2017-11-27 14:02:57 -08:00
Harshavardhana
8efa82126b
Convert errors tracer into a separate package ( #5221 )
2017-11-25 11:58:29 -08:00
Frank Wessels
6e6aeb6a9e
Updated version of klauspost/reedsolomon using proper AVX2 instructions as well a providing support for Cauchy matrices. ( #5215 )
2017-11-24 14:23:20 -08:00
Arnaud de Mouhy
c9e00ae0a5
Add support for dynamic address in docker healthcheck ( #5228 )
2017-11-24 14:11:26 +05:30
Nitish Tiwari
08e0698b7e
Update docs to latest Minio server release ( #5227 )
2017-11-23 10:19:53 +05:30
Harshavardhana
135a6a7bb4
Add chinese translation docs. ( #5224 )
2017-11-22 15:15:40 -08:00
Dee Koder
8b4d7048f8
fixed typo variable under domain heading. ( #5223 )
2017-11-22 12:37:00 -08:00
David G
f4d4ea5c36
Implement Sia Gateway ( #5114 )
2017-11-22 12:12:10 -08:00
Aditya Manthramurthy
d1a6c32d80
Improve make and make install messages ( #5207 )
2017-11-21 16:22:01 -08:00
Krishna Srinivas
bbd05a8f1c
gateway-gcs: Close the writer with error in case of any errors. ( #5217 )
...
fixes #5216
2017-11-21 14:45:37 -08:00
Krishna Srinivas
4393afb7e2
Remove checkGCSProjectID() as it needs extra permission setting ( #5210 )
...
fixes #5209
2017-11-21 10:43:39 -08:00
Krishna Srinivas
1a53734477
Rename UserDefined to UserMetadata for events ( #5206 )
...
fixes #5165
2017-11-20 15:32:25 -08:00
Andreas Auernhammer
e95c0bb913
return AWS compliant error if SSE-C key is wrong ( #5203 )
...
This PR changes the behavior of DecryptRequest.
Instead of returning `object-tampered` if the client provided
key is wrong DecryptRequest will return `access-denied`.
This is AWS S3 behavior.
Fixes #5202
2017-11-20 14:04:10 -08:00
Krishna Srinivas
fce556b8a0
Support for ListObjectParts in azure-gateway ( #5198 )
...
fixes #5169
2017-11-20 14:03:20 -08:00
Andreas Auernhammer
b97f99766f
add benchmarks for erasure backend ( #5084 )
...
This change adds benchmarks for erasure read/write in different setups.
2017-11-17 14:57:04 -08:00
Nitish Tiwari
f7b6f7b22f
Update getObjectInfo to stat for objects with trailing / ( #5179 )
...
Apache Spark sends getObject requests with trailing "/".
This PR updates the getObjectInfo to stat for files
even if they are sent with trailing "/".
Fixes #2965
2017-11-16 16:00:27 -08:00
Krishnan Parthasarathi
2a0a62b78d
Return ErrContentSHA256Mismatch when sha256sum is invalid ( #5188 )
2017-11-16 11:13:04 -08:00
Krishnan Parthasarathi
67f66c40c1
Fix ListenBucketNotification deadlock ( #5028 )
...
Previously ListenBucketNotificationHandler could deadlock with
PutObjectHandler's eventNotify call when a client closes its
connection. This change removes the cyclic dependency between the
channel and map of ARN to channels by using a separate done channel to
signal that the client has quit.
2017-11-16 10:56:06 -08:00
Krishna Srinivas
5a2bdf6959
Handle Path validation inside the PostPolicy handler ( #5192 )
2017-11-15 14:10:45 -08:00
silenceshell
51e78a3e20
fix a typo ( #5187 )
2017-11-15 15:12:14 +05:30
Harshavardhana
9eb52ec7c7
Remove release scripts for minio. ( #5181 )
...
Use `GOOS=<osname> go build` to build minio for any platform of choice.
2017-11-14 17:05:40 -08:00
Harshavardhana
0827a2747b
api: CopyObject should return NotImplemented for now ( #5183 )
...
Commit ca6b4773ed introduces SSE-C
support for HEAD, GET, PUT operations but since we do not
implement CopyObject() we should return NotImplemented.
2017-11-14 16:57:19 -08:00
Krishna Srinivas
e7a724de0d
Virtual host style S3 requests ( #5095 )
2017-11-14 16:56:24 -08:00
Krishna Srinivas
d57d57ddf5
Update ui-assets.go ( #5185 )
2017-11-14 16:22:35 -08:00
Harshavardhana
a4d6195244
Add public data-types for easier external loading ( #5170 )
...
This change brings public data-types such that
we can ask projects to implement gateway projects
externally than maintaining in our repo.
All publicly exported structs are maintained in object-api-datatypes.go
completePart --> CompletePart
uploadMetadata --> MultipartInfo
All other exported errors are at object-api-errors.go
2017-11-14 13:55:10 +05:30
Krishna Srinivas
7d3eaf79ff
Set Minio user-agent for GCS calls ( #5154 )
2017-11-13 19:06:51 -08:00
kannappanr
b63c37b28e
Return MethodNotAllowed error in PostPolicyBucketHandler if URL contains object name ( #5142 )
...
S3 spec requires that MethodNotAllowed error be return if object name is part
of the URL.
Fix postpolicy related unit tests to not set object name as part of target URL.
Fixes #5141
2017-11-13 16:30:20 -08:00
Harshavardhana
8d59f35523
Add GetInfo() support for solaris ( #5174 )
...
Fixes #5173
2017-11-13 12:54:38 -08:00
kannappanr
f460eceb6d
Check for value > 7 days in X-Amz-Expires header. ( #5163 )
...
Add a check to see if the X-Amz-Expires header in the presigned URL is less than 7 days.
Fixes #5162
2017-11-13 12:54:03 -08:00
Harshavardhana
d10679866c
Fix minio distributed setup to properly work on windows ( #5152 )
...
On windows having a preceding "/" will cause problems, if the
command line already has C:/<export-folder/ in it. Final resulting
path on windows might become C:/C:/ this will cause problems
of starting minio server properly in distributed mode on windows.
As a special case make sure to trim off the separator.
NOTE: It is also perfectly fine for windows users to have a path
without C:/ since at that point we treat it as relative path
and obtain the full filesystem path as well. Providing C:/
style is necessary to provide paths other than C:/,
such as F:/, D:/ etc.
Another additional benefit here is that this style also
supports providing UNC paths as well.
Fixes #5136
2017-11-12 08:09:53 +05:30
Andreas Auernhammer
a79a7e570c
replace SSE-C key derivation scheme ( #5168 )
...
This chnage replaces the current SSE-C key derivation scheme. The 'old'
scheme derives an unique object encryption key from the client provided key.
This key derivation was not invertible. That means that a client cannot change
its key without changing the object encryption key.
AWS S3 allows users to update there SSE-C keys by executing a SSE-C COPY with
source == destination. AWS probably updates just the metadata (which is a very
cheap operation). The old key derivation scheme would require a complete copy
of the object because the minio server would not be able to derive the same
object encryption key from a different client provided key (without breaking
the crypto. hash function).
This change makes the key derivation invertible.
2017-11-10 17:21:23 -08:00
Harshavardhana
16ecaac4fc
Help message should prioritize gateway after server ( #5153 )
...
Currently gateway is listed as a command after {version, update}
which is incorrect, fix it.
2017-11-08 13:38:53 -08:00
Andreas Auernhammer
ca6b4773ed
add SSE-C support for HEAD, GET, PUT ( #4894 )
...
This change adds server-side-encryption support for HEAD, GET and PUT
operations. This PR only addresses single-part PUTs and GETs without
HTTP ranges.
Further this change adds the concept of reserved object metadata which is required
to make encrypted objects tamper-proof and provide API compatibility to AWS S3.
This PR adds the following reserved metadata entries:
- X-Minio-Internal-Server-Side-Encryption-Iv ('guarantees' tamper-proof property)
- X-Minio-Internal-Server-Side-Encryption-Kdf (makes Key-MAC computation negotiable in future)
- X-Minio-Internal-Server-Side-Encryption-Key-Mac (provides AWS S3 API compatibility)
The prefix `X-Minio_Internal` specifies an internal metadata entry which must not
send to clients. All client requests containing a metadata key starting with `X-Minio-Internal`
must also rejected. This is implemented by a generic-handler.
This PR implements SSE-C separated from client-side-encryption (CSE). This cannot decrypt
server-side-encrypted objects on the client-side. However, clients can encrypted the same object
with CSE and SSE-C.
This PR does not address:
- SSE-C Copy and Copy part
- SSE-C GET with HTTP ranges
- SSE-C multipart PUT
- SSE-C Gateway
Each point must be addressed in a separate PR.
Added to vendor dir:
- x/crypto/chacha20poly1305
- x/crypto/poly1305
- github.com/minio/sio
2017-11-07 15:18:59 -08:00
Krishna Srinivas
7e7ae29d89
browser: Remove hardcoding of minioBrowserPrefix=/minio ( #5048 )
...
This enable reverse proxy of minio-browser. Fixes #5040
2017-11-06 15:59:37 -08:00
Krishnan Parthasarathi
7d18f00116
Make GCS multipart upload failures S3-compatible ( #5138 )
2017-11-06 10:09:21 -08:00
Harshavardhana
719f8c258a
fix content-sha256 verification for presigned PUT ( #5137 )
...
It is possible that x-amz-content-sha256 is set through
the query params in case of presigned PUT calls, make sure
that we validate the incoming x-amz-content-sha256 properly.
Current code simply just allows this without honoring the
set x-amz-content-sha256, fix it.
2017-11-05 16:32:19 +05:30
Harshavardhana
dcdb07433a
Support conditions for ListMultipartUploads and ListParts ( #5130 )
2017-11-02 11:39:48 -07:00
kannappanr
26e9f78a86
Display help when access/secret key is not set ( #5132 )
...
Display help message, when access and secret keys are not set in
any of the gateway.
Fixes #5124
2017-11-01 11:45:27 -07:00
Nitish Tiwari
34a1b58a75
Remove redirectHeaders method ( #5120 )
...
As of go 1.8, headers are copied on redirect, so we no longer need
to do this manually.
See https://github.com/golang/go/issues/4800 and
https://go-review.googlesource.com/c/go/+/28930 for more context on go
behaviour.
Fixes #5042
2017-11-01 12:43:13 +05:30
Bala FA
32c6b62932
move credentials as separate package ( #5115 )
2017-10-31 11:54:32 -07:00
Harshavardhana
8d584bd819
Remove unused value from retry-storage ( #5129 )
2017-10-29 15:43:16 +05:30
Nitish Tiwari
031767791d
Update GitHub PR template to include mint tests ( #4970 )
...
Mint is the functional testing platform for Minio server. So, it is
important that each PR to Minio server repository is checked for
addition/updating of Mint test cases.
2017-10-28 07:14:19 -07:00
Nitish Tiwari
3b917067d9
Update sample yaml files to latest release RELEASE.2017-10-27T18-59-02Z ( #5127 )
2017-10-28 07:11:53 -07:00
Harshavardhana
203ac8edaa
Bucket policies should use minio-go/pkg/policy instead. ( #5090 )
2017-10-27 16:14:06 -07:00
Harshavardhana
8bbfb1b714
Allow event notifications to work without region ( #5119 )
...
Fixes #5101
2017-10-27 15:09:55 -07:00
Harshavardhana
b4ddccc2f7
browser: Return a more descriptive error for HTTP 500 ( #5112 )
2017-10-27 15:09:14 -07:00
Timon Wong
6400f506da
Simplify gateway backend registration ( #5111 )
2017-10-27 15:07:46 -07:00
Frank Wessels
7195ac7f14
Add space to error message ( #5108 )
2017-10-27 15:07:14 -07:00
Krishna Srinivas
03df692ae2
Support for bosh/pcf user-agent when querying for updates. ( #5116 )
2017-10-26 18:53:45 -07:00
kannappanr
a011fe8450
"0" offset is ignored in GetObject method in Azure Gateway code ( #5118 )
...
In GetObject method, Check if startoffset is a non-negative value.
Ignore check for startOffset > and check for only length > 0.
Fixes minio/mint#191
2017-10-26 18:01:46 -07:00
kannappanr
95d97c2d6d
GCS gateway to return error in getBucketPolicy, when no policy is set ( #5117 )
...
Return NoSuchBucketPolicy error when there is no policy set.
Fixes minio/mint#199
2017-10-26 18:01:00 -07:00
Bala FA
bc8b936d4b
convert ETag properly for all gateways ( #5099 )
...
Previously ID/ETag from backend service is used as is which causes
failure on s3cmd like tools where those tools use ETag as checksum to
validate data. This is fixed by prepending "-1".
Refer minio/mint#193 minio/mint#201
2017-10-26 10:17:07 -07:00
Aditya Manthramurthy
d23ded0d83
Use retryableStorage after healing format.json ( #5105 )
...
- Previously networkStorage was being used and this lead to errors
when listing with a down server/disk
Fixes #5089
2017-10-26 09:52:23 -07:00
Julien Maitrehenry
db3fed2279
Fix s3MetaToAzureProperties Content-Md5 key ( #5068 )
2017-10-25 11:00:07 -07:00
A. Elleuch
866dffcd62
log: Store http request/responses in a log file ( #4804 )
...
When MINIO_TRACE_DIR is provided, create a new log file and store all
HTTP requests + responses data, body are excluded to reduce memory
consumption. MINIO_HTTP_TRACE=1 enables logging. Use non mem
consuming http req/resp recorders, the maximum is about 32k per request.
This logs to STDOUT, body logging is disabled for PutObject PutObjectPart
GetObject.
2017-10-25 10:59:53 -07:00
Harshavardhana
5eb210dd2e
Set etag properly to calculated value if available ( #5106 )
...
Fixes #5100
2017-10-24 12:25:42 -07:00
kula
758d5458f0
Update documenation to reflect correct default region of '' ( #5101 )
...
Ever since commit 5db1e9f3dd the
default region as been '', instead of 'us-east-1'. Update
documentation to reflect this, in particular, documentation about
notifications.
2017-10-23 11:57:40 -07:00
asubmani
8a40da3fd0
Update azure.md ( #5086 )
2017-10-23 14:38:38 +05:30
Harshavardhana
1d8a8c63db
Simplify data verification with HashReader. ( #5071 )
...
Verify() was being called by caller after the data
has been successfully read after io.EOF. This disconnection
opens a race under concurrent access to such an object.
Verification is not necessary outside of Read() call,
we can simply just do checksum verification right inside
Read() call at io.EOF.
This approach simplifies the usage.
2017-10-22 11:00:34 +05:30
Krishna Srinivas
65a817fe8c
Move packages between dependencies and devDependencies ( #5094 )
...
This change is needed to generate the right Open Source Disclosure File.
2017-10-21 12:21:32 +05:30
Julien Maitrehenry
1256b0b818
Fix multipart upload etag on azure gateway ( #5055 )
2017-10-20 14:00:18 -07:00
Krishna Srinivas
7e05b826fa
Figure out projectID for GCS automatically from credentials.json ( #5029 )
...
fixes #5027
2017-10-20 13:59:12 -07:00
Harshavardhana
d82a1da511
Fix notification unmarshalling, unmarshal only when size is > 0 ( #5087 )
...
Fixes #5085
2017-10-20 13:57:57 -07:00
Frank Wessels
f598f4fd1b
Fix typo in comment ( #5088 )
2017-10-20 15:08:15 +05:30
Krishna Srinivas
e65d8fdc7d
Remove unused packages from vendor.json ( #5079 )
2017-10-19 15:14:10 +05:30
A. Elleuch
b919462610
fix: Avoid teeing data into a null cache buffer ( #5070 )
...
In some cases, Cache manager returns ErrCacheFull error when creating a
new cache buffer but the code still sends object data to nil cache buffer data.
2017-10-18 14:42:10 -07:00
Nitish Tiwari
8287ab091c
Ignore file not found error for multipart-uploads ( #5065 )
...
Dont print the error errFileNotFound, as it is expected that concurrent
complete-multipart-uploads or abort-multipart-uploads would have deleted
the file, and the file may not be found
Fixes: https://github.com/minio/minio/issues/5056
2017-10-18 14:26:20 -07:00
Harshavardhana
f25bec6bf1
Do not attempt to generate URLToken for anonymous downloads ( #5078 )
...
This is a regression since last release - fixes #5076
2017-10-18 11:14:27 +05:30
Krishna Srinivas
75865efb0e
fs: All parts except the last part should be of the same size ( #5045 )
...
fixes #4881
2017-10-17 12:01:28 -07:00
Harshavardhana
3d2d63f71e
Fix gateway docs remove redundant files ( #5072 )
2017-10-17 08:12:44 +05:30
Harshavardhana
53e133e844
Remove NotSupported error redundant with NotImplemented ( #5074 )
2017-10-17 08:11:06 +05:30
Harshavardhana
b2cbade477
Support creating empty directories. ( #5049 )
...
Every so often we get requirements for creating
directories/prefixes and we end up rejecting
such requirements. This PR implements this and
allows empty directories without any new file
addition to backend.
Existing lower APIs themselves are leveraged to provide
this behavior. Only FS backend supports this for
the time being as desired.
2017-10-16 17:20:54 -07:00
Harshavardhana
0c0d1e4150
Implement backblaze-b2 gateway support ( #5002 )
...
Fixes https://github.com/minio/minio/issues/4072
2017-10-13 16:26:16 +05:30
Harshavardhana
3d0dced23c
Remove go1.9 specific code for windows ( #5033 )
...
Following fix https://go-review.googlesource.com/#/c/41834/ has
been merged upstream and released with go1.9.
2017-10-13 15:31:15 +05:30
Nitish Tiwari
ad53c5d859
Remove body from POST request in webhook ( #5067 )
...
When webhook notification is configured, Minio server tries to lookup the
webhook endpoint by making a POST request with body set as releasetag.
We can remove the body from the POST request as the POST body does not
add any specific value.
This discussion on IETF group says empty POSTs are okay
http://lists.w3.org/Archives/Public/ietf-http-wg/2010JulSep/0272.html
Fixes: https://github.com/minio/minio/issues/5066
2017-10-13 13:29:01 +05:30
Harshavardhana
45463b1d6b
Fix azure metadata handling for all incoming PUT requests ( #5038 )
...
s3cmd cli fails when trying to upload a file to azure gateway.
Previous fixes in azure to handle client side encryption alone
did not completely address the problem.
We need to possibilly convert all the x-amz-meta-<name>
, i.e specifically <name> should be converted into a
C# identifier as mentioned in the docs for `put-blob`.
https://docs.microsoft.com/en-us/rest/api/storageservices/put-blob
```
s3cmd put README.md s3://myanis/
upload: 'README.md' -> 's3://myanis/README.md' [1 of 1]
4598 of 4598 100% in 0s 47.24 kB/s done
upload: 'README.md' -> 's3://myanis/README.md' [1 of 1]
4598 of 4598 100% in 0s 50.47 kB/s done
ERROR: S3 error: 400 (InvalidArgument): Your metadata headers are not supported.
```
There is a separate issue with s3cmd after this fix is applied where
the ETag is wronly validated https://github.com/s3tools/s3cmd/issues/880
But that is an upstream s3cmd problem which wrongly interprets ETag
to be md5sum of the content that was uploaded.
2017-10-12 12:16:24 -07:00
Nitish Tiwari
d5895d3243
Update docker swarm doc to use docker volume prune ( #5053 )
...
docker prune command is an improvement over previous two step process
of removal of unused volumes in Swarm.
2017-10-11 10:17:57 -07:00
Krishna Srinivas
db1edfe487
Fix data race bug in the testcase TestHTTPListenerAcceptParallel ( #5043 )
2017-10-11 10:17:37 -07:00
Bala FA
9c16f73334
pkg/http: use port 65432 than 9000 for unit tests ( #5021 )
...
Fixes #5014
2017-10-11 10:16:38 -07:00
Nitish Tiwari
f50bec2987
Update Minio version in docs and example yaml files ( #5054 )
2017-10-10 17:35:00 -07:00
Harshavardhana
e2bff5cdc0
Move Dockerfile to go 1.9.1 ( #5047 )
...
Additionally add Dockerfile.dev to build local development images.
2017-10-10 14:17:16 -07:00
Julien Maitrehenry
02a5f1e96a
Add b2s method on pkg/disk/type_bsd.go ( #5036 )
2017-10-10 02:27:28 -07:00
Harshavardhana
4deefa3695
tests: Remove dependency on check.v1 ( #5034 )
...
This PR addresses a long standing dependency on
`gopkg.in/check.v1` project used for our tests.
All tests are re-written to use the go default
testing framework instead.
There was no reason for us to use an external
package where Go tools are sufficient for this.
2017-10-10 02:14:42 -07:00
Bala FA
d28b3d8801
Move to go1.9.1 as default environment. ( #5041 )
2017-10-09 22:23:59 -07:00
Harshavardhana
099b5293a9
Move gateway unsupported functions into a common struct. ( #5009 )
...
This is done to avoid repeated declaration of not-implemented
functions for each gateway. It also avoids a possible bug in go
https://github.com/golang/go/issues/18468 which is triggered on
our multiple PRs already.
2017-10-09 16:41:35 -07:00
Aditya Manthramurthy
4a0a491ca1
Refactor update check code ( #5020 )
...
- Add release-time conversion helpers
- Split GetCurrentReleaseTime() into two simpler functions.
- Avoid appending strings when assembling user-agent string.
- Reorder release info URLs to check the newer URLs earlier.
- Remove trivial low-level functions created solely for the purpose of
writing tests.
- Remove some unnecessary tests.
2017-10-09 16:12:13 -07:00
Harshavardhana
db6b6e9518
S3 peers should be initialized properly ( #5024 )
...
Fixes #4991
2017-10-08 20:23:42 -07:00
Bala FA
d8a11c8f4b
fix build failure for go1.9 ( #4872 )
2017-10-06 17:00:15 -07:00
Harshavardhana
0b546ddfd4
Return errors in PutObject()/PutObjectPart() if input size is -1. ( #5015 )
...
Amazon S3 API expects all incoming stream has a content-length
set it was superflous for us to support object layer which supports
unknown sized stream as well, this PR removes such requirements
and explicitly error out if input stream is less than zero.
2017-10-06 09:38:01 -07:00
Krishna Srinivas
d1712a46a7
Enable ListMultipartUploads and ListObjectParts for FS ( #4996 )
...
* Enable ListMultipartUploads and ListObjectParts for FS.
Previously we had disabled ListMultipartUploads and ListObjectParts
to see if any clients break. Docker registry broke. This patch
enables ListMultipartUploads and ListObjectParts, however
ListMultipartUploads with prefix based listing is not
supported (which is not used by docker registry anyway).
i.e ListMultipartUploads will need exact object name.
2017-10-05 16:08:25 -07:00
Harshavardhana
c46c2a6dd6
Fix healthcheck issue with Docker containers ( #5011 )
...
Fixes #5010
2017-10-05 13:52:57 -07:00
Bala FA
88938340b3
remove all dead codes ( #5019 )
...
Fixes #5012
2017-10-05 12:25:45 -07:00
Krishnan Parthasarathi
13a7033505
Translate s3 gateway errors at object layer ( #5006 )
2017-10-05 12:24:45 -07:00
Bala FA
670e3e0c8b
verify: add minio gateway s3 setup for verification tests ( #4976 )
2017-10-04 11:16:39 -07:00
Harshavardhana
89d528a4ed
Allow CopyObject() in S3 gateway to support metadata ( #5000 )
...
Fixes #4924
2017-10-03 10:38:25 -07:00
A. Elleuch
53f3d2fd65
Push max threads to little less than kernel limit ( #5001 )
...
Let Minio server use more threads than allowed by golang runtime. This
is important to better deal with high load.
2017-10-03 10:37:45 -07:00
A. Elleuch
a4f26aec00
fix: List buckets response should return UTC modtime ( #5004 )
2017-10-03 10:34:51 -07:00
Bala FA
60cc6184d2
azure: handle list objects properly ( #4953 )
...
When removing `minio.sys.tmp` prefixed entries and objects/prefixes is
empty, populate till we get all valid entries.
2017-09-29 12:08:23 -07:00
poornas
ce2d185211
Add maxKeys validation for azure and gcs gateway ( #4999 )
...
Gateway implementation of ListObjectsV1 does not validate maxKeys range.
Raise an InvalidArgument when maxKeys is negative so that ListObjects
call is compatible with S3 on all gateways.
2017-09-29 12:07:44 -07:00
Aditya Manthramurthy
b05351c420
Fix CopyObject with metadata for Azure gateway ( #4986 )
2017-09-29 10:58:40 -07:00
Harshavardhana
b415c600e1
Add bucketName checks for azure and s3 gateway in GetBucketInfo. ( #4992 )
...
Gateway interface implementations of GetBucketInfo() under
azure and s3 gateway did not perform any bucketname input
validation resulting in incorrect responses when the tests
are expecting InvalidBucketName.
Fixes #4983
2017-09-28 19:37:09 -07:00
Aditya Manthramurthy
4c9fae90ff
Optimize healObject by eliminating extra data passes ( #4949 )
2017-09-28 15:57:19 -07:00
Aditya Manthramurthy
94670a387e
Update Azure SDK ( #4985 )
2017-09-28 15:23:46 -07:00
Nitish Tiwari
789270af3c
Vendorize latest minio-go ( #4989 )
...
As minio-go behavior is fixed to treat empty byte arrays and nil byte
arrays in the same manner. These changes are needed in minio to
address the PutObject failure for S3 Gateway.
Fixes: https://github.com/minio/minio/issues/4974 ,
https://github.com/minio/minio-java/issues/615
2017-09-28 08:10:38 -07:00
fangyuxiang
a5fbe1e16c
fs: optimize multipart clean work ( #4944 )
2017-09-28 08:09:28 -07:00
Bala FA
3c836b5f34
tests: remove test cases not applicable for docker. ( #4951 )
...
When running `make test` in docker, two test cases cause hanging.
This Patch fixes the problem by removing those test cases.
Thanks to @ws141 for identifying the problem.
2017-09-27 13:51:26 -07:00
Andreas Auernhammer
02af37a394
optimize memory allocs during reconstruct ( #4964 )
...
The reedsolomon library now avoids allocations during reconstruction.
This change exploits that to reduce memory allocs and GC preasure during
healing and reading.
2017-09-27 10:29:42 -07:00
Harshavardhana
4879cd73f8
api: MakeBucket() should honor regions properly. ( #4969 )
...
Fixes #4967
2017-09-26 20:13:06 -07:00
Aditya Manthramurthy
b5dc4b5873
Fix CopyObject with metadata for GCS Gateway ( #4971 )
2017-09-26 11:04:42 -07:00
Harshavardhana
6dcfaa877c
Fix signature v2 handling for resource names ( #4965 )
...
Previously we were wrongly adding `?` as part
of the resource name, add a test case to check
if this is handled properly.
Thanks to @kannappanr for reproducing this.
Without this change presigned URL generated with following
command would fail with signature mismatch.
```
aws s3 presign s3://testbucket/functional-tests.sh
```
2017-09-26 11:00:07 -07:00
Tamer Fahmy
0bf981278e
Provide the correct free block size volume/disk information ( #4943 )
...
On *NIX platforms the statfs(2) system call returns a struct containing both the
free blocks in the filesystem (Statfs_t.Bfree) and the free blocks available to
the unprivileged or non-superuser (Statfs_t.Bavail).
The `Bfree` and `Bavail` fields (with `Bfree >= Bavail`) will be set to
different values on e.g. filesystems such as ext4 that reserve a certain
percentage of the filesystem blocks which may only be allocated by admnistrative
privileged processes.
The calculations for the `Total` disk space need to subtract the difference
between the `Bfree` and `Bavail` fields for it to correctly show the total
available storage space available for unprivileged users.
This implicitly fixes a bug where the `Used = Total - Free` calculation yielded
different (and also incorrect) results for identical contents stored when only
the sizes of the disks or backing volumes differed. (as can be witnessed in the
`Used:` value displayed in the Minio browser)
See:
- https://wiki.archlinux.org/index.php/ext4#Reserved_blocks
- http://man7.org/linux/man-pages/man2/statfs.2.html
- https://man.openbsd.org/statfs
- http://lingrok.org/xref/coreutils/src/df.c#893
2017-09-25 18:46:19 -07:00
Harshavardhana
d3eb5815d9
Avoid DDOS in PutObject() when objectName is '/' and size '0' ( #4962 )
...
It can happen that an incoming PutObject() request might
have inputs of following form eg:-
- bucketName is 'testbucket'
- objectName is '/'
bucketName exists and was previously created but there
are no other objects in this bucket. In a situation like
this parentDirIsObject() goes into an infinite loop.
Verifying that if '/' is an object fails on both backends
but the resulting `path.Dir('/')` returns `'/'` this causes
the closure to loop onto itself.
Fixes #4940
2017-09-25 14:47:58 -07:00
Andreas Auernhammer
7e6b5bdbb7
remove ReadFileWithVerify from StorageAPI ( #4947 )
...
This change removes the ReadFileWithVerify function from the
StorageAPI. The ReadFile was basically a redirection to ReadFileWithVerify.
This change removes the redirection and moves the logic of
ReadFileWithVerify directly into ReadFile.
This removes a lot of unnecessary code in all StorageAPI implementations.
Fixes #4946
* review: fix doc and typos
2017-09-25 11:32:56 -07:00
Harshavardhana
4cadb33da2
api/PostPolicy: Allow location header fully qualified URL ( #4926 )
...
req.Host is used to construct the final object location.
Fixes #4910
2017-09-24 16:43:21 -07:00
Harshavardhana
c3ff402fcb
Fix signature v2 and presigned query unescaping. ( #4936 )
...
Simplifies the testing code by using s3signer
package from minio-go library.
Fixes #4927
2017-09-24 14:20:12 -07:00
Davor Kapsa
aceaa1ec48
travis: update go versions ( #4945 )
2017-09-22 14:05:07 -07:00
Harshavardhana
330f79b40e
Remove pre go1.8 code and cleanup ( #4933 )
...
We don't need certain go1.7.x custom code anymore, since
we have migrated to go1.8
2017-09-22 14:03:31 -07:00
Bala FA
6d7df1d1cb
Use mc functional tests for verifying build ( #4934 )
2017-09-20 13:22:05 -07:00
Aditya Manthramurthy
3c0d3f7510
Fix bug in ErasureStorage.HealFile ( #4913 )
2017-09-20 09:50:27 -07:00
Krishnan Parthasarathi
f7ae3be586
Removing 100MB part and 1TB limitation ( #4939 )
...
With https://github.com/minio/minio/pull/4869 maximum size of a single
multipart upload part in not restricted to 100MB. 1TB maximum object
size limitation is no longer applicable too.
2017-09-20 09:48:48 -07:00
Bala FA
70fec0a53f
azure: add stateless gateway support ( #4874 )
...
Previously init multipart upload stores metadata of an object which is
used for complete multipart. This patch makes azure gateway to store
metadata information of init multipart object in azure in the name of
'minio.sys.tmp/multipart/v1/<UPLOAD-ID>/meta.json' and uses this
information on complete multipart.
2017-09-19 16:08:08 -07:00
Nitish Tiwari
fba1669966
Update docker-compose sample file to use local volume driver so data ( #4937 )
...
persists even after Minio container is stopped/deleted
2017-09-19 12:43:45 -07:00
Harshavardhana
71201273e2
Remove reference to go1.8 issue from homebrew ( #4931 )
2017-09-19 12:41:19 -07:00
Andreas Auernhammer
79ba4d3f33
refactor ObjectLayer PutObject and PutObjectPart ( #4925 )
...
This change refactor the ObjectLayer PutObject and PutObjectPart
functions. Instead of passing an io.Reader and a size to PUT operations
ObejectLayer expects an HashReader.
A HashReader verifies the MD5 sum (and SHA256 sum if required) of the object.
This change updates all all PutObject(Part) calls and removes unnecessary code
in all ObjectLayer implementations.
Fixes #4923
2017-09-19 12:40:27 -07:00
Harshavardhana
f8024cadbb
[security] rpc: Do not transfer access/secret key. ( #4857 )
...
This is an improvement upon existing implementation
by avoiding transfer of access and secret keys over
the network. This change only exchanges JWT tokens
generated by an rpc client. Even if the JWT can be
traced over the network on a non-TLS connection, this
change makes sure that we never really expose the
secret key over the network.
2017-09-19 12:37:56 -07:00
Evan
f680b8482f
Add a snapcraft build badge. ( #4805 )
2017-09-19 12:36:35 -07:00
Nitish Tiwari
a56f7e2f23
Refactor CONTRIBUTING.md to make it more reader friendly ( #4918 )
2017-09-18 11:02:45 -07:00
Nitish Tiwari
6d5d49bfb1
Update CLI examples to be in sync with examples used on Minio website ( #4920 )
2017-09-14 19:17:42 -07:00
fangyuxiang
8e4842b665
fs: multipart clean only trigger once ( #4915 )
2017-09-14 19:17:26 -07:00
ebozduman
b74ef6d5f4
Fixes the if condition when uploads.json file cannot be found ( #4883 )
2017-09-14 16:09:12 -07:00
fangyuxiang
9925640da8
fs: multipart clean doesn't work when object name has '/' ( #4919 )
2017-09-14 16:00:57 -07:00
Andrej Pregl
f45e0a44b8
Change average from int to int64 in order to support 32-bit systems. ( #4921 )
2017-09-14 10:23:23 -07:00
Krishna Srinivas
3e632a49ee
In gateway mode "continuation-token" will not contain "prefix" ( #4911 )
...
fixes #4900
2017-09-13 17:27:19 -07:00
Bala FA
1bb3a03099
build: add verify check on make test ( #4901 )
...
This patch adds basic tests for FS/XL/Distribute setups.
2017-09-12 16:56:33 -07:00
Krishna Srinivas
42b3795304
Set NextContinuationToken in ListObjectsV2 response for gateway ( #4908 )
...
fixes #4900
2017-09-12 16:19:58 -07:00
Bala FA
302fcb3b17
azure: handle encryption headers and azure InvalidMetadata error ( #4893 )
...
Previously minio gateway returns invalid bucket name error for invalid
meta data. This is fixed by returning BadRequest with 'Unsupported
metadata' in response.
Fixes #4891
2017-09-12 16:14:41 -07:00
Harshavardhana
b9fc4150f6
Fix preInit logic when mixed disk situations exist. ( #4904 )
...
When servers are started simultaneously across multiple
nodes or simulating a local setup, it can happen such
that one of the servers in setup reaches a following
situation where it observes
- Some servers are formatted
- Some servers are unformatted
- Some servers are offline
Current state machine doesn't handle this correctly, to fix
this situation where we have unformatted, formatted and
disks offline we do not decisively know the course of
action. So we wait for the offline disks to change their state.
Once the offline disks change their state to either one of these
states we can decisively move forward.
- nil (formatted disk)
- errUnformattedDisk
- Or any other error such as errCorruptedDisk.
Fixes #4903
2017-09-12 12:17:44 -07:00
Krishna Srinivas
f66239e82f
Expose common S3 headers in CORS setting ( #4839 )
...
fixes #4838
2017-09-11 08:15:51 -07:00
Harshavardhana
6c2bc0568b
Increase default read/write timeouts from 30sec to 15minutes ( #4888 )
...
The default timeout of 30secs is not enough for high latency
environments, change these values to use 15 minutes instead.
With 30secs I/O timeouts seem to be quite common, this leads
to pretty much most SDKs and clients reconnect. This in-turn
causes significant performance problems. On a low latency
interconnect this can be quite challenging to transfer large
amounts of data. Setting this value to 15minutes covers
pretty much all known cases.
This PR was tested with `wondershaper <NIC> 20000 20000` by
limiting the network bandwidth to 20Mbit/sec. Default timeout
caused a significant amount of I/O timeouts, leading to
constant retires from the client. This seems to be more common
with tools like rclone, restic which have high concurrency set
by default. Once the value was fixed to 15minutes i/o timeouts
stopped and client could steadily upload data to the server
even while saturating the network.
Fixes #4670
2017-09-07 11:16:45 -07:00
poornas
0d154871d5
Admin: Raise error if config and env credentials mismatch ( #4870 )
2017-09-07 11:16:13 -07:00
Bala FA
189b6682d6
azure: allow parts > 100MiB size to work properly ( #4869 )
...
Previously if any multipart part size > 100MiB is uploaded, azure
gateway returns error.
This patch fixes the issue by creating sub parts sizing each 100MiB of
given multipart part. On complete multipart, it fetches all uploaded
azure block ids for each parts and performs completion.
Fixes #4868
2017-09-05 16:56:23 -07:00
Harshavardhana
cf479eb401
Move to latest release of minio-go ( #4886 )
...
- Region handling can now use region endpoints directly.
- All uploads are streaming no more large buffer needed.
- Major API overhaul for CopyObject(dst, src)
- Fixes bugs present in existing code for copying
- metadata replace directive CopyObject
- PutObjectPart doesn't require md5Sum and sha256
2017-09-05 14:45:22 -07:00
Harshavardhana
72490bf8db
Implement proper reConnect logic for amqp notification target. ( #4867 )
...
Fixes #4597
2017-09-04 17:45:30 -07:00
Justin Clift
5a73aecb5c
fix: Trivial typo in error message ( #4878 )
2017-09-03 13:53:03 -07:00
Harshavardhana
e26a706dff
Ignore reservedBucket checks for net/rpc requests ( #4884 )
...
All `net/rpc` requests go to `/minio`, so the existing
generic handler for reserved bucket check would essentially
erroneously send errors leading to distributed setups to
wait infinitely.
For `net/rpc` requests alone we should skip this check and
allow resource bucket names to be from `/minio` .
2017-09-01 12:16:54 -07:00
Andrej Pregl
9e9c7b4f22
Lower object name length when running in docker to support aufs. ( #4879 )
2017-09-01 11:00:47 -07:00
Krishna Srinivas
ff8e2b5b4f
Init HTTP client and transport for azure sdk ( #4871 )
...
Fixes segfault
2017-08-31 17:19:03 -07:00
Frank Wessels
61e0b1454a
Add support for timeouts for locks ( #4377 )
2017-08-31 14:43:59 -07:00
Frank Wessels
93f126364e
Updated version of klauspost/reedsolomon with NEON support for ARM ( #4865 )
2017-08-30 09:49:00 -07:00
Harshavardhana
6dca044ea8
fs: Convert repeated code in rwpool.Open() into a single function. ( #4864 )
...
Refer https://github.com/minio/minio/issues/4658 for more information.
2017-08-30 09:48:19 -07:00
Harshavardhana
9e9faf7b53
Support source compilation on s390 and ppc64le ( #4859 )
...
NOTE: This doesn't validate that minio will work fine
on these platforms and is tested. Since we do not validate
on these architectures this is to be treated as just a build fix.
Fixes #4858
2017-08-30 09:45:36 -07:00
Nitish Tiwari
2bca51ab2c
Add steps to run GCS gateway on Kubernetes via YAML files ( #4819 )
2017-08-28 12:58:52 -07:00
Leo Arias
34e780e690
add the mount-observe plug to the snap ( #4853 )
2017-08-28 11:40:26 -07:00
Harshavardhana
6cab6d802d
api: Fix the conditional to check for reserved buckets. ( #4856 )
...
Current code was an logical `and` instead we should do `or`.
Fixes https://github.com/minio/mc/issues/2231
2017-08-28 11:39:48 -07:00
Andreas Auernhammer
ea65350308
add vscode config files to gitignore ( #4855 )
2017-08-27 13:44:39 -07:00
Harshavardhana
1bb9d49eaa
fs: ListObjects() was reading ETag at wrong offsets ( #4846 )
...
Current code was just using io.ReadAll() on an fd()
which might have moved underneath due to a concurrent
read operation. Subsequent read will result in EOF
We should always seek back and read again. pread()
is allowed on all platforms use io.SectionReader to
read from the beginning of the file.
Fixes #4842
2017-08-23 17:59:14 -07:00
Harshavardhana
db5af1b126
fix: tests error conditions should be used properly. ( #4833 )
2017-08-23 17:58:52 -07:00
Andreas Auernhammer
b233345f19
remove bcrypt code from code-base ( #4844 ) ( #4845 )
...
Bcrypt is not neccessary and not used properly. This change
replace the whole bcrypt hash computation through a constant time
compare and removes bcrypt from the code base.
2017-08-23 15:59:37 -07:00
Leo Arias
07ccf4c0d0
Add the install instructions for the snap ( #4843 )
2017-08-23 15:59:13 -07:00
Aditya Manthramurthy
77d2870f5b
Fix validation in PutBucketNotification handler ( #4841 )
...
Fixes #4813
If a TopicConfiguration element or CloudFunction element is found in
configuration submitted to PutBucketNotification API, an BadRequest
error is returned.
2017-08-23 15:58:02 -07:00
Andreas Auernhammer
3a73c675a6
restirct max size of http header and user metadata ( #4634 ) ( #4680 )
...
S3 only allows http headers with a size of 8 KB and user-defined metadata
with a size of 2 KB. This change adds a new API error and returns this
error to clients which sends to large http requests.
Fixes #4634
2017-08-22 16:53:35 -07:00
Bala FA
b694c1a4d7
fix: bufconn and listener tests for megacheck ( #4827 )
...
Fixes #4824
2017-08-20 12:25:08 -07:00
Harshavardhana
2e6ee68409
fix: [minor] Avoid unnecessary typecasting. ( #4828 )
...
We don't need to typecast identifiers from
their base to type to same type again. This
is not a bug and compiler is fine to skip
it but it is better to avoid if not needed.
2017-08-18 11:45:16 -07:00
Bala FA
7505bac037
tests: create temporary dir/files than /usr directory. ( #4820 )
...
Fixes #4816
2017-08-18 11:44:54 -07:00
Harshavardhana
9dca0c1889
fix: [minor] functions should take inputs with required functionality. ( #4823 )
2017-08-17 13:49:57 -07:00
Nitish Tiwari
69555f1224
Update Docker commands to use /data as example directory ( #4825 )
...
/data as default makes it easy to understand and shortens
the example Minio command for Docker.
2017-08-17 10:56:25 -07:00
Harshavardhana
879cef37a1
Fail to start server if detected cross-device mounts. ( #4807 )
...
Fixes #4764
2017-08-15 15:10:50 -07:00
wd256
3d21119ec8
Provide 200 response with per object error listing on access denied for delete multiple object request ( #4817 )
2017-08-15 12:49:31 -07:00
Frank Wessels
a2f2044528
Minor corrections in comments for xl utils ( #4815 )
2017-08-14 18:09:29 -07:00
Andreas Auernhammer
85fcee1919
erasure: simplify XL backend operations ( #4649 ) ( #4758 )
...
This change provides new implementations of the XL backend operations:
- create file
- read file
- heal file
Further this change adds table based tests for all three operations.
This affects also the bitrot algorithm integration. Algorithms are now
integrated in an idiomatic way (like crypto.Hash).
Fixes #4696
Fixes #4649
Fixes #4359
2017-08-14 18:08:42 -07:00
Leo Arias
617f2394fb
Hardcode snap version while a store bug is fixed ( #4806 )
...
* Hardcode snap version while a store bug is fixed
This is a workaround for for a failure in the store, which limits the version to 32 characters.
https://forum.snapcraft.io/t/versions-can-be-at-most-32-characters/1642
Capitalize the summary
* add the snapcraft dirs to gitignore bring back settings.json to ignore
2017-08-14 11:12:46 -07:00
Bala FA
1729e82361
tests: use port '0' for auto-detecting free port. ( #4803 )
...
Fixes #4774
2017-08-14 11:11:38 -07:00
Harshavardhana
58633a383a
Deprecate intel-32, arm64, arm support for minio builds. ( #4811 )
...
It was decided that we will be deprecating ARM support
for minio builds. ARM users should simply compile from source.
Additionally 32bit version of Linux, Windows and FreeBSD (64bit)
are deprecated.
2017-08-14 11:08:58 -07:00
Nitish Tiwari
d4b107adf4
Retry name lookup for kubernetes and docker swarm environment ( #4800 )
...
Wait for remote hosts to resolve instead of failing on first host
resolution error, when running in Kubernetes or Docker environment.
Note that
- Waiting is based on exponential back-off mechanism
- If run as a binary, server fails if remote host is not resolvable
This is needed because in orchestration platforms like Kubernetes, remote
hosts are started sequentially and all the hosts are not up initially,
though they are expected to come up in a short time frame
It is difficult to identify a cap on the waiting time due to
non-deterministic nature of infrastructure platforms, so the server waits
infinitely for the hosts to come up, while logging the error messages to
the console.
Fixes: https://github.com/minio/minio/issues/4669
2017-08-13 13:34:10 -07:00
Nitish Tiwari
53f84d6084
Report healthy status for initial 120s, then switch to healthcheck ( #4799 )
...
Swarm routes traffic only to containers that report healthy status,
while Minio in distributed mode needs to talk to other peers before
it can respond to healthcheck probe. As the Minio containers are not
able to talk to each other, distributed Minio is not getting started
on Docker Swarm.
With this PR, Minio Healthcheck report healthy status for initial
120s enough for distributed Minio to start. After that normal
Healthcheck resumes. Also changed the healthcheck method name
in accordance with Google shell styleguide.
Fixes: https://github.com/minio/minio/issues/4761
2017-08-12 19:26:35 -07:00
Harshavardhana
d864e00e24
posix: Deprecate custom removeAll/mkdirAll implementations. ( #4808 )
...
Since go1.8 os.RemoveAll and os.MkdirAll both support long
path names i.e UNC path on windows. The code we are carrying
was directly borrowed from `pkg/os` package and doesn't need
to be in our repo anymore. As a side affect this also
addresses our codecoverage issue.
Refer #4658
2017-08-12 19:25:43 -07:00
Harshavardhana
b69aa9c4d0
fs: Return errVolumeNotEmpty properly if path not empty. ( #4794 )
...
Refer #4770
2017-08-12 19:24:20 -07:00
Harshavardhana
d20cdac304
Move all Dockerfiles to alpine-3.6 ( #4797 )
...
Refer #4759 .
Fix this to avoid issues like below
```
docker run --rm minio/minio:edge-armhf version
minio: <ERROR> .... is not compiled by Go >= 1.8.3. ... recompile...
```
2017-08-12 19:17:37 -07:00
Frank Wessels
fffe4ac7e6
Prevent unnecessary verification of parity blocks while reading ( #4683 )
...
* Prevent unnecessary verification of parity blocks while reading erasure
coded file.
* Update klauspost/reedsolomon and just only reconstruct data blocks while
reading (prevent unnecessary parity block reconstruction)
* Remove Verification of (all) reconstructed Data and Parity blocks since
in our case we are protected by bit rot protection. And even if the
verification would fail (essentially impossible) there is no way to
definitively say whether the data is still correct or not, so this call
make no sense for our use case.
2017-08-11 18:25:46 -07:00
Frank Wessels
98b62cbec8
Implement an offline mode for a distributed node ( #4646 )
...
Implement an offline mode for remote storage to cache the
offline status of a node in order to prevent network calls
that are bound to fail. After a time interval an attempt
will be made to restore the connection and mark the node
as online if successful.
Fixes #4183
2017-08-11 11:49:35 -07:00
Dee Koder
1978b9d8f9
Prevent minio server starting in standalone erasure mode for wrong inputs. ( #4700 )
...
It is possible at times due to a typo when distributed mode was intended
a user might end up starting standalone erasure mode causing confusion.
Add code to check this based on some standard heuristic guess work and
report an error to the user.
Fixes #4686
2017-08-11 11:47:28 -07:00
Harshavardhana
3544e5ad01
fs: Fix Shutdown() behavior and handle tests properly. ( #4796 )
...
Fixes #4795
2017-08-10 14:11:57 -07:00
Harshavardhana
e7cdd8f02c
fs: Avoid non-idempotent code flow in ListBuckets() ( #4798 )
...
Under the call flow
```
Readdir
+
|
|
| path-entry
|
|
v
StatDir
```
Existing code was written in a manner where say
a bucket/top-level directory was indeed deleted
between Readdir() and before StatDir() we would
ignore certain errors. This is not a plausible
situation and might not happen in almost all
practical cases. We do not have to look for
or interpret these errors returned by StatDir()
instead we can just collect the successful
values and return back to the client. We do not
need to pre-maturely decide on bucket access
we just let filesystem decide subsequently for
real I/O operations.
Refer #4658
2017-08-10 13:36:11 -07:00
Leo Arias
3ff09b9b72
Add the packaging metadata to build the minio snap ( #4749 )
...
* Add the packaging metadata to build the minio snap.
2017-08-10 09:39:30 -07:00
Nitish
b3b42c72a9
Update orchestration examples to latest release
2017-08-08 17:15:46 -07:00
Aditya Manthramurthy
32da1aa9d6
XL: Simplify heal-format operations
...
This is in preparation for updated admin heal API.
* Improve case analysis of healFormatXL() - fixes a case where disks
could have unhandled errors.
* Simplify healFormatXLFreshDisks() and healFormatXLCorruptedDisks()
to share more code and handle fewer cases for improved simplicity
and reduced code repetition.
* Fix test cases.
2017-08-08 17:14:24 -07:00
Andreas Auernhammer
b10fa507b2
set http transport config for gateway ( #4765 )
...
This change sets the http config for the minio client used by the
minio server in gateway mode.
Fixes #4765
2017-08-08 16:23:52 -07:00
Andreas Auernhammer
1e8925cea0
add settings.json to .gitignore ( #4786 )
...
settings.json is the VS code workspace settings file
2017-08-08 13:06:57 -07:00
Harshavardhana
f346ca44f0
config: Avoid stale credentials in memory. ( #4466 )
2017-08-08 12:14:32 -07:00
A. Elleuch
6f7ace3d3e
Honor overriding response headers for HEAD ( #4784 )
...
Though not clearly mentioned in S3 specification, we should override
response headers for presigned HEAD requests as we do for GET.
2017-08-08 11:04:04 -07:00
Harshavardhana
812142f007
Fix typo in webhook docs ( #4787 )
2017-08-07 21:31:17 -07:00
Andrej Pregl
fa52d491c5
Added support for macOS in TestNewHTTPListener ( #4782 )
2017-08-07 16:02:34 -07:00
Harshavardhana
283ad7e5e5
browser: update ui-assets for new changes. ( #4780 )
2017-08-07 12:48:51 -07:00
poornas
748b1d6495
azure: For container access type private treat as no policy set. ( #4729 )
2017-08-06 22:24:40 -07:00
A. Elleuch
b4dc6df35c
go1.8: Changes to support golang 1.8 ( #4759 )
...
QuirkConn is added to replace net.Conn as a workaround to a golang bug:
https://github.com/golang/go/issues/21133
2017-08-06 11:27:33 -07:00
Aditya Manthramurthy
218049300c
Fix testcase to not overflow int type ( #4739 )
...
The int type is only 32-bits wide on 32-bit CPUs.
Set the type in the tests to int32 to avoid setting problematic
maxKeys values.
Fixes #4738
2017-08-05 02:36:47 -07:00
Aaron Walker
5db533c024
bucket-policy: Add IPAddress/NotIPAddress conditions support ( #4736 )
2017-08-05 01:00:05 -07:00
Brendan Ashworth
aeafe668d8
posix: do not upstream errors in deleteFile ( #4771 )
...
This commit changes posix's deleteFile() to not upstream errors from
removing parent directories. This fixes a race condition.
The race condition occurs when multiple deleteFile()s are called on the
same parent directory, but different child files. Because deleteFile()
recursively removes parent directories if they are empty, but
deleteFile() errors if the selected deletePath does not exist, there was
an opportunity for a race condition. The two processes would remove the
child directories successfully, then depend on the parent directory
still existing. In some cases this is an invalid assumption, because
other processes can remove the parent directory beforehand. This commit
changes deleteFile() to not upstream an error if one occurs, because the
only required error should be from the immediate deletePath, not from a
parent path.
In the specific bug report, multiple CompleteMultipartUpload requests
would launch multiple deleteFile() requests. Because they chain up on
parent directories, ultimately at the end, there would be multiple
remove files for the ultimate parent directory,
.minio.sys/multipart/{bucket}. Because only one will succeed and one
will fail, an error would be upstreamed saying that the file does not
exist, and the CompleteMultipartUpload code interpreted this as
NoSuchKey, or that the object/part id doesn't exist. This was faulty
behavior and is now fixed.
The added test fails before this change and passes after this change.
Fixes: https://github.com/minio/minio/issues/4727
2017-08-04 16:51:20 -07:00
Krishnan Parthasarathi
54f3a0946f
Avoid superfluous error messages after connect ( #4762 )
...
Peek could fail legitimately when clients abruptly close connection. So,
io.EOF and network timeout errors are not logged while all other errors
will be logged.
2017-08-04 14:35:07 -07:00
Krishna Srinivas
be6bd52978
fs: use keymarker and uploadidmarker in the multipart purging loop ( #4775 )
...
related to #4564
2017-08-04 14:14:23 -07:00
Krishnan Parthasarathi
75c43bfb6c
ListMultipartUploads, ListObjectParts return empty response ( #4694 )
...
Also, periodically removes incomplete multipart uploads older than 2 weeks.
2017-08-04 10:45:57 -07:00
ebozduman
0aca2ab970
Stop attempting to close nil Listener ( #4753 )
2017-08-04 10:44:46 -07:00
Brendan Ashworth
28bc5899fd
posix: test isDirEmpty, change error conditional ( #4743 )
...
This commit adds a new test for isDirEmpty (for code coverage) and
changes around the error conditional. Previously, there was a `return
nil` statement that would only be triggered under a race condition and
would trip up our test coverage for no real reason. With this new error
conditional, there's no awkward 'else'-esque condition, which means test
coverage will not change between runs for no reason in this specific
test. It's also a cleaner read.
2017-08-04 10:43:51 -07:00
Nitish Tiwari
fcc61fa46a
Remove minimum inodes reqd check ( #4747 )
2017-08-03 20:07:22 -07:00
Nitish Tiwari
f7889e1af7
Add TLS request for healthcheck ( #4740 )
2017-08-03 20:05:45 -07:00
Brendan Ashworth
bccc386994
fs: drop Stat() call from fsDeleteFile,deleteFile ( #4744 )
...
This commit makes fsDeleteFile() simply call deleteFile() after calling
the relevant path length checking functions. This DRYs the code base.
This commit removes the Stat() call from deleteFile(). This improves
performance and removes any possibility of a race condition.
This additionally adds tests and a benchmark for said function. The
results aren't very consistent, although I'd expect this commit to make
it faster.
2017-08-03 20:04:28 -07:00
ebozduman
0f401b67ad
Removes max limit requirement on accessKey and secretKey length ( #4730 )
2017-08-03 20:03:37 -07:00
Laurentiu Nicola
108decfa76
Fix sysctl proposed values ( #4741 )
...
`sched_wakeup_granularity_ns` and `sched_wakeup_granularity_ns` are measured in `ns`, so a value of `10` or `15` is way too low.
2017-08-02 14:08:58 -07:00
Nitish Tiwari
6afbd502e8
Add Docker command in erasure code document ( #4735 )
2017-07-31 11:40:14 -07:00
Brendan Ashworth
ec5293ce29
jwt,browser: allow short-expiry tokens for GETs ( #4684 )
...
This commit fixes a potential security issue, whereby a full-access
token to the server would be available in the GET URL of a download
request. This fixes that issue by introducing short-expiry tokens, which
are only valid for one minute, and are regenerated for every download
request.
This commit specifically introduces the short-lived tokens, adds tests
for the tokens, adds an RPC call for generating a token given a
full-access token, updates the browser to use the new tokens for
requests where the token is passed as a GET parameter, and adds some
tests with the new temporary tokens.
Refs: https://github.com/minio/minio/pull/4673
2017-07-24 12:46:37 -07:00
Harshavardhana
4785555d34
api: Upon bucket delete remove in-memory state properly. ( #4716 )
...
This PR fixes the issue of cleaning up in-memory state
properly. Without this PR we can lead to security
situations where new bucket would inherit wrong
permissions on bucket and expose objects erroneously.
Fixes #4714
2017-07-23 19:35:18 -07:00
A. Elleuch
b918a6592f
gcs: Better parsing of address flag ( #4709 )
2017-07-20 16:39:11 -07:00
Krishna Srinivas
eb787d8613
gateway-gcs: remove files older than 2 weeks in minio.sys.temp ( #4599 ).
...
Rename ##minio## to {minio}.
2017-07-20 15:36:48 -07:00
Aaron Kunz
0a1501bc1b
Fix typo ( #4695 )
2017-07-19 15:19:03 -07:00
Harshavardhana
f8bd9cfd83
rpc: Do not use read/write deadlines for rpc connections. ( #4647 )
...
Fixes #4626
2017-07-18 09:30:46 -07:00
Brendan Ashworth
c59b995f7b
build: ditch verifiers on make ( #4679 )
...
This commit ditches running verifiers automatically when just building
the server. It retains the verifiers when running tests.
There is very little point to running the verifiers each time a
developer builds the library but has no intent of running the tests.
They're expensive in time; this commit halves the build time on my
system, from 57 seconds to 29 seconds. This is because verifiers updates
the libraries from GitHub each time, which is slightly wasteful.
Additionally, computing cyclomatic complexity is expensive
computationally and isn't necessary to build the library.
Additionally, this allows the library to be built offline. It no longer
requires internet to run make.
2017-07-15 12:12:03 -07:00
Harshavardhana
bc73a1a1cb
gcs: Save partNumber as part of backend format. ( #4666 )
...
Fixes #4637
2017-07-13 23:20:16 -07:00
Krishna Srinivas
ce7c9c651d
gateway-azure: Return right error when Part size is > 100MB ( #4652 )
2017-07-12 16:42:14 -07:00
Bala FA
c3dd7c1f6c
Refactor HTTP server to address bugs ( #4636 )
...
* Refactor HTTP server to address bugs
* Remove unnecessary goroutine to start multiple TCP listeners.
* HTTP server waits for shutdown to maximum of Server.ShutdownTimeout
than per serverShutdownPoll.
* Handles new connection errors properly.
* Handles read and write timeout properly.
* Handles error on start of HTTP server properly by exiting minio
process.
Fixes #4494 #4476 & fixed review comments
2017-07-12 16:33:21 -07:00
Harshavardhana
2d23cd4f39
gcs: Fetch port as GlobalString(). ( #4657 )
...
Currently we were looking for `address` flag
under local flags. This PR fixes #4656
2017-07-11 18:06:26 -07:00
Harshavardhana
ce7af3aae1
gcs: Fix writer/reader go-routine leaks and code re-use ( #4651 )
...
This PR serves to fix following things in GCS gateway.
- fixes leaks in object reader and writer, not getting closed
under certain situations. This led to go-routine leaks.
- apparent confusing issue in case of complete multipart upload,
where it is currently possible for an entirely different
object name to concatenate parts of a different object name
if you happen to know the upload-id and parts of the object.
This is a very rare scenario but it is possible.
- succint usage of certain parts of code base and re-use.
2017-07-11 09:25:19 -07:00
Krishna Srinivas
1b92c5136b
Append "-1" to etag when it is not MD5 ( #4641 )
...
* gateway-azure: append "-1" to ETag so that clients do not interpret it as MD5. fixes #4537 . Added unit tests.
2017-07-10 18:21:12 -07:00
Harshavardhana
cc8a8cb877
posix: Check for min disk space and inodes ( #4618 )
...
This is needed such that we don't start or
allow writing to a posix disk which doesn't
have minimum total disk space available.
One part fix for #4617
2017-07-10 18:14:48 -07:00
Krishna Srinivas
ce403fdaa0
GCS documentation ( #4622 )
...
* GCS documentation and review fixes.
2017-07-10 09:35:20 -07:00
Nitish Tiwari
45fbb0d618
Add NATS Streaming doc to event notification doc ( #4645 )
2017-07-07 23:37:12 -07:00
Harshavardhana
f5ce685aa1
Remove dead unused errs and constants. ( #4627 )
2017-07-07 14:31:42 -07:00
Krishna Srinivas
c83055500d
fs: Fail CompleteMultipartUpload if partSize < 5M unless it is last part ( #4642 )
...
fixes #4625
2017-07-07 08:41:29 -07:00
Andreas Auernhammer
b0fbddc051
fix confusing code for http.Header handling ( #4623 )
...
Fixed header-to-metadat extraction. The extractMetadataFromHeader function should return an error if the http.Header contains a non-canonicalized key. The reason is that the keys can be manually set (through a map access) which can lead to ugly bugs.
Also fixed header-to-metadata extraction. Return a InternalError if a non-canonicalized key is found in a http.Header. Also log the error.
2017-07-05 16:56:10 -07:00
Harshavardhana
4e0c08e9c5
ListenBucketNotification should set proper MIME type. ( #4621 )
...
This is needed to avoid proxies buffering the connection
this is also a HTTP standard way to handle this situation
where server is sending back events in asynchronously.
For more details read https://goo.gl/RCML9f
Fixes - https://github.com/minio/minio-go/issues/731
2017-07-03 19:59:41 -07:00
Nitish Tiwari
344f9ec608
Fix gateway browser screenshot ( #4613 )
2017-06-30 09:47:40 -07:00
Dee Koder
28ff62716f
Fixed one of the images with missing details. ( #4612 )
2017-06-29 15:49:09 -07:00
Dee Koder
1f69a75efa
Updated docs with latest images. ( #4611 )
2017-06-29 11:41:21 -07:00
Nitish Tiwari
02a81ee564
Remove deployment scenarios from erasure code guide ( #4607 )
2017-06-29 11:40:59 -07:00
Nitish Tiwari
e91e9e8a38
GCS ListObjectV2 honours continuationToken ( #4608 )
2017-06-29 11:19:55 -07:00
Dee Koder
e45f9057d6
Added new optimized images. ( #4609 )
2017-06-29 09:10:47 -07:00
Bala FA
53e7fdc847
create subcomposed objects if total parts are > 32 ( #4593 )
2017-06-27 22:27:05 -07:00
Aditya Manthramurthy
1af331c05c
Remove unnecessary newline at beginning of server output ( #4600 )
2017-06-27 19:46:58 -07:00
ebozduman
0e5b9c7fe4
Adds 'gcs not ready for production' msg ( #4604 )
2017-06-27 19:44:47 -07:00
Harshavardhana
a86dc8a4c5
cleanup makefile and enable CGO_ENABLED=0 ( #4598 )
2017-06-26 18:07:06 -07:00
A. Elleuch
c88dca984d
web: Encode path in presigned GET urls ( #4596 )
...
When the browser asks for a GET presigned url, this latter is not
encoded and can be confusing when the user copies-pastes it somewhere,
especially when the path contains a space.
2017-06-25 18:39:14 -07:00
Harshavardhana
1054f9cbf0
fix: Remove adverb from erasure coded startup message. ( #4594 )
2017-06-25 18:38:55 -07:00
Harshavardhana
aaacce4c43
browser: update ui-assets with new changes. ( #4595 )
2017-06-25 18:09:24 -07:00
Nitish Tiwari
7bd1f44491
Add support for helm package info in useragent ( #4592 )
2017-06-24 13:17:28 -07:00
Harshavardhana
ba6d997d18
build: Error out properly when unsupported arch is given. ( #4585 )
2017-06-24 01:05:35 -07:00
A. Elleuch
eaa41e4086
gcs: Check if the given project id argument exists ( #4583 )
...
Using GCS resource manager API, check if the provided
project id is already created and associated to the current
user account.
2017-06-23 22:10:29 -07:00
Krishna Srinivas
6b70f429ed
gateway/azure: Parse error responses for anonymous requests ( #4543 )
...
fixes #4481
2017-06-23 22:07:46 -07:00
Krishna Srinivas
0a6e9a1834
gateway-gcs: cleanup minio.sys.temp before deleting the bucket ( #4582 )
...
fixes #4560
fixes #4569
2017-06-23 17:57:25 -07:00
Nitish Tiwari
15b65a8342
Added AnonListObjectsV2 support to GCS ( #4584 )
2017-06-23 17:35:45 -07:00
Harshavardhana
8b7df7da37
api: No need to set x-amz-bucket-region if region is empty. ( #4586 )
2017-06-23 16:05:40 -07:00
Krishnan Parthasarathi
237c8af5ef
Improve GCS gateway example instruction ( #4587 )
2017-06-23 13:13:26 -07:00
Krishna Srinivas
ff036c171f
gateway-gcs: double quotes should be striped from ETag ( #4590 )
2017-06-23 12:19:10 -07:00
Harshavardhana
a3b085300d
gcs: Add missing AnonListObjectsV2
2017-06-22 12:09:13 -07:00
Harshavardhana
b90cefdb88
Merge remote-tracking branch 'origin/feature-gcs'
2017-06-22 11:52:12 -07:00
Harshavardhana
f3506b8958
tests: Enable previously disabled UNC tests on CI. ( #4575 )
...
Windows CI had a bug which has been fixed, re-enable
those commented tests.
2017-06-22 07:54:22 -07:00
Frank Wessels
46897b1100
Name return values to prevent the need (and unnecessary code bloat) ( #4576 )
...
This is done to explicitly instantiate objects for every return statement.
2017-06-21 19:53:09 -07:00
Harshavardhana
cec8b238f3
sign: StreamingSign should use region from client. ( #4577 )
...
This is a fix to make streaming signature to behave
the same as regular signature and presigned signature.
Fixes https://github.com/minio/minio-go/issues/718
2017-06-21 11:30:34 -07:00
Krishna Srinivas
13ab8e17e2
gateway-gcs: use minio.sys.temp/multipart/v1 as url base ( #4562 )
2017-06-21 10:27:44 -07:00
Nitish Tiwari
9d3d64df2d
Added HEALTHCHECK in release Dockerfiles ( #4550 )
...
A new script `healthcheck.sh` checks for http status 200/404
2017-06-20 19:24:45 -07:00
Krishnan Parthasarathi
146bc3e638
Add MINIO_REGION to server help message ( #4558 )
...
* Add e.g for setting MINIO_REGION env variable
* Add MINIO_REGION to region table
2017-06-20 15:02:18 -07:00
Harshavardhana
0543d45fb3
fix: OwnerID in response should be 64 character in length. ( #4554 )
...
Rather than sending a custom "minio" string, we can
change this to `sha256('arn:aws:iam::minio:user/admin')`.
Fixes #4553
2017-06-20 15:01:13 -07:00
Krishnan Parthasarathi
fe426944ea
Fix GCS help message ( #4570 )
2017-06-20 14:25:16 -07:00
Aditya Manthramurthy
c1a6ca0c33
Fix spelling of function name to startLockMaintenance ( #4561 )
2017-06-20 12:10:02 -07:00
Harshavardhana
5a78266821
gateway/gcs: Complete minio browser support for gcs. ( #4552 )
...
Fixes #4460
2017-06-19 19:45:13 -07:00
Harshavardhana
f5b4b0765a
Update minio-go dependency ( #4551 )
...
This updates dependency for
- AWS S3 backend.
- pkg/madmin
```
- Relax isValidBucketName to allow reading existing buckets. (#708 ) (3 minutes ago) <Harshavardhana>
- For GCS the size limit of S3 is not useful. (#711 ) (3 days ago) <Harshavardhana>
- s3utils: Support AWS S3 US GovCloud endpoint. (#701 ) (3 days ago) <Harshavardhana>
- api: Always strip 80/443 port from host (#709 ) (3 days ago) <Anis Elleuch>
- Redact signature strings properly. (#706 ) (4 days ago) <Harshavardhana>
- api: Single putObject can use temporary file always. (#703 ) (6 days ago) <Harshavardhana>
- Spelling fix (#704 ) (7 days ago) <Jacob Taylor>
- api/encrypt: Get() on encrypted object should be a reader. (#699 ) (2 weeks ago) <Harshavardhana>
- get: Fix reading an object if its size is unknown (#694 ) (3 weeks ago) <Anis Elleuch>
- fixes #696 by updating the examples for put-encrypted-object and get-encrypted-object (#697 ) (3 weeks ago) <Tejay Cardon>
- fix InvalidAccessKeyId error according to amazon documentation (#692 ) (4 weeks ago) <samkevich>
- Add AWS S3 SSE-C example. (#689 ) (4 weeks ago) <Harshavardhana>
- According to RFC7232 Etag should be in quotes for If-Match. (#688 ) (5 weeks ago) <Harshavardhana>
- api: getReaderSize() should honor seeked file descriptors. (#681 ) (5 weeks ago) <Harshavardhana>
- tests: Use bytes.Repeat() when generating big data (#683 ) (5 weeks ago) <Anis Elleuch>
- api: Failed call retry with region only when http.StatusBadRequest. (#678 ) (5 weeks ago) <Harshavardhana>
- api: Add NewWithCredentials() (#646 ) (5 weeks ago) <Harshavardhana>
```
2017-06-19 16:02:35 -07:00
Krishna Srinivas
3928c1e14c
gateway/gcs: Change in multipart backend format ( #4455 )
2017-06-17 16:00:41 -07:00
Krishnan Parthasarathi
94241cd153
Update Minio on DC/OS doc for latest Minio package ( #4549 )
2017-06-17 12:21:37 -07:00
Harshavardhana
a86c2e2ce1
xl/fs: Return InvalidPart{} error for part ETag mismatch. ( #4541 )
...
Fixes #4539
2017-06-17 11:20:39 -07:00
Harshavardhana
e99244be02
xl: prepare storage should Abort properly. ( #4542 )
...
Current state-machine didn't honor a situation
which can arise when there is a combination of
- formatted
- unformatted
- corrupted
disks - this combination invariably goes into a
mode where all servers are waiting perpetually
forever thinking we will get quorum in future.
At this point there is a distant possibility of
ever getting a quorum since we don't even have
quorum number of disks offline.
We should exit and print a proper message per disk
to indicate what went wrong and what was detected
by the server.
Refer #4477
2017-06-17 11:20:12 -07:00
Nitish Tiwari
58833711e0
Added ListObjectsV2 and ListObjectsV2 Anon support to Gateway S3 and Azure. ( #4547 )
2017-06-16 22:17:00 -07:00
Harshavardhana
f99f218999
Add support for reading and saving config on Gateway. ( #4463 )
...
This is also a first step towards supporting bucket
notification for gateway.
2017-06-16 16:01:41 -07:00
Krishnan Parthasarathi
4fb5fc72d7
GCS gateway allows apps to supply their own marker ( #4495 )
...
Most s3 compatible apps use object keys returned in listing as
marker. This change allows this behaviour with gateway-gcs too.
2017-06-16 15:02:07 -07:00
Remco Verhoef
d86973dcca
Allow bucket creation in different regions, closes #4287 and #4241
...
* I needed to remove the region check from PutBucketHandler
2017-06-16 15:02:07 -07:00
Krishnan Parthasarathi
8085ba4494
Filter out internal object prefix during listing ( #4435 )
...
We use ZZZZ-Minio/ prefix internally in our GCS gateway which should be
filtered out in the response to ListObjects.
2017-06-16 15:02:07 -07:00
poornas
9bd0eb1a9e
Set default ETag value if vendor returns empty md5 string ( #4409 )
...
The ETag is constructed from md5 atttribute of object attributes
returned by the vendor's Composer. The md5 attribute comes back
as nil for large uploads. Instead the CRC32C should be used.
Refer to https://cloud.google.com/storage/docs/hashes-etags
Fixes #4397
2017-06-16 15:02:07 -07:00
Anis Elleuch
e4e0abfc05
fix: Check project id before starting gateway ( #4412 )
2017-06-16 15:02:07 -07:00
poornas
12b2fc894b
Remove profile option for gcs from gateway help message ( #4421 )
2017-06-16 15:01:34 -07:00
Krishna Srinivas
2aa76e7407
Change md5Sum to etag ( #4399 )
2017-06-16 14:58:49 -07:00
Remco Verhoef
0dab038858
Cleanup and update the PR with the master branch.
2017-06-16 14:55:32 -07:00
Remco Verhoef
a76556ec1b
Map only default region us-east-1 to gcs us region
2017-06-16 14:54:37 -07:00
Harshavardhana
91c7bb65c5
gateway/gcs: send proper error responses for Get/SetBucket policies. ( #4338 )
...
Fixes #4323
2017-06-16 14:54:37 -07:00
Anis Elleuch
5d602034ea
gateway: Use default params when no args provided ( #4315 )
...
For S3 & Azure, use default parameters when no arguments (endpoint) are
provided. This also avoids a crash.
2017-06-16 14:54:37 -07:00
Nitish Tiwari
b829ec4a6b
Fixes https://github.com/minio/minio/issues/4320 ( #4332 )
...
- Add description for error ErrBucketAlreadyExists
2017-06-16 14:54:37 -07:00
Remco Verhoef
9c50a9f567
Fix ListObjectParts to list properly all parts - closes #4322
2017-06-16 14:54:37 -07:00
Remco Verhoef
52122c0309
Fix uploadIDMarker handling.
2017-06-16 14:54:37 -07:00
Remco Verhoef
3b9d313c87
Fix issue with AbortMultipartUpload, closes #4322
2017-06-16 14:54:37 -07:00
Remco Verhoef
bfff251e2a
Fix issue with UNSIGNED payloads.
...
Additionally also fixes escaping slashes in
temporary multipart names
2017-06-16 14:54:37 -07:00
Remco Verhoef
52b500cce9
Verify multipart etag during complete, closes #4288
2017-06-16 14:54:37 -07:00
Remco Verhoef
c63cdca11f
Support iterating through ListObjectParts using NextPartNumberMarker, closes #4284
2017-06-16 14:54:37 -07:00
Remco Verhoef
4430085981
Add region to gcs gateway example
2017-06-16 14:54:37 -07:00
Remco Verhoef
5c78415b31
Verify md5 content hash, closes #4285
2017-06-16 14:54:37 -07:00
Remco Verhoef
bd67117756
Use maxKeys for iterator
2017-06-16 14:54:37 -07:00
Remco Verhoef
f3e5e9fb29
Support marker, closes #4286
2017-06-16 14:54:37 -07:00
Remco Verhoef
2de1921fe8
Use MINIO_REGION environment variable for region configuration, closes #4287
2017-06-16 14:54:37 -07:00
Remco Verhoef
dd7e47f264
Add access and secret key to example, needed to access Minio Gateway
2017-06-16 14:54:37 -07:00
Remco Verhoef
fe9d826bef
Implement bucket policies
2017-06-16 14:53:36 -07:00
Remco Verhoef
6dbc5aba09
Return correct error when PutObject fails
2017-06-16 14:53:36 -07:00
Remco Verhoef
de5374f74c
Map S3 regions to Google (multi)regions
2017-06-16 14:53:36 -07:00
Remco Verhoef
bf55591c64
Make every backend responsible for parsing its own arguments, fixes #4293
2017-06-16 14:53:36 -07:00
Remco Verhoef
2d814e340f
Return BucketAlreadyExists when bucket exists with another user
2017-06-16 14:53:36 -07:00
Remco Verhoef
0a8cf1a6b0
Allow bucket creation in different regions, closes #4287 and #4241
...
* I needed to remove the region check from PutBucketHandler
2017-06-16 14:53:36 -07:00
Remco Verhoef
07949f68d8
Translate gcs errors to S3 compatible errors, fixes #4278
2017-06-16 14:53:36 -07:00
Remco Verhoef
909a89647b
Use default endpoint when not supplied
2017-06-16 14:53:36 -07:00
Remco Verhoef
6508da5fde
Add usage for GCS gateway, closes #4280
2017-06-16 14:53:36 -07:00
Remco Verhoef
3379f005a5
Initial implementation of Google Cloud Storage
2017-06-16 14:47:02 -07:00
Remco Verhoef
4be609eb82
Added AllAccessDisabled error
2017-06-16 14:47:02 -07:00
Remco Verhoef
dd5b975001
Add comment, gateway should validate object name
2017-06-16 14:47:02 -07:00
Remco Verhoef
9ac3538141
Move anonymous error to object translation from Azure specific to gateway
2017-06-16 14:47:02 -07:00
Remco
ace4f9fd15
Implement gateway support Google Cloud Storage
2017-06-16 14:47:02 -07:00
Nitish Tiwari
b283a2c21f
Bump docs references to latest Minio release RELEASE.2017-06-13T19-01-01Z ( #4546 )
2017-06-15 14:20:36 -07:00
Anand Babu (AB) Periasamy
ed37bf3703
Reduce macOS instructions
2017-06-14 23:26:33 -07:00
Rushan
d0f18dc1b5
browser: Use custom input number step-up/down function ( #4524 )
2017-06-14 17:34:11 -07:00
splinter98
8293f546af
Add support for MQTT server as a notification target ( #4474 )
...
This implementation is similar to AMQP notifications:
* Notifications are published on a single topic as a JSON feed
* Topic is configurable, as is the QoS. Uses the paho.mqtt.golang
library for the mqtt connection, and supports connections over tcp
and websockets, with optional secure tls support.
* Additionally the minio server configuration has been bumped up
so mqtt configuration can be added.
* Configuration migration code is added with tests.
MQTT is an ISO standard M2M/IoT messaging protocol and was
originally designed for applications for limited bandwidth
networks. Today it's use is growing in the IoT space.
2017-06-14 17:27:49 -07:00
Anis Elleuch
af8071c86a
xl: Fix rare freeze after many disk/network errors ( #4438 )
...
xl.storageDisks is sometimes passed to some low-level XL functions. Some disks in
xl.storageDisks are set to nil when they encounter some errors. This means all
elements in xl.storageDisks will be nil after some time which lead to an unusable XL.
2017-06-14 17:14:27 -07:00
Daniel Lind
dce76d9307
Fix xl.diskWithAllParts to proper checksum algorithm ( #4509 )
2017-06-14 17:13:02 -07:00
Harshavardhana
11c4223f2c
Add docker release files. ( #4473 )
...
We used to release by building directly on the docker
hub auto build process, which is sufficient for edge
but it is not a good idea to do it for stable releases.
Do not build docker release binaries again, but instead
use the released binaries themselves which are signed
and validated.
2017-06-14 02:08:35 -07:00
Nitish Tiwari
26903da8c2
Add steps to remove Minio volumes in the swarm ( #4536 )
2017-06-13 16:10:11 -07:00
Harshavardhana
353f2d3a6e
fs: Hold format.json readLock ref to avoid GC. ( #4532 )
...
Looks like if we follow pattern such as
```
_ = rlk
```
Go can potentially kick in GC and close the fd when
the reference is lost, only speculation is that
the cause here is `SetFinalizer` which is set on
`os.close()` internally in `os` stdlib.
This is unexpected and unsual endeavour for Go, but
we have to make sure the reference is never lost
and always dies with the server.
Fixes #4530
2017-06-13 08:29:07 -07:00
Nitish Tiwari
c8947af227
Update Kubernetes-yaml deployment example and Helm deployment doc with Minio image update steps ( #4515 )
2017-06-13 01:37:14 -07:00
Harshavardhana
075b8903d7
fs: Add safe locking semantics for format.json ( #4523 )
...
This patch also reverts previous changes which were
merged for migration to the newer disk format. We will
be bringing these changes in subsequent releases. But
we wish to add protection in this release such that
future release migrations are protected.
Revert "fs: Migration should handle bucketConfigs as regular objects. (#4482 )"
This reverts commit 976870a391 .
Revert "fs: Migrate object metadata to objects directory. (#4195 )"
This reverts commit 76f4f20609 .
2017-06-12 17:40:28 -07:00
Harshavardhana
b8463a738c
Add support for DCOS host detection, improve Docker detection. ( #4525 )
...
isDocker was currently reading from `/proc/cgroup` file. But
this file alone is rather not conclusive evidence. Docker
internally has `.dockerenv` as a special file which we should
use instead.
Fixes #4456
2017-06-13 00:33:21 +00:00
Frank Wessels
6f4862659f
Investigate issue #4461 ( #4521 )
...
* Code to investigate issue #4461 (rare test failure in TestListenAndServeTLS)
* Use UTCNow() instead of time.Now().UTC()
2017-06-13 00:20:29 +00:00
Nitish Tiwari
f59d7a04b4
Removed references to docker-machine in Swarm guide ( #4502 )
2017-06-10 21:44:20 -07:00
Dee Koder
b28d5fa633
Clarify macOS instructions on brew paths. Deleted homebrew upgrade instructions. ( #4501 )
2017-06-09 14:24:32 -07:00
Harshavardhana
48dbd49980
Add support for kubernetes host detection ( #4514 )
...
Additionally improve what we print for `docker pull`
such that its precisely the relevant release tag.
Fixes #4456
2017-06-09 02:42:12 -07:00
Bala FA
3dfe254a11
gateway: make each backend as subcommands. ( #4506 )
...
Fixes #4450
2017-06-08 23:28:45 -07:00
Krishna Srinivas
ec2920e981
Allow "minio server ." to start minio in fs mode ( #4513 )
2017-06-08 18:58:51 -07:00
Rushan
0f5483f497
browser: Disable usage/free stats for browser-gateway ( #4497 )
2017-06-08 15:09:50 -07:00
Krishnan Parthasarathi
8a6b0cc0cd
TestInitListeners: Use port 0 pick available port ( #4508 )
2017-06-08 12:08:21 -07:00
Krishna Srinivas
2c56788f8d
Validate gateway arguments ( #4376 )
...
Fixes #4355
2017-06-08 11:20:56 -07:00
Frank Wessels
145328ac9f
tests: Run select statement in separate goroutine ( #4499 )
...
Instead of after the wg.Wait() so as to make sure that the 'earliest'
of the two select case that becomes active is selected..
2017-06-08 07:39:50 -07:00
poornas
45a568dd85
Give more specific error message on browser for nested policies ( #4488 )
2017-06-07 19:31:23 -07:00
Frank Wessels
7dcc1e92b4
Prevent unnecessary (superfluous) initialization of return variable ( #4490 )
2017-06-08 00:24:46 +00:00
poornas
999ae1cb96
Fix browser download returning zero bytes for s3 ( #4483 )
2017-06-06 18:19:35 -07:00
poornas
6651c2fc5f
disable settings change on browser in gateway mode ( #4472 )
2017-06-06 14:56:41 -07:00
Harshavardhana
976870a391
fs: Migration should handle bucketConfigs as regular objects. ( #4482 )
...
Current code failed to anticipate the existence of files
which could have been created to corrupt the namespace such
as `policy.json` file created at the bucket top level.
In the current release creating such as file conflicts
with the namespace for future bucket policy operations.
We implemented migration of backend format to avoid situations
such as these.
This PR handles this situation, makes sure that the
erroneous files should have been moved properly.
Fixes #4478
2017-06-06 12:15:35 -07:00
Harshavardhana
f99987e47c
Generate sha1sum as well for release for backward compatibility. ( #4475 )
...
Additionally remove support for arm6vl in release, since
go 1.8 the support for armv6 has been dropped and we do
not see high usage events from this platform.
2017-06-06 11:25:06 -07:00
Harshavardhana
1c3f244fc5
creds: Secretkey should be generated upto 40 characters in length. ( #4471 )
...
Current code allowed it wrongly to generate secret key upto 100
we should only use 100 as a value to validate but for generating
it should be 40.
Fixes #4470
2017-06-05 15:18:03 -07:00
Aditya Manthramurthy
986aa8fabf
Bypass network in lock requests to local server ( #4465 )
...
This makes lock RPCs similar to other RPCs where requests to the local
server bypass the network. Requests to the local lock-subsystem may
bypass the network layer and directly access the locking
data-structures.
This incidentally fixes #4451 .
2017-06-05 12:25:04 -07:00
poornas
2559614bfd
fix: Set UIversion in reply for policy API ( #4469 )
2017-06-05 08:11:54 -07:00
Harshavardhana
a4d1ef1b62
browser: update ui-assets with new changes. ( #4467 )
...
Fixes #4269
2017-06-02 15:11:47 -07:00
Harshavardhana
432bf7d99e
Fail if formatting is wrong in our CI tests. ( #4459 )
...
We didn't fail before, we should helps in avoiding
formatting issues to creep into the codebase.
2017-06-02 14:05:51 -07:00
poornas
18c4e5d357
Enable browser support for gateway ( #4425 )
2017-06-01 09:43:20 -07:00
Aditya Manthramurthy
64f4dbc272
Disable redirect of HTTP request to a HTTPS Minio server ( #4454 )
...
Fixes #4452
2017-05-31 20:33:13 -07:00
Frank Wessels
9ba57a8df0
Add errCorruptedFormat to list of ignored errors for metadata operations. ( #4447 )
...
Fixes listing of objects where xl.json is empty or corrupted to skip to the next disk/server (issue 4354).
2017-05-31 20:03:32 -07:00
Dee Koder
5621e6a494
Refactor service stop signal message. ( #4428 )
2017-05-31 11:53:04 -07:00
Frank Wessels
0f0758aece
Load IO error count for posix atomically ( #4448 )
...
* Load error count atomically in order to check for maximum allowed number of IO errors.
* Remove unused (previously atomic) network IO error count
2017-05-31 09:22:53 -07:00
Aditya Manthramurthy
a0e02f43e1
Fix and cleanup update message and improve related tests ( #4361 )
...
Fixes #4232
2017-05-31 09:22:00 -07:00
Harshavardhana
28352f3f5d
log: Startup banner should strip standard ports. ( #4443 )
...
APIEndpoints list should strip off standard ports
to avoid confusion with clients.
2017-05-31 09:21:28 -07:00
Harshavardhana
975972d57e
server: Redirection should use globalMinioPort with host without port. ( #4445 )
...
Currently redirection doesn't work in following scenarios
- server started with port ":80" and TLS is configured
client requested insecure request on port "80"
gets redirected to port 443 and fails.
2017-05-31 09:21:02 -07:00
Harshavardhana
458f22f37c
log: Fix printing of signature error request headers. ( #4444 )
...
The following commit f44f2e341c
fix was incomplete and we still had presigned URLs printing
in query strings in wrong fashion.
This PR fixes this properly. Avoid double encoding
percent encoded strings such as
`s3%!!(MISSING)A(MISSING)`
Print properly as json encoded.
`s3%3AObjectCreated%3A%2A`
2017-05-31 00:11:06 -07:00
Krishna Srinivas
0bba3cc8e3
gateway-azure: Convert S3 metadata to azure metadata ( #4384 )
...
fixes #4292
2017-05-30 20:05:41 -07:00
Cesar Alvernaz
bac5303b10
Some minor fixes ( #4441 )
...
Word terminations
2017-05-30 11:03:25 -07:00
Harshavardhana
e01b2fc06d
Disable network share test, appveyor bug. ( #4446 )
2017-05-30 11:02:31 -07:00
Harshavardhana
072fcf3ba6
fs: Make sure to validate bucket first in PutObject() ( #4427 )
...
Currently even when bucket doesn't exist we wrongly
return success, when an object is a directory prefix with
'/' as suffix and is of size 0.
This PR fixes this behavior.
2017-05-25 09:22:43 -07:00
Harshavardhana
b78f6fbcc5
Do not send envVars in ServerInfo() ( #4422 )
...
Sending envVars along with access and secret
exposes the entire minio server's sensitive
information. This will be an unexpected
situation for all users.
If at all we need to look for things like if
credentials are set through env, we should
only have access to only this information
not the entire set of system envs.
2017-05-24 21:09:23 -07:00
samkevich
99ca8a2928
fix InvalidAccessKeyId error according to amazon documentation ( #4404 )
...
http://docs.aws.amazon.com/AmazonS3/latest/API/ErrorResponses.html
2017-05-23 20:07:52 -07:00
poornas
9b3dd44607
Add dotnet library to minio startup message ( #4410 )
2017-05-23 13:57:27 -07:00
Krishnan Parthasarathi
3c5db69ffd
Treat 0.0.0.0 as local address in --address flag ( #4386 )
2017-05-23 12:07:39 -07:00
Krishna Srinivas
28c26a9e59
Generate random ETag if client does not provide MD5 for PutObjectPart ( #4385 )
...
fixes #4289
fixes #4290
2017-05-22 15:48:48 -07:00
morph027
9136734c02
added secrets to distributed swarm minio ( #4374 )
2017-05-22 11:30:30 -07:00
Harshavardhana
348aa6566c
Add nsswitch.conf to our docker image ( #4379 )
...
This will cause the alpine images to resolve DNS
using /etc/hosts along with the normal DNS resolver.
2017-05-22 07:05:39 -07:00
luomeiqin
9d98bf1c0f
Bucket names can contain hyphen ( #4324 )
2017-05-19 07:30:00 -07:00
Rushan
a767ad321a
Browser: Fix Safari Blob download issue ( #4357 )
2017-05-17 17:03:44 -07:00
Harshavardhana
1886d94e95
server/mux: Use constants provided by Go http ( #4360 )
2017-05-17 11:57:52 -07:00
Anis Elleuch
542f7ae42c
gateway: Reject endpoint pointing to local gateway ( #4310 )
...
Show an error when the user enters an endpoint url pointing
to the gateway server itself.
2017-05-16 21:13:29 -07:00
Harshavardhana
59b3e0b79b
auth/rpc: Add RWMutex instead of Mutex for granular locking. ( #4352 )
...
Refer https://github.com/minio/minio/issues/4345
2017-05-16 14:34:56 -07:00
Aditya Manthramurthy
8975da4e84
Add new ReadFileWithVerify storage-layer API ( #4349 )
...
This is an enhancement to the XL/distributed-XL mode. FS mode is
unaffected.
The ReadFileWithVerify storage-layer call is similar to ReadFile with
the additional functionality of performing bit-rot checking. It
accepts additional parameters for a hashing algorithm to use and the
expected hex-encoded hash string.
This patch provides significant performance improvement because:
1. combines the step of reading the file (during
erasure-decoding/reconstruction) with bit-rot verification;
2. limits the number of file-reads; and
3. avoids transferring the file over the network for bit-rot
verification.
ReadFile API is implemented as ReadFileWithVerify with empty hashing
arguments.
Credits to AB and Harsha for the algorithmic improvement.
Fixes #4236 .
2017-05-16 14:21:52 -07:00
Frank
cae4683971
Make clearing of stale debug lock info independent of deleting map entry of lock itself. ( #4353 )
...
This is believed to address issue #4337 where stale information for debug locks in shown.
2017-05-16 07:19:17 -07:00
Krishna Srinivas
5db1e9f3dd
signature: use region from Auth header if server's region not configured ( #4329 )
2017-05-15 18:17:02 -07:00
Anis Elleuch
465274cd21
server-info: Change Error type to string ( #4346 )
...
Golang std error type doesn't marshal/unmarshal with json. So errors
are not actually being sent when a client calls ServerInfo() API.
2017-05-15 07:28:47 -07:00
Harshavardhana
87fb911d38
Rename structs for azure and s3 gateway to be consistent. ( #4347 )
2017-05-15 00:52:33 -07:00
Harshavardhana
155a90403a
fs/erasure: Rename meta 'md5Sum' as 'etag'. ( #4319 )
...
This PR also does backend format change to 1.0.1
from 1.0.0. Backward compatible changes are still
kept to read the 'md5Sum' key. But all new objects
will be stored with the same details under 'etag'.
Fixes #4312
2017-05-14 12:05:51 -07:00
Harshavardhana
c63afabc9b
build/release: Generate sha256sums also without the release tag. ( #4318 )
...
Ref #4306
2017-05-12 21:40:22 -07:00
poornas
0404e747cf
Fix broken link ( #4344 )
2017-05-12 18:29:50 -07:00
Aditya Manthramurthy
3bc9e6101c
Add minimum requirements sections to notifications docs ( #4328 )
2017-05-11 17:34:27 -07:00
Anis Elleuch
f2ed149714
Add slack channel link to corrupted disk err msg ( #4270 )
2017-05-11 14:27:32 -07:00
Harshavardhana
5a16dcf4cf
Add a graceful msg when CTRL+C is pressed. ( #4248 )
2017-05-11 14:27:18 -07:00
Harshavardhana
fa3d5d0f46
build/release: Generate sha256sums for built binaries. ( #4311 )
...
We used to build sha1sum deprecate it and
use sha256sum instead.
Fixes #4306
2017-05-10 11:22:05 -07:00
Krishna Srinivas
bb292e4e38
web-handler: Allow anonymous download of zip ( #4309 )
...
fixes #4230
2017-05-10 09:54:24 -07:00
poornas
d1971b9a4d
Prevent duplicate policy rows from being created ( #4276 )
2017-05-10 09:52:31 -07:00
Harshavardhana
fa3f6d75b6
fs: Verify if parent is an object before i/o. ( #4304 )
...
PutObject() needs to verify and fail.
Fixes #4301
2017-05-09 17:46:46 -07:00
Harshavardhana
298b470f69
fs/erasure: Ignore objects with / even for DeleteObject() ( #4303 )
...
Additionally GetObject() also returns errFileNotFound similar
to HeadObject().
Fixes #4302
2017-05-09 14:32:24 -07:00
Krishna Srinivas
fc774957fe
gateway: reject requests with unknown authorization ( #4297 )
2017-05-09 07:53:31 -07:00
Nitish Tiwari
c6258f5e97
Multi tenancy doc ( #4215 )
...
* Add multi-tenancy doc
* Multi-tenancy documents
* Remove intro
* Update deploy-multiple-minio.md
* Update deploy-multiple-minio.md
* Update deploy-multiple-minio.md
* Update deploy-multiple-minio.md
* Update multi-tenant details
* Remove file
* Rename deploy-multiple-minio.md to README.md
* update ports
* Add multi-tenancy diagrams
* Link diagrams and update disk name in the commands
* Fix tenant config directory
2017-05-08 19:22:34 -07:00
Anis Elleuch
85bc6003e9
gateway-s3: Avoid x2 double quotes in ListParts ( #4295 )
...
ListParts response returns doubled double quotes in ETag field.
This commit cleans ETag when receiving it from minio client to
fix the issue.
2017-05-08 14:42:05 -07:00
Nitish Tiwari
0d9de50e21
Bump Docker compose file to latest release ( #4271 )
...
* Bump Docker compose file to latest release
* Bump Docker Swarm compose file to latest release
2017-05-07 18:58:24 -07:00
Rushan
d13aa1c42d
browser: make input number types readonly in share objects modal ( #4273 )
2017-05-07 18:02:19 -07:00
Harshavardhana
610dbe3479
config: Do not migrate config file if not needed. ( #4264 )
...
Also improve the error message returned by `pkg/quick`.
Fixes #4233
2017-05-06 10:16:59 -07:00
Anis Elleuch
2df1e2e9a9
doc: Fix pgsql cmd example ( #4265 )
2017-05-05 15:20:29 -07:00
Harshavardhana
e372b5ed67
build: Fix release build names. ( #4263 )
...
Currently due to the occurrence of 6 arguments from
`gen-ldflags.go` leads to a bug where the binaries
genenerated have wrong names.
As shown below.
```
If you want to build for all, Just press Enter: linux/amd64
--> linux/amd64:github.com/minio/minio
$ ls release/linux-amd64/
[2017-05-04 23:08:51 PDT] 17MiB minio
[2017-05-04 23:08:51 PDT] 17MiB minio.2017-05-05T06:08:22Z
[2017-05-04 23:08:51 PDT] 76B minio.shasum
```
This PR fixes this issue by retaining the previous release
binary names.
```
If you want to build for all, Just press Enter: linux/amd64
--> linux/amd64:github.com/minio/minio
$ ls release/linux-amd64/
[2017-05-04 23:08:51 PDT] 17MiB minio
[2017-05-04 23:08:51 PDT] 17MiB minio.RELEASE.2017-05-05T06-08-22Z
[2017-05-04 23:08:51 PDT] 76B minio.shasum
```
2017-05-05 13:16:58 -07:00
Harshavardhana
76f4f20609
fs: Migrate object metadata to objects directory. ( #4195 )
...
Fixes #3352
2017-05-05 08:49:09 -07:00
Harshavardhana
99ddd35343
docs: use IEC format such as iB everywhere. ( #4247 )
2017-05-05 08:28:08 -07:00
Remco Verhoef
01e9adc4b3
Implement anonymous uploads, fixes #4250 ( #4259 )
2017-05-04 20:03:56 -07:00
Nitish Tiwari
4c63fd06c6
Add Kubernetes yaml file deployment example ( #4262 )
...
* Add Kubernetes yaml file deployment example
* Change the Docker image tag to latest
* Update latest release tag
2017-05-04 19:30:46 -07:00
Nitish Tiwari
bb4efbf258
Add minimum requirements for MySQL notification ( #4260 )
2017-05-04 17:30:56 -07:00
Harshavardhana
a89c7299d1
browser: Update ui-assets with new fixes. ( #4246 )
...
Brings two fixes.
- browser: Listing should append instead of replacing previous listing (#4188 )
- browser: Make login form browser auto-fill compatible (#4091 ) fixes #4235
- browser: Selecting a new bucket appends objects list to previous bucket's list (#4252 )
2017-05-04 14:57:41 -07:00
Harshavardhana
df027a8f51
Webhook endpoints can fail, we must start the server. ( #4255 )
...
This PR fixes a regression introduced in #4060
2017-05-04 13:43:54 -07:00
Aditya Manthramurthy
a02575ebf9
Bump up minio-go to ( fixes #4243 ) ( #4256 )
2017-05-04 13:43:23 -07:00
Krishna Srinivas
972a527b66
browser: Selecting a new bucket appends objects list to previous bucket's list ( #4252 )
2017-05-04 11:12:46 -07:00
Krishnan Parthasarathi
02910725c5
Make gateway help for s3/azure similar ( #4249 )
2017-05-04 10:38:48 -07:00
Harshavardhana
0ea8bfaf78
Add waiting on hosts in docker entrypoint for distributed setups. ( #4244 )
...
Thanks to Remco Verhoef <remco@dutchcoders.io > for the script.
Fixes #4225
2017-05-04 00:48:13 -07:00
Remco Verhoef
069cf9e8aa
Use s3.amazonaws.com as default endpoint, fixes #4240 ( #4242 )
2017-05-03 22:41:03 -07:00
Aditya Manthramurthy
2121b78ea7
Fix bug in JSON representation of object properties ( #4238 )
...
Introduced in #4003
2017-05-03 20:10:00 -07:00
Remco Verhoef
5016649f47
Add s3 backend to help, fixes #4219 ( #4221 )
...
* Add s3 backend to help, fixes #4219
* Add samples for Gateway usage with S3
2017-05-03 17:55:30 -07:00
Bala FA
2b78444056
fix: ignore TLS handshake error. ( #4227 )
...
Fixes #4200
2017-05-03 03:23:15 -07:00
Karthic Rao
9b58a669e5
tests: Fix rare test crash ( #4175 )
...
Fix rare test crash by improving the randomness logic.
2017-05-02 23:54:22 -07:00
Krishna Srinivas
e5b2e25caf
gateway-s3: vendor-update minio-go ( #4220 )
2017-05-02 18:46:39 -07:00
Krishna Srinivas
4aa65910e5
gateway: Restore bucket policy functionality for Azure ( #4209 )
2017-05-02 12:27:25 -07:00
Harshavardhana
8b272a3163
config: Improve config migrate messaging. ( #4216 )
...
Previous message
```
Migration from version ‘17’ to ‘18’ completed successfully.
```
For example didn't provide any meaningful insights.
This PR attempts to improve this message as below
```
Configuration file '/home/harsha/.minio/config.json' migrated from version '17' to '18' successfully.
```
Fixes #4199
2017-05-02 11:43:27 -07:00
Harshavardhana
f0b5c0ec7c
windows: Support all REPARSE_POINT attrib files properly. ( #4203 )
...
This change adopts the upstream fix in this regard at
https://go-review.googlesource.com/#/c/41834/ for Minio's
purposes.
Go's current os.Stat() lacks support for lot of strange
windows files such as
- share symlinks on SMB2
- symlinks on docker nanoserver
- de-duplicated files on NTFS de-duplicated volume.
This PR attempts to incorporate the change mentioned here
https://blogs.msdn.microsoft.com/oldnewthing/20100212-00/?p=14963/
The article suggests to use Windows I/O manager to
dereference the symbolic link.
Fixes #4122
2017-05-02 02:35:27 -07:00
Remco Verhoef
44d53c9c67
cleanup and fix comments ( #4212 )
2017-05-01 14:44:31 -07:00
Krishna Srinivas
6cf6828a4c
gateway: Rename gateway files to have "gateway-" prefix ( #4207 )
2017-05-01 10:32:18 -07:00
Krishna Srinivas
01f04c717e
gateway: reject bad path segments in URL ( #4202 )
2017-04-28 17:17:18 -07:00
Krishna Srinivas
0d32b22359
gateway: Fix help message for gateway ( #4201 )
2017-04-28 16:42:16 -07:00
Harshavardhana
cab298d68f
pkg: Update the rpm spec with latest release. ( #4187 )
2017-04-28 12:35:02 -07:00
Krishna Srinivas
1ea53b4d9f
browser: Listing should append instead of replacing previous listing ( #4188 )
...
Fixes #4144
2017-04-28 09:30:26 -07:00
Anis Elleuch
d36dd80a8a
cors: Set Access-Control-Allow-Credentials to true ( #4185 )
...
This allow browsers to send credentials with preflighted requests.
2017-04-27 12:40:22 -07:00
Remco Verhoef
3a539ce660
Implement gateway S3 support ( #3940 )
2017-04-27 11:26:00 -07:00
Harshavardhana
57c5c75611
web: Simplify and converge common functions in web/obj API. ( #4179 )
...
RemoveObject() in webAPI currently re-implements some part
of the code to remove objects combine them for simplicity
and code convergence.
2017-04-26 23:27:48 -07:00
Bala FA
cf1fc45142
Improve duration humanization. ( #4071 )
2017-04-26 03:38:35 -07:00
Dee Koder
64c1c0f37d
docs: Update with home brew special note for macOS upgrades. ( #4180 )
2017-04-25 20:18:48 -07:00
Dee Koder
82857cd6df
docs: Document homebrew install path changes for minio. ( #4178 )
...
* docs: Document homebrew install path changes for minio.
* updates: Updated with the feedback provided.
* docs: Fix breaking change message as per feedback.
* docs: fix typos in readme.
* typo: grammar refactor of update instructions.
2017-04-25 19:43:22 -07:00
Krishnan Parthasarathi
3cdc0c57c8
Provide command to help fill issue template ( #4174 )
2017-04-25 00:58:11 -07:00
Harshavardhana
dc365bca44
build: -s -w should be added by gen-ldflags.go ( #4172 )
2017-04-24 23:01:38 -07:00
Harshavardhana
3b1626216d
docs: Point docker compose to new release. ( #4171 )
2017-04-24 21:37:13 -07:00
Harshavardhana
48aa2ac392
server: Validate path for bad components in a handler. ( #4170 )
2017-04-24 18:13:46 -07:00
Frank
0d1e2ab509
Remove hardcoded min and max limit for erasure coding ( #4157 )
2017-04-24 10:00:33 -07:00
Nitish Tiwari
ebf4c447bb
docs: Add Minikube deployment to k8s docs ( #4133 )
2017-04-24 09:20:51 -07:00
Peter Tribble
2b96d9f706
Enable build on solaris ( #4115 )
2017-04-23 11:10:18 -07:00
Anis Elleuch
83abad0b37
admin: ServerInfo() returns info for each node ( #4150 )
...
ServerInfo() will gather information from all nodes before returning
it back to the client.
2017-04-21 07:15:53 -07:00
Harshavardhana
df346753e1
api: Fix registering of s3 endpoint peers properly ( #4159 )
...
We need to have local peer initialized properly
for listen bucket to work, current code did initialize
properly but the resulting code was initializing
peer on a wrong target v/s what listen bucket expected
it to be.
This regression came in de204a0a52
Fixes #4158
2017-04-20 15:28:29 -07:00
Harshavardhana
f1d7780167
lock: Vendorize all the new changes made in minio/dsync ( #4154 )
...
Fixes #4139
2017-04-19 14:22:35 -07:00
Harshavardhana
5a3c5aec31
server/mux: Fix serverMux to set deadlines based on UTC time. ( #4146 )
...
Avoid using `time.Now()` instead rely on UTC time
for the final deadline, this is to be consistent with
all our internal functions.
Reduce the default read timeout to 15 seconds
in lieu with a newly discovered issue
- https://github.com/minio/minio/issues/4139
Additionally also change the Read() conn wrapper
to set deadline only upon successful Reads().
2017-04-19 13:16:06 -07:00
Aditya Manthramurthy
a4305742e8
Add key for Kafka messages ( fixes #4143 ) ( #4151 )
2017-04-19 11:26:35 -07:00
Harshavardhana
640ebb2f79
lock: Fix missing formatting directives while printing. ( #4147 )
...
Current log prints in this form
```
ERRO[8150] Lock maintenance failed to remove entry for write
lock (should never happen)%!!(MISSING)(EXTRA ....
```
Fix this by using proper formatting directive.
2017-04-19 10:37:56 -07:00
Harshavardhana
402a5e3bea
docs: Fix and reword FreeBSD documentation. ( #4145 )
2017-04-19 00:25:20 -07:00
Harshavardhana
f4dac979a2
server: Fix message when corrupted or unsupported format is found. ( #4142 )
...
Refer https://github.com/minio/minio/issues/4140
This is a fix to provide a little more elaborate message.
2017-04-18 10:35:17 -07:00
Krishnan Parthasarathi
3032f0f505
Remove duration field from lock instrumentation ( #4111 )
...
Duration for which a lock was held can be computed from the `Since`
field of `OpsLockState`. It is the difference between current time and
time at which the namespace lock was held. This change avoids
superfluous instrumentation.
2017-04-15 11:40:01 -07:00
Harshavardhana
7765081db7
cache: Increasing caching GC percent from 20 to 50. ( #4041 )
...
Previous value was set to avoid large cache value build
up but we can clearly see this can cause lots of GC
pauses which can lead to significant drop in performance.
Change this value to 50% and decrease the value to 25%
once the 75% cache size is used. To have a larger
window for GC pauses.
Another change is to only allow caching if a server has
more than 24GB of RAM instead of 8GB.
2017-04-15 02:16:49 -07:00
Anabel V
18bfe5cba6
docs: Created new illustration for docs. ( #4012 )
2017-04-14 17:21:57 -07:00
Anis Elleuch
14f0047295
fs: Remove fs meta lock when PutObject() fails ( #4114 )
...
Removing the fs meta lock file when PutObject() encounters any error
during its execution, such as upload getting permatuerly cancelled
by the client.
2017-04-14 12:06:24 -07:00
Krishna Srinivas
e6b2253da9
gateway: Fix help message for custom Azure Blob Storage endpoint. ( #4113 )
2017-04-14 11:02:43 -07:00
Krishnan Parthasarathi
ca64b86112
Return possible states a heal operation ( #4045 )
2017-04-14 10:28:35 -07:00
Karthic Rao
5f065e2a96
server: Fix CI build complaints ( #4119 )
...
- Ineffassign fixes.
- Spell check correction.
2017-04-14 08:00:04 -07:00
Harshavardhana
a7afa469e2
xl: Add stat calls to keep track of ignored errors. ( #4117 )
...
Such that in a situation where all errors were
ignored we need to reduce the errors using
readQuorum to get a consistent error value.
Without this change errors generated will
never be consistent with for an expected scenario.
For example in a 6 disk setup 1 disk is missing
and 5 do not have the volume (testbucket)
Without this change Stat() would result in different
errors depending on which disk died. Can cause
confusion to S3 client application.
This change addresses need to track type of
errors we ignored and bring readQuorum to
choose the maximally occuring as the value
of truth.
2017-04-14 01:46:16 -07:00
Bala FA
d103d5fb7c
server: Error out if loopback addr is used for Distributed Erasure ( #4105 )
2017-04-12 20:27:24 -07:00
Harshavardhana
6683247080
tests: Fix the sopradic test failure in TestListObjectPartsDiskNotFound ( #4107 )
...
getBucketInfo() should keep track errors ignored,
such that in a situation where all errors were
ignored we need to reduce the errors using readQuorum
to get a consistent error value.
This is the problem we see with DiskNotFound test
disks are randomly removed.
Fixes #4095
2017-04-12 15:38:35 -07:00
Anis Elleuch
e4bd882f11
handlers: Ignore malformatted datetime type header ( #4097 )
...
Ignore headers, such as If-Modified-Since, If-Unmodified-Since, etc.. when they
are received with a format other than HTTP date.
2017-04-12 12:34:57 -07:00
Nitish Tiwari
4448285a83
Cleanup service docs ( #4103 )
...
- Remove windows service doc and add to Minio-service
- Remove Linux service doc as it is duplicated
2017-04-12 12:18:34 -07:00
Harshavardhana
952c618441
server: Fix a regression in printing startup banner. ( #4100 )
...
Octect based sorting was lost in the previous commit
de204a0a52
This PR fixes a regression - fixes #4099
2017-04-12 09:22:35 -07:00
Krishna Srinivas
c5249c35d3
gateway: Support for custom endpoint. ( #4086 )
2017-04-11 17:44:26 -07:00
Bala FA
de204a0a52
Add extensive endpoints validation ( #4019 )
2017-04-11 15:44:27 -07:00
Harshavardhana
1b1b9e4801
lock/rpc: change rpcPath to be called serviceEndpoint. ( #4088 )
...
This is a cleanup to ensure proper naming.
2017-04-11 10:25:21 -07:00
Rushan
1b0b2c1c76
Browser: Make login form browser auto-fill compatible ( #4091 )
2017-04-11 10:11:42 -07:00
koolhead17
7f5e037846
docs: Update docker quick-start guide and adds relevant project URLS ( #4075 )
2017-04-11 00:53:33 -07:00
Karthic Rao
929a13f33f
Fix for writes from Apache Spark. ( #4074 )
...
- Due to usage of amazon SDK, spark expects md5sum of empty string to be
returned when it does PUT on a directory.
- The fix returns md5sum of a empty string for the above mentioned case.
- This fixes the issue of Apache Spark not being able to write into Minio.
2017-04-10 19:51:23 -07:00
Krishna Srinivas
a4209c10ea
signature-v4: Use sha256("") for calculating canonical request ( #4064 )
2017-04-10 09:58:08 -07:00
Harshavardhana
b927523223
server: Introduce a new env MINIO_REGION. ( #4078 )
...
This is implemented to be able to override region
through command line just like how access and
secret keys are provided.
2017-04-09 10:44:10 -07:00
Aditya Manthramurthy
604417baf4
Allow cluster to start when only n/2 servers are up ( #4066 )
...
Fixes #3234 .
Relaxes the quorum requirement to start the object layer, and skips
quick-healing at start-up (as no write quorum is present).
2017-04-09 00:28:27 -07:00
Harshavardhana
6e9ac8db59
docker: Support docker swarm secrets. ( #3977 )
...
Fixes #3896
2017-04-08 01:43:40 -07:00
Harshavardhana
0497d5c342
api: SourceInfo should be populated in GET/HEAD notification. ( #4073 )
...
Refer https://github.com/minio/mc/issues/2073
2017-04-08 01:39:20 -07:00
Harshavardhana
6b4f368dfe
notify: Webhook endpoints can fail, but we must start the server. ( #4060 )
...
Ignore any network errors when registering a webhook
notifier during Minio startup sequence. This way server
can be started even if the webhook endpoint is not available
and unreachable.
2017-04-08 01:13:55 -07:00
Nitish Tiwari
6507c30bbc
Add steps to run Minio distributed on Windows ( #4068 )
2017-04-08 01:12:00 -07:00
Harshavardhana
f44f2e341c
log: Dump signature request properly. ( #4063 )
...
Currently percent encoded strings are not properly encoded.
`s3%!!(MISSING)A(MISSING)`
Print properly as json encoded.
`s3%3AObjectCreated%3A%2A`
2017-04-07 14:37:32 -07:00
Krishnan Parthasarathi
60ea2b17ba
Fix xml block syntax in admin-api Readme ( #4062 )
2017-04-07 03:38:01 -07:00
Harshavardhana
5ed1a8ad23
docs: macOS brew now refers to Minio fork ( #4059 )
2017-04-07 01:42:59 -07:00
Harshavardhana
27749c2124
admin/info: Add HTTPStats value as part of serverInfo() struct. ( #4049 )
...
Remove our counter implementation instead use atomic external
package which supports more types and methods.
2017-04-06 23:08:33 -07:00
Krishna Srinivas
1d99a560e3
refactor: extractSignedHeaders() handles headers removed by Go http server ( #4054 )
...
* refactor: extractSignedHeaders() handles headers removed by Go http server.
* Cleanup extractSignedHeaders() TestExtractSignedHeaders()
2017-04-05 17:00:24 -07:00
Krishna Srinivas
af82d27018
signature-v4: Support for transfer-encoding request header ( #4053 )
2017-04-05 15:08:33 -07:00
Anis Elleuch
f205689ff5
build: Fix compilation in 32 bits platforms ( #4052 )
...
go fails to build Minio under at least, armv6 and 386 due to some
inconsistencies in the type of one syscall variable in different
architectures. This PR casts that variable to uint64 to achieve
the desired consistency.
2017-04-05 11:17:59 -07:00
Harshavardhana
393c01d078
browser: Generate new UI assets.
2017-04-04 23:03:36 -07:00
Rajeesh C V
e9037b9d36
fix: add white space in storage usage section ( #4038 )
...
Fix for #3962 - add whitespace before storage usage details section in
Browse component
2017-04-04 09:14:45 -07:00
Harshavardhana
4747adfcb4
fs: Enable returning ETag along with ListObjects() ( #4042 )
...
This is to comply with S3 behavior, we previously removed
reading `fs.json` for optimization reasons but we have a
reason to believe that providing ETag and using gjson
provides needed benefit of not having to deal with
unmarshalling overhead of golang stdlib.
Fixes #4028
2017-04-04 09:14:03 -07:00
Anis Elleuch
52d8f564bf
sigv2: Unespace canonicalized resources values ( #4034 )
...
Values of canonicalized query resources should be unescaped before calculating
the signature. This bug is not noticed before because partNumber and uploadID
values in Minio doesn't have characters that need to be escaped.
2017-04-03 17:55:14 -07:00
Harshavardhana
3fe33e7b15
handler: simplify parsing valid location constraint. ( #4040 )
...
Separate out validating v/s parsing logic in
isValidLocationConstraint() into parseLocationConstraint()
and isValidLocation()
Additionally also set `X-Amz-Bucket-Region` as part of the
common headers for the clients to fallback on in-case of any
region related errors.
2017-04-03 14:50:09 -07:00
Krishnan Parthasarathi
4041e5f20d
Provide mc-admin-heal command on start-up ( #4031 )
...
Healing of buckets, objects and incomplete uploads are implemented and
available via admin REST APIs. Additionally, it is available via mc admin
sub-command. The warning is no longer relevant.
Fixes #4030
2017-04-03 14:24:25 -07:00
Krishnan Parthasarathi
96c46c15e7
madmin: Rename HealObjectResult to HealResult ( #4035 )
...
madmin.HealObjectResult is used in HealObject and HealUpload. It only
makes sense to rename it to HealResult.
2017-04-03 08:25:32 -07:00
Harshavardhana
3bf67668b6
sys/stats: return cgroup mem limit, fall back to sysinfo() ( #4002 )
...
This is necessary where in certain environments where
cgroup is used to limit memory usage of a container or
a particular process.
GetStats() is used by caching module to figure out the
optimal cacheable size in memory with cgroup limits
what sysinfo reports might not be the right value set
for a given process.
Fixes #4001
2017-04-02 10:46:16 -07:00
Anis Elleuch
e31e2c3bc2
doc: Explain how to create certificate chain file ( #4032 )
...
public.crt needs sometimes to have a chain certificate, this PR
explains how to construct public.crt when certificate are issued
by a certificate authority.
2017-04-02 04:47:56 -07:00
Harshavardhana
214279aa57
build: Reduce binary size by using -s -w ( #4027 )
...
Refer #3939
2017-04-01 01:06:16 -07:00
Harshavardhana
4de6b15fca
vet: Fix all the go vet complaints ( #4029 )
...
```
go tool vet -atomic -bool -copylocks -nilfunc \
-printf -shadow -rangeloops -unreachable \
-unsafeptr -unusedresult cmd/
```
2017-04-01 01:06:06 -07:00
Krishnan Parthasarathi
2bd694dbc8
Add disksUnavailable healStatus const ( #3990 )
...
`disksUnavailable` healStatus constant indicates that a given object
needs healing but one or more of disks requiring heal are offline. This
can be used by admin heal API consumers to distinguish between a
successful heal and a no-op since the outdated disks were offline.
2017-03-31 17:55:15 -07:00
Aditya Manthramurthy
a2a8d54bb6
Add access format support for Elasticsearch notification target ( #4006 )
...
This change adds `access` format support for notifications to a
Elasticsearch server, and it refactors `namespace` format support.
In the case of `access` format, for each event in Minio, a JSON
document is inserted into Elasticsearch with its timestamp set to the
event's timestamp, and with the ID generated automatically by
elasticsearch. No events are modified or deleted in this mode.
In the case of `namespace` format, for each event in Minio, a JSON
document is keyed together by the bucket and object name is updated in
Elasticsearch. In the case of an object being created or over-written
in Minio, a new document or an existing document is inserted into the
Elasticsearch index. If an object is deleted in Minio, the
corresponding document is deleted from the Elasticsearch index.
Additionally, this change upgrades Elasticsearch support to the 5.x
series. This is a breaking change, and users of previous elasticsearch
versions should upgrade.
Also updates documentation on Elasticsearch notification target usage
and has a link to an elasticsearch upgrade guide.
This is the last patch that finally resolves #3928 .
2017-03-31 14:11:27 -07:00
Harshavardhana
2040d32ef8
server/tls: Do not rely on a specific cipher suite ( #4021 )
...
Do not rely on a specific cipher suite instead let the
go choose the type of cipher needed, if the connection
is coming from clients which do not support forward
secrecy let the go tls handle this automatically based
on tls1.2 specifications.
Fixes #4017
2017-03-31 13:28:45 -07:00
Harshavardhana
f1015a5096
notifiers: Stop using url.Parse in validating address format. ( #4011 )
...
url.Parse() wrongly parses an address of format "address:port"
which is fixed in go1.8. This inculcates a breaking change
on our end. We should fix this wrong usage everywhere so that
migrating to go1.8 eventually becomes smoother.
2017-03-31 04:47:40 -07:00
Aditya Manthramurthy
096427f973
Add deliveryMode parameter for AMQP notfication target ( #4008 )
...
Configuration migration was done.
Also adds documentation about AMQP configuration parameters.
Fixes #3982
2017-03-31 03:34:26 -07:00
Rushan
5cec6bd80d
Browser: Use object name with prefix to delete sub-path objects ( #4013 )
2017-03-30 23:28:28 -07:00
Bala FA
6e9c91f43a
fix: use its own lock in serverConfigV17 ( #4014 )
...
Previously serverConfigV17 used a global lock that made any instance of
serverConfigV17 depended on single global serverConfigMu.
This patch fixes by having individual lock per instances.
2017-03-30 22:26:24 -07:00
Bala FA
2df8160f6a
server: handle command line and env variables at one place. ( #3975 )
2017-03-30 11:21:19 -07:00
Romain Bouyé
447fdd4097
docs: Fix typo in docs/config/README.md ( #4009 )
2017-03-30 11:09:15 -07:00
Harshavardhana
28c5a887de
event: Set contentType as well under NotificationEvent. ( #4003 )
...
This is an enhancement change to to cater support all
the data fields present on the object. Currently
we only send a subset of data which object info
provides us.
It also helps us keep a full namespace mirror on
notification targets for efficient query.
2017-03-30 08:58:14 -07:00
Anis Elleuch
fbe8b3259d
webhook: Add support of custom CAs ( #4000 )
2017-03-29 13:42:55 -07:00
Anis Elleuch
e2aba9196f
obj-handlers: Rewrite src & dst path cmp in Copy() ( #3998 )
...
CopyObjectHandler() was incorrectly performing comparison
between destination and source object paths, which sometimes
leads to a lock race. This PR simplifies comparaison and add
one test case.
2017-03-29 09:21:38 -07:00
Aditya Manthramurthy
61b08137b0
Add access format support for Redis notification target ( #3989 )
...
This change adds `access` format support for notifications to a Redis
server, and it refactors `namespace` format support.
In the case of `access` format, a list is used to store Minio
operations in Redis. Each entry in the list is a JSON encoded list of
two items - the first is the Minio server timestamp of the event, and
the second is an object describing the operation that created/replaced
the object in the server.
In the case of `namespace` format, a hash is used. Entries in the hash
may be updated or removed if objects in Minio are updated or deleted
respectively. The field values in the Redis hash are JSON encoded.
Also updates documentation on Redis notification target usage.
Towards resolving #3928
2017-03-29 08:55:53 -07:00
Harshavardhana
1caad902cb
config/path: Figure out absolute paths properly on windows. ( #3996 )
...
The following form of arguments such as
```
minio.exe -C some_dir server dir
```
has stopped working because of lack of handling of
absolute paths for config directory. Always calculate
absolute path for any relative paths on any operating
system.
The following fix converts all config directory relative
paths into absolute paths.
Fixes #3991
2017-03-29 08:55:33 -07:00
Nitish Tiwari
d99efa2c93
Update filename ( #3995 )
2017-03-28 23:14:28 -07:00
Krishna Srinivas
9ee83b89bb
config: Appropriate error message when newer config file is found ( #3972 )
2017-03-28 18:41:16 -07:00
Nitish Tiwari
a8cb43926a
Docker guide fix ( #3992 )
...
* Update Docker quick start guide
- Add Compose to orchestration section.
- Refer Swarm from Docker quick start guide.
- Add common Docker commands to quick start guide.
* Paragraph cleanup
2017-03-28 13:54:19 -07:00
Harshavardhana
b62cd8ed84
sign/streaming: Content-Encoding is not set in newer aws-java-sdks ( #3986 )
...
We can't use Content-Encoding to verify if `aws-chunked` is set
or not. Just use 'streaming' signature header instead.
While this is considered mandatory, on the contrary aws-sdk-java
doesn't set this value
http://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-streaming.html
```
Set the value to aws-chunked.
```
We will relax it and behave appropriately. Also this PR supports
saving custom encoding after trimming off the `aws-chunked`
parameter.
Fixes #3983
2017-03-27 17:02:04 -07:00
Anis Elleuch
1b3a517683
lock, Windows: O_CREAT infers only GENERIC_WRITE ( #3981 )
...
Avoid using GENERIC_ALL flag when creating a file since it could
return permission denied in some Windows versions.
2017-03-27 12:47:39 -07:00
Nitish Tiwari
a9c0f1e0a4
Added kernel tuning docs ( #3921 )
2017-03-27 11:29:04 -07:00
Aditya Manthramurthy
a099319e66
Support access format for database notification targets ( #3953 )
...
* Add configuration parameter "format" for db targets and perform
configuration migration.
* Add PostgreSQL `access` format: This causes Minio to append all events
to the configured table. Prefix, suffix and event filters continue
to be supported for this mode too.
* Update documentation for PostgreSQL notification target.
* Add MySQL `access` format: It is very similar to the same format for
PostgreSQL.
* Update MySQL notification documentation.
2017-03-27 11:27:25 -07:00
Bala FA
6e63904048
browser-flag: wrapped bool type denotes browser on/off flag. ( #3963 )
...
Statically typed BrowserFlag prevents any arbitrary string value
usage. The wrapped bool marshals/unmarshals JSON according to the
typed value ie string value "on" represents boolean true and "off" as
boolean false.
2017-03-26 12:00:27 -07:00
Harshavardhana
565ac4c861
tests: use url.QueryEscape() when dealing with url query params. ( #3974 )
...
This is to keep the portability and also avoid errors that
might occur using the functions written for URL resource name
Since query param values have different escaping requirements.
2017-03-26 11:56:17 -07:00
Harshavardhana
a4ecd8bca2
docs: Add config directory documentation/guide. ( #3889 )
2017-03-25 02:34:04 -07:00
Harshavardhana
28eff0f6c1
build: Improve build messaging, say where we built Minio. ( #3973 )
2017-03-25 00:33:57 -07:00
Krishnan Parthasarathi
c27ece409b
heal: Check if all parts are available and valid ( #3967 )
...
In the algorithm to check if an object requires healing, in addition to
checking if all disks have xl.json present we should check if all parts
of the object are present and have valid blake2b checksums.
Also fixed a minor compilation error in heal-objects-list.go.
2017-03-24 08:40:44 -07:00
Dee Koder
8c0ce2fee9
docs: Removed space from code blocks. ( #3965 )
2017-03-23 19:06:31 -07:00
Bala FA
d3cb79a57c
Refactor logger ( #3924 )
...
This patch fixes below
* Previously fatalIf() never writes log other than first logging target.
* quiet flag is not honored to show progress messages other than startup messages.
* Removes console package usage for progress messages.
2017-03-23 16:36:00 -07:00
Anis Elleuch
11e15f9b4c
config: Remove level in console/file loggers ( #3938 )
...
Also rename fileName field in file logger to filename
2017-03-23 08:27:22 -07:00
Krishnan Parthasarathi
4e92b2ecb8
Fix listDirHealFactory merging of entries across disks ( #3959 )
...
For listing of objects needing heal, we list all objects present on all
the disks and return the set union. We were incorrectly dropping objects
that weren't already seen in disks so far.
Sample directory layout of disks in a 4-disk setup:
`/tmp/1`, `/tmp/2`, `/tmp/3`, `/tmp/4` are directories used as disks here.
`test` is the bucket, `obj1` and obj2` are the objects.
```
/tmp/1/test
└── obj2
├── part.1
├── part.2
└── xl.json
/tmp/2/test
└── obj1
├── part.1
├── part.2
└── xl.json
/tmp/3/test
├── obj1
│ ├── part.1
│ ├── part.2
│ └── xl.json
└── obj2
├── part.1
├── part.2
└── xl.json
/tmp/4/test
[This is empty]
```
2017-03-23 08:24:59 -07:00
koolhead17
80b83a51a3
Docs: Fix for Self signed certificate. ( #3957 )
2017-03-23 08:20:39 -07:00
Krishna Srinivas
777d12d928
browser: update ui-assets.go ( #3956 )
2017-03-22 23:17:14 -07:00
Krishnan Parthasarathi
607c8a9611
Add sourceInfo to NotificationEvent ( #3937 )
...
This change adds information like host, port and user-agent of the
client whose request triggered an event notification.
E.g, if someone uploads an object to a bucket using mc. If notifications
were configured on that bucket, the host, port and user-agent of mc
would be sent as part of event notification data.
Sample output:
```
"source": {
"host": "127.0.0.1",
"port": "55808",
"userAgent": "Minio (linux; amd64) minio-go/2.0.4 mc ..."
}
```
2017-03-22 18:44:35 -07:00
Anis Elleuch
7f5a5b5e9d
config: Do not validate creds when set via env ( #3955 )
...
It is useless to validate access/secret keys stored in
config file when the user sets them in the environment.
2017-03-22 16:11:58 -07:00
Bala FA
d4ca2ee1a3
pkg/quick: add Save() function and other enhancements. ( #3951 )
...
* Add a new function Save() which saves given configuration into given file.
* Simplify Load() function.
* Remove unused CheckVersion().
* CheckData() is a private function now.
* quick_test.go is part of quick package now.
* minio server uses top level quick.Load() and quick.Save() functions.
2017-03-22 10:23:25 -07:00
Krishnan Parthasarathi
417ec0df56
HealObject should succeed when only N/2 disks have data ( #3952 )
2017-03-22 10:15:16 -07:00
Rushan
fbfb4fc5a0
Browser: Use polyfill to support Object.assign in IE11 ( #3942 )
2017-03-21 13:48:07 -07:00
koolhead17
97dc34fe06
docs: Fix for README.md markdown table. ( #3948 )
2017-03-21 12:02:39 -07:00
Krishnan Parthasarathi
9f9ba1e984
XL: Return the right error ( #3944 )
2017-03-21 10:33:25 -07:00
Krishnan Parthasarathi
13c4ce3617
Add notification for object access via GET/HEAD ( #3941 )
...
The following notification event types are available for all targets,
s3:ObjectAccessed:Get
s3:ObjectAccessed:Head
s3:ObjectAccessed:*
2017-03-21 10:32:17 -07:00
Krishnan Parthasarathi
181e002c56
pkg/madmin: Set UploadID in ListUploadsHeal ( #3945 )
...
Without this fix, `mc admin heal -I` wouldn't be able to heal ongoing
uploads. `mc` depends on `ListUploadsHeal` API to identify ongoing
uploads to heal given a bucket and an object.
2017-03-21 10:32:02 -07:00
Anis Elleuch
9d6e226692
heal: Set truncate when no more walk entries ( #3932 )
2017-03-20 15:31:25 -07:00
Krishnan Parthasarathi
eb02261642
XL: Don't return ignored errors in listDirFactory ( #3935 )
...
Previously, erasure backend's `listDirFactory` may return errors which
were explicitly ignored. With this change, it returns nil. Superfluous
checks at higher-layers for ignored errors are removed as well.
2017-03-20 11:09:05 -07:00
Pawan Rawal
1396e91dd1
Update link for downloading minio server. ( #3934 )
...
This and the link for downloading Minio server at other places in the docs seems to be broken. I suppose this happened while updating the name of the page (which updated the url) in Doctor docs.
Might be nice for Doctor to update internal links if the name of a page is changed in a background job.
2017-03-19 18:26:23 -07:00
Bala FA
7ebf11b202
words: new package Damerau Levenshtein distance function. ( #3929 )
2017-03-19 14:23:05 -07:00
Bala FA
1c97dcb10a
Add UTCNow() function. ( #3931 )
...
This patch adds UTCNow() function which returns current UTC time.
This is equivalent of UTCNow() == time.Now().UTC()
2017-03-18 11:28:41 -07:00
Anis Elleuch
3a6111eff5
admin: Export HealStatus data type ( #3930 )
...
`healStatus` can be returned to the API caller. This commit will help
developers to declare a variable with HealStatus type.
2017-03-18 11:27:27 -07:00
Aditya Manthramurthy
2463ae243a
Add support for MySQL notifications ( fixes #3818 ) ( #3907 )
...
As a new configuration parameter is added, configuration version is
bumped up from 14 to 15.
The MySQL target's behaviour is identical to the PostgreSQL: rows are
deleted from the MySQL table on delete-object events, and are
created/updated on create/over-write events.
2017-03-17 09:29:17 -07:00
Krishnan Parthasarathi
c192e5c9b2
Implement heal-upload admin API ( #3914 )
...
This API is meant for administrative tools like mc-admin to heal an
ongoing multipart upload on a Minio server. N B This set of admin
APIs apply only for Minio servers.
`github.com/minio/minio/pkg/madmin` provides a go SDK for this (and
other admin) operations. Specifically,
func HealUpload(bucket, object, uploadID string, dryRun bool) error
Sample admin API request:
POST
/?heal&bucket=mybucket&object=myobject&upload-id=myuploadID&dry-run
- Header(s): ["x-minio-operation"] = "upload"
Notes:
- bucket, object and upload-id are mandatory query parameters
- if dry-run is set, API returns success if all parameters passed are
valid.
2017-03-17 09:25:49 -07:00
Nitish Tiwari
d4eea224d4
Remove white spaces ( #3922 )
2017-03-17 09:23:22 -07:00
Nitish Tiwari
e55421ebdd
Fixed Docker compose link ( #3920 )
2017-03-17 01:06:11 -07:00
Dee Koder
0a5d57a91e
docs: Update gateway doc with roadmap section. ( #3918 )
2017-03-16 18:25:01 -07:00
Krishna Srinivas
cea4cfa3a8
Implement S3 Gateway to third party cloud storage providers. ( #3756 )
...
Currently supported backend is Azure Blob Storage.
```
export MINIO_ACCESS_KEY=azureaccountname
export MINIO_SECRET_KEY=azureaccountkey
minio gateway azure
```
2017-03-16 12:21:58 -07:00
Anis Elleuch
8426cf9aec
config: Accept more address format + unit test ( #3915 )
...
checkURL() is a generic function to check if a passed address
is valid. This commit adds support for addresses like `m1`
and `172.16.3.1` which is needed in MySQL and NATS. This commit
also adds tests.
2017-03-16 11:44:01 -07:00
Harshavardhana
f3334159a4
config: Relax browser and region to be empty. ( #3912 )
...
- If browser field is missing or empty
then default to "on".
- If region field is empty or missing then
default to "us-east-1" (S3 spec behavior)
2017-03-16 11:06:17 -07:00
Harshavardhana
3edff1501e
Vendor upstream redis library instead of our fork. ( #3913 )
...
We forked the upstream to address a build issue on
go 1.6 - that is long done and we don't need
to manage our forks anymore.
2017-03-16 08:22:47 -07:00
Bala FA
21d73a3eef
Simplify credential usage. ( #3893 )
2017-03-16 00:16:06 -07:00
Krishnan Parthasarathi
051f9bb5c6
Implement list uploads heal admin API ( #3885 )
2017-03-16 00:15:06 -07:00
Harshavardhana
6509589adb
Use canonicalETag helper wherever needed. ( #3910 )
2017-03-15 20:48:49 -07:00
Anis Elleuch
ae4361cc45
config: Check for duplicated entries in all scopes ( #3872 )
...
Validate Minio config by checking if there is double json key
in any scope level. The returned error contains the json path
to the duplicated key.
2017-03-15 16:30:34 -07:00
Krishna Srinivas
cad0d0eb7a
browser: Update ui-assets.go ( #3902 )
...
fixes #3898
2017-03-15 11:32:51 -07:00
Nitish Tiwari
ba0c11757e
Added that no special config changes reqd ( #3906 )
...
Added a line saying no special config changes are required for Shared mode. Also the previous `Why Shared Backend` and current `Use cases` are already merged. This fixes the comment: https://github.com/minio/minio/pull/3888#discussion_r105573494
2017-03-15 08:22:03 -07:00
Krishna Srinivas
96050c1e21
browser: Do not show "Loading..." if there are no buckets. ( #3904 )
2017-03-14 19:07:23 -07:00
Rushan
a27d1b3d86
Browser: Use babel-polyfill to support new ES6 built-ins in older browsers ( #3900 )
2017-03-14 15:02:20 -07:00
Anis Elleuch
a5e60706a2
xl,fs: Return 404 if object ends with a separator ( #3897 )
...
HEAD Object for FS and XL was returning invalid object name when
an object name has a trailing slash separator, this PR changes the
behavior and will always return 404 object not found, this guarantees
a better compatibility with S3 spec.
2017-03-13 22:20:46 -07:00
Harshavardhana
5f7565762e
api: postPolicy cleanup. Simplify the code and re-use. ( #3890 )
...
This change is cleanup of the postPolicyHandler code
primarily to address the flow and also converting
certain critical parts into self contained functions.
2017-03-13 14:41:13 -07:00
Nitish Tiwari
3314501f19
Simplify shared mode document ( #3888 )
2017-03-12 16:17:03 -07:00
Harshavardhana
9d53a646a1
Simplify the title for orchestration and some words. ( #3887 )
2017-03-11 02:17:03 -08:00
Nitish Tiwari
5e0032e165
Update links ( #3886 )
2017-03-11 00:11:07 -08:00
Harshavardhana
305952d734
browser: Update ui-assets with new changes.
2017-03-10 15:13:26 -08:00
Rushan
e77885d671
Browser: Add Object.assign polyfill to support older browsers ( #3884 )
2017-03-10 14:30:23 -08:00
Nitish Tiwari
2410eb281e
Update kafkacat command with consumer flag ( #3882 )
2017-03-10 09:01:59 -08:00
Harshavardhana
e54025805f
browser: update ui-assets with new changes.
2017-03-09 15:26:25 -08:00
Anis Elleuch
d602495600
madmin: Do not require SSL to set credentials ( #3879 )
...
We need to relax this requirement and let the client decides
if it can allow to set credentials API over plain connection.
2017-03-09 14:08:33 -08:00
Nitish Tiwari
03937e7554
Added documentation for orchestration platforms ( #3684 )
2017-03-09 14:06:51 -08:00
Harshavardhana
e3b627a192
docker: Add ARM64 image build support ( #3876 )
2017-03-09 13:57:27 -08:00
Rushan
ccc3349f0c
Browser: Show complete bucket name by removing the ellipsis cut off ( #3877 )
2017-03-09 10:42:38 -08:00
Harshavardhana
85cbd875fc
cleanup: All conditionals simplified under pkg. ( #3875 )
...
Address all the changes reported/recommended by
`gosimple` tool.
2017-03-09 10:13:30 -08:00
Harshavardhana
43317530d5
Fix odd shadowing bug in XL init. ( #3874 )
...
Fixes #3873
2017-03-08 20:42:45 -08:00
Bala FA
8a9852220d
Make unit testable cert parsing functions. ( #3863 )
2017-03-08 19:20:01 -08:00
Harshavardhana
47ac410ab0
Code cleanup - simplify server side code. ( #3870 )
...
Fix all the issues reported by `gosimple` tool.
2017-03-08 10:00:47 -08:00
Harshavardhana
433225ab0d
docker: ca-certificates should not be removed. ( #3868 )
2017-03-07 16:30:19 -08:00
Harshavardhana
3e655a2c85
docker: Add ARM docker container dockerfile. ( #3574 )
2017-03-07 15:15:05 -08:00
Anis Elleuch
a2eae54d11
xl: Respect min. space by checking PrepareFile err ( #3867 )
...
It was possible to upload a big file which overcomes the minimal
disk space limit in XL, PrepareFile was actually checking for disk
space but we weren't checking its returned error. This patch fixes
this behavior.
2017-03-07 14:48:56 -08:00
Anis Elleuch
79e0b9e69a
Relax minio server start when disk threshold is reached and adds space check in FS ( #3865 )
...
* fs: Rename tempObjPath variable in fsCreateFile()
* fs/posix: Factor checkDiskFree() function
* fs: Add disk free check in fsCreateFile()
* posix: Move free disk check to createFile()
* xl: Relax free disk check in POSIX initialization
* fs: checkDiskFree checks for space to store data
2017-03-07 12:25:40 -08:00
Krishna Srinivas
29ff9674a0
browser: Humanize expiry time for Share-Object. ( #3861 )
2017-03-06 20:13:52 -08:00
Bala FA
bff4d29415
Remove commands and commandsTree global variables. ( #3855 )
2017-03-06 19:35:26 -08:00
Krishna Srinivas
436db49bf3
Share object expiry value modification modal was not working ( #3860 )
2017-03-06 18:50:25 -08:00
Rushan
966818955e
Browser: Implement multiple object delete ( #3859 )
2017-03-06 15:43:43 -08:00
Harshavardhana
e49efcb9d9
xl: quickHeal heal bucket only when needed. ( #3854 )
...
This improves the startup time significantly
for clusters which have lot of buckets.
Also fixes a bug where `.minio.sys` is created
on disks which do not have `format.json`
2017-03-06 02:00:15 -08:00
Harshavardhana
6f931d29c4
rpm: Add RPM spec for minio build. ( #3853 )
...
Currently the package is built and hosted at
https://copr.fedorainfracloud.org/coprs/minio/minio/
To enable minio repo one has to download.
Fedora - 25
https://copr.fedorainfracloud.org/coprs/minio/minio/repo/fedora-25/minio-minio-fedora-25.repo
Fedora - 26
https://copr.fedorainfracloud.org/coprs/minio/minio/repo/fedora-26/minio-minio-fedora-26.repo
Enables for both i386 and x86_64.
Fixes #3576
2017-03-05 13:09:31 -08:00
Krishnan Parthasarathi
e3fd4c0dd6
XL: Make listOnlineDisks and outDatedDisks consistent w/ each other. ( #3808 )
2017-03-04 14:53:28 -08:00
Harshavardhana
b05c1c11d4
browser: Update UI assets with new changes.
2017-03-03 18:05:02 -08:00
Krishna Srinivas
0bae3330e8
browser: Send correct arguments for RemoveObjects web handler. ( #3848 )
...
Fixes #3839
2017-03-03 17:07:17 -08:00
Harshavardhana
05e53f1b34
api: CopyObjectPart was copying wrong offsets due to shadowing. ( #3838 )
...
startOffset was re-assigned to '0' so it would end up
copying wrong content ignoring the requested startOffset.
This also fixes the corruption issue we observed while
using docker registry.
Fixes https://github.com/docker/distribution/issues/2205
Also fixes #3842 - incorrect routing.
2017-03-03 16:32:04 -08:00
Anis Elleuch
0c8c463a63
tests: Fix web handlers testing with faulty disks ( #3845 )
2017-03-03 15:46:28 -08:00
Aditya Manthramurthy
6df7bc42b8
Fix check for bucket name: ( #3832 )
...
* Do not allow bucket names with adjacent hypen and periods.
* Improve performance by eliminating the usage of regular expressions.
2017-03-03 10:23:41 -08:00
Anis Elleuch
6c00a57a7c
quick: Add yaml format support ( #3833 )
...
quick Save() and Load() infers config file's format from
file name extension.
2017-03-03 10:22:09 -08:00
Harshavardhana
bc52d911ef
api: Increase the maximum object size limit from 5GiB to 16GiB. ( #3834 )
...
The globalMaxObjectSize limit is instilled in S3 spec perhaps
due to certain limitations on S3 infrastructure. For minio we
don't have such limitations and we can stream a larger file
instead.
So we are going to bump this limit to 16GiB.
Fixes #3825
2017-03-03 10:14:17 -08:00
Anis Elleuch
28c53a3555
obj: Make checkBucketExist() returns all errors ( #3843 )
...
This function was returning BucketNotFound for all errors
which at least hides the fact that disks could be corrupted.
This commit fixes the behavior by returning all errors that,
are, by the way, Object API errors.
2017-03-03 10:12:43 -08:00
Harshavardhana
e5d4e7aa9d
web: Validate if bucket names are reserved ( #3841 )
...
Both '.minio.sys' and 'minio' should be never allowed
to be created from web-ui and then fail to list it
by filtering them out.
Fixes #3840
2017-03-03 03:01:42 -08:00
Anis Elleuch
cddc684559
admin: Set Config returns errSet and errMsg ( #3822 )
...
There is no way to see if a node encountered an error
when trying to set a new config set, this commit adds
a bool errSet field.
2017-03-03 02:53:48 -08:00
Zejun Li
32d0d3d4ac
Enhanced newObjectLayerFn ( #3837 )
2017-03-03 01:07:45 -08:00
Bala FA
98d17d2a97
Remove globalQuiet and globalConfigDir global variables ( #3830 )
2017-03-02 14:21:30 -08:00
Bala FA
208dd15245
Remove globalMaxCacheSize and globalCacheExpiry variables ( #3826 )
...
This patch fixes below
* Remove global variables globalMaxCacheSize and globalCacheExpiry.
* Make global variables into constant in objcache package.
2017-03-02 10:34:37 -08:00
Anis Elleuch
a179fc9658
quick: Simplify Load() and CheckVersion() ( #3831 )
2017-03-02 10:29:06 -08:00
Zejun Li
d1afd16955
Using RWMutex to guard closing and listeners ( #3829 )
2017-03-02 10:00:22 -08:00
Bala FA
2348ae7a19
Make default values as constants ( #3828 )
2017-03-02 04:58:39 -08:00
Bala FA
480ea826dc
Move rlimit functions into sys package. ( #3824 )
...
This patch addresses below
* go build works for bsd family
* probe total RAM size for bsd family
* make unit testable functions
2017-03-01 21:51:57 -08:00
Aditya Manthramurthy
09e9fd745c
Close client connection after checking for release update ( #3820 )
2017-03-01 09:18:55 -08:00
Anis Elleuch
77c1998a38
config: Fix creating new config with wrong version ( #3821 )
...
Simplify a little config code to avoid making mistake
next time.
2017-03-01 09:17:04 -08:00
Krishna Srinivas
91cf54f895
web-handlers: Support removal of multiple objects at once. ( #3810 )
2017-02-28 19:07:28 -08:00
Karthic Rao
2b0ed21f08
tests: Fix test server init - cleanup ( #3806 )
2017-02-28 18:05:52 -08:00
Harshavardhana
472fa4a6ca
api: Multi object delete should be protected. ( #3814 )
...
Add missing protection from deleting multiple objects
in parallel. Currently we are deleting objects without
proper locking through this API.
This can cause significant amount of races.
2017-02-28 18:00:24 -08:00
Bala FA
097cec676a
fix: Set globalMaxCacheSize to allowable value. ( #3816 )
...
If memory resource limit and total RAM are more than 8GiB, either 50%
of memory resource limit or total RAM is set to globalMaxCacheSize.
2017-02-28 16:51:52 -08:00
Rushan
fcad4a44fd
Browser: Remove duplicate object entries while sorting ( #3813 )
2017-02-28 13:11:34 -08:00
Anis Elleuch
9b3c014bab
config: Add browser parameter ( #3807 )
...
browser new parameter receives "on" or "off" parameter which is similar
to MINIO_BROWSER
2017-02-27 14:59:53 -08:00
Krishnan Parthasarathi
c9619673fb
Implement SetConfig admin API handler. ( #3792 )
2017-02-27 11:40:27 -08:00
Anis Elleuch
dce0345f8f
Set disk to nil after write which needs quorum ( #3795 )
...
Ignore a disk which wasn't able to successfully perform an action to
avoid eventual perturbations when the disk comes back in the middle
of write change.
2017-02-26 11:58:32 -08:00
Anis Elleuch
461b2bbd37
admin: Move SetCredentials from Service to Generic ( #3805 )
...
Setting credentials doesn't belong to service management API
anymore.
2017-02-25 11:06:08 -08:00
Bala FA
69777b654e
event: use common initialization logic ( #3798 )
...
Previously creating and adding targets for each notification type was
repeated. This patch fixes it.
2017-02-24 18:27:52 -08:00
Harshavardhana
70d2cb5f4d
rpc: Remove time check for each RPC calls. ( #3804 )
...
This removal comes to avoid some redundant requirements
which are adding more problems on a production setup.
Here are the list of checks for time as they happen
- Fresh connect (during server startup) - CORRECT
- A reconnect after network disconnect - CORRECT
- For each RPC call - INCORRECT.
Verifying time for each RPC aggravates a situation
where a RPC call is rejected in a sequence of events
due to enough load on a production setup. 3 second
might not be enough time window for the call to be
initiated and received by the server.
2017-02-24 18:26:56 -08:00
Harshavardhana
cff45db1b9
cli: Use ADDRESS:PORT to clarify --address behavior ( #3803 )
...
Currently we document as IP:PORT which doesn't provide
if someone can use HOSTNAME:PORT. This is a change
to clarify this by calling it as ADDRESS:PORT which
encompasses both a HOSTNAME and an IP.
Fixes #3799
2017-02-24 14:19:20 -08:00
Harshavardhana
bcc5b6e1ef
xl: Rename getOrderedDisks as shuffleDisks appropriately. ( #3796 )
...
This PR is for readability cleanup
- getOrderedDisks as shuffleDisks
- getOrderedPartsMetadata as shufflePartsMetadata
Distribution is now a second argument instead being the
primary input argument for brevity.
Also change the usage of type casted int64(0), instead
rely on direct type reference as `var variable int64` everywhere.
2017-02-24 09:20:40 -08:00
Harshavardhana
25b5a0534f
browser: Update ui-assets and fix the copyright header. ( #3790 )
2017-02-22 17:27:26 -08:00
Rushan
52d6678bf0
Browser: Implement multi select user interface for object listings ( #3730 )
2017-02-22 14:51:38 -08:00
Nitish Tiwari
d8950ba7c5
Added server times note and fix Notes rendering for Doctor. ( #3787 )
2017-02-22 02:12:03 -08:00
Harshavardhana
cc28765025
xl/multipart: Make sure to delete temp renamed object. ( #3785 )
...
Existing objects before overwrites are renamed to
temp location in completeMultipart. We make sure
that we delete it even if subsequenty calls fail.
Additionally move verifying of parent dir is a
file earlier to fail the entire operation.
Ref #3784
2017-02-21 19:43:44 -08:00
Harshavardhana
fe86319c56
ci: For windows builds stick to go1.7.5 ( #3786 )
2017-02-21 17:24:11 -08:00
Harshavardhana
99a12613a3
update: For source builds look for absolute path. ( #3780 )
...
os.Args[0] doesn't point to absolute path we need
use exec.LookPath to find the absolute path before
sending os.Stat().
2017-02-21 01:32:05 -08:00
Nitish Tiwari
097dd7418a
Remove unused erasure diagram ( #3783 )
2017-02-20 21:01:08 -08:00
Nitish Tiwari
a7d3ea8c15
Update erasure code image ( #3782 )
2017-02-20 20:12:21 -08:00
Krishnan Parthasarathi
2745bf2f1f
Implement ServerConfig admin REST API ( #3741 )
...
Returns a valid config.json of the setup. In case of distributed
setup, it checks if quorum or more number of nodes have the same
config.json.
2017-02-20 12:58:50 -08:00
Anis Elleuch
70d825c608
doc: Small rewrite of bucket events notif intro ( #3775 )
2017-02-20 12:07:27 -08:00
Harshavardhana
6b68c0170f
For streaming signature do not save content-encoding in PutObject() ( #3776 )
...
Content-Encoding is set to "aws-chunked" which is an S3 specific
API value which is no meaning for an object. This is how S3
behaves as well for a streaming signature uploaded object.
2017-02-20 12:07:03 -08:00
Aditya Manthramurthy
0a905e1a8a
Fix rabbitmq reconnect problem ( #3778 )
2017-02-20 12:05:21 -08:00
Harshavardhana
9eb8e375c5
cli: Make sure to add --help flag for subcommands. ( #3773 )
...
--help is now back and prints properly with command
help template.
2017-02-19 20:46:06 -08:00
Harshavardhana
7ea1de8245
copyObject: Be case sensitive for windows only server. ( #3766 )
...
For case sensitive platforms we should honor case.
Fixes #3765
```
1) python s3cmd -c s3cfg_localminio put logo.png s3://testbucket/xyz/etc2/logo.PNG
2) python s3cmd -c s3cfg_localminio ls s3://testbucket/xyz/etc2/
2017-02-18 10:58 22059 s3://testbucket/xyz/etc2/logo.PNG
3) python s3cmd -c s3cfg_localminio cp s3://testbucket/xyz/etc2/logo.PNG s3://testbucket/xyz/etc2/logo.png
remote copy: 's3://testbucket/xyz/etc2/logo.PNG' -> 's3://testbucket/xyz/etc2/logo.png'
4) python s3cmd -c s3cfg_localminio ls s3://testbucket/xyz/etc2/
2017-02-18 10:58 22059 s3://testbucket/xyz/etc2/logo.PNG
2017-02-18 11:10 22059 s3://testbucket/xyz/etc2/logo.png
```
2017-02-18 13:41:59 -08:00
Anis Elleuch
54a18592e9
flags: Fix --version output ( #3772 )
2017-02-18 13:41:33 -08:00
Anis Elleuch
7e84c7427d
server-mux: Rewrite graceful shutdown mechanism ( #3771 )
...
Old code uses waitgroup Add() and Wait() in different threads,
which eventually can lead to a race.
2017-02-18 13:28:54 -08:00
Bala FA
d12f3e06b1
config-old: Use interface to avoid code repetition. ( #3769 )
2017-02-18 10:45:37 -08:00
Harshavardhana
0137ff498a
auth/rpc: Token can be concurrently edited protect it. ( #3764 )
...
Make sure we protect when we access `authToken` in authClient.
Fixes #3761
2017-02-18 03:15:42 -08:00
Harshavardhana
34d9a6b46a
Make sure client initializes to proper lock RPC path. ( #3763 )
...
Fixes a regression introduced in previous commit.
2017-02-18 02:52:11 -08:00
Harshavardhana
50b4e54a75
fs: Do not return reservedBucket names in ListBuckets() ( #3754 )
...
Make sure to skip reserved bucket names in `ListBuckets()`
current code didn't skip this properly and also generalize
this behavior for both XL and FS.
2017-02-16 14:52:14 -08:00
Harshavardhana
8816b08aae
Fix the systemd config path to the new URL
2017-02-15 21:28:06 -08:00
Harshavardhana
271e3ecde5
Fix tests from cli changes
2017-02-15 18:05:55 -08:00
Harshavardhana
611bd68739
cli: Add new features from CLI.
...
use `.HelpName` for template to pass down
proper command names. Also treat flags
to be optional in all of them.
2017-02-15 17:45:11 -08:00
Harshavardhana
25eeb88a8f
cli: Bring upstream changes to minio server.
2017-02-15 17:30:31 -08:00
Krishna Srinivas
3e770defae
browser: Update ui assets with new changes ( #3751 )
2017-02-15 14:44:34 -08:00
Krishna Srinivas
ae4656e699
browser: Remove currentPath prefix in object names from list result. ( #3750 )
2017-02-15 13:07:47 -08:00
Harshavardhana
f6a9e690bf
browser: Update ui assets with new changes.
2017-02-15 11:29:23 -08:00
Anis Elleuch
7f86a21317
admin: Add ServerInfo API() ( #3743 )
2017-02-15 10:45:45 -08:00
Harshavardhana
fb39c7c26b
sRPC/client: Properly trim storageRPCPath for actual disk path. ( #3749 )
...
Never print internal RPC endpoint paths.
2017-02-15 03:47:47 -08:00
Harshavardhana
13c3b9cbcb
main/cli: Don't print hidden flags/commands in help template. ( #3748 )
...
Always use .VisibleFlags and .VisibleCommands to not print
Hidden flags as expected from help template.
2017-02-15 02:25:38 -08:00
Bala FA
602dac8773
mainUpdate: refactor to handle quiet flag properly ( #3744 )
2017-02-15 00:31:00 -08:00
Dee Koder
c6e76160ad
Update README.md
2017-02-14 14:20:29 -08:00
Bala FA
a53b909fcd
mainVersion: Remove minio init and quiet handling ( #3739 )
2017-02-13 00:24:34 -08:00
Harshavardhana
22909c849e
objcache: Return io.ReaderAt to avoid Seeking and Reading. ( #3735 )
2017-02-11 17:17:58 -08:00
Bala FA
440866d26c
Move go version check into main() ( #3734 )
...
Previously the check was done in init(). This patch moves into main
for unit testable friendly function.
2017-02-11 01:27:27 -08:00
Krishna Srinivas
25b936c369
browser: Implement infinite scrolling for object listing. ( #3720 )
...
fixes #2831
2017-02-10 22:54:42 -08:00
Nitish Tiwari
8f66cfa316
Fix https://github.com/minio/minio/issues/3453 ( #3733 )
2017-02-10 22:53:17 -08:00
Bala FA
8912b6bf3b
trie: new package ( #3729 )
...
This implements a simple trie tree for minio server/tools.
This package borrows idea from
https://godoc.org/golang.org/x/text/internal/triegen .
2017-02-10 11:51:41 -08:00
Karthic Rao
a799351926
Fix macOS builds ( #3728 )
2017-02-10 00:37:22 -08:00
Harshavardhana
9df01035da
Remove XL references in public docs to Erasure. ( #3725 )
...
Ref #3722
2017-02-09 23:26:44 -08:00
Krishnan Parthasarathi
1ad96ee09f
Handle uptime for single-node instances differnt from distributed setup ( #3726 )
2017-02-09 20:38:14 -08:00
Harshavardhana
1b4bb94ac4
config: setter/getter for Notifier and Logger into its own struct. ( #3721 )
...
This is an attempt cleanup code and keep the top level config
functions simpler and easy to understand where as move the
notifier related code and logger setter/getter methods as part
of their own struct.
Locks are now held properly not globally by configMutex, but
instead as private variables.
Final fix for #3700
2017-02-09 15:20:54 -08:00
Andreas Auernhammer
f38222c0cc
update the blake2b implementation ( #3724 )
...
Fixes a performance bug caused by SSE-AVX register savings on amd64.
2017-02-09 15:01:00 -08:00
Anis Elleuch
c9b1468c3b
Presigend Post: Error out when File is not found ( #3723 )
...
Follow S3 behavior when no File is sent in the presigned
post request form.
2017-02-09 12:37:32 -08:00
Krishna Srinivas
6800902b43
web-handlers: Implement API to download files as a zip file. ( #3715 )
2017-02-08 23:39:08 -08:00
Krishnan Parthasarathi
e5773e11c6
Make minio server compile on OpenBSD, NetBSD, Solaris ( #3719 )
2017-02-08 22:27:35 -08:00
Nitish Tiwari
0c7694894b
Added spaces for formatting ( #3717 )
2017-02-08 00:51:08 -08:00
Krishnan Parthasarathi
ce9aa2f2b2
Add uptime to ServiceStatus ( #3690 )
2017-02-08 00:13:02 -08:00
Nitish Tiwari
7547f3c8a3
Added Webhook notification details ( #3706 )
2017-02-07 23:19:32 -08:00
Harshavardhana
31dff87903
Honor envs properly for access and secret key. ( #3703 )
...
Also changes the behavior of `secretKeyHash` which is
not necessary to be sent over the network, each node
has its own secretKeyHash to validate.
Fixes #3696
Partial(fix) #3700 (More changes needed with some code cleanup)
2017-02-07 12:51:43 -08:00
Anis Elleuch
fd72c21e0e
tests: Reduce two functions work to gain test time ( #3712 )
...
TestListObjectsHeal and TestFSShutdown takes around 3 min,
this PR reduces the number of created test objects
2017-02-07 12:51:23 -08:00
Harshavardhana
310bf5bd36
auth/rpc: Make auth rpc client retry configurable. ( #3695 )
...
Currently the auth rpc client defaults to to a maximum
cap of 30seconds timeout. Make this to be configurable
by the caller of authRPCClient during initialization, if no
such config is provided then default to 30 seconds.
2017-02-07 02:16:29 -08:00
Harshavardhana
a170e44689
Let hijacker and flusher interfaces to be reflected. ( #3709 )
...
Ideally here if the interface is not found it would
fail the server, as it should be because without these
we can't even have a working server in the first place.
Just like how it fails in master invariably inside Go
net/http code path.
Fixes #3708
2017-02-06 23:52:47 -08:00
Anis Elleuch
70e70446bb
signv4: Read always returns EOF when stream ends ( #3692 )
...
When EOF is reached, further calls of Read() doesn't return io.EOF
but continue to work as it expects to have more data, this PR fixes
the behavior
2017-02-06 14:19:27 -08:00
Krishna Srinivas
45d9cfa0c5
signature-v4: stringToSign and signingKey should use Scope's date. ( #3688 )
...
fixes #3676
2017-02-06 13:09:09 -08:00
Anis Elleuch
93fd269329
stats: Add network and http statisics ( #3686 )
...
Network: total bytes of incoming and outgoing server's data
by taking advantage of our ConnMux Read/Write wrapping
HTTP: total number of different http verbs passed in http
requests and different status codes passed in http responses.
This is counted in a new http handler.
2017-02-06 09:29:53 -08:00
Harshavardhana
6717a0b68c
Add consistency guarantees
2017-02-05 18:31:48 -08:00
Harshavardhana
533338bdeb
all/windows: Be case in-sensitive about pattern matching. ( #3682 )
...
Resource strings and paths are case insensitive on windows
deployments but if user happens to use upper case instead of
lower case for certain configuration params like bucket
policies and bucket notification config. We might not honor
them which leads to a wrong behavior on windows.
This is windows only behavior, for all other platforms case
is still kept sensitive.
2017-02-03 23:27:50 -08:00
Anis Elleuch
b6ebf2aba8
server-mux: Simplify graceful shutdown behavior ( #3681 )
...
`*http.Server` is no more used, doing some cleanup.
2017-02-03 22:53:30 -08:00
Anis Elleuch
ed4fcb63f7
Require content-length in POST & Upload requests ( #3671 )
...
Avoid passing size = -1 to PutObject API by requiring content-length
header in POST request (as AWS S3 does) and in Upload web handler.
Post handler is modified to completely store multipart file to know
its size before sending it to PutObject().
2017-02-02 10:45:00 -08:00
Harshavardhana
4b4cb07fb6
Update browser assets with new changes.
2017-02-02 03:01:49 -08:00
Harshavardhana
fc446e0b81
docs: Add browser doc for running dev in custom port ( #3675 )
...
Fixes #3674
2017-02-01 13:12:13 -08:00
Anis Elleuch
f612a7dd85
madmin: Fix a typo in Locks duration query name ( #3673 )
2017-02-01 11:46:49 -08:00
Krishnan Parthasarathi
0472e5c1e1
Change query param name to duration in list/clear locks API ( #3664 )
...
Following is a sample list lock API request schematic,
/?lock&bucket=mybucket&prefix=myprefix&duration=holdDuration
x-minio-operation: list
The response would contain the list of locks held on mybucket matching
myprefix for a duration longer than holdDuration.
2017-02-01 11:17:30 -08:00
Harshavardhana
6a6c930f5b
xl: Abort multipart upload should honor quorum properly. ( #3670 )
...
Current implementation didn't honor quorum properly and didn't
handle the errors generated properly. This patch addresses that
and also moves common code `cleanupMultipartUploads` into xl
specific private function.
Fixes #3665
2017-02-01 11:16:17 -08:00
Karthic Rao
35ca3e5d9b
Fix unresponsiveness of doneCh due to Sleep call. ( #3667 )
2017-02-01 09:17:32 -08:00
Harshavardhana
a9ab01731f
docs: Remove reference word from notification title
2017-01-31 18:17:31 -08:00
Harshavardhana
cd80e6df29
docs: Move the notifications into docs/bucket
...
Cleanup some formatting issues.
2017-01-31 18:07:39 -08:00
koolhead17
4dea4f3b89
docs: added event-notification back end doc. ( #3510 )
2017-01-31 17:04:36 -08:00
Harshavardhana
1b30a3be2b
xl/utils: getPartSizeFromIdx should return error. ( #3669 )
2017-01-31 15:34:49 -08:00
Krishna Srinivas
f7f103725b
browser: Make logo and "Minio Browser" text non-clickable. ( #3668 )
...
fixes #3601
2017-01-31 13:44:07 -08:00
Harshavardhana
77a192a7b5
Implement CopyObjectPart API ( #3663 )
...
This API is implemented to allow copying data from an
existing source object to an ongoing multipart operation
http://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadUploadPartCopy.html
Fixes #3662
2017-01-31 09:38:34 -08:00
Harshavardhana
cb48517a78
Update browser assets with new changes.
2017-01-30 16:38:49 -08:00
Anis Elleuch
e9394dc22d
xl PutObject: Split object into parts ( #3651 )
...
For faster time-to-first-byte when we try to download a big object
2017-01-30 15:44:42 -08:00
Krishna Srinivas
46743c7918
browser: Component attr names should not clash with redux state keys. ( #3656 )
...
fixes #270
2017-01-30 14:25:55 -08:00
Krishna Srinivas
b288eaddb3
xl: bit-rot algo was not set in get-object. ( #3652 )
...
fixes #3650
2017-01-30 14:25:28 -08:00
Krishnan Parthasarathi
2665aba555
Fail PutBucketPolicy if conditions are incompatible with actions. ( #3659 )
2017-01-30 09:20:16 -08:00
Krishnan Parthasarathi
9b6bcb30d9
Fix handling of StringNotEquals condition operator ( #3660 )
2017-01-30 09:18:10 -08:00
Harshavardhana
ac9ba13c19
build: Increase the travis build test timeout to 15mins
2017-01-30 01:39:53 -08:00
Krishnan Parthasarathi
864b8795aa
heal: Should delete stale object parts before healing ( #3649 )
2017-01-30 00:45:56 -08:00
Bala FA
cc1575f944
fix: do port availability check only on macOS. ( #3654 )
...
On macOS, if a process already listens on 127.0.0.1:PORT, net.Listen() falls back
to IPv6 address ie minio will start listening on IPv6 address whereas another
(non-)minio process is listening on IPv4 of given port.
To avoid this error sutiation we check for port availability only for macOS.
Note: checkPortAvailability() tries to listen on given port and closes it.
It is possible to have a disconnected client in this tiny window of time.
2017-01-30 00:44:36 -08:00
Krishnan Parthasarathi
b408d0e87d
Add aws:Referer condition key support. ( #3641 )
...
This change implements bucket policy enhancements required to restrict access based on HTTP referer.
See https://docs.aws.amazon.com/AmazonS3/latest/dev/example-bucket-policies.html#example-bucket-policies-use-case-4 for more information.
Fixes #3540
2017-01-29 19:45:11 -08:00
Nitish Tiwari
69b81af93e
Fix Minio quick start guide URL ( #3655 )
2017-01-29 07:21:30 -08:00
Harshavardhana
cf558ff6d1
docs: Fix TLS doc to be docs.minio.io compatible
2017-01-28 22:46:29 -08:00
Harshavardhana
5af7bd6a01
docs: Fix a typo in TLS doc description
2017-01-28 22:41:52 -08:00
Harshavardhana
73f4f29110
docs: Restructure docs, move the files to their relevant location. ( #3648 )
...
Also combines windows TLS docs with single doc with Linux.
2017-01-28 19:45:30 -08:00
Harshavardhana
ccd949d8ca
Add -v for race tests to avoid travis and appveyor timeouts. ( #3647 )
...
This is added so that CI's don't wrong fully timeout on
little longer running tests.
2017-01-27 12:31:02 -08:00
Anis Elleuch
e1bc99e4fe
xl: Fix GET of an empty multiparted object ( #3646 )
...
GetObject returns unsatisfied range error when we try to download an object
uploaded using multipart mechanism.
2017-01-27 10:51:02 -08:00
Alex Ellis
02194ee3c6
Fix issue with webhook events not closing Response body, leaving connections open with remote server ( #3645 )
2017-01-27 09:50:08 -08:00
Harshavardhana
85f2b74cfd
jwt: Cache the bcrypt password hash. ( #3526 )
...
Creds don't require secretKeyHash to be calculated
everytime, cache it instead and re-use.
This is an optimization for bcrypt.
Relevant results from the benchmark done locally, negative
value means improvement in this scenario.
```
benchmark old ns/op new ns/op delta
BenchmarkAuthenticateNode-4 160590992 80125647 -50.11%
BenchmarkAuthenticateWeb-4 160556692 80432144 -49.90%
benchmark old allocs new allocs delta
BenchmarkAuthenticateNode-4 87 75 -13.79%
BenchmarkAuthenticateWeb-4 87 75 -13.79%
benchmark old bytes new bytes delta
BenchmarkAuthenticateNode-4 15222 9785 -35.72%
BenchmarkAuthenticateWeb-4 15222 9785 -35.72%
```
2017-01-26 16:51:51 -08:00
Krishna Srinivas
152cdf1c05
fs: Move traceError() to lower functions where possible. ( #3633 )
2017-01-26 15:40:10 -08:00
Krishna Srinivas
17dd1c19df
cleanup: refactor common code between FS and XL listDirFactory. ( #3639 )
2017-01-26 15:39:22 -08:00
Anis Elleuch
8e49a3d047
Simplify running cmd.Main() for external tests ( #3636 )
...
An external test that runs cmd.Main() has a difficulty to set cmd arguments
and MINIO_{ACCESS,SECRET}_KEY values, this commit changes a little the current
behavior in a way that helps external tests.
2017-01-26 15:22:41 -08:00
Krishna Srinivas
cccf77229d
cleanup: Move code in *-multipart-common.go to *-multipart.go ( #3638 )
...
The code in *-multipart-common.go is not common anymore.
2017-01-26 12:51:12 -08:00
Harshavardhana
dafdc74605
fs: if fs.json is empty ignore it while reading metadata. ( #3634 )
...
This is needed so that we don't send wrong errors
on previously failed PutObject() which would have
left a stale `fs.json` entry.
2017-01-26 10:19:07 -08:00
Anis Elleuch
c753b5dfeb
madmin: All APIs return ErrorResponse error type ( #3632 )
...
Avoid returning errors.New("Got HTTP Status") in all APIs
2017-01-25 16:01:29 -08:00
Krishna Srinivas
82373e3d50
fs: cleanup - do not cache size of metafiles ( #3630 )
...
* Remove Size() method and size field from lock.LockedFile
* WriteTo method of fsMeta and uploadsV1 now takes concrete type *lock.LockedFile
2017-01-25 12:29:06 -08:00
Anis Elleuch
76b8abcd8e
madmin: Fix args order in listObjectsHeal() ( #3629 )
...
The order of marker and delimiter and in listObjectsHeal() internal function
are switched. That will give wrong result in case of a non recursive objects
heal list.
2017-01-25 09:58:49 -08:00
Harshavardhana
dbb5408906
Add ARM64 build binary for README.md
...
Fixes #3626
2017-01-24 20:28:54 -08:00
Harshavardhana
18c9d49a32
Build browser with new changes and update yarn.lock
2017-01-24 19:11:37 -08:00
Harshavardhana
d41dcb784b
Move to blake2b-simd due to perf problems in golang.org/x/crypto
...
Ref https://github.com/golang/go/issues/18563
2017-01-24 18:07:34 -08:00
Krishna Srinivas
659d5aabd1
browser: Access to private paths redirects to login. ( #3622 )
2017-01-24 12:08:00 -08:00
Krishna Srinivas
b4343a28b7
browser: add yarn.lock and .gitignore files. ( #3621 )
2017-01-24 11:56:30 -08:00
Krishnan Parthasarathi
0e693e0284
Add dry-run query param for HealFormat API ( #3618 )
2017-01-24 08:11:05 -08:00
Anis Elleuch
fc880f9b23
admin: Enhance set credentials test ( #3619 )
...
Add more test cases and ignore access and secret keys set from the env
2017-01-24 08:08:36 -08:00
Krishna Srinivas
cead24b0f7
miniobrowser: Bring Minio browser source into minio repo. ( #3617 )
2017-01-23 18:07:22 -08:00
Krishna Srinivas
8489f22fe2
signature-v2: Use request.RequestURI for signature calculation. ( #3616 )
...
* signature-v2: Use request.RequestURI for signature calculation.
* Use splitStr instead of strings.Split
2017-01-23 17:01:44 -08:00
Anis Elleuch
fc6f804865
server-mux: Keep listening after Accept() err ( #3613 )
...
Accept() can return errors like: `too many open files`, no need to totally quit listening in this case.
2017-01-23 09:55:34 -08:00
Anis Elleuch
d1d89116f1
admin: Add version to service Status API response ( #3605 )
...
Add server's version field to service status API:
"version":{
"version":"DEVELOPMENT.GOGET",
"commitID":"DEVELOPMENT.GOGET"
}
2017-01-23 08:56:06 -08:00
Anis Elleuch
e1f64141a2
presign-v2: Compute signature on encoded URL path ( #3612 )
...
Encode the path of the passed presigned url before calculating the signature. This fixes
presigning objects whose names contain characters that are found encoded in urls.
2017-01-23 08:54:32 -08:00
Krishnan Parthasarathi
586058f079
Implement mgmt REST APIs to heal storage format. ( #3604 )
...
* Implement heal format REST API handler
* Implement admin peer rpc handler to re-initialize storage
* Implement HealFormat API in pkg/madmin
* Update pkg/madmin API.md to incl. HealFormat
* Added unit tests for ReInitDisks rpc handler and HealFormatHandler
2017-01-23 00:32:55 -08:00
Krishna Srinivas
4e926b292f
vendor-update: Minio Browser ( #3609 )
2017-01-22 21:02:09 -08:00
Anis Elleuch
47358e3104
server-mux: Add tcp idle read timeout ( #3607 )
...
Avoid many idle client connections i.e client didn't send any data until a given stipulated amount of time.
Default chosen here is 30 seconds.
2017-01-22 14:48:27 -08:00
Harshavardhana
3640c63289
server/mux: PeekProtocol() should return error and connection be closed. ( #3608 )
...
For TLS peekProtocol do not assume the incoming request to be a TLS
connection perform a handshake() instead and validate.
Also add some security related defaults to `tls.Config`.
2017-01-22 12:14:00 -08:00
Harshavardhana
51fa4f7fe3
Make PutObject a nop for an object which ends with "/" and size is '0' ( #3603 )
...
This helps majority of S3 compatible applications while not returning
an error upon directory create request.
Fixes #2965
2017-01-20 16:33:01 -08:00
Andrei Kopats
c3f7d1026f
fs: start even if there are not enough free space ( #3606 )
2017-01-20 09:30:20 -08:00
Harshavardhana
80f1387877
Initialize peers properly for localhost. ( #3600 )
...
This introduced a regression.
Fixes #3594
2017-01-19 11:32:13 -08:00
Andreas Kohn
0674fa43ff
Handle the region for GetBucketLocation and PutBucket properly ( #3596 )
...
This adjusts the code for these two handlers to match the logic in ListBucketHandler.
Fixes #3595
2017-01-19 11:31:51 -08:00
Harshavardhana
a17f1e875c
server/mux: Close the connection even if buffer.Flush() returns error. ( #3599 )
...
It is possible that buf.Flush() might return an error, leading to a
potential leak in active sockets.
2017-01-19 11:19:57 -08:00
Jeffery Utter
9e1f1b50e0
Don't Check Available Inodes on NFS ( #3598 )
...
In some cases (such as with VirutualBox, this value gets hardcoded
to 1000, which is less than the required minimum of 10000.
Fixes #3592
2017-01-19 10:39:44 -08:00
Anis Elleuch
0715032598
heal: Add ListBucketsHeal object API ( #3563 )
...
ListBucketsHeal will list which buckets that need to be healed:
* ListBucketsHeal() (buckets []BucketInfo, err error)
2017-01-19 09:34:18 -08:00
Harshavardhana
dfc2ef3004
storage/rpc: Remove network error restriction. ( #3591 )
...
This restriction has lots of side affects, since
we do not have a mechanism to clear states like
this it is better not to keep them.
Network errors are common and can occur with
simple cable removal etc. Since we already have
a retry mechanism this error count and stateful
nature can bring problems on a long running
cluster.
2017-01-18 12:55:57 -08:00
Harshavardhana
62f8343879
Add constants for commonly used values. ( #3588 )
...
This is a consolidation effort, avoiding usage
of naked strings in codebase. Whenever possible
use constants which can be repurposed elsewhere.
This also fixes `goconst ./...` reported issues.
2017-01-18 12:24:34 -08:00
Anis Elleuch
f803bb4b3d
admin: Add service Set Credentials API ( #3580 )
2017-01-17 14:25:59 -08:00
Andy Brown
20a65981bd
add delete and options methods to CORS whitelist ( #3589 )
2017-01-17 14:20:05 -08:00
Harshavardhana
09b450d610
Fix fs tests to avoid deleting /usr to certain systems.
2017-01-17 14:05:07 -08:00
Krishnan Parthasarathi
c194b9f5f1
Implement mgmt REST APIs for heal subcommands ( #3533 )
...
The heal APIs supported in this change are,
- listing of objects to be healed.
- healing a bucket.
- healing an object.
2017-01-17 10:02:58 -08:00
Harshavardhana
98a6a2bcab
obj: Return objectInfo for CompleteMultipartUpload(). ( #3587 )
...
This patch avoids doing GetObjectInfo() in similar way
how we did for PutOject().
2017-01-16 19:23:43 -08:00
Harshavardhana
1c699d8d3f
fs: Re-implement object layer to remember the fd ( #3509 )
...
This patch re-writes FS backend to support shared backend sharing locks for safe concurrent access across multiple servers.
2017-01-16 17:05:00 -08:00
Harshavardhana
a054c73e22
Add slack replace gitter ( #3584 )
2017-01-16 15:26:26 -08:00
Harshavardhana
b580ad24e2
server/http: Add missing keep alive for incoming tcp connections. ( #3585 )
...
This was seen reproducing a bug with @gowithplanb. Windows Cloud
Berry clients do not close their respective connections.
2017-01-16 03:38:06 -08:00
Harshavardhana
f8a3b1e164
Fix a bug in previous patch.
2017-01-16 01:48:34 -08:00
Harshavardhana
bf2b8879b7
config: Allow non-standard config dir to be configured with SSL. ( #3583 )
2017-01-15 16:53:01 -08:00
Harshavardhana
caecd75a2a
Deprecate and remove service stop API. ( #3578 )
...
Fixes #3570
2017-01-14 14:48:52 -08:00
Anis Elleuch
2959c104b3
peer rpc: Fix typo in cluster credentials update ( #3579 )
...
Update credentials in cluster wasn't working due to a typo
2017-01-14 14:06:23 -08:00
Harshavardhana
50796e481d
build: Add ARM binary builds for ARMv6 and Aarch64. ( #3577 )
2017-01-13 18:51:17 -08:00
Anis Elleuch
f64f8b03cb
admin: Enhance locks list json response ( #3573 )
2017-01-13 14:25:34 -08:00
Harshavardhana
7b85756c64
notify/webhook: Handle webendpoints without port ( #3568 )
...
Fixes and issue initializing webhook notification
```
FATA[0000] Initializing object layer failed cause=Unable to initialize event \
notification. dial tcp: missing port in address requestb.in source=[server-main.go:448:serverMain()]
```
2017-01-12 20:08:00 -08:00
Alex
d6a327fbc5
Add notifications by webhook.
...
Add a new config entry moving to version 13.
```
"webhook": {
"1": {
"enable": true,
"address": "http://requestb.in/1i9al7m1 "
}
}
```
2017-01-12 10:19:59 -08:00
Anis Elleuch
f24753812a
nats: Add support of NATS.io Streaming server ( #3494 )
2017-01-11 16:41:05 -08:00
Harshavardhana
08b6cfb082
ssl: Set a global boolean to enable SSL across Minio ( #3558 )
...
We have been using `isSSL()` everywhere we can set
a global value once and re-use it again.
2017-01-11 13:59:51 -08:00
Krishna Srinivas
12a7a15daa
browser: Allow anonymous browsing of readable buckets. ( #3515 )
2017-01-11 13:26:42 -08:00
Harshavardhana
7c6d77734a
Fix vendor.json to remove minio/blake2b-simd ref
2017-01-11 00:26:00 -08:00
Harshavardhana
b0cfceb211
event: Enhance event message struct to provide origin server. ( #3557 )
...
`principalId` i.e user identity is kept as AccessKey in
accordance with S3 spec.
Additionally responseElements{} are added starting with
`x-amz-request-id` is a hexadecimal of the event time itself in nanosecs.
`x-minio-origin-server` - points to the server generating the event.
Fixes #3556
2017-01-10 16:43:48 -08:00
Harshavardhana
0563a9235a
handlers: Handle crash if r.URL.Path is empty. ( #3554 )
...
URL paths can be empty and not have preceding separator,
we do not yet know the conditions this can happen inside
Go http server.
This patch is to ensure that we do not crash ourselves
under conditions where r.URL.Path may be empty.
Fixes #3553
2017-01-10 11:01:23 -08:00
Anis Elleuch
eb6d53d2f5
heal: Fix new entries computation in listDirHeal ( #3551 )
...
A crash was happening due to an incorrect interpreation of the return value of sort.SearchString()
2017-01-09 15:48:13 -08:00
Anis Elleuch
95d9e47353
Presign V2: Unescape non-std queries in urls ( #3549 )
...
A client sends escaped characters in values of some query parameters in a presign url.
This commit properly unescapes queires to fix signature calculation.
2017-01-09 14:22:20 -08:00
Harshavardhana
e1142e99f2
rpc/lock: Make sure to capitalize for proper marshalling. ( #3544 )
...
Distributed setup stopped working for certain
types of operations `6d10f4c19af6861e4de1b22ac20a3e5136f69d67`
This is a regression.
Fixes #3543
2017-01-08 20:37:53 -08:00
Harshavardhana
a091fe3ed6
docs: Fix caching docs to comply with current features.
2017-01-08 11:17:02 -08:00
koolhead17
f37f56ac15
docs: Added guide for distributed minio on Windows ( #3507 )
2017-01-07 18:35:12 -08:00
Karthic Rao
2f4a7483ea
Test Function to reset globals. ( #3538 )
...
- Adding reset functions for important global variables.
- Using them in tests.
2017-01-07 11:27:01 -08:00
Harshavardhana
464f9d34d6
Remove all references to GO15VENDOREXPERIMENT
2017-01-06 18:42:32 -08:00
Harshavardhana
926c75d0b5
api: Set appropriate content-type for success/error responses. ( #3537 )
...
Golang HTTP client automatically detects content-type but
for S3 clients this content-type might be incorrect or
might misbehave.
For example:
```
Content-Type: text/xml; charset=utf-8
```
Should be
```
Content-Type: application/xml
```
Allow this to be set properly.
2017-01-06 00:37:00 -08:00
Krishnan Parthasarathi
c8f57133a4
Implement list, clear locks REST API w/ pkg/madmin support ( #3491 )
...
* Filter lock info based on bucket, prefix and time since lock was held
* Implement list and clear locks REST API
* madmin: Add list and clear locks API
* locks: Clear locks matching bucket, prefix, relTime.
* Gather lock information across nodes for both list and clear locks admin REST API.
* docs: Add lock API to management APIs
2017-01-03 23:39:22 -08:00
Harshavardhana
cae62ce543
browser: Handle proper login errors. ( #3518 )
...
Also additionally log the remote address.
Fixes #3514
2017-01-03 01:33:00 -08:00
Harshavardhana
7bbb532b4b
Add a isErr function to check for errs.
...
DisksInfo() should handle collection of some
base errors as offlineDisks.
2017-01-02 10:52:43 -08:00
Bala.FA
6d10f4c19a
Adopt dsync interface changes and major cleanup on RPC server/client.
...
* Rename GenericArgs to AuthRPCArgs
* Rename GenericReply to AuthRPCReply
* Remove authConfig.loginMethod and add authConfig.ServiceName
* Rename loginServer to AuthRPCServer
* Rename RPCLoginArgs to LoginRPCArgs
* Rename RPCLoginReply to LoginRPCReply
* Version and RequestTime are added to LoginRPCArgs and verified by
server side, not client side.
* Fix data race in lockMaintainence loop.
2017-01-02 20:57:42 +05:30
Matthew Hall
cde6496172
checkdeps.sh: support unusual git version strings (for OS X et. al.) ( #3512 )
2016-12-31 19:57:33 -08:00
Harshavardhana
8562b22823
Fix delays and iterim fix for the partial fix in #3502 ( #3511 )
...
This patch uses a technique where in a retryable storage
before object layer initialization has a higher delay
and waits for longer period upto 4 times with time unit
of seconds.
And uses another set of configuration after the disks
have been formatted, i.e use a lower retry backoff rate
and retrying only once per 5 millisecond.
Network IO error count is reduced to a lower value i.e 256
before we reject the disk completely. This is done so that
combination of retry logic and total error count roughly
come to around 2.5secs which is when we basically take the
disk offline completely.
NOTE: This patch doesn't fix the issue of what if the disk
is completely dead and comes back again after the initialization.
Such a mutating state requires a change in our startup sequence
which will be done subsequently. This is an interim fix to alleviate
users from these issues.
2016-12-30 17:08:02 -08:00
Harshavardhana
dd68cdd802
Auto-reconnect for regular authRPC client. ( #3506 )
...
Implement a storage rpc specific rpc client,
which does not reconnect unnecessarily.
Instead reconnect is handled at a different
layer for storage alone.
Rest of the calls using AuthRPC automatically
reconnect, i.e upon an error equal to `rpc.ErrShutdown`
they dial again and call the requested method again.
2016-12-29 19:42:02 -08:00
Harshavardhana
41cf580bb1
Improve reconnection logic, allow jitters. ( #3502 )
...
Attempt a reconnect also if disk not found.
This is needed since any network operation error
is converted to disk not found but we also need
to make sure if disk is really not available.
Additionally we also need to retry more than
once because the server might be in startup
sequence which would render other servers to
wrongly think that the server is offline.
2016-12-29 03:13:51 -08:00
Dee Koder
00baec1746
syntax highlighting : Remove sh from code block which incorrectly highlights shell commands. ( #3504 )
2016-12-28 16:07:05 -08:00
Harshavardhana
f9025652df
Support building specific architectures.
2016-12-27 17:52:24 -08:00
Bala FA
ee0172dfe4
Have simpler JWT authentication. ( #3501 )
2016-12-27 08:28:10 -08:00
Harshavardhana
69559aa101
objAPI: Implement CopyObject API. ( #3487 )
...
This is written so that to simplify our handler code
and provide a way to only update metadata instead of
the data when source and destination in CopyObject
request are same.
Fixes #3316
2016-12-26 16:29:26 -08:00
Krishnan Parthasarathi
36fd317eb2
Clean up lock-instrumentation and improve comments ( #3499 )
...
- Add a lockStat type to group counters
- Remove unnecessary helper functions
- Fix stats computation on force unlock
- Removed unnecessary checks and cleaned up comments
2016-12-26 10:29:55 -08:00
Bala FA
e8ce3b64ed
Generate and use access/secret keys properly ( #3498 )
2016-12-26 10:21:23 -08:00
Andreas Auernhammer
6ee27daac1
fix blake2b tests on non-amd64 machines ( #3496 )
...
Fix the TestHashes Test for non-amd64 machines
2016-12-25 17:03:08 -08:00
koolhead17
cdc6c2d578
docs: Removed unmaintained docs. ( #3493 )
...
* docs: Removed unmaintained docs.
* docs: removed associated screenshots with the older docs.
2016-12-23 22:06:01 -08:00
Harshavardhana
855ef4f1aa
Fix typo in erasure-utils.go
2016-12-22 10:47:10 -08:00
Harshavardhana
5878fcc086
bit-rot: Default to sha256 on ARM64. ( #3488 )
...
This is to utilize an optimized version of
sha256 checksum which @fwessels implemented.
blake2b lacks such optimizations on ARM platform,
this can provide us significant boost in performance.
blake2b on ARM64 as expected would be slower.
```
BenchmarkSize1K-4 30000 44015 ns/op 23.26 MB/s
BenchmarkSize8K-4 5000 335448 ns/op 24.42 MB/s
BenchmarkSize32K-4 1000 1333960 ns/op 24.56 MB/s
BenchmarkSize128K-4 300 5328286 ns/op 24.60 MB/s
```
sha256 on ARM64 is faster by orders of magnitude giving close to
AVX performance of blake2b.
```
BenchmarkHash8Bytes-4 1000000 1446 ns/op 5.53 MB/s
BenchmarkHash1K-4 500000 3229 ns/op 317.12 MB/s
BenchmarkHash8K-4 100000 14430 ns/op 567.69 MB/s
BenchmarkHash1M-4 1000 1640126 ns/op 639.33 MB/s
```
2016-12-22 08:25:03 -08:00
Andreas Auernhammer
1ac36a95aa
replace blake2b implementation ( #3481 )
...
* replace blake2b implementation
replace the blake2b-simd with the golang/x/crypto implementation
```
name old time/op new time/op delta
Size64-8 715ns ±13% 614ns ± 3% ~ (p=0.084 n=6+6)
Size128-8 612ns ± 5% 634ns ± 8% ~ (p=0.084 n=6+6)
Size1K-8 2.18µs ± 5% 2.09µs ± 7% ~ (p=0.084 n=6+6)
Size8K-8 13.1µs ± 2% 13.4µs ± 3% ~ (p=0.084 n=6+6)
Size32K-8 48.5µs ± 1% 49.5µs ± 3% ~ (p=0.775 n=6+6)
Size128K-8 199µs ± 0% 198µs ± 3% ~ (p=0.468 n=6+6)
name old speed new speed delta
Size64-8 92.6MB/s ±11% 104.2MB/s ± 3% ~ (p=0.139 n=6+6)
Size128-8 208MB/s ± 6% 202MB/s ± 8% ~ (p=0.102 n=6+6)
Size1K-8 466MB/s ± 7% 492MB/s ± 7% ~ (p=0.139 n=6+6)
Size8K-8 621MB/s ± 2% 610MB/s ± 3% ~ (p=0.102 n=6+6)
Size32K-8 672MB/s ± 2% 669MB/s ± 1% ~ (p=0.818 n=6+6)
Size128K-8 657MB/s ± 1% 672MB/s ± 0% +2.28% (p=0.002 n=6+6)
name old time/op new time/op delta
Size64-4 334ns ± 1% 243ns ± 0% -27.14% (p=0.029 n=4+4)
Size128-4 296ns ± 1% 242ns ± 0% -18.21% (p=0.029 n=4+4)
Size1K-4 1.44µs ± 0% 1.28µs ± 0% -10.83% (p=0.029 n=4+4)
Size8K-4 10.0µs ± 0% 9.4µs ± 0% -6.23% (p=0.029 n=4+4)
Size32K-4 39.8µs ± 1% 37.3µs ± 0% -6.31% (p=0.029 n=4+4)
Size128K-4 162µs ± 3% 149µs ± 0% -7.72% (p=0.029 n=4+4)
name old speed new speed delta
Size64-4 192MB/s ± 1% 263MB/s ± 0% +37.24% (p=0.029 n=4+4)
Size128-4 431MB/s ± 0% 526MB/s ± 0% +22.04% (p=0.029 n=4+4)
Size1K-4 713MB/s ± 0% 800MB/s ± 0% +12.17% (p=0.029 n=4+4)
Size8K-4 815MB/s ± 0% 869MB/s ± 0% +6.64% (p=0.029 n=4+4)
Size32K-4 823MB/s ± 1% 878MB/s ± 0% +6.72% (p=0.029 n=4+4)
Size128K-4 810MB/s ± 3% 877MB/s ± 0% +8.23% (p=0.029 n=4+4)
```
See: https://go-review.googlesource.com/#/c/34319/
2016-12-21 14:20:01 -08:00
Harshavardhana
15b4c49621
fs/xl: Simplify bucket metadata reading. ( #3486 )
...
ObjectLayer GetObject() now returns the entire object
if starting offset is 0 and length is negative. This
also allows to simplify handler layer code where
we always had to use GetObjectInfo() before proceeding
to read bucket metadata files examples `policy.json`.
This also reduces one additional call overhead.
2016-12-21 11:29:32 -08:00
Harshavardhana
f57f773189
admin: Add missing madmin examples and API docs. ( #3483 )
2016-12-20 18:49:48 -08:00
Harshavardhana
e7b4e4e105
admin: ServiceStatus() shouldn't have to write double http headers. ( #3484 )
...
Fixes #3482
2016-12-20 18:05:25 -08:00
Harshavardhana
1d134c1a94
admin: Fix wrong vendorized location for s3signer and s3utils.
2016-12-20 15:18:37 -08:00
Anis Elleuch
329a910b86
Admin Lib: Implement Service API ( #3426 )
...
Three APIs were added to control a minio server
* NewAdminClient()
* ServiceStop()
* ServiceRestart()
* ServiceStatus()
2016-12-20 14:45:17 -08:00
Harshavardhana
4309727354
docs: Add docs for minio limitations. ( #3477 )
2016-12-20 14:42:13 -08:00
Anis Elleuch
ef3319a49d
Fix: Typo in non canonicalized header extraction ( #3480 )
...
Extracting metadata from headers was doing wrong when Headers are not well canonicalized, fixing typo.
2016-12-20 11:59:08 -08:00
Anis Elleuch
d8e4d3c9c8
POSTForm: Return http 303 if redirect is specified ( #3479 )
...
success_action_redirect in the sent Form means that the server needs to return 303 in addition to a well specific redirection url, this commit adds this feature
2016-12-20 09:32:17 -08:00
Harshavardhana
faa6b1e925
vendorize deps for snappy, blake2b and sha256 ( #3476 )
...
Bring in new optimization and portability changes.
Fixes https://github.com/minio/minio-go/issues/578
2016-12-19 19:32:55 -08:00
Krishnan Parthasarathi
85c6bb9809
server: Sort disk arguments for consistent ordering ( #3469 )
...
This is important in a distributed setup, where the server hosting the
first disk formats a fresh setup. Sorting ensures that all servers
arrive at the same 'first' server.
Note: This change doesn't protect against different disk arguments
with some disks being same across servers.
2016-12-19 17:04:31 -08:00
Anis Elleuch
5404dddcea
PostForm: Save supported headers in obj metadata ( #3474 )
...
Supported Headers like Content-Type, Cache-Control, Content-Encoding, X-Amz-* , etc.. are now saved in object metadata
2016-12-19 16:14:04 -08:00
Anis Elleuch
4692fdbb8f
PostForm: Follow success_action_status requirement ( #3467 )
...
S3 spec requires that Post Object response depends on the passed success_action_status. This commit implements that requirement.
2016-12-18 13:39:56 -08:00
Bala FA
1875a47495
rpcclient: fix leaky connection ( #3471 )
...
Previously, more than one goroutine calls RPCClient.dial(), each
goroutine gets a new rpc.Client but only one such client is stored
into RPCClient object. This leads to leaky connection at the server
side. This is fixed by taking lock at top of dial() and release on
return.
2016-12-17 18:17:40 -08:00
Harshavardhana
9c9f390350
fs: validate filesystem path argument properly. ( #3470 )
...
FS should fail for invalid paths like
- file:///
- ftp://
- http://
2016-12-17 13:43:26 -08:00
Bala FA
1b2b16998f
Remove regexp usage ( #3456 )
...
This patch removes regexp usage in cacheControlHandler.ServeHTTP() and
server-main.checkEndpointsSyntax()
2016-12-17 11:00:16 -08:00
koolhead17
7a17b2a585
Browser: Added character limit for Bucket Name. ( #3454 )
2016-12-16 09:59:37 -08:00
Aditya Manthramurthy
f7766b49aa
Amend ZFS line in README - to make it about Minio on FreeNAS ( #3461 )
2016-12-16 09:59:02 -08:00
Krishnan Parthasarathi
b2f920a868
Add service API handler stubs for status, stop and restart ( #3417 )
2016-12-15 22:26:15 -08:00
Anis Elleuch
8ceb969445
tests: Use testTmpDir var to specify tmp directory ( #3459 )
...
To be able to specify the directory where tests will be done. This way,
it will be easier to run Minio tests on a mounted directory like NFS, ..
2016-12-15 22:25:05 -08:00
Harshavardhana
0db484c8f6
signv2: Do not use path encoding for query values. ( #3458 )
...
Use query unescape before comparing signature.
2016-12-15 14:56:18 -08:00
dcharbonnier
5c481fbf6e
Fix Minio Quickstart Guide link ( #3457 )
2016-12-15 11:59:16 -08:00
Aditya Manthramurthy
8e6e9301ce
Add support for Kafka as a notifications target ( #2869 ) ( #3439 )
2016-12-15 08:23:48 -08:00
Harshavardhana
664ff063a1
server: checkEndpoints syntax properly. ( #3451 )
2016-12-14 20:42:19 -08:00
koolhead17
d9fd6f9a96
docs: Removed $/# from code block to make code copy easier. ( #3448 )
2016-12-14 17:45:47 -08:00
Harshavardhana
b28ff50126
lock/server: Check if the lock server itself is skewed back. ( #3447 )
2016-12-14 08:02:32 -08:00
Karthic Rao
3fe2d77b70
Adding functions for resetting globals. ( #3421 )
2016-12-13 11:51:48 -08:00
Krishnan Parthasarathi
ab49498fc3
server: Exit gracefully if no endpoint is local to it. ( #3442 )
2016-12-13 11:18:31 -08:00
Minio Trusted
46fdd70114
Fix docker file and avoid creating README.md
2016-12-13 09:31:19 -08:00
Krishnan Parthasarathi
29d72b84c0
loginServer should be a member of lockServer ( #3441 )
...
Add a unit test to catch a missing LoginHandler method in lockServer.
2016-12-13 08:01:47 -08:00
Anand Babu (AB) Periasamy
f53fcdf10e
keep make messages consistent ( #3437 )
2016-12-13 00:26:45 -08:00
Krishna Srinivas
8e665105b1
Presign-v4: Allow requests that were signed slightly ahead of the current time. ( #3435 )
2016-12-12 16:32:00 -08:00
Harshavardhana
2062add05f
fs/posix: On windows use helpers and init format.json properly. ( #3434 )
...
Fixes #3433
2016-12-12 15:43:41 -08:00
Krishnan Parthasarathi
6b4e6bcebf
Move LoginHandler into LoginServer which others embed ( #3431 )
...
* Move LoginHandler into LoginServer which others embed
* Add unit tests for loginServer
2016-12-12 08:11:23 -08:00
Harshavardhana
2d6f8153fa
format: Check properly for disks in valid formats. ( #3427 )
...
There was an error in how we validated disk formats,
if one of the disk was formatted and was formatted with
FS would cause confusion and object layer would never
initialize essentially go into an infinite loop.
Validate pre-emptively and also check for FS format
properly.
2016-12-11 15:18:55 -08:00
Anis Elleuch
5c10f4adf0
presign v2: include resp headers in signature calc ( #3428 )
...
Include response headers when presigning an url using signature v2 algorithm
2016-12-11 14:32:25 -08:00
Harshavardhana
4daa0d2cee
lock: Moving locking to handler layer. ( #3381 )
...
This is implemented so that the issues like in the
following flow don't affect the behavior of operation.
```
GetObjectInfo()
.... --> Time window for mutation (no lock held)
.... --> Time window for mutation (no lock held)
GetObject()
```
This happens when two simultaneous uploads are made
to the same object the object has returned wrong
info to the client.
Another classic example is "CopyObject" API itself
which reads from a source object and copies to
destination object.
Fixes #3370
Fixes #2912
2016-12-10 16:15:12 -08:00
Harshavardhana
cd0f350c02
env: Bring back MINIO_BROWSER env. ( #3423 )
...
Set MINIO_BROWSER=off to disable web browser completely.
Fixes #3422
2016-12-10 00:42:22 -08:00
Krishna Srinivas
ac554bf663
FS/Multipart: Fix race between PutObjectPart and Complete/Abort multi… ( #3419 )
...
FS/Multipart: Fix race between PutObjectPart and Complete/Abort multipart. close(timeoutCh) on complete/abort so that a racing PutObjectPart does not leave a dangling go-routine.
Fixes #3351
2016-12-09 16:10:18 -08:00
Harshavardhana
b363709c11
caching: Optimize memory allocations. ( #3405 )
...
This change brings in changes at multiple places
- Reuse buffers at almost all locations ranging
from rpc, fs, xl, checksum etc.
- Change caching behavior to disable itself
under low memory conditions i.e < 8GB of RAM.
- Only objects cached are of size 1/10th the size
of the cache for example if 4GB is the cache size
the maximum object size which will be cached
is going to be 400MB. This change is an
optimization to cache more objects rather
than few larger objects.
- If object cache is enabled default GC
percent has been reduced to 20% in lieu
with newly found behavior of GC. If the cache
utilization reaches 75% of the maximum value
GC percent is reduced to 10% to make GC
more aggressive.
- Do not use *bytes.Buffer* due to its growth
requirements. For every allocation *bytes.Buffer*
allocates an additional buffer for its internal
purposes. This is undesirable for us, so
implemented a new cappedWriter which is capped to a
desired size, beyond this all writes rejected.
Possible fix for #3403 .
2016-12-08 20:35:07 -08:00
Anis Elleuch
410b579e87
startup: Show elapsed time in disks format process ( #3413 )
2016-12-07 10:22:00 -08:00
Karthic Rao
7b7c0bba58
Use a non member mutex lock for serverConfig access. ( #3411 )
...
- This is to ensure that the any new config references made to the
serverConfig is also backed by a mutex lock.
- Otherwise any new config assigment will also replace the member mutex
which is currently used for safe access.
2016-12-07 03:41:54 -08:00
Anis Elleuch
0cef971832
Fix max cache size calculation when system RAM is inferior to the default cache size ( #3410 )
2016-12-06 16:09:26 -08:00
Anis Elleuch
5c9a95df32
srv-mux: do not print peek protocol EOF err msg ( #3402 )
...
EOF err message in Peek Protocol is shown when a client closes the
connection in the middle of peek protocol, this commit hides it since it
doesn't make sense to show it
2016-12-05 14:49:32 -08:00
Anis Elleuch
3b455d6137
tests: Add tests for xl-v1-list-objects-heal ( #3399 )
2016-12-05 09:40:33 -08:00
Anis Elleuch
b2a0e5754b
bucket-handlers: More tests for post form handler ( #3392 )
2016-12-04 12:23:19 -08:00
Anis Elleuch
63d9bb626a
postform: fix check when ${filename} is provided ( #3391 )
...
Checking key condition when ${filename} is provided wasn't working well,
this patch fixes the wrong behavior
2016-12-04 10:30:52 -08:00
Anis Elleuch
372da5eaf5
tests: Enhance checkPostPolicy() coverage ( #3389 )
2016-12-03 12:41:07 -08:00
Harshavardhana
cf17fc7774
fs: PutObject create 0byte objects properly. ( #3387 )
...
Current code always appends to a file only if 1byte or
more was sent on the wire was affecting both PutObject
and PutObjectPart uploads.
This patch fixes such a situation and resolves #3385
2016-12-03 11:53:12 -08:00
Krishnan Parthasarathi
67509453d3
FS: sync abortMultipart cleanup and bg append ( #3388 )
...
backgroundAppend type's abort method should wait for appendParts to finish
writing ongoing appending of parts in the background before cleaning up
the part files.
2016-12-02 23:33:06 -08:00
Harshavardhana
d31f256020
Fail on lint errors during CI build.
2016-12-02 18:08:12 -08:00
Harshavardhana
d67f47927c
api: Fix the formatting issues in last patch.
2016-12-02 17:39:21 -08:00
Anis Elleuch
85bb5870a9
Post Policy Form: exhaustive post policy check ( #3386 )
...
Add support of all conditions check described in
http://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-HTTPPOSTConstructPolicy.html
and simplify a little bit the existing code
2016-12-02 17:00:33 -08:00
Harshavardhana
4a9b205a15
docs: Add missing comments for exported functions.
2016-12-02 14:39:23 -08:00
koolhead17
f3b346cbb3
docs: removed the Edge tag reference. ( #3366 )
2016-12-02 10:47:18 -08:00
Harshavardhana
ff4ce0ee14
fs/xl: Combine input checks into re-usable functions. ( #3383 )
...
Repeated code around both object layers are moved
and combined into simple re-usable functions.
2016-12-01 23:15:17 -08:00
Anis Elleuch
918924796f
fs: Enable shutdown test with faulty disks ( #3380 )
2016-12-01 13:59:06 -08:00
Krishnan Parthasarathi
feb6685359
posix: Use preparePath only for paths used with syscall or os functions ( #3377 )
2016-11-30 20:56:15 -08:00
Bala FA
0d59ea1e94
postpolicy: handle Amazon S3 compatible content-length-range condition ( #3376 )
...
Previously minio server expects content-length-range values as integer
in JSON. However Amazon S3 handles content-length-range values as
integer and strings.
This patch adds support for string values.
2016-11-30 18:30:59 -08:00
Krishna Srinivas
38edd94282
ListBuckets: Allow listBuckets request to be signed with region configured in config.json ( #3374 )
...
Fixes #3373
2016-11-30 13:55:56 -08:00
Krishna Srinivas
8021061bd8
Implement BucketUpdater interface to call BucketMetaState methods. ( #3375 )
2016-11-30 13:37:38 -08:00
Anand Babu (AB) Periasamy
bc9509bc8a
edits on Limits
2016-11-30 01:58:31 -08:00
Krishna Srinivas
e3b4910b66
FS/CompleteMultipart: lock the namespace before renaming the appended tmp file. ( #3371 )
2016-11-29 23:26:36 -08:00
Harshavardhana
d056f19d07
api: Allow reconnection of policy/notification rpc clients. ( #3368 )
...
Since we moved out reconnection logic from net-rpc-client.go
we should do it from the top-layer properly and bring back
the code to reconnect properly in-case the connection is lost.
2016-11-29 22:39:32 -08:00
Harshavardhana
834007728c
fs: Do not print redundant md5Sum response header. ( #3369 )
...
For both GET and HEAD requests.
2016-11-29 16:47:01 -08:00
Krishnan Parthasarathi
a609a4126c
Vendorize recent fixes in dsync ( #3365 )
...
* Update 'i' only if lock grant read from buffered channel
2016-11-28 23:03:46 -08:00
Krishna Srinivas
bcd1a2308b
FS/Shutdown: cleanup and delete .minio.sys during Shutdown() ( #3360 )
2016-11-28 22:54:48 -08:00
koolhead17
694bad434c
docs: Modified the Docker doc to reflect distributed release image & ( #3362 )
...
modification.
2016-11-28 12:45:35 -08:00
Anis Elleuch
01f625824a
args: Honor config-dir & quiet wherever they are ( #3356 )
...
setGlobalsFromContext() is added to set global variables after parsing
command line arguments. Thus, global flags will be honored wherever
they are placed in minio command.
2016-11-28 12:15:36 -08:00
Bala FA
9ccfb70104
Minor cleanup. ( #3361 )
2016-11-28 12:14:24 -08:00
Harshavardhana
201a20ac02
handlers: Handle re-direction properly for S3 requests. ( #3355 )
...
Make sure all S3 signature requests are not re-directed
to `/minio`. This should be only done for JWT and some
Anonymous requests.
This also fixes a bug found from https://github.com/bji/libs3
```
$ s3 -u list
ERROR: XmlParseFailure
```
Now after this fix shows proper output
```
$ s3 -u list
Bucket Created
-------------------------------------------------------- --------------------
andoria 2016-11-27T08:19:06Z
```
2016-11-27 16:30:46 -08:00
Krishna Srinivas
f3322e94c8
FS: Skip creating fs.json for objects created by minio (ex. policy.json) ( #3353 )
2016-11-27 11:33:08 -08:00
Harshavardhana
46a6fde813
xl/fs: Fix initializing meta volume bug.
2016-11-25 18:17:53 -08:00
Anis Elleuch
fd1f09a66c
log: Enable loggers just after configuration load ( #3348 )
...
It would make sense to enable logger just after config initialisation.
That way, errorIf() and fatalIf() will be usable and can catch error
like invalid access and key errors.
2016-11-25 10:39:00 -08:00
Remco Verhoef
752ed7915b
Add windows services docs with nssm instead of sc.
2016-11-24 16:53:10 -08:00
Bala FA
d3064e40b3
isDocker() logs error than fatal error. ( #3347 )
2016-11-24 16:06:49 -08:00
Bala FA
39f9324616
Remove uncessary err != nil check. ( #3346 )
2016-11-24 15:22:33 -08:00
Aditya Manthramurthy
a822b8e782
docs: Fix command sample to be copy-pastable ( #3345 )
2016-11-24 11:38:19 -08:00
Harshavardhana
a6922f94ee
docs: Fix more formatting issues.
2016-11-23 20:25:05 -08:00
Harshavardhana
b4d57bf4c4
docs: Fix some formatting issues for rendering
2016-11-23 20:20:12 -08:00
Bala FA
0f2e493c9a
Use isErrIgnored() function wherever applicable. ( #3343 )
2016-11-23 20:05:04 -08:00
Nitish Tiwari
4ef2d8940c
Rename quick-start-guide.md to README.md ( #3344 )
2016-11-23 20:04:29 -08:00
Nitish Tiwari
96099a1e97
Added distributed Minio architecture images and quick start guide ( #3342 )
2016-11-23 19:30:23 -08:00
Harshavardhana
0e87f29de9
Disable heal message printing, comment it out as todo.
2016-11-23 17:54:29 -08:00
Harshavardhana
dd74e5a809
Revert "init: Honor config-dir flag when it is passed as global or local flag ( #3337 )"
...
This reverts commit e2ef95af7d .
This is reverted since the previous patch caused crashes.
2016-11-23 17:31:36 -08:00
Harshavardhana
12c1abed98
Vendorize with bug fixes from minio browser. ( #3341 )
...
This patch brings in changes from miniobrowser repo.
- Bucket policy UI and functionality fixes by @krishnasrinivas
- Bucket policy implementation by @balamurugana
- UI changes and new functionality changing password etc. @rushenn
- UI and new functionality for sharing URLs, deleting files
@rushenn and @krishnasrinivas.
- Other misc fixes by @vadmeste @brendanashworth
2016-11-23 17:31:11 -08:00
Anis Elleuch
e2ef95af7d
init: Honor config-dir flag when it is passed as global or local flag ( #3337 )
...
setGlobalsFromContext() is added to sets global variable after parsing
command line arguments.
2016-11-23 17:13:40 -08:00
Harshavardhana
d711ff454e
logs: Do not log common successful errors. ( #3340 )
...
Errors like `BucketNotFound`, `BucketExists` shouldn't be logged.
Fixes #3229
2016-11-23 16:36:26 -08:00
Anis Elleuch
c667d20dfc
config-migrate: Fix buggy continuous re-migration of v9 to v10 config ( #3338 )
2016-11-23 15:53:55 -08:00
Harshavardhana
6efee2072d
objectLayer: Check for format.json in a wrapped disk. ( #3311 )
...
This is needed to validate if the `format.json` indeed exists
when a fresh node is brought online.
This wrapped implementation also connects to the remote node
by attempting a re-login. Subsequently after a successful
connect `format.json` is validated as well.
Fixes #3207
2016-11-23 15:48:10 -08:00
Harshavardhana
7a5bbf7a2e
Revert "Vendor update for dsync, fixing major go routine leak issue. ( #3308 )"
...
This reverts commit 273228fafa .
2016-11-23 15:17:41 -08:00
Anis Elleuch
14cb3645a3
config/logger: remove syslogger and upgrade to config v10 which eliminates syslog config ( #3336 )
2016-11-23 15:00:53 -08:00
Krishna Srinivas
f4f512fedd
FS/multipart: Bug fix related to part path. Hold lock on part while appending. ( #3335 )
2016-11-23 12:50:09 -08:00
Nitish Tiwari
1983925dcf
Added distributed Minio architecture files. ( #3330 )
2016-11-23 11:36:52 -08:00
Anis Elleuch
22c98d3fa2
logger: Disassociate shared log config between console, file and syslog ( #3333 )
...
logurs is not helping us to set different log formats (json/text) to
different loggers. Now, we create different logurs instances and call
them in errorIf and fatalIf
2016-11-23 11:35:04 -08:00
Krishna Srinivas
01ae5bb39c
FS/multipart: Fix append-parts to use minioMetaTmpBucket. ( #3304 )
2016-11-23 03:04:04 -08:00
koolhead17
11faf3f16d
docs: Modified FreeBSD guide with official how to install golang link, ( #3293 )
2016-11-23 02:35:19 -08:00
Remco Verhoef
d3df6c711a
How to run Minio as a service on Windows ( #3327 )
2016-11-23 02:27:54 -08:00
Bala FA
ed6e781679
globals: make read only variables as constants. ( #3326 )
2016-11-22 20:13:20 -08:00
Bala FA
baf1c1638d
server: set maximum allowed request body. ( #3324 )
...
This patch sets the value as 5GiB + 64MiB. 5GiB is the maximum
allowed object size and 64MiB is for form data fields and headers.
2016-11-22 19:58:51 -08:00
Bala FA
825000bc34
Use humanize constants for KiB, MiB and GiB units. ( #3322 )
2016-11-22 18:18:22 -08:00
Bala FA
c1ebcbcda2
Remove usused code. ( #3321 )
2016-11-22 17:44:18 -08:00
Anis Elleuch
41a3a9e402
server: forbid zero port in address flag since it confuses clients and ( #3318 )
2016-11-22 17:01:15 -08:00
Bala FA
1d4ac4b084
Rename getUUID() into mustGetUUID() ( #3320 )
...
In case of UUID generation failure mustGetUUID() will panic than
infinitely trying in for loop.
2016-11-22 16:52:37 -08:00
Bala FA
71b357e4f2
Remove uploadIDChange structure. ( #3309 )
...
addUploadID() and removeUploadID() are wrappers to updateUploadJSON()
which is called with respective arguments.
2016-11-22 15:29:39 -08:00
Anis Elleuch
339c9019b9
Protect multipart directory from removing when it is empty ( #3315 )
2016-11-22 13:15:06 -08:00
Harshavardhana
dd93f808c8
web: Add more data for jsonrpc responses. ( #3296 )
...
This change adds more richer error response
for JSON-RPC by interpreting object layer
errors to corresponding meaningful errors
for the web browser.
```go
&json2.Error{
Message: "Bucket Name Invalid, Only lowercase letters, full stops, and numbers are allowed.",
}
```
Additionally this patch also allows PresignedGetObject()
to take expiry parameter to have variable expiry.
2016-11-22 11:12:38 -08:00
Anis Elleuch
4098025c11
Remove XL multipart tmp files when the latter is canceled ( #3214 )
...
XL multipart fails to remove tmp files when an error occurs during upload, this case covers the scenario where an upload is canceled manually by the client in the middle of job.
2016-11-21 16:34:57 -08:00
Bala FA
bef0a50bc1
Cleanup and fixes ( #3273 )
...
* newRequestID() (previously generateUploadID()) returns string than byte array.
* Remove unclear comments and added appropriate comments.
* SHA-256, MD5 Hash functions return Hex/Base64 encoded string than byte array.
* Remove duplicate MD5 hasher functions.
* Rename listObjectsValidateArgs() into validateListObjectsArgs()
* Remove repeated auth check code in all bucket request handlers.
* Remove abbreviated names in bucket-metadata
* Avoid nested if in bucketPolicyMatchStatement()
* Use ioutil.ReadFile() instead of os.Open() and ioutil.ReadAll()
* Set crossDomainXML as constant.
2016-11-21 13:51:05 -08:00
Anis Elleuch
71ada9d6f8
commitXLMetadata() expects src and dst buckets in its arguments ( #3307 )
2016-11-21 12:45:02 -08:00
tibbes
33c022fcec
Fix checkdeps.sh on Mac ( #3306 )
...
Update the check_minimum_version function to use numeric comparison (not
string comparison) on components of version numbers. Fixes the following
output:
```
$ make
Checking deps:
ERROR
OSX version '10.11.6' not supported.
Minimum supported version: 10.8
make: *** [checks] Error 1
```
2016-11-21 12:25:46 -08:00
Karthic Rao
273228fafa
Vendor update for dsync, fixing major go routine leak issue. ( #3308 )
2016-11-21 10:47:48 -08:00
Harshavardhana
aa98702908
api: Handle content-length-range policy properly. ( #3297 )
...
content-length-range policy in postPolicy API was
not working properly handle it. The reflection
strategy used has changed in recent version of Go.
Any free form interface{} of any integer is treated
as `float64` this caused a bug where content-length-range
parsing failed to provide any value.
Fixes #3295
2016-11-21 04:15:26 -08:00
Harshavardhana
5197649081
utils: reduceErrs returns and validates quorum errors. ( #3300 )
...
This is needed as explained by @krisis
Lets say we have following errors.
```
[]error{nil, errFileNotFound, errDiskAccessDenied, errDiskAccesDenied}
```
Since the last two errors are filtered, the maximum is nil,
depending on map order.
Let's say we get nil from reduceErr. Clearly at this point
we don't have quorum nodes agreeing about the data and since
GetObject only requires N/2 (Read quorum) and isDiskQuorum
would have returned true. This is problematic and can lead to
undersiable consequences.
Fixes #3298
2016-11-21 01:47:26 -08:00
Harshavardhana
066f64d34a
bootup: MetaVolume init should use isErrIngored helper. ( #3303 )
2016-11-21 01:46:55 -08:00
Krishna Srinivas
afa4c7c3ef
fs/multipart: Append multipart parts in a proper Go routine in background. ( #3282 )
2016-11-20 23:42:53 -08:00
Krishna Srinivas
38537c7df2
Print line numbers to give more info on the failed tests in ExecObjectLayerAPIAnonTest() ( #3302 )
2016-11-20 23:41:39 -08:00
Krishnan Parthasarathi
eed9ab0464
XL: pickValidXLMeta should return error instead of panic'ing ( #3277 )
2016-11-20 20:56:44 -08:00
Harshavardhana
0b9f0d14a1
auth/rpc: Take remote disk offline after maximum allowed attempts. ( #3288 )
...
Disks when are offline for a long period of time, we should
ignore the disk after trying Login upto 5 times.
This is to reduce the network chattiness, this also reduces
the overall time spent on `net.Dial`.
Fixes #3286
2016-11-20 16:57:12 -08:00
Anis Elleuch
ffbee70e04
Avoid removing 'tmp' directory inside '.minio.sys' ( #3294 )
2016-11-20 14:25:43 -08:00
Harshavardhana
2c3a2241e7
update: Change update notifier for new style banner. ( #3289 )
...
For binary releases and operating systems it would be
All operating systems.
```
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Minio is 25 days 12 hours 30 minutes old ┃
┃ Update: https://dl.minio.io/server/minio/release/linux-amd64/minio ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛
```
On docker.
```
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Minio is 25 days 12 hours 32 minutes old ┃
┃ Update: docker pull minio/minio ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛
```
2016-11-19 23:20:13 -08:00
Harshavardhana
1c47365445
xl/bootup: Upon bootup handle errors loading bucket and event configs. ( #3287 )
...
In a situation when we have lots of buckets the bootup time
might have slowed down a bit but during this situation the
servers quickly going up and down would be an in-transit state.
Certain calls which do not use quorum like `readXLMetaStat`
might return an error saying `errDiskNotFound` this is returned
in place of expected `errFileNotFound` which leads to an issue
where server doesn't start.
To avoid this situation we need to ignore them as safe values
to be ignored, for the most part these are network related errors.
Fixes #3275
2016-11-19 17:37:57 -08:00
Krishnan Parthasarathi
81eb7c0301
Have separate directory names for docker-compose and Dockerfile ( #3292 )
2016-11-19 10:42:22 -08:00
Anis Elleuch
4a926d19a7
Add GnuTLS config documentation for Windows ( #3285 )
2016-11-18 12:03:23 -08:00
Bala FA
05dc52a206
fix: use constants for access/secret key min/max length ( #3271 )
2016-11-16 17:33:55 -08:00
Harshavardhana
c91d3791f9
heal: Add healing support for bucket, bucket metadata files. ( #3252 )
...
This patch implements healing in general but it is only used
as part of quickHeal().
Fixes #3237
2016-11-16 16:42:23 -08:00
Bala FA
df8153859c
docs: Add comments of using canonical import paths. ( #3269 )
2016-11-16 16:23:32 -08:00
Aditya Manthramurthy
2f43709f85
Prevent gorilla mux from normalizing path ( Fixes #3256 ) ( #3268 )
...
Also fix test to not use a bucket name with a leading slash - this
causes the bucket name to become empty and go to an unintended API
call (listbuckets).
2016-11-16 16:23:22 -08:00
Bala FA
61d67a061c
tests: add unit test for DeleteMultipleObjectsHandler. ( #3267 )
...
Fixes #3058
2016-11-16 09:46:09 -08:00
Harshavardhana
1b85302161
Fix spelling and golint errors. ( #3266 )
...
Fixes #3263
2016-11-15 18:14:23 -08:00
Anis Elleuch
6512d9978e
Add support of user self signed certificates
...
Additionally add documentation about how to configure TLS with Minio
2016-11-15 16:15:23 -08:00
Aditya Manthramurthy
e216201901
Remove control command from minio binary ( Fixes #3264 ) ( #3265 )
2016-11-15 13:39:02 -08:00
Krishnan Parthasarathi
7abcededf2
Add tests for storage rpc handlers ( #3262 )
2016-11-15 12:12:06 -08:00
Harshavardhana
398421b9f5
xl/bootup: Server bootup shouldn't return for missing buckets. ( #3255 )
...
Ref #3196
2016-11-14 15:45:00 -08:00
Anis Elleuch
b8f0d9352f
signature-v2: encode path and query strings when calculating signature ( #3253 )
2016-11-14 10:23:21 -08:00
Harshavardhana
f234c35020
lock: slice length of lock clients should be precisely urls. ( #3254 )
...
This patch fixes a possible bug, reproduced rarely only seen
once.
```
panic: runtime error: index out of range
goroutine 136 [running]:
panic(0xac1a40, 0xc4200120b0)
/usr/local/go/src/runtime/panic.go:500 +0x1a1
github.com/minio/minio/vendor/github.com/minio/dsync.lock.func1(0xc4203d2240, 0x4, 0xc420474080, 0x4, 0x4, 0xc4202abb60, 0x0, 0xa86d01, 0xefcfc0, 0xc420417a80)
/go/src/github.com/minio/minio/vendor/github.com/minio/dsync/drwmutex.go:170 +0x69b
created by github.com/minio/minio/vendor/github.com/minio/dsync.lock
/go/src/github.com/minio/minio/vendor/github.com/minio/dsync/drwmutex.go:191 +0xf4
```
2016-11-14 10:18:56 -08:00
koolhead17
3bcf7b7593
Docs: Modified Docker instruction adding distributed image tag. ( #3250 )
2016-11-13 13:40:56 -08:00
Anis Elleuch
0c042a622a
Return objects content types in Web List Objects handler ( #3249 )
2016-11-13 12:26:40 -08:00
Harshavardhana
c57a358c9d
Validate date header only for Signed{,V2} and StreamingSigned. ( #3248 )
...
Fixes #2941
2016-11-13 12:08:24 -08:00
Anis Elleuch
380d6c6435
Use getObjectInfo() in both FS and XL ListObjects() to simplify and to return complete object information ( #3247 )
2016-11-13 11:48:02 -08:00
Harshavardhana
716316f711
Reduce number of envs and options from command line. ( #3230 )
...
Ref #3229
After review with @abperiasamy we decided to remove all the unnecessary options
- MINIO_BROWSER (Implemented as a security feature but now deemed obsolete
since even if blocking access to MINIO_BROWSER, s3 API port is open)
- MINIO_CACHE_EXPIRY (Defaults to 72h)
- MINIO_MAXCONN (No one used this option and we don't test this)
- MINIO_ENABLE_FSMETA (Enable FSMETA all the time)
Remove --ignore-disks option - this option was implemented when XL layer
would initialize the backend disks and heal them automatically to disallow
XL accidentally using the root partition itself this option was introduced.
This behavior has been changed XL no longer automatically initializes
`format.json` a HEAL is controlled activity, so ignore-disks is not
useful anymore. This change also addresses the problems of our documentation
going forward and keeps things simple. This patch brings in reduction of
options and defaulting them to a valid known inputs. This patch also
serves as a guideline of limiting many ways to do the same thing.
2016-11-11 16:40:55 -08:00
Anis Elleuch
98e79b4b50
Use 307 StatusTemporaryRedirect to redirect clients from http to https with forcing them to keep HTTP Verb ( #3246 )
2016-11-11 15:04:51 -08:00
Anis Elleuch
e2216a0936
exponentialBackoffWait returns zero after some retries, limit attempt number inside ( #3245 )
2016-11-11 09:49:30 -08:00
Harshavardhana
e8d9d710d0
rpc: Protect racy access of internal auth states. ( #3238 )
...
Fixes #3232
2016-11-11 00:14:32 -08:00
Harshavardhana
a8ab02a73a
v4/presign: Fix presign requests when there are more signed headers. ( #3222 )
...
This fix removes a wrong logic which fails for requests which
have more signed headers in a presign request.
Fixes #3217
2016-11-10 21:57:15 -08:00
Aditya Manthramurthy
cf022de4d5
Add tests for s3PeerAPIHandlers ( Fixes #3067 ) ( #3242 )
2016-11-10 16:43:04 -08:00
Bala FA
3995e21c5b
fix: Ignore object not found error in RemoveObject() in web-handler. ( #3228 )
...
Fixes #3181
2016-11-10 15:02:03 -08:00
Harshavardhana
2f7fb78692
rpc: Our rpcClient should make an attempt to reconnect. ( #3221 )
...
rpcClient should attempt a reconnect if the call fails
with 'rpc.ErrShutdown' this is needed since at times when
the servers are taken down and brought back up.
The hijacked connection from net.Dial is usually closed.
So upon first attempt rpcClient might falsely indicate that
disk to be down, to avoid this state make another dial attempt
to really fail.
Fixes #3206
Fixes #3205
2016-11-10 07:44:41 -08:00
Bala FA
cf2fb30ac7
event: Add event notification for object deletion in web browser. ( #3226 )
...
Fixes #3201
2016-11-10 07:42:55 -08:00
Harshavardhana
51d1e6f75b
Fix missing CompleteMultipartUpload Etag. ( #3227 )
...
Fixes #3224
2016-11-10 07:41:02 -08:00
Harshavardhana
2f373684f5
Fix the server startup messages and help text. ( #3211 )
2016-11-09 23:37:12 -08:00
Anis Elleuch
5741a53d46
More portable way to list files to be spellchecked and include docs/ directory ( #3220 )
2016-11-09 15:17:47 -08:00
Aditya Manthramurthy
dd0698d14c
Improve namespace lock API: ( #3203 )
...
- abstract out instrumentation information.
- use separate lockInstance type that encapsulates the nsMutex, volume,
path and opsID as the frontend or top-level lock object.
2016-11-09 10:58:41 -08:00
Harshavardhana
3e67bfcc88
heal: Print heal command appropriately without export path. ( #3208 )
...
Fixes #3204
2016-11-09 10:50:14 -08:00
Anis Elleuch
ea579f5b69
Avoid shutdown fs multiple times and create a new fs in each loop ( #3213 )
2016-11-09 10:10:14 -08:00
Anis Elleuch
daf6f3a5c0
Fix path comparing in checkgopath ( #3215 )
2016-11-09 10:09:42 -08:00
Aditya Manthramurthy
d44e9d6da9
Prevent weird messages from rpc lib on start ( Fixes #3209 ): ( #3212 )
...
This is done by not making the methods of the BucketMetaState interface
as methods (via type nesting) on the type implementing
RPCs (s3PeerAPIHandlers).
2016-11-08 23:47:44 -08:00
Bala FA
9c2cfb5cb6
tests: Add missing unit test cases for AbortMultipartUploadHandler(). ( #3200 )
...
Fixes #3070
2016-11-08 16:25:00 -08:00
Bala FA
91a0ade908
tests: add unit test for HeadObjectHandler ( #3197 )
...
Fixes #3068
2016-11-07 16:02:27 -08:00
Bala FA
5eb4002bf7
cleanup build scripts. ( #3192 )
2016-11-07 13:28:41 -08:00
Aditya Manthramurthy
85a5c358d8
Add bucket metadata state client/handler ( Fixes #3022 ) ( #3152 )
...
- Adds an interface to update in-memory bucket metadata state called
BucketMetaState - this interface has functions to:
- update bucket notification configuration,
- bucket listener configuration,
- bucket policy configuration, and
- send bucket event
- This interface is implemented by `localBMS` a type for manipulating
local node in-memory bucket metadata, and by `remoteBMS` a type for
manipulating remote node in-memory bucket metadata.
- The remote node interface, makes an RPC call, but the local node
interface does not - it updates in-memory bucket state directly.
- Rename mkPeersFromEndpoints to makeS3Peers and refactored it.
- Use arrayslice instead of map in s3Peers struct
- `s3Peers.SendUpdate` now receives an arrayslice of peer indexes to
send the request to, with a special nil value slice indicating that
all peers should be sent the update.
- `s3Peers.SendUpdate` now returns an arrayslice of errors, representing
errors from peers when sending an update. The array positions
correspond to peer array s3Peers.peers
Improve globalS3Peers:
- Make isDistXL a global `globalIsDistXL` and remove from s3Peers
- Make globalS3Peers an array of (address, bucket-meta-state) pairs.
- Fix code and tests.
2016-11-07 12:09:24 -08:00
Harshavardhana
33c771bb3e
tests: Add tests for browser peer rpc. Fixes #3062 ( #3189 )
2016-11-07 11:43:35 -08:00
Karthic Rao
286a8924fd
Add leak detection to object-handler tests. ( #3195 )
2016-11-06 21:53:50 -08:00
Karthic Rao
efca29b00e
Fix typos and comments in leak_detect_test.go. ( #3193 )
2016-11-06 21:02:29 -08:00
Harshavardhana
9161016962
tests: Improve coverage on signature v4 tests. ( #3188 )
...
Fixes #3065
2016-11-06 11:47:16 -08:00
Anis Elleuch
5ff30777e1
Rewrite connection muxer peek process to avoid server blocking by silent clients ( #3187 )
2016-11-06 11:41:01 -08:00
Anis Elleuch
754c0770d6
Merge ListenAndServe and ListenAndServeTLS for simplification purpose ( #3186 )
2016-11-05 20:32:13 -07:00
Harshavardhana
1105508453
connection muxer should use bufio.Reader to be simpler. ( #3177 )
2016-11-05 12:57:31 -07:00
Harshavardhana
1ba497950c
Fix net.Listener to fully close the underlying socket. ( #3171 )
...
Leads to races and accepting connections. This patch implements
a way to reject accepting new connections.
2016-11-05 09:43:28 -07:00
Harshavardhana
bf3c93a8cc
Fix docker file to use binary endpoint. ( #3180 )
2016-11-04 18:25:30 -07:00
Aditya Manthramurthy
eb1bc67db1
Refactor: simplify signV4TrimAll() ( #3179 )
2016-11-04 13:52:22 -07:00
Krishna Srinivas
8408dfaa6c
bootup-validation: Allow port configuration only using --address option. ( #3166 )
2016-11-04 12:14:19 -07:00
Harshavardhana
d192044915
router: PathPrefix router was wrong. ( #3172 )
2016-11-04 12:13:22 -07:00
Anis Elleuch
d9bab6b3bd
sigv4: Trim and shrink spaces in headers values ( #3162 )
2016-11-03 16:41:25 -07:00
koolhead17
4503491a0a
docs:Browser/added storageInfo content to the doc. ( #3146 )
2016-11-03 01:58:55 -07:00
Anis Elleuch
e6965ca066
Quit initializing disks process when term signal is invoked ( #3163 )
2016-11-02 15:27:36 -07:00
Harshavardhana
d9674f7524
Improve coverage of web-handlers.go ( #3157 )
...
This patch additionally relaxes the requirement for
accesskeys to be in a regexy set of values.
Fixes #3063
2016-11-02 14:45:11 -07:00
koolhead17
f024deb1f8
docs: Fixed path for docker compose ( #3156 )
2016-11-02 08:54:20 -07:00
Anis Elleuch
79601d27f2
Use endpoint url when printing disks status in distributed mode ( #3151 )
2016-11-02 08:51:06 -07:00
Aditya Manthramurthy
226a69fe15
Add test for updateUploadJSON ( Fixes #3060 ) ( #3155 )
2016-11-01 20:04:32 -07:00
Harshavardhana
9bb799462e
Start all listeners when a given host resolves to multiple IPs. ( #3145 )
...
Default golang net.Listen only listens on the first IP when
host resolves to multiple IPs.
This change addresses a problem for example your ``/etc/hosts``
has entries as following
```
127.0.1.1 minio1
192.168.1.10 minio1
```
Trying to start minio as
```
minio server --address "minio1:9001" ~/Photos
```
Causes the minio server to be bound only to "127.0.1.1" which
is an incorrect behavior since we are generally interested in
`192.168.1.10` as well.
This patch addresses this issue if the hostname is resolvable
and gives back list of addresses associated with that hostname
we just bind on all of them as it is the expected behavior.
2016-11-01 15:38:28 -07:00
Karthic Rao
8bffa78f7f
Fix Instance type during benchmarks. ( #3147 )
...
- The benchmark initialization function was not taking into account the
instance type (FS/XL), was using XL ObjectLayer even for FS
benchmarks.
- This was leading to incorrect benchmark results for FS related
benchmarks.
- The fix takes into account the instance type (FS/XL) and correctly
returns FS backend for FS benchmarks.
2016-11-01 10:21:16 -07:00
Anis Elleuch
4b302173ae
fs: Remove object append file if found since it is not always used ( #3139 )
2016-11-01 00:56:03 -07:00
Karthic Rao
9db2205db3
Fix List-Object Benchmark failure. ( #3142 )
2016-10-31 13:57:30 -07:00
Aditya Manthramurthy
6914fe436c
Remove unused function. ( #3143 )
2016-10-31 12:18:15 -07:00
Anis Elleuch
807cc3c28d
Tracing locking errors for better debugging ( #3140 )
2016-10-31 10:17:14 -07:00
Harshavardhana
f3c6c55719
posix: Fix windows performance issues. ( #3132 )
...
Do not attempt to fetch volume/drive information for
each i/o situation. In our case we do this in all calls
`posix.go` this in-turn created a terrible situation for
windows. This issue does not affect the i/o path on Unix
platforms since statvfs calls are in the range of micro
seconds on these platforms.
This verification is only needed during startup and we
let things fail at a later stage on windows.
2016-10-31 09:34:44 -07:00
Harshavardhana
a773a6dce6
docker: Fix docker build.
2016-10-31 02:51:00 -07:00
Harshavardhana
62dcee3b14
dist: Moved systemd scripts to minio-systemd. ( #3136 )
...
Removing this from the repo.
2016-10-31 02:37:32 -07:00
Harshavardhana
490159ea89
docker: Move the docker document and compose to a separate dir. ( #3135 )
2016-10-31 02:08:16 -07:00
Karthic Rao
3e8cb8c937
build: ineffassign fixes. ( #3134 )
2016-10-30 23:32:46 -07:00
Aditya Manthramurthy
dd6ecf1193
Read/write quorum algo with uploads.json in xl ( Fixes #3123 ) ( #3124 )
...
- Reads and writes of uploads.json in XL now uses quorum for
newMultipart, completeMultipart and abortMultipart operations.
- Each disk's `uploads.json` file is read and updated independently for
adding or removing an upload id from the file. Quorum is used to
decide if the high-level operation actually succeeded.
- Refactor FS code to simplify the flow, and fix a bug while reading
uploads.json.
2016-10-30 09:27:29 -07:00
Anis Elleuch
90e1803798
Fix docker-compose to follow the new server args style ( #3128 )
2016-10-29 16:53:09 -07:00
Anis Elleuch
a47ce7ab22
Add support of fallocate for FS and XL backends ( #3032 )
2016-10-29 12:44:44 -07:00
Krishna Srinivas
0b3282ac9f
Logging: errorIf fatalIf print in the format [file.go:82:funcName()] ( #3127 )
2016-10-29 02:34:16 -07:00
Krishna Srinivas
79b98b5c25
boot: getPath() should take care of simple directory exports. ( #3122 )
...
fixes #3120
2016-10-28 13:08:56 -07:00
Krishnan Parthasarathi
6a57f2c1f0
XL: Add more information to panic msg ( #3119 )
2016-10-28 08:46:03 -07:00
Krishna Srinivas
0fa2477cb0
boot: check parameter syntax before initializing the system. ( #3114 )
...
fixes #3112
2016-10-28 08:45:32 -07:00
Anis Elleuch
7ffb337cd7
Update control cmd USAGE to indicate that it is possible to put access secret keys in URL ( #3115 )
2016-10-27 13:26:11 -07:00
Karthic Rao
a9dd2793f9
Better formatting and adding systemd config to run Minio server on custom port. ( #3113 )
2016-10-27 10:04:06 -07:00
Harshavardhana
9e2d0ac50b
Move to URL based syntax formatting. ( #3092 )
...
For command line arguments we are currently following
- <node-1>:/path ... <node-n>:/path
This patch changes this to
- http://<node-1>/path ... http://<node-n>/path
2016-10-27 03:30:52 -07:00
Aditya Manthramurthy
30dc11a931
No listener.json for single-node mode ( Fixes #3052 ) ( #3108 )
...
In FS or single-node XL mode, there is no need to save listener
configuration to persistent storage. As there is only one server, if it
is restarted, any connected listenBucketAPI clients were disconnected
and will have to reconnect - so there is nothing to actually store.
This incidentally solves #3052 by avoiding the problem.
2016-10-26 20:13:00 -07:00
Anis Elleuch
a15dc5fed5
Print message when creating the config file ( #3089 )
2016-10-26 18:44:22 -07:00
Anis Elleuch
f7c20b97a1
control cmds: Extract access and secret keys from URL if specified ( #3109 )
2016-10-26 18:42:12 -07:00
Harshavardhana
e9c45102b0
posix: Use sync.Pool buffers to copy in large buffers. ( #3106 )
...
These fixes are borrowed from the fixes required for GlusterFS i/o throughput.
2016-10-26 17:14:05 -07:00
Anis Elleuch
8871eb8e1e
Show offline nodes after a fixed number of init retry ( #3107 )
2016-10-26 16:09:06 -07:00
Krishna Srinivas
0f32efb825
PostPolicy - rename of files/functions + add testcases ( #3104 )
2016-10-26 10:15:57 -07:00
Karthic Rao
63f1b4fdf4
mispell fixes. ( #3100 )
2016-10-26 08:46:14 -07:00
Krishnan Parthasarathi
31f2db6880
Remove leftover debug statement from PutObject StreamingSignature unit-test ( #3099 )
2016-10-26 03:17:47 -07:00
Krishnan Parthasarathi
2c9b406f6c
Add TLS based tests to functional test suite ( #3083 )
2016-10-26 02:30:31 -07:00
Krishnan Parthasarathi
12cd2da265
Add PutObjectHandler unit tests covering failure cases ( #3096 )
2016-10-26 02:06:22 -07:00
Harshavardhana
485c0ea8bf
tests: Combine v2 tests with the Suite itself. ( #3088 )
2016-10-25 13:34:14 -07:00
Krishnan Parthasarathi
49ba07d1d6
Use net.ParseCIDR instead of custom-built parsers ( #3055 )
...
Removes avoidable conversion to and from net.IP to string.
2016-10-25 11:14:47 -07:00
Krishna Srinivas
35e541e0b1
content-length-range policy should be honored for the uploaded object sizes. ( #3076 )
2016-10-24 23:47:03 -07:00
Aditya Manthramurthy
3977d6b7bd
Lock bucket while modifying its metadata ( Fixes #2979 ) ( #3019 )
...
- When modifying notification configuration
- When modifying listener configuration
- When modifying policy configuration
With this change we also stop early checking if the bucket exists, since
that uses a Read-lock and causes a deadlock due to the outer Write-lock.
2016-10-24 19:52:24 -07:00
Harshavardhana
0905398459
Fix benchmark tests. ( #3082 )
...
Fixes #3081
2016-10-24 18:45:06 -07:00
Aditya Manthramurthy
f41faf96b7
Fix newMultipartUpload to not leave stale uploads.json ( Fixes #3014 ) ( #3079 )
2016-10-24 17:37:18 -07:00
Bala FA
36639b65a9
rename completeMultipartMD5() into getCompleteMultipartMD5(). ( #3051 )
2016-10-24 13:56:13 -07:00
Harshavardhana
7fc598b73f
Fix user-agent prefix to have docker instead of suffix. ( #3074 )
2016-10-24 13:44:15 -07:00
Krishna Srinivas
21d41ad7fd
init[windows]: Fix to handle the case when export path is a relative path. ( #3054 )
...
ex. to handle "minio server export"
2016-10-24 08:26:28 -07:00
Harshavardhana
fe56220d1a
Do not print nil when hostname is provided as --address ( #3053 )
...
Fixes #3018
2016-10-23 23:55:12 -07:00
Harshavardhana
5782ec3ada
Fix peers and web UIVersion validation. ( #3048 )
2016-10-23 12:32:35 -07:00
Krishnan Parthasarathi
8839c5105a
Pass values to closures esp. when passed to defer statement. ( #3050 )
...
opsID, a variable on the stack, changes over the course of
Completemultipartupload function in xl-v1-multipart.go. This was
being used in a function closure which was passed to defer
statement. The variables used in the closure depend on their values at
the time of evaluation which is indeterminate behaviour. It is
incorrect to depend on values of variables on stack at the end of
function, when deferred functions are executed.
2016-10-23 09:57:52 -07:00
Harshavardhana
e293f079f5
github: Add PR and issue templates. ( #3046 )
2016-10-22 17:35:52 -07:00
Anis Elleuch
6c2d5e3d05
Correct the number of failed disks that we can withstand in startup message ( #3045 )
2016-10-22 10:36:50 -07:00
Krishna Srinivas
5999a23d3e
When object whose size is greater than 5G is uploaded using presigned POST we should return error. ( #3033 )
...
fixes #2961
2016-10-22 09:05:01 -07:00
Krishna Srinivas
e51be73ac7
PresignedPost: Support for Signature V2 presigned POST Policy. ( #3043 )
...
fixes #2993
2016-10-22 08:57:12 -07:00
Harshavardhana
4b5b363c6c
tests: Fix race between SetBucketListenerConfig and eventNotifyForBucketNotifications ( #3041 )
2016-10-22 02:35:33 -07:00
Krishna Srinivas
f2b0c08e34
logging: print file:line:funcName of the caller of errorIf and fatalIf ( #3035 )
2016-10-22 02:24:34 -07:00
Harshavardhana
83b364891d
tests: Fix a potential race in ListenBucketNotificationHandler. ( #3040 )
2016-10-21 22:56:27 -07:00
Karthic Rao
87af2dbc43
dist: Adding systemd script for running Minio distributed. ( #3034 )
2016-10-21 20:56:56 -07:00
Justin Clift
885bac330b
Added a "catch all" documentation link ( #3038 )
...
This should help guide other newbies ;)
2016-10-21 19:31:21 -07:00
Harshavardhana
e3ab478c70
tests: Fix a potential race in RemoveBucketNotification. ( #3037 )
...
Fixes #3036
2016-10-21 17:12:56 -07:00
Harshavardhana
ecaccefd2e
tests: Implement GetBucketNotification handler tests. ( #3029 )
2016-10-21 02:39:37 -07:00
Harshavardhana
ece559afe2
api: Do not use sqs for ListenBucketNotification. ( #3023 )
...
Add more tests. Fixes #3024
2016-10-21 01:25:17 -07:00
Krishna Srinivas
d3aaf50a40
posix: Split on ":" in path d:\export makes minio use wrong disk. ( #3027 )
...
As the host/path split happens at a higher layer now, split at posix is not needed.
fixes part of #2987
2016-10-20 23:39:33 -07:00
Karthic Rao
43ce028840
Point link of the docs to docs.minio.io. ( #3025 )
2016-10-20 22:28:22 -07:00
Krishna Srinivas
32c3a558e9
distributed-XL: Support to run one minio process per export even on the same machine. ( #2999 )
...
fixes #2983
2016-10-20 18:31:02 -07:00
Anis Elleuch
41f9ab1c69
Translate storage access denied error to S3 Access Denied response ( #3015 )
2016-10-20 16:09:55 -07:00
Aditya Manthramurthy
8876e0a80a
Delete bucket listener config file from disk ( #3016 )
2016-10-20 16:09:19 -07:00
Anis Elleuch
c21ac80268
Validate access/secret keys found in the config file and enhance invalid keys messages ( #3017 )
2016-10-20 16:07:24 -07:00
Frank
0e2cd1a64d
Added clear subcommand for control lock ( #3013 )
...
Added clear subcommand for control lock with following options:
```
3. Clear lock named 'bucket/object' (exact match).
$ minio control lock clear http://localhost:9000/bucket/object
4. Clear all locks with names that start with 'bucket/prefix' (wildcard match).
$ minio control lock --recursive clear http://localhost:9000/bucket/prefix
5. Clear all locks older than 10minutes.
$ minio control lock --older-than=10m clear http://localhost:9000/
6. Clear all locks with names that start with 'bucket/a' and that are older than 1hour.
$ minio control lock --recursive --older-than=1h clear http://localhost:9000/bucket/a
```
2016-10-20 13:15:28 -07:00
Aditya Manthramurthy
6274727b71
Pick up server address from --address option ( #3002 ) ( #3008 )
...
This makes sure that when SSL is enabled (for FS/single node mode),
the server address is picked up from the --address option (that needs
to include the hostname for SSL verification, and has to be input
appropriately by user), instead of just using ":<port>".
2016-10-20 11:39:10 -07:00
Harshavardhana
95567c68bf
posix: Do not print errors in expected errors. ( #3012 )
...
Fixes #3011
2016-10-20 09:26:18 -07:00
Anis Elleuch
c189337b6e
rpc: Support SNI in TLS certificates ( #3009 )
2016-10-20 07:43:31 -07:00
Krishnan Parthasarathi
6fc81dc162
Delete temp object/part when PutObject{,Part} fails ( #3004 )
2016-10-19 22:52:03 -07:00
Krishnan Parthasarathi
7d50361ca9
Move housekeeping before object layer initialization ( #3001 )
...
In a distributed setup that the server should not perform any operation
on the storage layer after it is exported via RPC. e.g, cleaning up of
temporary directories under .minio.sys/tmp may interfere with ongoing
PUT objects being served by the distributed setup.
2016-10-19 19:59:48 -07:00
Frank
19c51f3f3c
Added ForceUnlock to namespace-lock ( #2990 )
2016-10-19 09:27:36 -07:00
Aditya Manthramurthy
c3bbadacbf
Improve Peer RPC error handling ( Fixes #2992 ) ( #2995 )
...
* Check for RPC connection shutdown and try again just once.
* Refactor SendRPC to use sync.WaitGroup
2016-10-18 21:26:58 -07:00
Anis Elleuch
2208992e6a
More informative message when erasure fails to read a part of an object ( #2989 )
2016-10-18 13:09:26 -07:00
Anis Elleuch
bbba8e432a
Add ssl support to s3/web peers connections ( #2988 )
2016-10-18 11:46:33 -07:00
Harshavardhana
39331b6b4e
xl: GetCheckSumInfo() shouldn't fail if hash not available. ( #2984 )
...
In a multipart upload scenario disks going down and coming backup
can lead to certain parts missing on the disk/server which was
going down. This is a valid case since these blocks can be
missing and should be healed through heal operation. But we are
not supposed to fail prematurely since we have enough data on
the other disks as well within read-quorum.
This fix relaxes previous assumption, fixes a major corruption
issue reproduced by @vadmeste.
Fixes #2976
2016-10-18 11:13:25 -07:00
Mike Ralphson
6e748cb1cf
Report when invalid bucket names are skipped in FS backend. ( #2947 )
2016-10-18 01:42:46 -07:00
Anis Elleuch
2005d656e6
Properly load creds from env and save them when server cmd is executed ( #2970 )
2016-10-17 23:14:41 -07:00
Aditya Manthramurthy
0f26ec8095
Propagate creds change to cluster ( Fixes #2855 ) ( #2929 )
2016-10-17 20:18:08 -07:00
Harshavardhana
8d2347bc7b
storage: DeleteFile should return errFileNotFound for ENOENT. ( #2978 )
2016-10-17 16:38:46 -07:00
Aditya Manthramurthy
0ff359ca0e
Fix early init. problem for notifications ( Fixes #2972 ) ( #2977 )
2016-10-17 16:38:29 -07:00
Harshavardhana
f8e13fb00e
server: Startup sequence should be more idempotent. ( #2974 )
...
Fixes #2971 - honors ignore-disks option properly.
Fixes #2969 - change the net.Dial to have a timeout of 3secs.
2016-10-17 14:31:33 -07:00
Harshavardhana
686a610fc3
api: Nanosecond precision for API responses is valid with S3. ( #2957 )
...
Wqe need to be compatible as well fixes #2955
2016-10-17 08:44:55 -07:00
Mike Ralphson
7fc1685b7a
Allow Travis builds from GitHub forks ( #2958 )
...
Set the go_import_path explicitly. See
https://docs.travis-ci.com/user/languages/go#Go-Import-Path
2016-10-17 08:42:20 -07:00
Krishnan Parthasarathi
b89609dc2e
XL: Filter out md5Sum from user defined headers ( #2962 )
2016-10-17 08:41:33 -07:00
Anis Elleuch
fa50312220
Avoid returning disk corrupted by servers in the middle of init all disks formats ( #2964 )
2016-10-17 08:39:55 -07:00
Harshavardhana
fee3f99a6e
xl: heal bucket should validate if bucket exists first. ( #2953 )
...
Fixes #2944
2016-10-17 02:10:23 -07:00
Frank
ea406754a6
New dsync and added ForceUnlock to lock rpc server ( #2956 )
...
* Update dsync and added ForceUnlock function
* Added test cases for ForceUnlock
2016-10-17 01:53:29 -07:00
Aditya Manthramurthy
d02cb963d5
Fix listen-bucket ( Fixes #2942 ) ( #2949 )
...
Don't close socket while re-initializing notify-listeners, as the rpc
client object is shared between notify-listeners and peer clients.
Also, improves SendRPC() readability by using GetPeerClient().
2016-10-16 20:52:10 -07:00
Anis Elleuch
334cdb5d64
XL total/free space calculation is done inside xl module ( #2945 )
2016-10-16 14:24:15 -07:00
Harshavardhana
a681af6953
Update minio browser with new changes. ( #2940 )
...
- Bucket policy set/unset support.
- Shareable URL support.
- Delete object support.
2016-10-15 08:51:53 -07:00
Anis Elleuch
5c3639c1b7
Redirect /minio to /minio/ when requests come from browsers ( #2937 )
2016-10-15 06:21:51 -07:00
Krishna Srinivas
903574db90
copy-object: Do not use ETag of source as MD5 as it will not be MD5 if source was uploaded as multipart. ( #2938 )
...
fixes #2934
2016-10-15 06:20:55 -07:00
Anis Elleuch
f463d3ce42
Fix a crash when service shutdown is signaled and object API is not ready yet ( #2939 )
2016-10-15 06:20:16 -07:00
Aditya Manthramurthy
17eeec6895
Bucket policy propagation ( Fixes #2930 ) ( #2932 )
...
Fixes a serialisation bug - encoding/gob does not directly support
serializing `map[string]interface{}`, so we serialise to JSON and send a
byte array in the RPC call, and deserialize and update on the receiver.
2016-10-14 22:49:51 -07:00
Karthic Rao
070d3610ff
tests: V2 Signature tests for object-handlers. ( #2931 )
2016-10-14 20:52:46 -07:00
Harshavardhana
f22862aa28
heal: Refactor heal command. ( #2901 )
...
- return errors for heal operation through rpc replies.
- implement rotating wheel for healing status.
Fixes #2491
2016-10-14 19:57:40 -07:00
koolhead17
18be3bc95a
docs: added space in README.md so that its compatible with doctor. ( #2927 )
2016-10-14 13:20:33 -07:00
Harshavardhana
2f520ed92f
Remove errors package, add comments and simplify. ( #2925 )
2016-10-14 12:31:00 -07:00
Mateusz Gajewski
c03ce0f74a
Display SSL expiry warnings ( #2925 )
2016-10-14 12:30:36 -07:00
Krishna Srinivas
0320a77dc0
HealBucket: create the bucket if it is missing in one of the disks. ( #2924 )
2016-10-14 11:12:17 -07:00
koolhead17
3349153058
docs: added space in source download steps so it appears as desired in ( #2923 )
...
doctor.
2016-10-14 08:49:12 -07:00
Harshavardhana
5e86352464
doc: Fix docker.md instructions and words.
2016-10-13 21:34:03 -07:00
Harshavardhana
18d125ef1c
doc: Redo install instructions ( #2922 )
2016-10-13 19:44:27 -07:00
Aditya Manthramurthy
31be826f51
Fix missing error check for jsonrpc.Server.RegisterService() ( #2921 )
2016-10-13 17:34:10 -07:00
Harshavardhana
eb372d53df
Fix docker release titles
2016-10-13 16:24:18 -07:00
Harshavardhana
1788c58d5c
Add docker edge instructions
2016-10-13 16:19:14 -07:00
Karthic Rao
17e49a9ed2
signature-v2 fix. ( #2918 )
...
- Return errors similar to V4 Sign processsing.
- Return ErrMissing fields when Auth Header fields are missing.
- Return InvalidAccessID when accessID doesn't match.
* tests: Adding V2 signature tests for bucket handler API's.
2016-10-13 09:25:56 -07:00
Aditya Manthramurthy
0aabc1d8d9
Use Peer RPC to propagate bucket policy changes ( #2891 )
2016-10-13 09:19:04 -07:00
Harshavardhana
55f6828750
Do not print update message unless there is an update. ( #2919 )
2016-10-13 09:17:08 -07:00
Aditya Manthramurthy
6303f26330
Protect map from concurrent access ( Fixes #2915 ) ( #2916 )
...
Protects the Peers RPC clients map from concurrent access to fix a data race condition.
2016-10-13 01:33:50 -07:00
Krishnan Parthasarathi
b59bac670a
Handle err returned by rpc.Server.RegisterName ( #2910 )
2016-10-12 23:13:24 -07:00
Anis Elleuch
84acc820c7
Fix free drive space calculation in XL mode ( #2917 )
2016-10-12 20:22:15 -07:00
Harshavardhana
92858c7db2
Fix docker documentation.
2016-10-12 18:31:23 -07:00
Harshavardhana
fdaa129a5b
Fix dockerfile container image. ( #2892 )
2016-10-12 18:09:08 -07:00
Anis Elleuch
df59967f59
Avoid checking date header of web requests by properly applying generic handlers ( #2914 )
2016-10-12 12:58:36 -07:00
Mateusz Gajewski
73982c8cb6
Listen bucket notification for multiple prefixes/suffixes ( #2911 )
...
* Listen bucket notification for multiple prefixes/suffixes
* After review fixes
2016-10-12 11:02:15 -07:00
Aditya Manthramurthy
6199aa0707
Peer RPCs for bucket notifications ( #2877 )
...
* Implements a Peer RPC router that sends info to all Minio servers in the cluster.
* Bucket notifications are propagated to all nodes via this RPC router.
* Bucket listener configuration is persisted to separate object layer
file (`listener.json`) and peer RPCs are used to communicate changes
throughout the cluster.
* When events are generated, RPC calls to send them to other servers
where bucket listeners may be connected is implemented.
* Some bucket notification tests are now disabled as they cannot work in
the new design.
* Minor fix in `funcFromPC` to use `path.Join`
2016-10-12 01:03:50 -07:00
Krishnan Parthasarathi
a5921b5743
Use same timestamp for all chunks in chunked signature ( #2908 )
2016-10-11 23:46:51 -07:00
Karthic Rao
f0538dbb5c
fix broken link for Go Installation in CONTRIBUTING.md ( #2907 )
2016-10-11 22:19:35 -07:00
Karthic Rao
ff91ecb177
tests: Adding unknown signature type test for API handlers. ( #2905 )
2016-10-11 20:38:10 -07:00
Frank
a6357502c1
Correct typo in error string ( #2902 )
2016-10-11 08:56:02 -07:00
Harshavardhana
fa8ea41cd9
lock/instrumentation: Cleanup and print in user friendly form. ( #2807 )
2016-10-11 00:50:27 -07:00
Karthic Rao
3ac6790ca2
tests: Add Object Layer nil test for bucket-handler API's ( #2899 )
2016-10-11 00:00:02 -07:00
Krishna Srinivas
268b96058f
ns-lock: lock namespace during FS object operations. ( #2896 )
2016-10-10 10:20:04 -07:00
Frank
0d031c432b
Fix typo in serverVersion ( #2894 )
2016-10-10 10:11:56 -07:00
Frank
6e8f3224c5
Test coverage for lock rpc server ( #2893 )
...
* Add test coverage for removeEntry and removeEntryIfExists
* Initial test framework for Lock/Unlock functionality
* Add clarification comments
* Add test coverage code for RLock() and RUnlock()
* Add test coverage for Expired() function
* Have all lock-rpc-server test functions start with the same prefix
* Properly initialize JWT security token
2016-10-10 10:11:29 -07:00
Karthic Rao
9c53e9f4c3
tests: Enhance coverage for bucket policy handlers. ( #2895 )
2016-10-10 09:29:56 -07:00
Krishnan Parthasarathi
2d5e988a6d
Refactor streaming signatureV4 w/ state machine ( #2862 )
...
* Refactor streaming signatureV4 w/ state machine
- Used state machine to make transitions between reading chunk header,
chunk data and trailer explicit.
* debug: add print/panic statements to gather more info on CI failure
* Persist lastChunk status between Read() on ChunkReader
... remove panic() which was added as interim aid for debugging.
* Add unit-tests to cover v4 streaming signature
2016-10-10 01:42:32 -07:00
Harshavardhana
3cfb23750a
control: Implement service command 'stop,restart,status'. ( #2883 )
...
- stop - stops all the servers.
- restart - restart all the servers.
- status - prints status of storage info about the cluster.
2016-10-09 23:03:10 -07:00
Anis Elleuch
57f75b1d9b
Ignore copy conditions when ETag is not available ( #2888 )
2016-10-09 16:21:42 -07:00
Anand Babu (AB) Periasamy
4560cbc20c
List docker as first example
2016-10-09 13:06:25 -07:00
Karthic Rao
e213172431
tests: Missing anonymous tests for bucket-handlers. ( #2885 )
2016-10-09 09:21:37 -07:00
Krishna Srinivas
f5f007e183
Test: Add test case for xl.HealObject() ( #2884 )
...
fixes #2842
2016-10-08 17:08:17 -07:00
Karthic Rao
09463265ce
tests: Adding anonymous requests tests for bucket policy handlers. ( #2882 )
2016-10-08 01:04:26 -07:00
Karthic Rao
8f4cf2a7d0
tests: anonymous/unsigned tests for object handler API's . ( #2881 )
2016-10-07 23:28:50 -07:00
Karthic Rao
30183c4a9a
tests: cleanup and unsigned request test. ( #2880 )
...
- Cleaning up of ListMultipartUpload API test for improving readability,
code maintainance and extensibility.
- Moving ListMultipartUploads to Go 1.7 sub tests.
- Using the new Anonymous request helper function for
ListMultipartUploads.
2016-10-07 20:16:57 -07:00
Karthic Rao
d1df5e0ae1
tests: Add helper function for API handler anonymous request tests. ( #2876 )
...
- Add helper function for API handler anonymous request tests.
- Add PutObject Part Anonymous request case using the new helper
function to validate its functionality.
2016-10-07 11:16:11 -07:00
Harshavardhana
f1bc9343a1
prep: Initialization should wait instead of exit the servers. ( #2872 )
...
- Servers do not exit for invalid credentials instead they print and wait.
- Servers do not exit for version mismatch instead they print and wait.
- Servers do not exit for time differences between nodes they print and wait.
2016-10-07 11:15:55 -07:00
Frank
e53a9f6cab
Update vendorized version of dsync that relaxes read quorum to N/2 ( #2874 )
2016-10-07 08:02:59 -07:00
Karthic Rao
97f4989945
tests: cleaning up. ( #2875 )
...
- Clean up PutObjectPart and ListObjectPart API handler tests.
- Add more comments, make the tests more readable.
- Add verification for HTTP response status code.
- Initialize the test using object Layer.
- Move to Go 1.7 sub tests.
2016-10-07 08:02:37 -07:00
Harshavardhana
ed676667d0
vendor: update reedsolomon package with new changes. ( #2870 )
...
- Cached inverse matrices for better reconstruct performance.
- New error reconstruction required is returned, helpful in
initiating healing.
2016-10-06 21:57:42 -07:00
Harshavardhana
1e5e213d24
auth: Make sure we initialize or change config before RPC requests. ( #2867 )
2016-10-06 13:35:56 -07:00
Karthic Rao
a8105ec068
- Test utility function for easy asserting of cases wherein objectLayer ( #2865 )
...
is `nil` in API handlers.
- Remove the existing tests for the `nil` check and use the new method
to test for object layer being `nil`.
2016-10-06 13:34:33 -07:00
Krishna Srinivas
c6d2967b84
Doc: Document list of supported environmental varaibles. ( #2864 )
...
fixes #2773
2016-10-06 09:30:08 -07:00
Krishna Srinivas
bb9be02228
minio-browser: do not redirect to /minio if MINIO_BROWSER=off ( #2863 )
...
fixes #2837
2016-10-06 08:30:32 -07:00
Harshavardhana
64f37bbf5b
rpc: Add RPC client tests. ( #2858 )
2016-10-06 02:30:54 -07:00
Karthic Rao
0fc96fa25c
Refactor bucket policy handler test to use API test initializer. ( #2859 )
2016-10-06 02:02:42 -07:00
Karthic Rao
2d8c6f8288
unit test for bucketPolicyConditionMatch function. ( #2857 )
2016-10-06 00:23:46 -07:00
Harshavardhana
b94211bd66
api: ListObjectsV1 compliance with AWS S3. ( #2856 )
...
XSD - xml schema definition for SOAP operations
on S3 provides positional restrictions on XML
output.
Fix the response by re-arranging the positions in
accordance with S3 behavior.
Fixes #2849
2016-10-05 20:12:47 -07:00
Harshavardhana
6494b77d41
server: Add more elaborate startup messages. ( #2731 )
...
These messages based on our prep stage during XL
and prints more informative message regarding
drive information.
This change also does a much needed refactoring.
2016-10-05 12:48:07 -07:00
Bala FA
63a7ca1af0
web: fix jwt token expiry set to one day by default. ( #2819 )
...
Fixes #2818
2016-10-05 10:18:55 -07:00
Krishna Srinivas
95f544657a
Signature-V2: use raw resource/query from the request for signature calculation. ( #2833 )
2016-10-05 09:18:53 -07:00
Harshavardhana
740a919e25
config: Use migrateV8 to v9 function properly. ( #2852 )
2016-10-05 02:28:04 -07:00
Krishnan Parthasarathi
402c92beda
Add listObjectParts test w/ unknown request signature type ( #2847 )
2016-10-04 07:57:35 -07:00
Karthic Rao
6a9013b97c
misspell fixes. ( #2835 )
2016-10-04 00:09:21 -07:00
Krishnan Parthasarathi
73b50aea2d
Add preSign auth type tests for ListObjectPartsHandler and PutObjectPartHandler ( #2834 )
2016-10-03 22:05:33 -07:00
Aditya Manthramurthy
315e66858c
Add PostgreSQL notifier ( #2739 ) ( #2824 )
...
* The user is required to specify a table name and database connection
information in the configuration file.
* INSERTs and DELETEs are done via prepared statements for speed.
* Assumes a table structure, and requires PostgreSQL 9.5 or above due to
the use of UPSERT.
* Creates the table if it does not exist with the given table name using
a query like:
CREATE TABLE myminio (
key varchar PRIMARY KEY,
value JSONB
);
* Vendors some required libraries.
2016-10-03 17:29:55 -07:00
Krishnan Parthasarathi
4f902d42b2
Add unit-tests for ListObjectParts API handler ( #2826 )
...
* Add missing uploadID test
... make variables in test code unexported.
* Add ServerNotInitialized test for ListObjectPartsHandler
* Add tests for ListObjectParts with signatureV2 and Anonymous requests
* Add failure test cases for ListObjectParts
2016-10-03 08:54:57 -07:00
Krishna Srinivas
61a18ed48f
sha256: Verify sha256 along with md5sum, signature is verified on the request early. ( #2813 )
2016-10-02 15:51:49 -07:00
Anis Elleuch
b5a6dd1395
Avoid path-cleaning policy resources for a better compliance with S3 ( #2823 )
2016-10-01 21:30:25 -07:00
Krishnan Parthasarathi
83e6e1060e
Layer LimitReader responsibly allowing sign verification to work ( #2821 )
2016-10-01 09:37:40 -07:00
Krishnan Parthasarathi
ddeb8242d8
PutObjectPartHandler unit-tests ( #2810 )
2016-10-01 08:23:26 -07:00
Harshavardhana
a08052f640
Add docker pulls badge.
2016-09-30 19:19:19 -07:00
Harshavardhana
5ecba587f7
api: Relax object name validation. ( #2814 )
...
Fixes #2812
2016-09-30 16:56:36 -07:00
Harshavardhana
db3da97a50
signature/v2: Fix presigned requests.
2016-09-30 15:22:00 -07:00
Harshavardhana
5885ffc8ae
signature: Add legacy signature v2 support transparently. ( #2811 )
...
Add new tests as well.
2016-09-30 14:32:13 -07:00
Anis Elleuch
9fb1c89f81
Add TLS encryption capability to RPC clients ( #2789 )
2016-09-29 23:42:37 -07:00
Anis Elleuch
1e6afac3bd
Add NATS notifier ( #2795 )
2016-09-29 23:42:10 -07:00
Harshavardhana
64083b9227
signature: Region changes should be handled just like AWS. ( #2805 )
...
- PutBucket happens with 'us-east-1'.
- ListBuckets happens with any region.
- GetBucketLocation happens with 'us-east-1' and location is returned.
2016-09-29 15:51:00 -07:00
Krishnan Parthasarathi
5fdd768903
Make addition of TopicConfig to globalEventNotifier go-routine safe ( #2806 )
2016-09-28 22:46:19 -07:00
Harshavardhana
f72163f856
build: Deprecate requirement of GOROOT ( #2803 )
2016-09-28 18:49:16 -07:00
Krishnan Parthasarathi
428629f577
Add unit tests for server-main.go ( #2802 )
2016-09-28 11:19:07 -07:00
Harshavardhana
1edd74dda2
update: Deprecate the usage of update=yes query param. ( #2801 )
...
Fixes #2799
2016-09-28 02:41:21 -07:00
Krishnan Parthasarathi
740ecf530c
Add PutBucketNotification, ListenBucketNotification handler unit-tests. ( #2787 )
2016-09-28 01:08:03 -07:00
Aditya Manthramurthy
10d2ef5449
Remove comments relating to deprecated MINIO_DEBUG envvar ( #2797 )
2016-09-27 18:28:46 -07:00
Aditya Manthramurthy
8ea571c7f7
Remove MINIO_DEBUG environment variable ( #2794 )
...
Removes the unimplemented settings of MINIO_DEBUG=mem and makes
MINIO_DEBUG=lock the default behaviour.
2016-09-27 14:35:43 -07:00
Harshavardhana
ca3022d545
api: Change ListenBucketNotification with new API format. ( #2791 )
...
Take prefix, suffix and events as query params.
2016-09-27 13:17:43 -07:00
Anis Elleuch
9417614a8e
Recalculate free minimum disk space ( #2788 )
...
* Fix calculating free space disk by using blocks available for unprivileged user
* Use fixed minimal free disk space instead of percentage
2016-09-27 12:46:38 -07:00
Aditya Manthramurthy
70d52bbc4c
Add unit test for rate-limit-handler ( #2661 ) ( #2784 )
2016-09-26 21:31:12 -07:00
Harshavardhana
6aa2fc95c0
Revert "bucket: refactor policies and fix bugs related to enforcing policies. ( #2766 )"
...
This reverts commit ca5ca8332b .
2016-09-26 19:32:33 -07:00
Harshavardhana
cfbab22237
web: Remove bucket policy when we have no more statements. ( #2779 )
2016-09-26 03:11:22 -07:00
Harshavardhana
be0e06c0aa
web: Simplify and rename GetAllBucketPolicy --> ListAllBucketPolicies. ( #2778 )
2016-09-25 21:53:19 -07:00
Harshavardhana
1c941fd787
rpc: Should validate server versions. ( #2775 )
...
Fixes #2764
2016-09-24 03:34:45 -07:00
Krishnan Parthasarathi
669783f875
Purge stale object cache entry ( #2770 )
2016-09-23 19:55:28 -07:00
Krishnan Parthasarathi
27e474b3d2
Improve code coverage in bucket-notification-handlers.go ( #2759 )
...
* Fix incorrect test cases for bucket-notification handler
* Add tests covering failure cases for bucket notification
2016-09-23 13:32:51 -07:00
Krishna Srinivas
1e53316241
Add tests for presigned-get ( #2767 )
...
* web-handlers: support for presigned-get json-rpc call for MinioBrowser's "share" feature.
* Add tests for presigned-get
2016-09-23 01:25:49 -07:00
Harshavardhana
ca5ca8332b
bucket: refactor policies and fix bugs related to enforcing policies. ( #2766 )
...
This patch also addresses the problem of double caching at
object layer once at XL and another at handler layer.
2016-09-22 23:47:48 -07:00
Bala FA
aa579bbc20
web: add method to get all policies for given bucket name. ( #2756 )
...
Refer #1858
2016-09-22 23:06:45 -07:00
Harshavardhana
e375d822da
bucket: SetBucketPolicy should save a valid Version and validate. ( #2762 )
2016-09-22 22:27:21 -07:00
astaxie
4bc2eb9a4d
chinese translate ( #2637 )
2016-09-22 20:58:10 -07:00
Anis Elleuch
fc783f8407
More tests for web handlers ( #2755 )
...
* Return negative values of Total and Free in StorageInfo() when we fail to get disk info
* Return consistent messages in web handlers when the server is not initialized
2016-09-22 16:35:12 -07:00
Anis Elleuch
79e0c52fc2
Increment golang version for docker images ( #2761 )
2016-09-22 16:32:54 -07:00
Anis Elleuch
ef22330563
Require go 1.7.1 to build Minio server ( #2727 )
2016-09-22 10:33:52 -07:00
Harshavardhana
e1adbf83d8
docker: Compose config should point to edge tag.
2016-09-22 09:55:45 -07:00
Karthic Rao
1148f95292
ineffassign fixes ( #2758 )
2016-09-21 23:03:54 -07:00
Karthic Rao
f7430ec09c
use runtime/debug.Stack() in leak detect test ( #2757 )
2016-09-21 22:04:35 -07:00
Karthic Rao
b8903d842c
api/complete-multipart: fixes and tests. ( #2719 )
...
* api/complete-multipart: tests and simplification.
- Removing the logic of sending white space characters.
- Fix for incorrect HTTP response status for certain cases.
- Tests for New Multipart Upload and Complete Multipart Upload.
* tests: test for Delelete Object API handler
2016-09-21 20:08:08 -07:00
Aditya Manthramurthy
32f097b4d6
Controller rpc tests ( #2709 )
...
* Test code for controller-handler operations:
* Heal operations
* List operation
* Switch to "testing" lib, moving away from gocheck
* Minor refactors
* Remove extra call to initGracefulShutdown
* Remove dead code in mainControl:
Dead code found by the TestControlMain() test function that always
passes.
* Add tests for control-*-main.go
2016-09-21 19:58:50 -07:00
Krishnan Parthasarathi
559ad38b8c
Add bucket-notification-handler tests ( #2750 )
2016-09-21 17:41:34 -07:00
Anis Elleuch
90417d2dd6
Check for bucket existence in Set/Get/Remove bucket policy workflow + tests ( #2745 )
2016-09-21 16:38:50 -07:00
Anis Elleuch
e66fb4bd7b
configMigrate() returns errors + tests ( #2735 )
2016-09-21 09:44:57 -07:00
Harshavardhana
018c90dae7
events: ElasticSearch doesnt support objects with '/' in them. ( #2747 )
...
Fix this by using a unique sha256 generated for each unique key.
2016-09-20 16:36:18 -07:00
Anis Elleuch
a5066e8f76
Better code coverage of handler-utils.go ( #2746 )
2016-09-20 10:40:46 -07:00
Harshavardhana
0a3448c8b6
events: Change event notifiers to delete and update keys. ( #2742 )
...
ElasticSearch and Redis are both treated like a database.
Each indexs are based on the object names uniquely indentifying
the event. Upon each delete event of the named object deletes
the index on elasticsearch and redis respectively.
2016-09-20 02:11:17 -07:00
Harshavardhana
c4964232eb
config: Fail to start for config mistakes. ( #2740 )
2016-09-19 15:23:49 -07:00
Harshavardhana
ef3c807b4a
policies: Parser should handle Principals with various forms. ( #2733 )
...
Handles cases for these three combinations
- "Principal": "*",
- "Principal": { "AWS" : "*" }
- "Principal": { "AWS" : [ "*" ]}
Fixes #2732
2016-09-19 13:52:28 -07:00
Harshavardhana
113b93346b
lock: Make some cleanup and moving the code around. ( #2718 )
...
This patch just avoids lot of ifs and inverts some logic.
2016-09-19 13:14:55 -07:00
Krishna Srinivas
a955676986
Signature-V4: Dump the request with error message on signature mismatch. ( #2734 )
...
fixes #2691
2016-09-19 10:17:46 -07:00
Harshavardhana
725df557b5
tests: Add tests for bucket-notification-utils ( #2726 )
...
Part - 2 final fix #2711
2016-09-17 03:19:39 -07:00
koolhead17
250ac644d6
docs: Modified README.md with alias addition and bucket creation steps. ( #2725 )
2016-09-17 02:57:36 -07:00
Harshavardhana
490056eee3
tests: Add tests for bucket-notification-utils ( #2723 )
...
Part fix - 1 for #2711
2016-09-16 17:26:27 -07:00
Harshavardhana
797d749322
tests: Add tests for filterRuleMatch ( #2722 )
...
Part-1 fix for #2418
2016-09-16 16:44:44 -07:00
Harshavardhana
79888bfff7
tests: Add auth-handler. ( #2721 )
...
Fixes #2658
2016-09-16 15:17:49 -07:00
koolhead17
6ca57e81f1
docs: Fixed markdown typo in README.md. ( #2720 )
2016-09-16 14:26:28 -07:00
Anis Elleuch
010f61e91f
Add more tests for event-notifier code ( #2716 )
2016-09-16 14:26:05 -07:00
Harshavardhana
9216981262
tests: Add test for diskCount. ( #2717 )
...
Fixes #2312
2016-09-16 13:44:52 -07:00
Anis Elleuch
b89a1cd482
tests: Implemented more tests for fs-v1*.go ( #2686 )
2016-09-16 13:06:49 -07:00
Harshavardhana
7d37dea449
tests: Add more streaming signature tests. ( #2713 )
...
Part fix for #2621
2016-09-16 02:45:42 -07:00
Frank
df2ef64d20
Upgrade to new dsync version incl. stale lock detection ( #2708 )
2016-09-16 00:30:55 -07:00
Anis Elleuch
7a549096de
XL and FS use different tree walk ignored errors ( #2707 )
2016-09-15 13:43:40 -07:00
Harshavardhana
a1ff351f21
tests: Fix ListMultipartUploadsHandler tests. ( #2705 )
2016-09-15 01:44:19 -07:00
Harshavardhana
03430d0db8
tests: Add ListBucketHandler tests. ( #2701 )
...
part-3 final fix for #2412
2016-09-14 23:53:42 -07:00
Anis Elleuch
6f73d597e0
Fix tracing twice an error in fs Complete Multipart Upload ( #2703 )
2016-09-14 21:24:54 -07:00
koolhead17
e273a40345
docs: Modified README.md by providing information about Minio server ( #2704 )
...
data directory.
2016-09-14 21:24:12 -07:00
Anis Elleuch
a84548d7ea
Fix FS remove bucket regression bug ( #2693 )
2016-09-14 16:41:39 -07:00
Kartik Lunkad
19e01ceb19
QuickStart docs for Minio Server Setup needs update #2698 ( #2700 )
2016-09-14 15:16:59 -07:00
Harshavardhana
1e6d67b16d
server: Remove deadcode. ( #2699 )
2016-09-14 13:43:08 -07:00
Aditya Manthramurthy
6533927237
Lock-free rate-limit algorithm + bug-fix ( #2694 )
2016-09-14 11:27:37 -07:00
Harshavardhana
da9ae574df
server: We should fail properly during server startup. ( #2689 )
...
Fixes #2688
2016-09-14 01:11:03 -07:00
Harshavardhana
16e4a7c200
Merge pull request #2657 from minio/distributed
...
Distributed XL support
2016-09-13 22:34:49 -07:00
Harshavardhana
ee7e70c992
tests: Add tests for ListMultipartUploads, DeleteMultipleObjects. ( #2649 )
...
Additionally adds PostPolicyHandler tests.
2016-09-13 21:22:31 -07:00
Krishna Srinivas
54a9f59a13
Init: Print SQS ARNs after globalEventNotifier is inited. ( #2682 )
...
fixes #2681
2016-09-13 21:18:30 -07:00
Harshavardhana
e6fd664331
tests: Fix format-config tests.
2016-09-13 21:18:30 -07:00
Karthic Rao
b247ec9352
tests: refactor object-handler tests. ( #2656 )
...
- Move the initialization to a common executor for object Layer API
tests.d
2016-09-13 21:18:30 -07:00
Harshavardhana
43befab8ef
Change distributed server wording.
2016-09-13 21:18:30 -07:00
Harshavardhana
eae0281c64
tests: Add GetBucketLocation, HeadBucket tests. ( #2644 )
2016-09-13 21:18:30 -07:00
Karthic Rao
8bd78fbdfb
performance: gjson parsing for readXLMeta, listParts, getObjectInfo. ( #2631 )
...
- Using gjson for constructing xlMetaV1{} in realXLMeta.
- Test for parsing constructing xlMetaV1{} using gjson.
- Changes made since benchmarks showed 30-40% improvement in speed.
- Follow up comments in issue https://github.com/minio/minio/issues/2208
for more details.
- gjson parsing of parts from xl.json for listParts.
- gjson parsing of statInfo from xl.json for getObjectInfo.
- Vendorizing gjson dependency.
2016-09-13 21:18:30 -07:00
Krishnan Parthasarathi
66459a4ce0
Add unit-tests for formatting disks during initialization ( #2635 )
...
* Add unit-tests for formatting disks during initialization
- Fixed corresponding code at places where it was deviating from the
tabular spec.
* Added more test cases and simplified algo
... based on feedback from ``go test -coverprofile``.
2016-09-13 21:18:30 -07:00
Harshavardhana
182109f0de
xl: Heal format.json properly on fresh disks.
2016-09-13 21:18:30 -07:00
Harshavardhana
9998e9ea19
api: Response timeFormat do not need to have nano-second precision.
...
Fixes an error reported by s3verify.
2016-09-13 21:18:30 -07:00
Harshavardhana
ba2ba328da
server: Fixes for various conditions
...
- Fix distributed branch to be able to run FS version.
- Fix distributed branch to be able to run XL local disks.
- Ignore initialization failures of notification and bucket
policies, the codepath should load whatever is possible.
2016-09-13 21:18:30 -07:00
Anis Elleuch
67b8080144
Fix control lock rpc name in control lock cmd ( #2627 )
2016-09-13 21:18:30 -07:00
Anis Elleuch
239a34ca97
Add tests for regular and streaming v4 PutObject Handler ( #2618 )
2016-09-13 21:18:30 -07:00
Krishna Srinivas
81d8263ae2
binary-update: Do not fetch update info for minio binary compiled from source.
...
fixes #2494
2016-09-13 21:18:30 -07:00
Krishna Srinivas
b4e4846e9f
PutObject: object layer now returns ObjectInfo instead of md5sum to avoid extra GetObjectInfo call. ( #2599 )
...
From the S3 layer after PutObject we were calling GetObjectInfo for bucket notification. This can
be avoided if PutObjectInfo returns ObjectInfo.
fixes #2567
2016-09-13 21:18:30 -07:00
Krishna Srinivas
92e49eab5a
FS/Multipart: Do not rename append files to another tmp file as the append files are already in tmp location. ( #2612 )
2016-09-13 21:18:30 -07:00
Harshavardhana
c4a7b950a0
fs: Fix asynchronous multipart bug.
...
Construct part path properly.
2016-09-13 21:18:30 -07:00
Karthic Rao
1ce339abeb
Fixing ineffssaign errors ( #2608 )
2016-09-13 21:18:30 -07:00
Aditya Manthramurthy
a1f922315b
Add docker-compose file to run Minio in distributed mode ( #2606 )
...
Serves as a starting point to run a Minio cluster using Docker. The
file can be used as configuration for the docker-compose tool to start
4 Minio servers in distributed mode.
* Add a docker-compose.yml file to run 4 minio server instances in
distributed mode
* Update Docker.md with command to use the file
2016-09-13 21:18:30 -07:00
Anis Elleuch
3e284162d7
Add global flags to all commands and subcommands ( #2605 )
2016-09-13 21:18:30 -07:00
Anis Elleuch
ff99392102
Enhance minio server help template ( #2603 )
2016-09-13 21:18:30 -07:00
Krishna Srinivas
9358ee011b
logging: Print stack trace in case of errors.
...
fixes #1827
2016-09-13 21:18:30 -07:00
Harshavardhana
37cbcae6ba
xl: Remove an unecessary lock with isBucketExist() ( #2593 )
...
Fixes #2566
2016-09-13 21:18:30 -07:00
Harshavardhana
ae64b7fac8
XL: Handle object layer initialization properly.
...
Initialization when disk was down the network disk
reported an incorrect error rather than errDiskNotFound.
This resulted in incorrect error handling during
prepInitStorage() stage.
Fixes #2577
2016-09-13 21:18:30 -07:00
Anis Elleuch
d936ed90ae
Avoid testing on system errors strings in posix ( #2583 )
2016-09-13 21:18:30 -07:00
Krishna Srinivas
7cc77eba45
XL/Healing: errDiskNotFound is the only pardonable error in xlShouldHeal. ( #2586 )
...
This is so that we try to heal a file for all the "bad" cases except when the disk is down.
2016-09-13 21:18:30 -07:00
Karthic Rao
07d232c7b4
instrumentation: instrumentation for locks. ( #2584 )
...
- Instrumentation for locks.
- Detailed test coverage.
- Adding RPC control handler to fetch lock instrumentation.
- RPC control handlers suite tests with a test RPC server.
2016-09-13 21:18:30 -07:00
Krishnan Parthasarathi
de67bca211
Move formatting of disks out of object layer initialization ( #2572 )
2016-09-13 21:18:30 -07:00
Anis Elleuch
5c4dbc966f
Add Howto setup minio distributed with Docker
2016-09-13 21:18:30 -07:00
Harshavardhana
bca1385683
lock: Fix support single node XL locking as well.
2016-09-13 21:18:30 -07:00
Harshavardhana
f655592ff5
vendorize new dsync with new changes of logging.
2016-09-13 21:18:30 -07:00
Harshavardhana
cbe87cb2ed
Fix fd-leak in rpcClient close it pro-actively.
2016-09-13 21:18:30 -07:00
Anis Elleuch
0513b3ed07
Add Heal Disk Metadata RPC API + tests ( #2556 )
2016-09-13 21:18:30 -07:00
Frank
7f92165c79
Single use DRWMutex, RPCClient refactor and added missing cases to lock-rpc-server ( #2560 )
...
This PR contains various fixes for the distributed release:
- Use DRWMutex in namespace-lock only for a single Lock()/RLock() call in conformance to server-side rw-locking as implemented in minio/dsync
- Implement missing cases in lock-rpc-server to catch Unlock() for active read locks and RUnlock() for an active write lock
- Refactor RPCClient to release local mutex while making actual RPC.Call()
2016-09-13 21:18:30 -07:00
Harshavardhana
780ccc26f7
server: Validate server arguments for duplicates. ( #2554 )
...
- Validates invalid format inputs.
- Validates duplicate entries.
- Validates sufficient amount of disks.
Partially fixes #2502
2016-09-13 21:18:30 -07:00
Harshavardhana
339425fd52
server: Fetch StorageInfo() from underlying disks transparently. ( #2549 )
...
Fixes #2511
2016-09-13 21:18:30 -07:00
Harshavardhana
fa6e9540a8
server: We shouldn't exit the server in lazy init. ( #2548 )
...
Avoid fatalIf instead these are non-critical errors,
continue running the server.
- initializing bucket notifications
- initializing bucket policies.
- migrating bucket policies failure.
Fixes #2547
2016-09-13 21:18:30 -07:00
Harshavardhana
9605fde04d
controller/auth: Implement JWT based authorization for controller. ( #2544 )
...
Fixes #2474
2016-09-13 21:18:30 -07:00
Anis Elleuch
200d327737
List only objects that need healing ( #2546 )
2016-09-13 21:18:30 -07:00
Harshavardhana
e1b0985b5b
rpc: Refactor authentication and login changes. ( #2543 )
...
- Cache login requests.
- Converge validating distributed setup.
2016-09-13 21:18:30 -07:00
Krishnan Parthasarathi
c8dfc4cda4
auth-rpc: Reset token on disconnect ( #2542 )
2016-09-13 21:18:30 -07:00
Bala FA
7922a54c9a
rpc-client: remove unwanted nil check of rpcClient. ( #2538 )
2016-09-13 21:18:30 -07:00
Krishnan Parthasarathi
bda6bcd5be
Layered rpc-client implementation ( #2512 )
2016-09-13 21:18:30 -07:00
Harshavardhana
7e3e24b394
rpc: client login should ignore server versions.
2016-09-13 21:18:30 -07:00
Harshavardhana
bb0466f4ce
control: Fix controller CLI handling with distributed server object layer.
...
Object layer initialization is done lazily fix it.
2016-09-13 21:18:30 -07:00
Harshavardhana
8797952409
server: Add server command line for running in distributed mode
2016-09-13 21:18:30 -07:00
Krishnan Parthasarathi
c33d1b8ee6
Vendorize minio/dsync for server-side read lock ( #2484 )
...
- Prevention of stale lock accumulation.
- Removal of dead code.
2016-09-13 21:18:30 -07:00
Krishnan Parthasarathi
1f67c18222
Lock/Rlock rpc reply was incorrect ( #2479 )
2016-09-13 21:18:30 -07:00
Krishnan Parthasarathi
a4691611a7
Move initGracefulShutdown before objectLayer initialization ( #2468 )
2016-09-13 21:18:30 -07:00
awwalker
7c7eb1475d
splitNetPath: Add support for windows paths including volumeNames e.g ip:C:\network\path
2016-09-13 21:18:30 -07:00
Harshavardhana
0bce3d6d63
Fix web handler initialize with distributed lazy init.
2016-09-13 21:18:30 -07:00
Krishnan Parthasarathi
804d91ef61
storage/rpc-client: Reconnect on network disconnect ( #2436 )
2016-09-13 21:18:30 -07:00
Krishnan Parthasarathi
229600ce9b
Implement RLock, RUnlock rpc handlers ( #2437 )
...
This would make it simplify dsync.RWMutex's algorithm to acquire a
distributed read lock and be tolerant to N/2-1 failures.
2016-09-13 21:18:30 -07:00
Harshavardhana
43098df9d8
rpc: Re-factor ReadFile behavior client <--> server.
...
Current code did not marshal/unmarshal buffers properly from
the server, The reason being buffers have to be allocated and
sent properly back to client to be consumable.
2016-09-13 21:18:30 -07:00
Harshavardhana
6908a0dcd4
Extract rpc server wrapped errors and translate to storage error.
2016-09-13 21:18:30 -07:00
Harshavardhana
cae5761f16
rpc/client: Add missing rpcTokens for each rpc calls.
2016-09-13 21:18:30 -07:00
Harshavardhana
83074ed57e
Add missing rpc-client.go - missing in previous commit.
2016-09-13 21:18:30 -07:00
Krishnan Parthasarathi
64a7e6992e
Vendorize rpc reconnect changes from minio/dsync ( #2405 )
2016-09-13 21:18:30 -07:00
Harshavardhana
61af764f8a
Add rpc layer authentication.
2016-09-13 21:18:30 -07:00
Harshavardhana
b4172ad3c8
Rename rpc-{client,server} storage-rpc-{client,server}
2016-09-13 21:18:30 -07:00
Harshavardhana
4917038f55
Move the ObjectAPI() resource to be beginning of each handlers.
...
We should return back proper errors so that the clients can
retry until server has been initialized.
2016-09-13 21:18:30 -07:00
Krishnan Parthasarathi
b0f3f94163
unify single-node and distributed namespace locking ( #2401 )
2016-09-13 21:18:30 -07:00
Krishnan Parthasarathi
b7c169d71d
object layer initialization using distributed locking ( #2397 )
...
* vendorized latest minio/dsync
* wip - object layer initialization using distributed locking
2016-09-13 21:18:30 -07:00
Frank
3939c75345
Added distributed RWMutex ( #2369 )
2016-09-13 21:18:30 -07:00
Krishnan Parthasarathi
e55926e8cf
distribute: Make server work with multiple remote disks
...
This change initializes rpc servers associated with disks that are
local. It makes object layer initialization on demand, namely on the
first request to the object layer.
Also adds lock RPC service vendorized minio/dsync
2016-09-13 21:18:30 -07:00
Anis Elleuch
f82f535509
Fix possible race in shutdown callbacks process and simplify shuttting down profiler ( #2684 )
2016-09-13 18:43:45 -07:00
Anis Elleuch
51e337228e
Avoid hiding disk errors in some cases in FS Shutdown ( #2668 )
2016-09-13 11:01:10 -07:00
Kevin Qiu
241c56e6d7
Use Set instead of Add in the event that the request already contains the content-length ( #2683 )
2016-09-13 10:59:40 -07:00
Aditya Manthramurthy
895471afa1
Fixes #2678 ( #2679 )
...
Refactor `ratelimit.acquire()` to properly enforce the *globalMaxConn*
limit.
2016-09-12 15:53:54 -07:00
Harshavardhana
85e2d886bc
bucketPolicy: Trim incoming paths properly for policy verification.
2016-09-11 10:36:54 -07:00
Harshavardhana
07aa02f196
build: Fix appveyor.yml to report windows coverage as well. ( #2666 )
2016-09-10 21:05:00 -07:00
Anis Elleuch
8acf4d112a
Return Owner information in ListObjectV2 only when fetch-owner is specified ( #2654 )
2016-09-10 10:44:38 -07:00
Anis Elleuch
e6abfb3b67
More tests for format-config for a better coverage ( #2655 )
2016-09-10 09:54:22 -07:00
Harshavardhana
040af08473
server: Startup message now prints configured ARNs. ( #2653 )
...
Fixes #2628
2016-09-10 02:23:28 -07:00
Harshavardhana
e3de237eb8
Move config to v7 to fix previous v6 bug on null Notification entries. ( #2650 )
...
Thanks to @koolhead17 for reporting this.
2016-09-10 00:51:25 -07:00
Anis Elleuch
11785529fc
Add Request Not Ready error when presigned date request is not valid ( #2646 )
2016-09-10 00:38:07 -07:00
Anis Elleuch
32201a18ab
Forbid requests generated at least 15 min in the past or in the future ( #2648 )
2016-09-09 15:33:35 -07:00
Anis Elleuch
421cccb1d7
Add naughty disk to simulate disk errors ( #2645 )
2016-09-09 12:53:09 -07:00
Krishna Srinivas
400e9309f1
Signature-V4: Include content-length for signature calculation. ( #2643 )
...
This is to be compatible with clients which includes content-length for signature calculation
(and hence deviating from AWS Signature-v4 spec)
2016-09-09 09:38:07 -07:00
Krishna Srinivas
a4afb312d4
S3-layer: Multiple-delete should return success for non-existent objects. ( #2633 )
...
Fixes #2630
2016-09-07 11:49:12 -07:00
Harshavardhana
afe874f15a
api: CopyObject pipe should be closed pro-actively upon error. ( #2625 )
...
Fixes a deadlock reproduced while running s3verify during
RemoveObject(). Previously held lock by GetObject() inside
the go-routine was never relenquished.
2016-09-06 02:23:32 -07:00
Harshavardhana
b291dbe9c5
s3: Fix compliance bugs reported by s3verify. ( #2624 )
...
- Fixes couple of error strings reported are mismatching.
- Fixes a error HTTP status which was wrong fixed.
- Remove usage of an deprecated PostResponse, au contraire
to their documentation there is no response body in
PostPolicy.
2016-09-06 02:18:02 -07:00
Harshavardhana
bc8f34bfe7
server/mux: Remove unused waitgroup from listenerMux. ( #2614 )
...
Change struct names to be more meaningful.
2016-09-02 19:07:42 -07:00
Harshavardhana
2dc7ecc59b
notifications: Fix bucket notifications for DeleteMultipleObjects. ( #2609 )
...
Now reports bucket notifications for DeleteMultipleObjects API.
Also deletes multiple objects in parallel.
2016-09-02 01:59:08 -07:00
Hugo Jobling
600551feb9
Make export directory an exposed volume ( #2610 )
...
This exposes the /export directory, making it possible to mount it from other
containers. It also makes it possible to bind a local folder to the internal /export
mount point using the native Docker for Mac/Windows and Kinematic.
2016-09-02 01:58:15 -07:00
Harshavardhana
7398d737b5
profiler: Fix it properly and generate/save profiles even failure situations. ( #2607 )
...
Fixes #2594
2016-09-01 20:13:11 -07:00
Harshavardhana
bf62ba57cf
docs: Update minimum drives in erasure docs.
2016-09-01 11:35:53 -07:00
Harshavardhana
511a8cbe04
profiler: Fix how we generate profiles. ( #2596 )
...
Fixes #2594
2016-09-01 03:50:31 -07:00
koolhead17
790ad68d4d
Doc: Fixed Minio install steps from source. ( #2590 )
2016-08-31 18:00:11 -07:00
Anis Elleuch
e79d2381fc
Fix rare 'go test -race' failure in ListenServe{Plain,TLS} ( #2588 )
2016-08-31 16:07:44 -07:00
Krishna Srinivas
a3b4199e9b
FS/Multipart: Allow for parallel uploads of parts with same part number. ( #2587 )
...
CBL client does not close connection when the backup process is stopped, this causes
read() on the stream on the server side to block and hence the lock held on the part
is not released. When the backup process is restarted, we again try to lock on the
part and this will block. Using a unique tmp name and not locking it fixes the problem.
2016-08-31 16:01:32 -07:00
Krishna Srinivas
3aa0574c6b
FS/multipart: Append the parts to a file as and when the parts arrive. ( #2513 )
2016-08-31 13:42:57 -07:00
Andrew Stone
3118195e92
docker: Minimize image size ( #2585 )
...
This takes the image size from 823.1MB (current edge) to 281.8MB.
The following steps are taken:
0) Use go-alpine for a much smaller base image
1) Perform all actions in 1 step to eliminate diff images
2) Remove unused files created after setup
2016-08-31 13:32:41 -07:00
Harshavardhana
5716f1c199
Fix incorrect error strings. ( #2580 )
2016-08-30 13:47:09 -07:00
Harshavardhana
2020afd200
server: http.Server do not add deadlines causes issues. ( #2581 )
...
Adding deadlines is a no go since Golang doesn't back off
the timers if there is an active i/o in progress.
It is meant to be for applications to handle this themselves
and manually progress the deadlines.
Fixes #2561
2016-08-30 12:52:19 -07:00
Bala FA
7431acb2c4
web: add handlers for set/get bucket policy. ( #2486 )
...
Refer #1858
2016-08-30 10:04:50 -07:00
Harshavardhana
2f9975c76c
notifications: GetObjectInfo() later if bucket notifications are set. ( #2575 )
2016-08-30 03:42:37 -07:00
Dee Koder
d9bd73f4e8
docs: Simplified install instructions for home-brew. ( #2571 )
2016-08-29 14:08:04 -07:00
koolhead17
8484d1c0ca
Doc: Modified README.md to add brew package manager install steps. ( #2557 )
2016-08-28 15:44:56 -07:00
Harshavardhana
7270ca4157
pkg/wildcard: Simplify the wildcard logic further. ( #2555 )
2016-08-27 00:27:17 -07:00
Brendan Ashworth
b67c8970c9
bucket-handlers: fix missing return statement ( #2553 )
...
If the location was invalid, it would write an error response but then
continue to attempt to make the bucket. Whether or not it would succeed,
it would attempt to call response.WriteHeaders twice in a row, which
would cause a message to be logged to the server console (bad).
Here is the relevant Go code:
https://github.com/golang/go/blob/c80e0d374ba3caf8ee32c6fe4a5474fa33928086/src/net/http/server.go#L878-L881
2016-08-25 20:00:47 -07:00
Mohit Agarwal
418921de89
minor cleanup
...
- Reused contains() from utils.go at a couple of places
- Cleanup in return statements and boolean checks
2016-08-24 22:54:34 -07:00
Harshavardhana
ec4260d260
api: BucketNotification should disallow duplicate notification. ( #2539 )
...
Added checks to look for duplicated notification configs.
Fixes #2472
2016-08-23 18:42:30 -07:00
Harshavardhana
c39d3db7a0
server/tls: allocate tls.Config{} properly. ( #2537 )
...
There is a golang bug which crashes the server, allocate tls.Config properly to avoid
this problem.
```
2016/08/22 20:21:03 http: panic serving 127.0.0.1:40744: runtime error: invalid memory address or nil pointer dereference
goroutine 38 [running]:
net/http.(*conn).serve.func1(0xc820526680)
/home/harsha/.gimme/versions/go1.6.2.linux.amd64/src/net/http/server.go:1389 +0xc1
panic(0xbefa80, 0xc820010140)
/home/harsha/.gimme/versions/go1.6.2.linux.amd64/src/runtime/panic.go:443 +0x4e9
crypto/tls.(*Conn).serverHandshake(0xc820368300, 0x0, 0x0)
/home/harsha/.gimme/versions/go1.6.2.linux.amd64/src/crypto/tls/handshake_server.go:43 +0x4d6
crypto/tls.(*Conn).Handshake(0xc820368300, 0x0, 0x0)
/home/harsha/.gimme/versions/go1.6.2.linux.amd64/src/crypto/tls/conn.go:1035 +0x169
net/http.(*conn).serve(0xc820526680)
/home/harsha/.gimme/versions/go1.6.2.linux.amd64/src/net/http/server.go:1405 +0x382
created by net/http.(*Server).Serve
/home/harsha/.gimme/versions/go1.6.2.linux.amd64/src/net/http/server.go:2137 +0x44e
```
Fixes #2536
2016-08-23 00:20:15 -07:00
Harshavardhana
2da0cfc904
signature: Use the client sent region if region is properly validated. ( #2534 )
...
A properly validated region string from the client should be once
validated properly.
Fixes #2521
2016-08-22 19:43:36 -07:00
Harshavardhana
9dd0e3dc44
config: Initialize notify for fresh configs. ( #2531 )
2016-08-22 16:04:04 -07:00
Dee Koder
4722c94653
typo: Fix capitalization in heading. ( #2529 )
2016-08-22 12:01:21 -07:00
Krishna Srinivas
45c928e2f5
boot: checkPortAvailability() should fail only for EADDRINUSE error and ignore other errors. ( #2527 )
...
fixes #2510
2016-08-22 10:20:01 -07:00
Anis Elleuch
07506358ff
Refactor Heal RPC and add Shutdown RPC ( #2488 )
2016-08-21 12:06:53 -07:00
Harshavardhana
975eb31973
api/bucketPolicy: Use minio-go/pkg/set and fix bucket policy regression. ( #2506 )
...
Current master has a regression 'mc policy <policy-type> alias/bucket/prefix'
does not work anymore, due to the way new minio-go changes do json marshalling.
This led to a regression on server side when a ``prefix`` is provided
policy is rejected as malformed from th server which is not the case with
AWS S3.
This patch uses the new ``minio-go/pkg/set`` package to address the
unmarshalling problems.
Fixes #2503
2016-08-20 03:16:38 -07:00
koolhead17
a3c509fd23
Doc: Modified examples in how-to-run-multiple-minio-server-instances-on-single-machine.md ( #2497 )
2016-08-19 16:09:21 -07:00
GarimaKapoor
63bb78cfc6
Update README.md
2016-08-19 11:46:03 -07:00
Jesse Lucas
f2fd8b0265
Adding test coverage for server-mux.go by creating TestListenAndServeTLS(). ( #2493 )
...
Editing config.go to add lock on customConfigPath to avoid race condition
when setGlobalConfigPath() and getConfigPath() are called concurrently.
2016-08-19 01:29:50 -07:00
Jesse Lucas
a8052889fe
server-mux_test.go updating test to fix race condition with TestListenAndServe ( #2485 )
2016-08-18 19:45:03 -07:00
Harshavardhana
bccf549463
server: Move all the top level files into cmd folder. ( #2490 )
...
This change brings a change which was done for the 'mc'
package to allow for clean repo and have a cleaner
github drop in experience.
2016-08-18 16:23:42 -07:00
Anis Elleuch
73d1a46f3e
Allow asterisk character in key names ( #2487 )
2016-08-18 12:00:01 -07:00
Harshavardhana
95c16f51cb
notification: Handle ARN with empty account and name types. ( #2482 )
...
Fixes #2481
2016-08-17 16:34:03 -07:00
Harshavardhana
810dcbf34b
fs: ListObjects should populate ETag properly if fs.json is available. ( #2480 )
...
Fixes #2470
2016-08-17 13:26:08 -07:00
Harshavardhana
cb77586508
controller: Fix controller cli arguments.
2016-08-17 11:41:04 -07:00
Krishna Srinivas
e2498edb45
contoller: Implement controlled healing and trigger ( #2381 )
...
This patch introduces new command line 'control'
- minio control
TO manage minio server connecting through GoRPC API frontend.
- minio control heal
Is implemented for healing objects.
2016-08-17 11:36:33 -07:00
Jesse Lucas
0b7dfab17a
Refactoring MuxServer Close() method to always wait for conns to close before returning. Adding lock around ServerMux listener setting to protect against data race. Adding additional tests to server-mux_test.go to make sure open connections are closed and for ListenAndServe. ( #2467 )
2016-08-17 00:18:23 -07:00
Krishna Srinivas
674fdc4304
redirect-middleware: Redirect to /minio only if client is a browser and the request is not V4 signed. ( #2469 )
...
fixes #2356
2016-08-17 00:16:59 -07:00
karthic rao
10feb1af3f
tests: object handlers: Unit tests for Get and Copy Object handlers ( #2451 )
2016-08-16 19:24:23 -07:00
GarimaKapoor
d2b924cca8
Added Goreportcard badge.
2016-08-16 17:59:59 -07:00
Brendan Ashworth
13390d0c95
tests: add tests for signature-v4.go ( #2458 )
...
This adds tests for the functions `doesPresignedSignatureMatch` and
`doesPolicySignatureMatch`.
2016-08-16 17:02:35 -07:00
Harshavardhana
0a3d43273f
vendor: sha256 32bit updated. ( #2459 )
2016-08-16 16:19:29 -07:00
Harshavardhana
4dec50ba51
build: Add platform specific fixes.
2016-08-16 14:40:41 -07:00
Krishna Srinivas
f2bffe6086
fs/delete-object: fs.json filepath was incorrect. ( #2448 )
2016-08-16 10:05:08 -07:00
Krishna Srinivas
8e2f64aea4
fs/multipart: save metadata(fs.json) for multipart uploads. ( #2450 )
2016-08-16 10:04:40 -07:00
Harshavardhana
c054e633fd
utils: Shutdown channel should be bufferred.
2016-08-15 21:01:24 -07:00
Harshavardhana
e86dfcf41e
api: Change listen bucket notification to be TopicConfiguration. ( #2447 )
2016-08-15 20:56:43 -07:00
Anis Elleuch
3b9dbd748b
tests: Web handlers ( #2429 )
2016-08-15 16:13:03 -07:00
Harshavardhana
3d1bb8f439
tests: Fix hasExtendedHeader tests with env variable.
2016-08-15 16:09:08 -07:00
Krishna Srinivas
bb8a425d49
When updating the meta file, write to temp file first and then rename to the actual location.
...
This prevents appending the metadata to the metadata-file when a file is reuploaded.
2016-08-15 15:55:59 -07:00
Harshavardhana
0e745fdb05
fs: Enable fs.json with env MINIO_ENABLE_FSMETA
2016-08-15 15:53:48 -07:00
Anis Elleuch
51d7749c3e
Check if eventN is initialized before notifying in Upload web handler ( #2435 )
2016-08-15 12:15:46 -07:00
Harshavardhana
76d56c6ff2
typo: Fix typos across the codebase. ( #2442 )
2016-08-15 02:44:48 -07:00
Harshavardhana
b41bfcbf2f
utils: Fix unit tests issue. ( #2441 )
2016-08-15 01:59:28 -07:00
Yurii Rashkovskii
341171f326
Problem: AWS documentation defines event timestamp as 1970-01-01T00:00:00.000Z ( #2440 )
...
While Minio is using 20160814T124605Z
(See http://docs.aws.amazon.com/AmazonS3/latest/dev/notification-content-structure.html )
Solution: adhere to AWS documentation
2016-08-15 01:50:07 -07:00
karthic rao
a3592228f5
bug-fix: fix for tests failure when cache is disabled ( #2439 )
2016-08-15 01:25:41 -07:00
Anis Elleuch
5526ac13d2
Protect shutdown callbacks lists with a mutex ( #2432 )
2016-08-14 23:55:48 -07:00
Harshavardhana
9606cb9bcd
posix: Disk free verification should have relaxed handling of inodes. ( #2431 )
...
Some filesystems do not implement a way to provide total inodes available, instead inodes
are allocated based on available disk space. For example CephFS, StoreNext CVSFS, AzureFile
driver. Allow for the available disk to be separately validate and we will validate inodes
only if the total inodes are provided by the underlying filesystem.
Fixes #2364
2016-08-13 02:30:15 -07:00
Yurii Rashkovskii
7829ccea2c
Routing key was misspelled as routine key ( #2430 )
2016-08-12 22:23:06 -07:00
Anis Elleuch
723153951c
Test api responses ( #2427 )
...
* Test List Multipart Uploads with correct max-keys
* Test List Objects V1 V2 with valid parameters
2016-08-12 11:28:27 -07:00
Anis Elleuch
64dc2a2e7f
Heal format by inspection should avoid picking minioMetaBucket dir if the latter comes first in the list (listDir not ordered) ( #2426 )
2016-08-12 08:38:04 -07:00
Anis Elleuch
fdab984c8d
Add test for fresh disks healing ( #2424 )
2016-08-12 08:36:43 -07:00
karthic rao
70fd38818e
clean up: ineffassign fixes ( #2411 )
2016-08-12 00:26:30 -07:00
Jesse Lucas
ef0a108dde
Graceful shutdown for ServerMux ( #2341 )
2016-08-11 21:33:55 -07:00
karthic rao
0b225269e1
tests: posix: tests cleaning up and enhancing coverage. ( #2410 )
2016-08-11 19:57:14 -07:00
Anis Elleuch
fe62688683
Add tests for Damerau Levenshtein algorithm ( #2407 )
2016-08-11 17:50:04 -07:00
Anis Elleuch
fadb71351c
Test Post policy parsing and checking conditions ( #2408 )
2016-08-11 17:49:40 -07:00
Matthieu Fronton
402af93da2
Update how-to-install-golang URL ( #2406 )
2016-08-11 12:01:12 -07:00
Harshavardhana
d1bb8a5b21
api: refactor the bucket policy reading and writing. ( #2395 )
...
Policies are read once during server startup and subsequently
managed through in memory map. In-memory map is updated as
and when there are new changes coming in.
2016-08-10 20:10:47 -07:00
Harshavardhana
97c1289659
handlers: GetObject and HeadObject support more responses. ( #2403 )
...
- response-content-encoding.
- response-content-language.
Fixes #2393
2016-08-10 17:36:28 -07:00
Harshavardhana
8274ac2e5a
tests: Make sure we try tests on free ports. ( #2402 )
...
Fixes #2376
2016-08-10 16:28:05 -07:00
Brendan Ashworth
758aa21b9c
tests: add tests for certs.go and trie.go ( #2394 )
2016-08-10 02:26:40 -07:00
koolhead17
0dc5662f9b
Doc: Fixed screenshot path for modified docs. ( #2390 )
2016-08-09 15:04:44 -07:00
Harshavardhana
82cd38e959
handlers: Remove 'notification.xml' when bucket is deleted. ( #2389 )
...
Do not pass around objectHandlers object, input argument
should comply to a type for only that would be used inside
the function body.
2016-08-09 11:33:45 -07:00
karthic rao
e0cf4ee9fc
presignV4: fix errors response and tests. ( #2375 )
...
- Fix error response when one of the query params in the presign URL is
missing.
- Exhasutive test coverage for presignv4.
2016-08-09 09:13:15 -07:00
Dee Koder
2a920e568c
docs: Readded code coverage badge in github Readme. ( #2391 )
...
We have the fix in place to hide this on docs.minio.io
2016-08-08 22:14:38 -07:00
Harshavardhana
9c7f75d1e7
handler: Remove unused accesslog handler ( #2388 )
2016-08-08 21:33:21 -07:00
Harshavardhana
7e46055a15
api/handlers: Implement streaming signature v4 support. ( #2370 )
...
* api/handlers: Implement streaming signature v4 support.
Fixes #2326
* tests: Add tests for quick/safe
2016-08-08 20:56:29 -07:00
koolhead17
0c125f3596
Doc: This patch new guides with titles mentioned below ( #2382 )
2016-08-08 19:39:01 -07:00
GarimaKapoor
a1f3bf57c7
Update README.md
2016-08-07 11:08:42 -07:00
Harshavardhana
0188cd0b84
utils: Take monitorShutdownSignal to take an exitFunc which would executed upon error. ( #2378 )
...
This hook approach allows program to keep running but being able to handle exiting
of the program in the dynamic way.
Fixes #2377
2016-08-06 23:53:10 -07:00
Harshavardhana
b23605a2b5
pkg/objcache: Add more tests. ( #2371 )
2016-08-06 10:22:14 -07:00
koolhead17
8c2985a9f5
Doc: README.md/Removed codecov badge from title. ( #2367 )
2016-08-05 16:53:29 -07:00
Anis Elleuch
d28fb5fe23
Add a generic registerShutdown function for graceful exit ( #2344 )
...
* Add a generic registerShutdown function for graceful exit
* Add shutdown callback test case
2016-08-05 13:48:31 -07:00
GarimaKapoor
62c0612eac
Update README.md
2016-08-05 11:35:03 -07:00
karthic rao
d63ce9d60d
tests: tests for signature v4 parser ( #2362 )
2016-08-05 08:03:18 -07:00
Remco Verhoef
5a44c34fd7
fixed some issues in readme ( #2363 )
...
thx to @MartijnVogel_twitter
2016-08-05 08:02:11 -07:00
Harshavardhana
064c51162d
api: Add new ListenBucketNotificationHandler. ( #2336 )
...
This API is precursor before implementing `minio lambda` and `mc` continous replication.
This new api is an extention to BucketNofication APIs.
// Request
```
GET /bucket?notificationARN=arn:minio:lambda:us-east-1:10:minio HTTP/1.1
...
...
```
// Response
```
{"Records": ...}
...
...
...
{"Records": ...}
```
2016-08-04 22:01:58 -07:00
Harshavardhana
90c20a8c11
Add codecov for minio. ( #2359 )
2016-08-04 16:48:50 -07:00
Harshavardhana
e783d77c3d
Add codecov for minio.
2016-08-04 15:56:26 -07:00
Krishna Srinivas
e887fea485
getbucketlocation: should handle UNSIGNED-PAYLOAD for sha256 header for signature calculation. ( #2358 )
...
fixes #2355
2016-08-04 15:49:35 -07:00
Harshavardhana
de5d5ff241
pkg/crypto: Deprecate cgo sha256 version. ( #2354 )
2016-08-04 03:19:36 -07:00
karthic rao
2e0742e309
bucket policy: Support for '?' wildcard. ( #2353 )
...
- Support for '?' wildcard for resource matching.
- Wildcard package is added with Match functions.
- Wildcard.Match supports '*' and wild.MatchExtended supports both '*'
and '?' wildcards in the pattern string.
- Tests for the same for the wide range of cases.
2016-08-04 00:41:32 -07:00
Anis Elleuch
cc0d5b6fe0
webapi: ServerInfo returns the list variables/values in the environnment of the Minio server ( #2351 )
2016-08-03 13:47:03 -07:00
Harshavardhana
2db51e9d61
server/config: config should be updated only when we edit the credentials. ( #2345 )
2016-08-02 16:48:21 -07:00
Krishna Srinivas
69fd196471
Object-cache: enforce cache size to be less than RAM. ( #2338 )
2016-08-02 10:04:35 -07:00
karthic rao
1494af485e
tests: signature-utils test ( #2342 )
2016-08-01 20:54:11 -07:00
Harshavardhana
c1d70f1f9e
server/config: Create 'certs' directory in initServerConfig(). ( #2331 )
...
certs directory was created only if config was not present, our
expectancy is we need 'certs' directory to be present all the
time making it easier to be documented.
2016-07-30 14:55:20 -07:00
karthic rao
9baf599c91
tests: Unit tests and fixes for copyBuffer. ( #2333 )
...
- Unit tests for copyBuffer.
- Shadowing fix for copyBuffer.
2016-07-30 13:36:43 -07:00
Harshavardhana
8d090a20ce
server: set globalCacheSize honoring system limits for max memory. ( #2332 )
...
On unix systems it is possible to set max memory used by
running processes using 'ulimit -m' or 'syscall.RLIMIT_AS'.
A process whence exceeds this limit, kernel would pro-actively
kill such a server with OOM. To avoid this problem of defaulting
our cache size to 8GB we should look for if the current system
limits are lower and set the cache size appropriately.
2016-07-30 08:50:49 -07:00
karthic rao
5b86dd7659
Tests: Cleanup/Enhancement: Add few more cases to posix.ReadFile tests and use a cleaner posixTestSetup for posix tests ( #2330 )
2016-07-30 01:26:19 -07:00
Jesse Lucas
4b05b6a6c1
Refactoring checkPortAvailability to check each tcp network (tcp, tcp4, tcp6) if a port is taken. ( #2325 )
2016-07-29 18:11:00 -07:00
Jesse Lucas
851d05161a
Adding return error value to checkPortAvailability to enable testing of function. Adding checkport_test.go to test checkPortAvailability. Updated server-main.go to use error value from checkPortAvailability and calls fatalIf if an error is returned. ( #2322 )
2016-07-29 14:05:31 -07:00
Harshavardhana
cf9ba7b88f
tests: Add missing unit test for posix.ReadFile. ( #2319 )
2016-07-28 21:57:11 -07:00
Krishnan Parthasarathi
50dae0ab04
bucket-policy: Migrate bucket policy to minioMetaBuket/buckets ( #2321 )
2016-07-28 20:49:08 -07:00
Anis Elleuch
14cefd352c
Heal corrupted formats of disks already containing objects ( #2297 )
2016-07-28 16:49:58 -07:00
Frank
f239fcac67
Switched to faster minio/sha256-simd implementation ( #2320 )
2016-07-28 14:44:37 -07:00
Anis Elleuch
dcc3463e48
Limit POST form fields and file size + Generic Request Size limiter ( #2317 )
...
* Use less memory when receiving a file via multipart
* Add generic http request maximum size limiter to secure against malicious clients
2016-07-28 12:02:22 -07:00
Krishna Srinivas
7850d17f48
web-browser: disable minio browser when environmental variable MINIO_BROWSER=off ( #2315 )
...
fixes #2314
2016-07-28 04:00:33 -07:00
Harshavardhana
f503ac3db8
XL/Erasure: Make bit-rot verification based on xl.json algo. ( #2299 )
...
Currently `xl.json` saves algorithm information for bit-rot
verification. Since the bit-rot algo's can change in the
future make sure the erasureReadFile doesn't default to
a particular algo. Instead use the checkSumInfo.
2016-07-28 02:20:34 -07:00
Harshavardhana
65f71ce0c5
browser: Object upload should save metadata and notify. ( #2309 )
...
Object upload from browser should save additional
incoming metadata. Additionally should also notify
through bucket notifications once they are set.
Fixes #2292
2016-07-27 21:11:15 -07:00
Harshavardhana
ad19bf0ec1
server: Add update referral for update notification URL. ( #2308 )
2016-07-27 19:59:19 -07:00
Harshavardhana
f0067babe0
handlers: Add 'crossdomain.xml' handler. ( #2305 )
...
Fixes #2301
2016-07-27 19:53:55 -07:00
karthic rao
6b5fce826b
placing defer file.Close() right after opening it ( #2306 )
2016-07-27 19:22:32 -07:00
Anis Elleuch
8b3cb3a0de
POST Policy, multiple fixes: AccessDenied with unmet conditions, ${filename} in Key, missing filename in multipart ( #2304 )
...
* Unsatisfied conditions will return AccessDenied instead of MissingFields
* Require form-field `file` in POST policy and make `filename` an optional attribute
* S3 feature: Replace in Key by filename attribute passed in multipart
2016-07-27 17:51:55 -07:00
Harshavardhana
2f7358a8a6
XL: erasure Index should have its corresponding distribution order. ( #2300 )
2016-07-27 11:57:08 -07:00
Harshavardhana
77248bd6e8
api: Notify events only if bucket notifications are set. ( #2293 )
...
While the existing code worked, it went to an entire cycle
of constructing event structure and end up not sending it.
Avoid this in the first place, but returning quickly if
notifications are not set on the bucket.
2016-07-26 19:10:02 -07:00
Harshavardhana
3054b74260
docs: Fix startup message for server as well.
2016-07-26 15:54:11 -07:00
koolhead17
7d42d09da8
Doc: Replaced README & FreeBSD docs with updated minio server splash ( #2298 )
...
screen.
2016-07-26 15:46:41 -07:00
Anis Elleuch
95ddf061ab
Rate limit is working and supports limited waiting clients ( #2295 )
2016-07-26 14:17:11 -07:00
karthic rao
5fe72cf205
Removing readAllMeta from xl-v1-healing.go and placing it in xl-v1-utils.go ( #2296 )
2016-07-26 11:34:48 -07:00
Harshavardhana
1e3d80552f
XL: format.json healing should cater for mismatching order. ( #2285 )
...
Fresh disks can be provided in any order, we need to make sure
to preserve existing disk order and populate the fresh disks
in new positions.
Thanks for Anis Elleuch <vadmeste@gmail.com > for finding this issue.
2016-07-26 03:18:47 -07:00
Harshavardhana
f253dfc922
docs: Fix erasure code image embedding issue.
2016-07-26 00:09:57 -07:00
Harshavardhana
1f9e38e3cd
api: Add bucket notification util tests. ( #2289 )
2016-07-26 00:01:35 -07:00
karthic rao
530ed67b59
Adding leak test framework ( #2254 )
2016-07-25 20:39:14 -07:00
Harshavardhana
a2b6f0524d
XL/erasure: Remove deprecated copyN function. ( #2288 )
2016-07-25 20:36:56 -07:00
karthic rao
091d80666a
Enhancement for Erasure encode test. ( #2287 )
2016-07-25 20:36:41 -07:00
Harshavardhana
efbf7dbc0f
api: Bucket notification add filter rules check and validate. ( #2272 )
...
These filtering techniques are used to validate
object names for their prefix and suffix.
2016-07-25 17:53:55 -07:00
Krishna Srinivas
043ddbd834
optimize memory allocation during erasure-read by using temporary buffer pool. ( #2259 )
...
* XL/erasure-read: optimize memory allocation during erasure-read by using temporary buffer pool.
With the change the buffer needed during GetObject by erasureReadFile is allocated only once.
2016-07-25 14:17:01 -07:00
Dee Koder
04f90bd463
doc: Broken links fixed in the Explore further section. ( #2281 )
2016-07-24 22:53:35 -07:00
Harshavardhana
9212e11b90
XL/GetObject: When disk is not available, checksum should be empty. ( #2276 )
2016-07-24 22:49:27 -07:00
Harshavardhana
79bab6b561
XL: Operations on uploads.json should cater for disk being unavailable. ( #2277 )
2016-07-24 18:08:15 -07:00
Krishnan Parthasarathi
7e5a78985d
tests: Using listObjects clean up remaining tree walk go routines. ( #2278 )
...
* fs: Set nextMarker independent of it having a slash or not.
* tests: Using listObjects clean up remaining tree walk go routines.
* tests: Use slices to hold data instead of enumerating test cases by hand
... also fixed numbering of test cases.
2016-07-24 15:52:12 -07:00
Anis Elleuch
b0b919a1d6
Server http and https on the same port using a customized server ( #2247 )
2016-07-24 12:30:57 -07:00
Harshavardhana
6c2fb19ed7
docs: Removed and purged uneeded docs. ( #2273 )
2016-07-24 03:32:45 -07:00
Harshavardhana
f248089523
api: Implement bucket notification. ( #2271 )
...
* Implement basic S3 notifications through queues
Supports multiple queues and three basic queue types:
1. NilQueue -- messages don't get sent anywhere
2. LogQueue -- messages get logged
3. AmqpQueue -- messages are sent to an AMQP queue
* api: Implement bucket notification.
Supports two different queue types
- AMQP
- ElasticSearch.
* Add support for redis
2016-07-23 22:51:12 -07:00
Harshavardhana
f85d94288d
api: extract http headers with some supported header list. ( #2268 )
2016-07-22 20:31:45 -07:00
Harshavardhana
55cb55675c
api/multipart: Send S3 compatible error message, missing second sentence. ( #2270 )
2016-07-22 17:05:40 -07:00
koolhead17
7e076577de
Update Minio-erasure-code-quickStart-guide.md ( #2269 )
...
Minor update to link the URL.
2016-07-22 13:35:27 -07:00
Harshavardhana
5d118141cd
XL: Remove deadcode unionChecksumInfo. ( #2261 )
2016-07-21 19:07:00 -07:00
karthic rao
646ff2c64d
Get Object disk not found test ( #2264 )
...
Test: GetObject disk not found test
2016-07-21 19:06:50 -07:00
Harshavardhana
0add96f655
fs: Save metadata for objects in minioMetaBucket directory. ( #2251 )
2016-07-21 17:31:14 -07:00
Krishna Srinivas
303f216150
tests: xl-v1-metadata.go, xl-v1-multipart-common.go - remove unused methods, add enhance tests to improve code coverage. ( #2260 )
2016-07-21 15:00:11 -07:00
koolhead17
a7b5b8e63f
Doc: Modified the contents for Doctor. ( #2262 )
2016-07-21 14:58:16 -07:00
Krishnan Parthasarathi
5730d40478
tests: Added GetObject, DeleteObject and PutObject unit-tests ( #2222 )
2016-07-21 13:15:54 -07:00
karthic rao
0eaf684777
Remove consuming benchmarks, clean up closures, correct Get and PutObject Parallel benchmarks ( #2258 )
2016-07-21 11:17:28 -07:00
Harshavardhana
a0635dcdd9
XL: Do not rely on getLoadBalancedQuorumDisks for NS consistency. ( #2243 )
...
The reason is any function relying on `getLoadBalancedQuorumDisks`
cannot possibly have an idempotent behavior.
The problem comes from given a set of N disks returning just a
shuffled N/2 disks. In case of a scenario where we have N/2
number of failed disks, the returned value of `getLoadBalancedQuorumDisks`
is not equal to the same failed disks so essentially calls using such
disks might succeed or fail randomly at different intervals in time.
This proposal change is we move to `getLoadBalancedDisks()`
and use the shuffled N disks as a whole. Since most of the time we might
hit a good disk since we are not reducing our solution space. This
also provides consistent behavior for all the functions which rely
on shuffled disks.
Fixes #2242
2016-07-21 00:27:08 -07:00
Dee Koder
41f4f2806d
screenshots: update with the latest optimized image. ( #2249 )
2016-07-20 16:15:26 -07:00
Dee Koder
2a972ef1fd
images: Move screenshot for docs inside docs/screenshots directory. ( #2248 )
...
* images: Move screenshot for docs inside docs/screenshots directory. Use optimized images.
* images: This fix optimizes the images for the Erasure Code Quick Start Guide
2016-07-20 13:52:30 -07:00
Harshavardhana
c1e953b368
api: Set content-encoding properly if set. ( #2245 )
...
Additionally don't set content-type if not present, golang http
server automaticaly handles this and sets it automatically.
2016-07-20 12:40:20 -07:00
Krishna Srinivas
18728a0b59
XL/erasure-read: refactor erasure read and add tests ( #2232 )
2016-07-20 01:30:30 -07:00
Harshavardhana
cef26fd6ea
XL: Refactor usage of reduceErrs and consistent behavior. ( #2240 )
...
This refactor is also needed in lieu of our quorum
requirement change for the newly understood logic behind
klauspost/reedsolom implementation.
2016-07-19 19:24:32 -07:00
Dee Koder
f67c930731
doc: Fixed spacing with respect to code blocks. ( #2241 )
2016-07-19 19:08:43 -07:00
GarimaKapoor
3589a58179
Update Minio-erasure-code-quickStart-guide.md
2016-07-19 17:59:05 -07:00
Dee Koder
e8155abc18
screenshot: Use the full path to the screenshot when embedding images ( #2239 )
2016-07-19 17:48:18 -07:00
Dee Koder
2e8360120d
headings: We need to add a consistent heading for all docs. Adding Minio FreeBSD QuickStart Guide in the title. ( #2233 )
2016-07-19 14:56:34 -07:00
Dee Koder
02b191222c
headings: Added standardized heading for this document. ( #2234 )
2016-07-19 14:56:20 -07:00
Dee Koder
b699795901
docs: Remove additional headings. Added standard heading. Include numbering. ( #2235 )
2016-07-19 14:30:32 -07:00
Harshavardhana
86d31e99d5
api: use checkAuth now at PutBucket, DeleteBucket handlers. ( #2225 )
...
Additionally add a unit test for isReqAuthenticated function.
2016-07-18 23:56:27 -07:00
Krishna Srinivas
897d78d113
erasureReadFile and erasureCreateFile testcases. ( #2229 )
...
* unit-tests: Unit tests for erasureCreateFile and erasureReadFile.
* appendFile() should return errXLWriteQuorum.
* TestErasureReadFileOffsetLength() tests erasureReadFile() for different offset and lengths.
* Fix for the failure seen in the erasure read unit test case. Issue #2227
* Move common erasure setup code to newErasureTestSetup()
* Review fixes. Add few more test cases for erasureReadFile.
2016-07-18 23:56:16 -07:00
Harshavardhana
1f706e067d
api: xmlDecoder should honor contentLength. ( #2226 )
...
This is needed so that we avoid reading large amounts
of data from compromised clients.
2016-07-18 21:20:17 -07:00
Krishnan Parthasarathi
5cc9e4e214
fs/XL: Return IncompleteBody{} error for short writes ( #2228 )
2016-07-18 19:06:48 -07:00
Krishna Srinivas
27a5b61f40
tree-walk: optimize tree walk such that leaf detection of entries is delayed till the entry is sent on the treeWalkResult channel. ( #2220 )
2016-07-17 15:16:52 -07:00
Harshavardhana
aeac902747
API: ListBuckets doesn't have a body, we should never read the body. ( #2218 )
...
ListBuckets was incorrectly reading the body of the request, fix it.
2016-07-17 13:23:15 -07:00
Harshavardhana
aaf7803831
api: Requests should be differentiated if possible based on http router. ( #2219 )
...
In current master ListObjectsV2 was merged into ListObjectsHandler
which also implements V1 API as well.
Move the detection of ListObject types to its rightful place
in http router.
2016-07-17 12:32:05 -07:00
Krishna Srinivas
8cc163e51a
Refactor xl.GetObject and erasureReadFile. ( #2211 )
...
* XL: Refactor xl.GetObject and erasureReadFile. erasureReadFile() responsible for just erasure coding, it takes ordered disks and checkSum slice.
* move getOrderedPartsMetadata and getOrderedDisks to xl-v1-utils.go
* Review fixes.
2016-07-16 08:35:30 -07:00
Harshavardhana
2d38046a5a
utils: BucketNames with double periods and ip address should be rejected. ( #2213 )
...
Fixes #2212
2016-07-15 17:30:37 -07:00
Harshavardhana
cbb6b48b94
doc: update README.md
2016-07-15 16:12:54 -07:00
Harshavardhana
d0636d633d
doc: Move FreeBSD.md to docs.
2016-07-15 16:09:01 -07:00
Harshavardhana
41187fc2ef
docker: Fix docker edge build
2016-07-15 15:10:38 -07:00
koolhead17
204ec2c6c0
doc:README.md/Updated to sync with docs.minio.io ( #2210 )
...
* doc:README.md/Updated to sync with docs.minio.io
* doc:README.me/Modified the minio server output terminal to reflect new release changes.
* docs:README.md/Modified and changed location of other markdown files.
2016-07-15 15:03:59 -07:00
Krishnan Parthasarathi
3bce5db6d1
tests: Add tests to treeWalk for sortedness, recursive listing and isEnd behaviour ( #2209 )
2016-07-14 18:37:43 -07:00
Harshavardhana
35d438e0ff
vendorize: update all vendorized packages. ( #2206 )
...
Bring in new changes from upstream for all the packages.
Important ones include
- gorilla/mux
- logrus
- jwt
2016-07-14 14:59:20 -07:00
Krishna Srinivas
b090c7112e
Refactor of xl.PutObjectPart and erasureCreateFile. ( #2193 )
...
* XL: Refactor of xl.PutObjectPart and erasureCreateFile.
* GetCheckSum and AddCheckSum methods for xlMetaV1
* Simple unit test case for erasureCreateFile()
2016-07-14 14:59:01 -07:00
Harshavardhana
af6109f89a
update: Remove extraneous '/' in update message. ( #2207 )
2016-07-14 14:08:16 -07:00
Anis Elleuch
3f27734c22
Use normal color instead of forced white for users who have bright terminal background ( #2200 )
2016-07-13 14:27:36 -07:00
Harshavardhana
cdf1373f8e
XL: Ignore and continue for cases when bucket does not exist. ( #2205 )
...
Fixes #2201
Fixes #2204
2016-07-13 13:44:33 -07:00
Krishnan Parthasarathi
45240f158d
xl: Make namespace locking granular for PutObject ( #2199 )
2016-07-13 11:56:25 -07:00
Harshavardhana
0bd6b67ca5
server: Sort ips based on their last octet value. ( #2198 )
2016-07-13 06:34:59 -07:00
Harshavardhana
8c84df5e74
server: Change color codes for headings and sub-headings. ( #2197 )
...
This patch changes the color coding used for headings, sub-headings
and values as finalized.
2016-07-13 00:56:00 -07:00
Harshavardhana
dc3bafb194
XL: isQuorum rename as isDiskQuorum, word it properly. ( #2196 )
2016-07-13 00:29:48 -07:00
Harshavardhana
3b69b4ada4
server: Change server startup message. ( #2195 )
...
This change brings in the new agreed startup message
for the server.
Adds additional links point to Minio SDKs as well.
2016-07-12 23:21:18 -07:00
Krishnan Parthasarathi
0610527868
XL: PutObjectPart update checksum, re-read from xl.json for the part being written. ( #2191 )
2016-07-12 18:23:40 -07:00
Harshavardhana
0fcfb5df3b
XL/fs: Change minioMetaBucket different than '.minio' config dir. ( #2190 )
...
This fixes corruption of config directory seen when minio server
exports 'home' directory.
```
minio server ~
```
2016-07-12 15:21:29 -07:00
Harshavardhana
623e0f9243
XL: listOnlineDisks should use modTime instead of version. ( #2166 )
...
This change is needed to make reading from objects future proof
in-terms of handling online disks. Our current counter is not
based on affirmative knowledge and relies on arithmetic sequence
which can lead to bugs.
Using modTime simplifies the understanding of `xl.json` and future
tooling / debugging of the format.
2016-07-12 15:20:31 -07:00
utsl42
e5cd35aad0
XL: GetObjectInfo() store and retrieve user-defined object metadata. ( #2189 )
2016-07-12 12:45:17 -07:00
Anis Elleuch
5cd629adca
XL/fs: DeleteVol should not return error cleaning multipart dir for errVolumeNotFound ( #2188 )
2016-07-12 10:07:32 -07:00
Anis Elleuch
0fddf3fe17
Avoid creating tmp directories under .minio/tmp/ to facilitate cleaning ( #2187 )
2016-07-12 09:38:45 -07:00
karthic rao
ac6ff67546
Tool for running benchmark comparison of 2 commits ( #2161 )
2016-07-12 02:08:38 -07:00
Harshavardhana
126865e8df
XL/bucket: Remove bucket should cleanup incomplete uploads as well. ( #2173 )
...
This behavior is in accordance with S3.
Fixes #2170
2016-07-12 01:01:47 -07:00
Krishnan Parthasarathi
1c82b81408
Rename parts/objects only on onlineDisks ( #2185 )
2016-07-11 22:53:54 -07:00
Bala FA
749a94f6c9
tests: Add tests for signature-jwt code ( #2169 )
...
Fixes #1989
2016-07-11 21:57:40 -07:00
Harshavardhana
e9647b5f12
API/CopyObject: Refactor the code and handle if-modified-since as well. ( #2183 )
...
This completes the S3 spec behavior for CopyObject API as reported
by `s3verify`.
2016-07-11 19:24:34 -07:00
Krishnan Parthasarathi
bef72f26db
xl: Make locking more granular for PutObjectPart requests ( #2168 )
2016-07-11 17:24:49 -07:00
Harshavardhana
ede4dd0f9c
server: update command should check for 3s from 1ms. ( #2175 )
...
Programmer error :-)
2016-07-11 16:22:10 -07:00
Bala FA
bfc59b7d50
tests: improve unit tests for xl-v1-metadata. ( #2172 )
...
Fixes #2124
2016-07-11 11:42:01 -07:00
Remco Verhoef
a162198623
implemented systemd script ( #2167 )
2016-07-11 03:55:40 -07:00
Harshavardhana
de468f92ec
posix: ReadAll should handle the case when parent is not a dir. ( #2163 )
...
It can happen so that a read request can come for a file which
already has a parent i.e a file.
This fix handles this scenario - fixes #2047
2016-07-11 00:15:37 -07:00
Harshavardhana
d676e660c9
API/CopyObject: If-None-Match should return Precondition failed. ( #2164 )
2016-07-10 17:32:59 -07:00
Krishna Srinivas
aa7079fc7b
XL/GetObject: If quorum not available during GetObject appropriate error should be returned. ( #2135 )
2016-07-10 17:12:22 -07:00
Harshavardhana
bdff0848ed
server: Implement --ignore-disks for ignoring disks from healing. ( #2158 )
...
By default server heals/creates missing directories and re-populates
`format.json`, in some scenarios when disk is down for maintainenance
it would be beneficial for users to ignore such disks rather than
mistakenly using `root` partition.
Fixes #2128
2016-07-10 14:38:15 -07:00
Bala FA
0793237d94
tests: Move signature calculation in separate function. ( #2160 )
...
Previously newTestRequest() creates request object and returns
signature v4 signed request. In TestCopyObject(), its required to add
headers later to the request and sign the request.
This patch introduces two new functions
* signRequest(): signs request using given access/secret keys.
* newTestSignedRequest(): returns new request object signed with given
access/secret keys.
Fixes #2097
2016-07-10 11:10:59 -07:00
Bala FA
2a95eabb8a
benchmarks: add parallel benchmarks for PutObject/GetObject. ( #2159 )
...
Fixes #2092
2016-07-10 11:08:45 -07:00
Krishnan Parthasarathi
bc8720406d
Added specific error for InvalidObjectName ( #2157 )
2016-07-09 17:11:08 -07:00
Krishna Srinivas
ae80f8ca35
ObjectLayer/GetObject: Should return the right error value. Fix done in FS and XL. ( #2133 )
...
fixes #2117
2016-07-09 13:01:32 -07:00
Harshavardhana
5102a5877e
API/handler: CopyObject make it behave in accordance with S3 spec. ( #2155 )
...
Fixes bugs found while running s3verify tool - fixes #2152
2016-07-09 12:13:40 -07:00
karthic rao
3341fe9b28
organizing the benchmarks in the right test files ( #2154 )
2016-07-09 00:45:49 -07:00
Harshavardhana
c0c8a8430e
XL/PutObject: Add single putObject and multipart caching. ( #2115 )
...
- Additionally adds test cases as well for object cache.
- Adds auto-expiry with expiration and cleanup time interval.
Fixes #2080
Fixes #2091
2016-07-08 20:34:27 -07:00
karthic rao
b0c180b77c
Test for ObjectLayer.GetObject() ( #2153 )
2016-07-08 18:26:04 -07:00
karthic rao
778b870b77
placing the http range error in objct-api-errors. ( #2150 )
2016-07-08 17:22:55 -07:00
Harshavardhana
cb415ef12e
Merge pull request #2149 from harshavardhana/hash-order
...
XL/metadata: use new hashOrder algorithm for newXLMeta. (#2147 )
2016-07-08 15:57:16 -07:00
Harshavardhana
6266328a85
XL/metadata: use new hashOrder algorithm for newXLMeta. ( #2147 )
2016-07-08 15:39:21 -07:00
frankw
63b3f1dcfd
Use new algorithm to get fixed random order of disks ( #2147 )
2016-07-08 15:38:47 -07:00
Anis Elleuch
5ff1203fc0
Add PutObjectPart benchmark ( #2145 )
2016-07-08 14:28:06 -07:00
Harshavardhana
c3ab8bbd51
Turning off OSX builds for now.
2016-07-08 11:22:25 -07:00
Harshavardhana
a4a55bf134
tests: Fix erasure-readfile test formatting.
2016-07-08 11:05:30 -07:00
Harshavardhana
ec35330ebb
XL/fs: GetObject should validate all its inputs. ( #2142 )
...
Fixes #2141
Fixes #2139
2016-07-08 07:46:49 -07:00
Harshavardhana
ca1b1921c4
XL: Implement ignore errors. ( #2136 )
...
Each metadata ops have a list of errors which can be
ignored, this is essentially needed when
- disks are not found
- disks are found but cannot be accessed (permission denied)
- disks are there but fresh disks were added
This is needed since we don't have healing code in place where
it would have healed the fresh disks added.
Fixes #2072
2016-07-07 22:10:27 -07:00
Harshavardhana
4c21d6d09d
tests: Remove racey failedDisks behavior in RenameObject tests. ( #2138 )
...
Additionally also initialize namespace lock only once per test
run, there is no reason to initialize it multiple times to avoid
races.
Fixes #2137
2016-07-07 19:50:44 -07:00
Bala FA
5ec7734d88
FS: Check offset is within object size in GetObject() ( #2123 )
...
Fixes #2118
2016-07-07 19:49:45 -07:00
karthic rao
2c837128ef
Object layer tests revamp for individual execution ( #2134 )
2016-07-07 15:05:51 -07:00
Bala FA
282044d454
http: handle possible range requests properly. ( #2122 )
...
Previously range string is not validated against various combination
of values. This patch fixes the issue.
Fixes #2098
2016-07-07 15:05:18 -07:00
Harshavardhana
8e53064bb4
XL/format: Initialize missing meta volume. ( #2131 )
...
Fixes #2127
2016-07-07 15:01:54 -07:00
Krishna Srinivas
f55093cdd6
multipart: During multipart list the listing go-routine should be saved to the List-pool. ( #2130 )
2016-07-07 09:06:35 -07:00
Harshavardhana
ddf3245677
xl/fs: offset and length cannot be negative. ( #2121 )
...
Fixes #2119
2016-07-07 01:30:34 -07:00
Harshavardhana
169c72cdab
vendor: Bring new updates from blake2b-simd repo. ( #2094 )
...
This vendorization is needed to bring in new improvements
and support for AVX2 and SSE.
Fixes #2081
2016-07-06 18:24:31 -07:00
Nick Craig-Wood
8c767218a4
URL Encode X-Amz-Copy-Source as per the spec ( #2114 )
...
The documents for COPY state that the X-Amz-Copy-Source must be URL encoded.
http://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectCOPY.html
2016-07-06 15:42:17 -07:00
Bala FA
a51bb1d728
http: handle request range like Amazon S3. ( #2112 )
...
Fixes #2098
2016-07-06 12:50:24 -07:00
Krishna Srinivas
26b7d79a85
XL/ObjectCache: DeleteObject() should delete the object from the object cache. ( #2113 )
...
fixes #2103
2016-07-06 10:25:42 -07:00
Krishna Srinivas
01cbacd803
object-cache: use golang bytes.Buffer and bytes.NewReader instead of custom implementation. ( #2108 )
2016-07-06 01:29:49 -07:00
Harshavardhana
7bde27032d
signv4: Validate preSigned payload properly. ( #2106 )
...
We need to only validate presigned payload only
if the payload is requested for, with default payload
i.e 'UNSIGNED-PAYLOAD' we don't need to validate.
Fixes #2105
2016-07-05 21:00:20 -07:00
Bala FA
44ae7a037b
fix: allocate buffer to required size than readSizeV1 ( #2095 )
...
Refer #2077
2016-07-05 20:59:54 -07:00
karthic rao
a35341448f
XL: Cache: Purging partially cached content upon erasureReadFile failure ( #2104 )
2016-07-05 12:48:54 -07:00
Krishna Srinivas
d6dfcd0ba7
unit-tests: Unit tests for functions in erasure-readfile.go ( #2090 )
2016-07-05 11:41:25 -07:00
Harshavardhana
8ddf52021a
server: Bring in s3 compatibility fixes. ( #2099 )
...
This patch fixes majority of discrepant messages and responses
previously reported.
There are few discrepancies observed
- S3 is not honoring 'If-Modified-Since' header.
- We do not implement object policy, S3 returns a different response in this category.
- Adding new headers causes signature mismatch, but Minio server is fine for example
TestCopyObject() to be fixed by moving the signature logic out.
Relevant bug - https://github.com/minio/minio/issues/2097
Fixes #1955
2016-07-05 01:06:30 -07:00
Harshavardhana
8a028a9efb
handler/PUT: Handle signature verification through a custom reader. ( #2066 )
...
Change brings in a new signVerifyReader which provides a io.Reader
compatible reader, additionally implements Verify() function.
Verify() function validates the signature present in the incoming
request. This approach is choosen to avoid complexities involved
in using io.Pipe().
Thanks to Krishna for his inputs on this.
Fixes #2058
Fixes #2054
Fixes #2087
2016-07-05 01:04:50 -07:00
Bala FA
0540863663
fix: use readSizeV1 wherever applicable. ( #2093 )
2016-07-04 19:21:15 -07:00
Harshavardhana
4cfbdb1bf0
server: Remove deadcode/deprecated code. ( #2088 )
2016-07-04 14:46:38 -07:00
Krishna Srinivas
1ec86dac2c
server-tests: unify XL and FS tests into common code. server_test.go contains common test code. server_xl_test.go contains XL tests specific to XL. ( #2089 )
2016-07-04 13:18:41 -07:00
Krishna Srinivas
7a8b8cd0a1
tree-walk: unify FS and XL tree-walk with functional approach. ( #2027 )
2016-07-04 01:49:27 -07:00
karthic rao
a8a3e95835
Api/Bucket-Policy: Handler tests ( #2074 )
2016-07-03 22:35:30 -07:00
Bala FA
1ad5fb8f76
posix: checkDiskFree() also checks free inodes. ( #2086 )
...
Previously checkDiskFree() checks for free available space. This
patch enables checkDiskFree() also checks for free inodes in linux and
free clusters in windows.
Fixes #2075
2016-07-03 22:34:45 -07:00
Bala FA
52b55afce0
FS: check whether disk format is FS or not. ( #2083 )
...
Fixes #2060
2016-07-03 20:01:40 -07:00
Harshavardhana
355f06cfea
tests: Add urlEncode tests. ( #2078 )
2016-07-03 19:25:04 -07:00
Harshavardhana
d2557bb538
XL: GetObject caching implemented for XL. ( #2017 )
...
The object cache implementation is XL cache, which defaults
to 8GB worth of read cache. Currently GetObject() transparently
writes to this cache upon first client read and then subsequently
serves reads from the same cache.
Currently expiration is not implemented.
2016-07-03 17:15:38 -07:00
Bala FA
8d4365d23c
tests: add unit test for posix functions. ( #2037 )
...
Unit tests for posix operations.
* MakeVol
* DeleteVol
* StatVol
* ListVols
* DeleteFile
* AppendFile
* RenameFile
* StatFile
Fixes #2021
2016-07-03 11:17:08 -07:00
karthic rao
55ae7cac42
api/bucket-policy: Refactor, Handler test. ( #2071 )
...
PR contains,
- New setup utilities for running object handler tests. Here is why they are essential,
- Unit tests have to be run in isolation without being have to be associated with other functionalities which are not under test.
- The integration tests follows the philosophy of running a Test Server and registers all handlers and fires HTTP requests over the socket to simulate the system functionality under usual work load scenarios and test for correctness. But this philosophy cannot be adopted for running unit tests for HTTP handlers.
- Running Unit tests for API handlers,
- Shouldn't run a test server. Should purely call the handlers `ServeHTTP` under isolation.
- Shouldn't register all handlers, should only register handlers under test and so that the system is close to be in an isolated setup.
- As an example PutBucketPolicy test is illustrated using the new setup. Exhaustive cases has to be added and has been listen in TODO for now.
2016-07-02 19:05:16 -07:00
Harshavardhana
48ac34919f
browser: Add new release for ui-assets.go ( #2070 )
...
update `ui-assets.go` using `x-amz-date` for JSON rpc.
2016-07-02 10:54:17 -07:00
Harshavardhana
d64c3fd464
posix: Return errDiskNotWritable during disk initialization. ( #2048 )
...
It can happen that minio server might not have
writable permissions on the export paths command line.
Fixes #2035
2016-07-02 01:59:28 -07:00
Harshavardhana
e5dd917c37
handlers/generic: Remove support for 'x-minio-date' ( #2064 )
2016-07-02 01:51:17 -07:00
Krishna Srinivas
eb5f782c74
object-handler: skip sha256 calculation if x-amz-content-sha256=="UNSIGNED-PAYLOAD" ( #2038 )
...
fixes #2024 #2056
2016-07-01 14:34:40 -07:00
Harshavardhana
734e779b19
XL/erasureCreate: Create a limit reader if size is specified. ( #2059 )
...
This is needed so that we only write data which was requested
for, using a limit reader avoids spurious reads on the incoming
client data. Additionally using limit reader provides server
safety from rogue clients sending copious amounts of data (for
example a denial of service attack).
This patch also caters for size == -1 when content encoding from
a client is set as chunked, we happily read till io.EOF
2016-07-01 14:33:28 -07:00
Krishna Srinivas
3f2b4d9dc2
Show "https" in the "minio server export/" output if certificates are available. ( #2065 )
...
fixes #2036
2016-07-01 14:32:53 -07:00
Bala FA
cd1c2db864
posix-utils: fix path handling in windows. ( #2053 )
2016-07-01 11:44:07 -07:00
karthic rao
48aa5f2199
api/bucket-policy: Add unit tests for more coverage, fixes couple of bugs. ( #2055 )
...
Changes to ResourceMatch logic.
Test for action match function.
2016-06-30 23:49:59 -07:00
Krishnan Parthasarathi
bcb822c390
Send XML header before the first of whitespace chars ( #2046 )
...
* Sent XML header before the first of whitespace chars
XML parsing fails in aws cli due to unexpected whitespace character. To
fix this, we send the xml header before we send the first whitespace
character, if any.
* Fix race between sendWhiteSpaceChars and completeMultiUploadpart
2016-06-30 18:48:50 -07:00
Krishnan Parthasarathi
285a94d2c0
Ignored errors in cleanup in commitXLMetadata ( #2044 )
...
Deletion of tmp files where xl metadata was saved before the commit
operation doesn't change the error returned to the caller. So, it is to
be ignored.
2016-06-30 16:37:51 -07:00
Krishnan Parthasarathi
64899e5197
xl: Used unique tmp file to update xl.json in putObjectPart ( #2043 )
...
An in-place update to xl.json amidst concurrent PutObjectPart operations
lead to racy updates to xl.json making it un-parseable. To avoid this,
we create a unique tmp file where updates to xl.json are staged before
renaming it to the final location.
2016-06-30 16:28:01 -07:00
Danilo Pereira
812554087f
getCertsPath should use getConfigPath instead of defaulting to users homedir. ( #2039 )
...
Fixes #2028
2016-06-30 15:49:18 -07:00
Bala FA
57bc08cc7e
posix: remove disk free space check for read-only and delete methods. ( #2033 )
2016-06-29 11:25:35 -07:00
Harshavardhana
0e3907072c
XL/fs: Initialize export paths supplied on command line ( #2020 )
...
Fixes #2013
2016-06-29 03:13:44 -07:00
karthic rao
8e8f6f90a4
adding detailed comments to server_test ( #2032 )
2016-06-29 02:30:36 -07:00
Bala FA
4c1a11aae6
XL: allow meta bucket name appended with tmp meta prefix. ( #2007 )
2016-06-29 02:28:46 -07:00
Harshavardhana
ae936a0147
XL: Relax write quorum further to N/2 + 1. ( #2018 )
...
This changes behavior in some parts of the code
as well address it.
Fixes #2016
2016-06-29 02:10:40 -07:00
Harshavardhana
d484157d67
XL/bitrot: Migrate to new blake2b-simd SIMD optimized implementation. ( #2031 )
...
Thanks for Frank Wessels <fwessels@xs4all.nl > for all the heavy lifting work.
Comparative benchmarks are as below.
```
benchmark old ns/op new ns/op delta
BenchmarkHash64-4 742 411 -44.61%
BenchmarkHash128-4 681 346 -49.19%
BenchmarkWrite1K-4 4239 1497 -64.69%
BenchmarkWrite8K-4 33633 11514 -65.77%
BenchmarkWrite32K-4 134091 45947 -65.73%
BenchmarkWrite128K-4 537976 183643 -65.86%
benchmark old MB/s new MB/s speedup
BenchmarkHash64-4 86.18 155.51 1.80x
BenchmarkHash128-4 187.96 369.10 1.96x
BenchmarkWrite1K-4 241.55 683.87 2.83x
BenchmarkWrite8K-4 3897.06 11383.41 2.92x
BenchmarkWrite32K-4 977.48 2852.63 2.92x
BenchmarkWrite128K-4 243.64 713.73 2.93x
```
Fixes #2030
2016-06-29 02:06:35 -07:00
Harshavardhana
796fe165c7
server: Minor command line doc change for XL. ( #2022 )
2016-06-29 02:05:57 -07:00
Harshavardhana
3ac39ff107
XL: Change minimum disks supported to 6 now. ( #2023 )
...
This change co-incides with another patch set which
reduces the writeQuorum requirement. With the
write quorum change it is now possible to support
6 disk configuration.
2016-06-29 02:05:29 -07:00
Krishnan Parthasarathi
b6b9e88e47
Added unit-tests for treeWalkPool ( #1969 )
...
* Added unit-tests for treeWalkPool
* Added unit tests for tree-walk-fs
* Added period at the end of all comments.
* FS/XL: Unified tree walk tests for both backends
* Added disk failure related tests for treewalk
Replaced removeRandomDisks with removeDiskN. There is no need to
randomize disks that fail while the distribution of chunks in XL during
erasure coding data is random.
2016-06-28 22:32:00 -07:00
karthic rao
59366d8f4c
Benchmarks for ObjectLayer.PutObject() ( #2029 )
2016-06-28 22:12:36 -07:00
Harshavardhana
748dc80047
API: add writePartTooSmallErrorResponse to extend standard error responses. ( #2005 )
...
This function is added to extend the standard error responses.
Which is needed in some cases for example CompleteMultipartUpload
should respond with ErrPartTooSmall error when parts uploaded are
lesser than 5MB (i.e minimum allowed size per part).
Fixes #1536
2016-06-28 14:51:49 -07:00
karthic rao
6dcfa7b046
Fix for tests leaving out temp directories ( #2025 )
2016-06-28 04:21:52 -07:00
Krishnan Parthasarathi
a854e8cc5c
api: Sent ErrPreconditionFailed on If-Match failure ( #2009 )
...
* api: Sent ErrPreconditionFailed on If-Match failure
ref:
http://docs.aws.amazon.com/AmazonS3/latest/API/ErrorResponses.html#ErrorCodeList
* tests: Added functional tests for GetObject w/ If-Match headers set
* tests: Used verifyError to simplify errorCode and description matching on error
2016-06-28 01:18:18 -07:00
karthic rao
76f6533f8d
Adding detailed comments for server_xl_test. ( #2011 )
2016-06-27 23:54:56 -07:00
Harshavardhana
4db2b03312
XL: Rename objectN to part.N ( #2019 )
...
Fixes #2015
2016-06-27 21:42:33 -07:00
Krishna Srinivas
5291db60c6
XL/erasure: refactor erasureReadFile. Move parallelRead into a separate function. ( #2008 )
2016-06-27 13:24:55 -07:00
Harshavardhana
2e1f66c37d
XL: Handle quorum situations properly for write operations. ( #1986 )
...
Adds two test cases one for
- PutObject when write quorum is not available.
- PutObjectPart when write quorum is not available.
Fixes #1951
2016-06-27 10:01:09 -07:00
Bala FA
c88720ea2c
XL/listObjects: Ignore entry if getObjectInfo() returns errFileNotFound ( #2004 )
...
Fixes #1956
2016-06-26 22:10:22 -07:00
karthic rao
ce7d5eddbc
Misspell warnings fix ( #2001 )
2016-06-26 22:05:48 -07:00
Harshavardhana
0d3a9c9438
XL: Add tests for checkSufficientDisks, storageInfo. ( #1988 )
2016-06-26 19:48:02 -07:00
Harshavardhana
293ba00249
posix: Re-do tests for readDir(). ( #1996 )
2016-06-26 19:31:53 -07:00
Krishnan Parthasarathi
d0be09fdd3
object: checkETag compares quoted ETags properly ( #1997 )
...
Previously, checkETag didn't handle ETags with leading and trailing
double quotes. e.g "abcdef1234" == "\"abcdef1234\"" would return false.
Now, checkETag function canonicalizes the ETags passed as arguments by
removing one leading/trailing double quote.
2016-06-26 18:10:08 -07:00
Remco Verhoef
1e52759c3c
fix typo ( #1987 )
2016-06-26 11:27:04 -07:00
Harshavardhana
9add048f3c
erasure: Add erasure encode/decode unit tests. ( #1911 )
...
Fixes #1819
2016-06-26 03:32:49 -07:00
Harshavardhana
936a916e78
server: Add connection rate limiter for server. ( #1977 )
2016-06-26 03:18:07 -07:00
Harshavardhana
57146fbbb8
main: minio --help should print regardless of root. ( #1985 )
...
Remove root check entirely.
Fixes #1964
2016-06-26 03:03:52 -07:00
karthic rao
3d02f7471e
Benchmarks for various object sizes for FS/XL GetObject ( #1984 )
2016-06-25 20:22:04 -07:00
karthic rao
b2d9a46cbb
Cleaning up/refactoring tests to be more extensible ( #1970 )
2016-06-25 19:07:44 -07:00
Harshavardhana
42286cba70
XL: Implement new ReadAll API for files which are read in single call. ( #1974 )
...
Add a unit test as well.
2016-06-25 14:51:06 -07:00
karthic rao
ed2fdd90b0
fs: Fix GetObject failure to read large blocks. ( #1982 )
...
Add relevant test cases as well for verifying this
part of the codebase.
Fixes #1979
2016-06-25 03:03:27 -07:00
Krishna Srinivas
cb1200a66d
XL/erasure-read: read disks in parallel. ( #1975 )
...
On read failure, fallback to reads from other
disks also happen in parallel.
2016-06-24 18:02:10 -07:00
Krishnan Parthasarathi
a3a310cde8
Moved tree-walk-fs to use tree-walk-pool ( #1978 )
2016-06-24 16:41:57 -07:00
Bala FA
f625392211
tests: add unit test for posix.readDir() ( #1967 )
...
Fixes #1820
2016-06-24 14:32:08 -07:00
Harshavardhana
e8990e42c2
XL: Make allocations simpler avoid redundant allocs. ( #1961 )
...
- Reduce 10MiB buffers for loopy calls to use 128KiB.
- start using 128KiB buffer where needed.
2016-06-24 02:06:23 -07:00
Harshavardhana
ff9fc22c72
posix: Mkdir() and OpenFile() should honor umask. ( #1972 )
...
Adds two unit tests for validation as well.
Fixes #1965
2016-06-23 20:19:27 -07:00
Harshavardhana
41c089a7e0
XL: Add mis-aligned GetObject() test. ( #1960 )
2016-06-22 21:42:24 -07:00
Harshavardhana
5725f3c809
Merge pull request #1958 from krisis/unittest/unc-path
...
Added unit tests for UNC path handling in windows
2016-06-22 17:19:40 -07:00
Harshavardhana
f4830162a4
XL: Format heal should re-allocate new UUIDs not reuse. ( #1953 )
...
This patch also supports writing to a temporary file and renaming
rather than appending to an existing file. This helps in avoiding
inconsistent files.
2016-06-22 17:18:31 -07:00
Harshavardhana
e10934a88e
bitrot: Start using blake2b algorithm and remove sha512 usage. ( #1957 )
...
Fixes #1952
2016-06-22 17:13:26 -07:00
Krishnan Parthasarathi
a07751f61b
Added tests to validate disk name length near MAX_PATH
2016-06-22 15:30:41 -07:00
Krishnan Parthasarathi
0766e903e3
Added unit tests for UNC path handling in windows
2016-06-22 15:30:41 -07:00
Harshavardhana
75dddfb2ae
Merge pull request #1959 from krishnasrinivas/parallel-reads
...
Parallel reads in erasure-read
2016-06-22 15:05:35 -07:00
Harshavardhana
9b82e64a11
XL/erasure-read: Avoid memory copy, write to writer directly all the dataBlocks.
2016-06-23 02:06:57 +05:30
Harshavardhana
bdf8738076
lock: Add unit tests for namespace lock ( #1922 )
...
Fixes #1821
2016-06-22 12:27:47 -07:00
Krishna Srinivas
d4bea5fbf8
XL/erasure-read: Add Comments and enable bitrot detection.
2016-06-23 00:34:47 +05:30
Krishna Srinivas
17efaaa902
XL/erasure-read: Support parallel reads from disks.
2016-06-23 00:34:47 +05:30
Krishnan Parthasarathi
78ae696749
Added couple of unit-tests to xlObjects ( #1950 )
...
* Added couple of units to xlObjects
* Used test_utils for initialize/destroying xlObjects
2016-06-21 15:48:27 -07:00
Harshavardhana
3fa95f5263
docker: Remove unneeded docker files and makefile tags.
2016-06-21 15:31:30 -07:00
karthic rao
ba5bb4a127
TestServer introduction and revamp of functional tests. ( #1940 )
...
Allows for easy creation of Test server with temp backend.
changes
2016-06-21 12:10:18 -07:00
Harshavardhana
409b4ddecb
api: MakeBucket should set proper bucket location. ( #1948 )
...
Fixes #1942
2016-06-20 23:25:18 -07:00
Harshavardhana
ad779a8ba4
XL: Enable tests for large GetObject. ( #1947 )
...
Ref #1946
2016-06-20 22:08:36 -07:00
Bala FA
7d757033f2
erasure-readfile: Use chunk size to read from each disk for a block. ( #1949 )
...
A block of data is split into data chunk and each data chunk is
written to each disk. Previously block size was used to read data
chunk which returns corrupted data.
This patch fixes the issue by reading chunk sized data from each disk
and assembles a block.
Fixes #1939
2016-06-20 21:40:10 -07:00
Krishnan Parthasarathi
393c504de0
Renaming a part from tmp namespace needs to be handled different from… ( #1944 )
...
* Renaming a part from tmp namespace needs to be handled different from renaming of an object
* Made argument passing in xl.rename and xl.undoRename explicit
2016-06-20 19:11:55 -07:00
Krishnan Parthasarathi
6143c87c3a
Make ioErrCount updates go-routine safe ( #1943 )
...
* Make ioErrCount updates go-routine safe
* Made ioErrCount int32 instead of *int32
... and implemented StorageAPI on *posix as opposed to posix type.
This is consistent with the thumb-rule that if a value of a type is
modified as part of the interface implementation then we implement the
interface on pointer to that type.
2016-06-20 16:57:14 -07:00
Bala FA
8559e89494
XL: fix getBlockInfo() to return correct end block ( #1941 )
...
If requested offset/length of an object is equal to
erasureInfo.BlockSize, getBlockInfo() returns one more block added to
actual end block. This patch fixes the issue.
This patch also adds unit test for get objects with big files.
2016-06-20 14:23:25 -07:00
Krishna Srinivas
468ca4ccda
XL/Unittest: Add testcase for xlMetaV1{} and its methods. ( #1938 )
...
fixes #1822
2016-06-20 07:35:41 -07:00
Aakash Muttineni
4ee2136b28
Unit tests for PUT object when object already exists ( #1904 )
...
* fs/xl tests for multiple put object requests
* xl fix for put object on directory
* Unit tests fix windows test issue.
2016-06-20 06:18:47 -07:00
Yurii Rashkovskii
80d83220ad
INSTALLGO.md mentions Go 1.5+ for OS X ( #1936 )
...
However, current requirement is 1.6, so the file has been updated to reflect that.
2016-06-20 06:17:57 -07:00
Bala FA
fb10c09da7
posix-utils: remove unused isValidPath() ( #1937 )
2016-06-20 06:17:36 -07:00
Bala FA
2f136e92f7
posix: cleanup usage of fmt.Println() ( #1934 )
2016-06-19 18:52:19 -07:00
Harshavardhana
50d25ca94a
XL: Change AppendFile() to return only error ( #1932 )
...
AppendFile ensures that it appends the entire buffer. Returns
an error otherwise, this patch removes the necessity for the
caller to look for 'n' return on short writes.
Ref #1893
2016-06-19 15:31:13 -07:00
Harshavardhana
e1aad066c6
XL: CompleteMultipart should ignore last part is 0bytes. ( #1931 )
...
Fixes #1917
2016-06-19 14:51:20 -07:00
Bala FA
1ea1dba528
erasure-readfile: write to given Writer than returning buffer. ( #1910 )
...
Fixes #1889
2016-06-19 13:35:26 -07:00
Krishna Srinivas
c41bf26712
Unit tests: add unit tests for listv1/v2 for list bucket handler. ( #1933 )
...
fixes #1818
2016-06-19 13:33:00 -07:00
Harshavardhana
8c0942bf0d
XL: Remove usage of reduceErr and make it isQuorum verification. ( #1909 )
...
Fixes #1908
2016-06-18 00:27:51 +05:30
Harshavardhana
7f38f46e20
vendor: update klauspost/reedsomon package with upstream changes. ( #1912 )
2016-06-17 15:16:26 +05:30
Krishna Srinivas
466a2e01f1
XL/Erasure: Blocksize for object-part should be derived from what was decided during xl.NewMultipartUpload which creates xl.json. ( #1920 )
...
fixes #1919
2016-06-17 12:47:15 +05:30
Krishna Srinivas
d31b38aac8
XL/GetObject: pick the xl.json with highest version for metadata information. ( #1914 )
...
fixes #1913
2016-06-17 10:56:18 +05:30
Krishna Srinivas
365f80efa3
XL/DeleteObject: delete call on a prefix should not delete the entire tree structure. ( #1916 )
...
fixes #1915
2016-06-17 10:48:43 +05:30
Anand Babu (AB) Periasamy
f51d34cedd
Do not guess content-type for objects with no extension ( #1918 )
2016-06-17 10:12:02 +05:30
Krishnan Parthasarathi
129ebbd685
object layer: Send 200 OK and whitespace chars ( #1897 )
2016-06-16 09:01:06 +05:30
Krishna Srinivas
e2743d05e8
FS: remove .minio directory if .minio/multipart is empty. ( #1899 )
...
fixes #1886
2016-06-16 08:50:38 +05:30
Krishna Srinivas
de1c7d33eb
XL: appendFile should return error if quorum is not met. ( #1898 )
...
Fixes #1890
2016-06-15 00:24:49 +05:30
karthic rao
afc3102488
Adding format.json during FS initialization ( #1896 )
2016-06-14 14:09:40 +05:30
Harshavardhana
23c88ffb1d
XL/format: Fix a bug in checkDisksConsistency. ( #1894 )
2016-06-14 01:12:15 -07:00
Harshavardhana
ed4fe689b4
posix: Support UNC paths on windows. ( #1887 )
...
This allows us to now use 32K paths names on windows.
Fixes #1620
2016-06-13 02:53:09 -07:00
Harshavardhana
4ab57f7d60
server: terminal width should fallback to 80x25. ( #1895 )
...
Some environments might disable access to `/dev/tty`, fall
back to '80' in such scenarios.
Move to 'cheggaaa/pb' package for better cross platform
support on fetching terminal width.
Fixes #1891
2016-06-12 19:35:28 -07:00
karthic rao
276282957e
Test for Complete Multipart Upload. ( #1888 )
2016-06-10 18:43:16 +05:30
Harshavardhana
71632b375e
docs: Add comments for each data types. ( #1881 )
2016-06-09 06:24:11 -07:00
Aakash Muttineni
6f3bd76754
api: Add new bucket policy nesting error ( #1883 )
...
* Added ErrPolicyNesting which is returned when nesting of policies has occured
* Replaces ErrMalformedPolicy in the case of nesting
* Changed test case in bucket-policy-parser_test.go (ErrMalformedPolicy -> ErrPolicyNesting)
2016-06-09 01:53:56 -07:00
Bala FA
f2765d98a8
XL: set write quorum (no. of disk / 2) + 2 ( #1876 )
...
Previously write quorum was set to (no. of disk / 2) + 3. As per new
change, the write quorum is set to (no. of disk / 2) + 2. This helps
to accommodate one more failure of disk.
2016-06-08 22:12:36 -07:00
Bala FA
61598ed02f
posix: return errFaultyDisk on I/O errors. ( #1885 )
...
When I/O error is occured more than allowed limit, posix returns
errFaultyDisk.
Fixes #1884
2016-06-08 22:02:10 -07:00
Krishna Srinivas
1b9db9ee6c
FS/PutObject: Read() data should be handled even in case of EOF. ( #1864 )
...
Fixes #1710
2016-06-08 22:00:31 -07:00
Harshavardhana
51f3d4e0ca
XL/multipart: statPart should ignore errDiskNotFound. ( #1862 )
...
startPart should also take uploadId and partName as arguments.
2016-06-07 18:15:04 -07:00
Bala FA
d13e6e7156
XL: return error if DeleteObject() fails. ( #1878 )
...
Previously DeleteObject() does not return any error if write quorum is
not available. This patch fixes the issue by returning errors.
2016-06-07 11:35:03 -07:00
Bala FA
d32f3288f8
XL: return false only if given prefix doesn't exist in all disks ( #1877 )
...
Previously xl.isObject() returns false if one of the disk doesn't have
the object. Its possible that object may be present in another disk.
This patch fixes the issue by returning false only if given prefix
doesn't exist in all disks.
Fixes #1855
2016-06-07 11:02:12 -07:00
Harshavardhana
c5b6cb2420
Merge pull request #1867 from balamurugana/devel
...
cleanup: remove unused waitCloser.
2016-06-06 20:32:33 -07:00
Bala.FA
2eb6fa3fce
cleanup: remove unused waitCloser.
2016-06-07 07:38:18 +05:30
Krishna Srinivas
acc393ba8b
XL/tree-walk: Added comments, changed variable names and structure fields to improve code readability. ( #1856 )
2016-06-05 11:55:45 -07:00
Harshavardhana
37551a2ad3
Merge pull request #1857 from harshavardhana/erasure
...
erasure: Fix block index matching.
2016-06-05 09:35:31 -07:00
Harshavardhana
c6ac3fa6db
erasure: Fix block index matching.
...
This patch fixes an important issue of block reconstruction upon block corruption.
2016-06-05 06:19:58 -07:00
Harshavardhana
18b3871705
Add erasure code.
2016-06-03 12:50:36 -07:00
Harshavardhana
73ddb5be75
Merge pull request #1850 from harshavardhana/list-rewrite
...
XL: Implement ListObjects channel and pool management.
2016-06-03 12:30:06 -07:00
Krishna Srinivas
002c5bf7dd
XL: Treewalk handle all the race conditions and blocking channels.
2016-06-03 12:17:54 -07:00
Harshavardhana
1cf1532ca3
XL: Implement ListObjects channel and pool management.
2016-06-03 12:17:54 -07:00
Harshavardhana
70a1231f02
Merge pull request #1849 from harshavardhana/multipart
...
XL/PutObject: Handle all pending cases of DiskNotFound.
2016-06-03 11:58:04 -07:00
Harshavardhana
82fd907933
XL/PutObject: Handle all pending cases of DiskNotFound.
2016-06-03 11:40:44 -07:00
Harshavardhana
f39a6f8df7
Merge pull request #1852 from minio/harshavardhana-patch-1
...
Fix download link
2016-06-03 01:17:54 -07:00
Harshavardhana
f6013c46ea
Fix download link
2016-06-03 00:05:32 -07:00
Harshavardhana
da069a18c4
Merge pull request #1847 from krisis/patch-1
...
Created ISSUE_TEMPLATE with basic information
2016-06-03 00:01:48 -07:00
Harshavardhana
5108ba6eb1
Merge pull request #1728 from minio/rewrite-xl
...
XL/FS: Rewrite in new format.
2016-06-02 23:19:17 -07:00
Krishnan Parthasarathi
1213bf9fa1
Created ISSUE_TEMPLATE with basic information
...
Added an issued template to gather as much useful information that we may need to resolve an issue while the user has access to the relevant details.
2016-06-03 10:31:08 +05:30
Krishna Srinivas
b00ac40c35
XL/PutObject: Calculate size if not provided by the client and update xl.json with the correct size. ( #1844 )
2016-06-02 17:09:47 -07:00
Harshavardhana
fb95c1fad3
XL: Bring in some modularity into format verification and healing. ( #1832 )
2016-06-02 16:34:15 -07:00
Krishna Srinivas
aa1d769b1e
FS/Multipart: remove uploads.json on complete-multipart if no more uploadIDs are present for the object. ( #1843 )
...
Fixes #1835
2016-06-02 15:54:00 -07:00
Krishna Srinivas
611c892f8f
FS/Multipart: Lock() to avoid race during PutObjectPart. ( #1842 )
...
Fixes #1839
2016-06-02 15:19:13 -07:00
Harshavardhana
67bba270a0
FS: Cleanup and Fix all multipart related operations. ( #1836 )
2016-06-02 12:18:56 -07:00
Harshavardhana
de21126f7e
XL: Re-align the code again.
2016-06-02 01:54:06 -07:00
Harshavardhana
ae311aa53b
XL: Cleanup, comments and all the updated functions. ( #1830 )
2016-06-01 16:43:31 -07:00
Krishna Srinivas
9b79760dcf
XL/heal: heal missing format.json on replaced drives. ( #1828 )
...
fixes #1817
2016-06-01 16:15:56 -07:00
Bala FA
116b5607d7
server: fix to have readable timeout value ( #1823 )
2016-06-01 09:14:50 -07:00
Krishna Srinivas
614c770b5d
List Objects version 2. ( #1815 )
...
object: List Objects v2 support
2016-05-31 22:10:55 -07:00
Harshavardhana
c493ab5d0d
XL: Bring in sha512 checksum support. ( #1797 )
2016-05-31 20:23:31 -07:00
Bala FA
db2fdbf38d
erasure: allocate buffer only for non-nil disk ( #1811 )
2016-05-31 11:55:50 -07:00
Krishna Srinivas
89f65333fb
XL/Multipart: Introduce "deleted" field for uploads.json ( #1810 )
...
To future proof backend in case #1805 becomes an issue.
2016-05-31 11:54:01 -07:00
Krishna Srinivas
22511dc4c7
XL/Multipart: During list-multipart-uploads ignore errFileNotFound and errDiskNotFound errors. ( #1813 )
...
Fixes #1795
2016-05-31 11:53:28 -07:00
karthic rao
1947ae198e
Adding read nad write timeout for unresponsive client connectinos ( #1809 )
2016-05-31 11:53:21 -07:00
Harshavardhana
2e4ab71303
Web: Update with ui changes. ( #1808 )
2016-05-31 02:01:02 -07:00
Harshavardhana
445dc22118
XL: Cleanup and add more comments. ( #1807 )
2016-05-30 16:51:59 -07:00
karthic rao
ffc2b3c304
Test for ListObjectParts. ( #1802 )
2016-05-30 14:36:33 -07:00
Krishnan Parthasarathi
967c2b2940
Handled possible short writes to httpResponseWriter ( #1804 )
...
* XL: Handled possible short writes to httpResponseWriter
* Added tests for Range Header combinations
2016-05-30 11:27:15 -07:00
Krishna Srinivas
b466f27705
Nslock fixes ( #1803 )
...
* XL/Multipart: Support parallel upload of parts by doing NS locking appropriately.
* XL/Multipart: hold lock on the multipart upload while aborting.
2016-05-30 11:26:10 -07:00
Harshavardhana
a4a0ea605b
XL: Fix GetObject erasure decode issues. ( #1793 )
2016-05-29 15:38:14 -07:00
Harshavardhana
5e8de786b3
XL: Truly use unique id's in temp directory. ( #1790 )
...
This also helps in avoiding cleaning up directories after.
Additionally this patch also fixes the problem of Range offsets.
2016-05-29 00:42:09 -07:00
Harshavardhana
feb337098d
XL: bring in new storage API. ( #1780 )
...
Fixes #1771
2016-05-28 16:12:51 -07:00
Krishnan Parthasarathi
c87f259820
Remove parts that are missing in CompleteMultipartUpload ( #1786 )
...
* Remove parts that are missing in CompleteMultipartUpload
* Moved isUploadIDExists under proper namespace locks
* Moved code that deletes part files to a function
2016-05-28 15:15:53 -07:00
karthic rao
7278b90fe1
Adding defer to the lock ( #1785 )
2016-05-28 15:15:53 -07:00
Krishna Srinivas
41a5b3908b
XL/ListParts: take the size from xl.json instead of backend file size as it will be different. ( #1781 )
...
Fixes #1779
2016-05-28 15:15:53 -07:00
Krishna Srinivas
3fb0b5e455
XL/Multipart: check existance upload uploadID after lock. ( #1778 )
...
Fixes #1767
2016-05-28 15:15:53 -07:00
Harshavardhana
ba8bdec077
XL: ListObjects should not list when delimiter and prefix are '/'. ( #1777 )
2016-05-28 15:15:53 -07:00
Harshavardhana
27cc8a6529
erasure: read only dataBlocks if we have enough. ( #1776 )
...
Reconstruct with parity blocks if we don't have enough data blocks.
2016-05-28 15:15:53 -07:00
Krishnan Parthasarathi
302ec27fa2
Fixed race during parallel PutObjectPart requests ( #1775 )
...
The race is between two parallel PutObjectPart requests updating partsInfo in xl.json.
Previously, it was being updated under a RLock().
2016-05-28 15:15:53 -07:00
Krishnan Parthasarathi
5f679d9d1e
Rename back multipart objects if read/write Quorum was unavailable ( #1773 )
2016-05-28 15:15:53 -07:00
Bala FA
51bb613fdf
pkg/safe: remove temporary file on failure ( #1774 )
2016-05-28 15:15:53 -07:00
Harshavardhana
d65101a8c8
XL: Implement strided erasure distribution. ( #1772 )
...
Strided erasure distribution uses a new randomized
block distribution for each Put operation. This
information is captured inside `xl.json` for subsequent
Get operations.
2016-05-28 15:15:53 -07:00
Krishna Srinivas
6dc8323684
FS/ListMultipart: Fix FS list-multipart to work for unit test cases.
2016-05-28 15:15:53 -07:00
Krishna Srinivas
616a257bfa
XL/Multipart: isMultipartUpload() checks for presence of uploads.json on a random disk.
2016-05-28 15:15:53 -07:00
Krishna Srinivas
3487b3c095
Multipart: Disable FS tests and certain test cases for list-incomplete-uploads.
2016-05-28 15:15:53 -07:00
Karthic Rao
1f51af6f37
Listmultipart tests.
2016-05-28 15:15:53 -07:00
Krishna Srinivas
b1e2b7dea2
Fix list-incomplete uploads for XL.
2016-05-28 15:15:53 -07:00
Harshavardhana
34e9ad24aa
XL: Introduce new API StorageInfo. ( #1770 )
...
This is necessary for calculating the total storage
capacity from object layer. This value is also needed for
browser UI.
Buckets used to carry this information, this patch
deprecates this feature.
2016-05-28 15:15:53 -07:00
Harshavardhana
b2293c2bf4
XL: Rename, cleanup and add more comments. ( #1769 )
...
- xl-v1-bucket.go - removes a whole bunch of code.
- {xl-v1,fs-v1}-metadata.go - add a lot of comments and rename functions
appropriately.
2016-05-28 15:15:53 -07:00
Harshavardhana
553fdb9211
XL: Bring in support for object versions written during writeQuorum. ( #1762 )
...
Erasure is initialized as needed depending on the quorum and onlineDisks.
This way we can manage the quorum at the object layer.
2016-05-28 15:15:53 -07:00
Harshavardhana
cae4782973
XL: explicit deleteObject is not needed after rename failure. ( #1760 )
...
Reason is renameObject() does deleteObject() upon writeQuorum
failure if not keeps the successfully renamed parts if we have
reached readQuorum.
2016-05-28 15:15:53 -07:00
Krishnan Parthasarathi
3550660163
Return error for empty parts in multipartupload complete ( #1758 )
2016-05-28 15:15:53 -07:00
Harshavardhana
a4771265cf
XL: Abortmultipart should update uploads.json properly. ( #1757 )
2016-05-28 15:15:53 -07:00
Harshavardhana
a9e778f460
XL/fs: initObjectLayer should cleanup tmpMetaPrefix in parallel. ( #1752 )
...
Fixes #1747
2016-05-28 15:15:53 -07:00
Harshavardhana
ee6645f421
XL: Add additional PartNumber variable as part of xl.json ( #1750 )
...
This is needed for verification of incoming parts and to
support variadic part uploads. Which should be sorted
properly.
Fixes #1740
2016-05-28 15:15:53 -07:00
Harshavardhana
a97230dd56
XL/erasure: Reset dataBlocks to reduce the memory usage. ( #1749 )
...
Fixes #1748
2016-05-28 15:15:53 -07:00
Harshavardhana
1e393c6c5b
XL: Add new metadata for checksum. ( #1743 )
2016-05-28 15:15:53 -07:00
Krishna Srinivas
b38b9fea79
XL/erasure: fix for skipping 0 padding. ( #1737 )
...
Fixes #1736
2016-05-28 15:15:53 -07:00
Krishna Srinivas
6d84e84b3c
XL/mutltipart: fix partnumber to partname association. ( #1739 )
...
Fixes #1738
2016-05-28 15:15:53 -07:00
Harshavardhana
a00a5c6e7e
XL: Multipart update uploads.json properly. ( #1741 )
2016-05-28 15:15:53 -07:00
Harshavardhana
ed43d5e02b
No need to delete file inside erasure code ( #1732 )
2016-05-28 15:15:53 -07:00
Harshavardhana
293d246f95
XL/FS: Rewrite in new format.
2016-05-28 15:15:53 -07:00
Anand Babu (AB) Periasamy
63c65b4635
filter GOPATH from stack trace ( #1755 )
2016-05-25 02:32:35 -07:00
Harshavardhana
64b0976e1b
Remove probe and tasker. ( #1733 )
...
Fixes #1717
2016-05-24 18:43:33 -07:00
Aakash Muttineni
b48b2e7f7c
Part ID check ( #1730 )
...
* Added check in PutObjectPartHandler to make sure part ID does not exceed 10000. ErrInvalidMaxParts written to response if part ID exceeds the maximum value.
2016-05-24 01:52:47 -07:00
Krishnan Parthasarathi
584813e214
Used MINIO_PROFILE_DIR for saving profile information of a minio server ( #1722 )
...
To specify the directory where profiling information should be saved
```
export MINIO_PROFILE_DIR=/path/to/profile/dir
```
By default, profiling information would be saved in a directory created
using ioutil.TempDir, which would be displayed in stdout on starting the
minio server.
2016-05-22 22:11:39 -07:00
Krishnan Parthasarathi
3a980eac1a
Fix shadowing of err variable ( #1718 )
2016-05-21 00:43:47 -07:00
Harshavardhana
5a4b074ca0
XL: PutObject incorrectly returned after deleting multipart object. ( #1715 )
...
Fixes #1714
2016-05-20 17:27:48 -07:00
Harshavardhana
f76d975304
xl: StatVol and ListVols should handle cases when disks are missing. ( #1703 )
...
Remove unnecessary code, handle cases of missing disk.
2016-05-20 16:45:53 -07:00
Harshavardhana
6015a7a3cd
Add mention-bot config
2016-05-20 13:53:15 -07:00
Krishna Srinivas
b83b87a7f6
XL/Incompleteuploads: list should save the tree-walk go routine to the map if eof is not reached. ( #1695 )
2016-05-20 12:09:21 -07:00
Krishna Srinivas
5b95f097d4
multipart: listing does not skip uploadIDmarker. ( #1708 )
...
Fixes #1706
2016-05-20 11:48:28 -07:00
Harshavardhana
e4240aa58f
XL/objects: Initialize format.json outside of erasure. ( #1640 )
...
Fixes #1636
New format now generates a UUID and includes it along with
the order of disks. So that UUID is the real order of disks
and on command line user is able to specify disks in any order.
This pre-dominantly solves our dilemma.
```
{
"format" : "xl",
"xl" : {
"version" : "1",
"disk": "00e4cf06-5bf5-4bb5-b885-4b2fff4a7959",
"jbod" : [
"00e4cf06-5bf5-4bb5-b885-4b2fff4a7959",
....
"c47d2608-5067-4ed7-b1e4-fb81bdbb549f",
"a543293e-99f1-4310-b540-1e450878e844",
"18f97cbe-529a-456a-b6d4-0feacf64534d"
]
},
"version" : "1"
}
```
2016-05-20 02:22:22 -07:00
Andreas Linz
f5dfa895a5
Exit with error code if minio server fails to start ( #1704 )
...
This commit replaces the call to `errorIf` with `fatalIf`, so that the
minio server exits with an non-zero exit status if something fails, e.g.
the port was already openend by another process.
2016-05-20 02:05:52 -07:00
Harshavardhana
50c328ff19
XL: RenameFile should rename and cleanup on writeQuorum. ( #1702 )
...
Fixes #1683
2016-05-20 01:56:46 -07:00
Anand Babu (AB) Periasamy
b8405ca172
simplify profiler cleanup
2016-05-19 19:19:32 -07:00
Harshavardhana
f6d9e73548
posix: Do not lowercase names, return as is. Object layer will filter them out. ( #1699 )
2016-05-19 18:52:55 -07:00
Harshavardhana
7ae5470395
XL: simplify isMultipartObject not need to handle unknown errors. ( #1686 )
...
Unknown errors are just logged with errorIf.
2016-05-19 17:10:33 -07:00
Harshavardhana
9fdb69563d
handler: CopyObject should save metadata. ( #1698 )
...
- Content-Type
- Content-Encoding
- ETag
Fixes #1682
2016-05-19 17:10:08 -07:00
Harshavardhana
00d0558131
XL: Enable tests for content-type for Head and Get
...
Fixes #1674
2016-05-19 15:34:18 -07:00
Bala.FA
13e4618309
XL/fs: Return saved content-type during GetObject
...
Fixes #1674
2016-05-19 15:34:18 -07:00
Krishnan Parthasarathi
2f05aacbf2
Stop profiling on exit of main goroutine ( #1670 )
...
* Stop profiling on exit of main goroutine
Previously, profiling was stopped since Stop() method was called on exit of cli.BeforeFunc.
This lead to profiling to be stopped prematurely.
* Moved profiling switch statement to a separate func
2016-05-19 14:50:54 -07:00
Krishna Srinivas
dc36594ef4
XL/heal: Should skip healing if CreateFile() failed on the part which needed healing. ( #1693 )
...
Fixes #1684
2016-05-19 11:32:47 -07:00
Krishna Srinivas
537568f9f9
XL/ListVols: Fix panic. Skip if slice is nil. ( #1694 )
...
Fixes #1692
2016-05-19 11:32:19 -07:00
Harshavardhana
62b4fd6964
XL: Close the reader properly.
2016-05-18 20:16:19 -07:00
Harshavardhana
7d6ed50fc2
objects: Save all the incoming metadata properly. ( #1688 )
...
For both multipart and single put operation
2016-05-18 19:54:25 -07:00
Frank
af85acf388
Added ETag as an exposed header (required for multi part uploads) ( #1681 )
...
* Added ETag as an exposed header (required for multi part uploads)
* Fix formatting on adding ETag for exposed headers
2016-05-18 19:17:32 -07:00
Frank
a4fef436c8
Fix formatting for adding ETag for exposed headers ( #1687 )
2016-05-18 19:17:17 -07:00
Harshavardhana
404364ba73
XL/fs: ListMultipartUploads should list only requested entries. ( #1668 )
...
Fixes #1665
2016-05-18 15:06:29 -07:00
Krishna Srinivas
3c1ef3fee2
Locking: move locking code from xl-erasure to xl-objects. ( #1673 )
...
Fixes #1639 #1649 #1650 #1624
2016-05-18 15:05:23 -07:00
Bala FA
a0865122a7
XL/objectLayer: Save additional meta data during PutObject ( #1672 )
...
Fixes #1602
2016-05-18 13:56:11 -07:00
Krishna Srinivas
824c8a39f1
XL/Multipart: If the part is already uploaded ignore the newly uploaded part. ( #1677 )
...
Fixes #1644
2016-05-18 13:37:28 -07:00
Krishna Srinivas
71b9341fc7
XL/Multipart: Cleanup uploads.json after abort-multipart-upload. ( #1678 )
...
Fixes #1663
2016-05-18 13:30:58 -07:00
Harshavardhana
b69a97aed4
server: Set rLimit properly to the max. ( #1676 )
...
4000 is too small to handle 500 go-routines.
Fixes #1666
2016-05-18 11:34:24 -07:00
karthic rao
2da34e4668
vendor changes to pkg/profile ( #1671 )
2016-05-18 09:22:06 -07:00
Harshavardhana
4bc923e63b
XL/fs: Optimize calling isBucketExist() ( #1656 )
...
* posix: Avoid using getAllVolumeInfo() in getVolumeDir()
This is necessary compromise to avoid significant slowness this
causes under load. The compromise is also substantial in a way
so that to avoid penalizing common cases v/s special cases.
For buckets with Caps on Unixes, we filter buckets based on the
latest anyways, so this is completely acceptable.
* XL/fs: Change the usage of verification of existance of buckets.
Optimize calling isBucketExists, it is not needed for all call
paths. isBucketExist should be called only for calls which use
temporary volume location for operations, for the rest rely on
the errors returned on their original call path.
Remove usage of filtering as well across all volume names.
2016-05-17 21:22:27 -07:00
Harshavardhana
4214da65af
XL/fs: MakeVol replies should be consistent. ( #1667 )
...
Fixes #1658
Fixes #1633
2016-05-17 18:41:17 -07:00
Krishnan Parthasarathi
596fe65e84
Write pprof output files under config dir supplied ( #1660 )
...
Since config dir, supplied as command line argument, is parsed after pprof
output directory is determined, pprof output files are written in ~/.minio/profile
directory instead of <configDir>/profile/. This change fixes this behaviour.
2016-05-17 11:44:40 -07:00
Krishna Srinivas
39865c0d2e
XL/Multipart: Fix list multipart output. ( #1661 )
...
Fixes #1541
2016-05-17 11:44:32 -07:00
Harshavardhana
1760687c83
XL: Make sure to create proper temporary files for renames to succeed. ( #1654 )
...
Renames work in a special manner, temporary location files should
be created properly.
Fixes #1653
Fixes #1651
2016-05-16 15:40:57 -07:00
Harshavardhana
9472299308
logging: Log only for unhandled errors, remove all the debug logging. ( #1652 )
...
This patch brings in the removal of debug logging altogether, instead
we bring in the functionality of being able to trace the errors properly
pointing back to the origination of the problem.
To enable tracing you need to enable "MINIO_TRACE" set to "1" or "true"
environment variable which would print back traces whenever there is an
error which is unhandled or at the handler layer.
By default this tracing is turned off and only user level logging is
provided.
2016-05-16 14:31:28 -07:00
Harshavardhana
8828fd1e5c
vendor: Remove unused packages.
2016-05-15 00:04:56 -07:00
Harshavardhana
7de206cb85
XL: ListVols should provide consistent view. ( #1648 )
...
Additionally get list of all volumes in parallel for aggregation
and quorum verification.
Fixes #1647
2016-05-14 23:57:57 -07:00
Harshavardhana
498ce1e9bb
handler: Add a waitgroup to avoid expect100Continue crash. ( #1623 )
...
This waitgroup allows for safe blocking operation where we can cleanly
control the flow of the writes and the underlying pipe altogether.
Fixes #1553
2016-05-14 17:18:00 -07:00
Harshavardhana
5b29cefd40
api: DeleteObject should always return 204. ( #1645 )
...
Fixes #1643
2016-05-14 15:47:19 -07:00
Hori Ryota
e03ebfd13b
Add default cmd ( #1625 )
2016-05-14 02:58:50 -07:00
Harshavardhana
74c23a3544
docs: Move developer docs from top-level to its own directory. ( #1642 )
2016-05-14 02:47:16 -07:00
Harshavardhana
025054fb36
XL: CreateFile/ReadFile should write and read from all disks in parallel. ( #1612 )
...
* XL: CreateFile should write to all disks in parallel.
* XL: ReadFile should read from all disks in parallel.
2016-05-14 01:57:04 -07:00
Bala FA
7264cd2ab3
Fix error message when wrong set of disks are passed ( #1634 )
...
Previously when wrong set of disks are given with last minio server
run, it throws unclear error message. This is fixed by returning
appropriate errors.
Fixes #1591
2016-05-13 23:04:10 -07:00
Harshavardhana
0e4e9c4bc1
XL: ListDir should return each List from a random disk in the set. ( #1613 )
...
Fixes #1609
2016-05-13 18:12:26 -07:00
Krishna Srinivas
8099396ff0
xl/putObject: Should take care of the situation if an object already exists at the location. ( #1606 )
...
Fixes #1598 #1594 #1595
2016-05-13 11:52:36 -07:00
Krishna Srinivas
d267696110
Validation: Reject object names with trailing "/". ( #1619 )
...
Fixes #1616
2016-05-13 11:43:06 -07:00
Bala FA
43539a0c86
posix: parseDirents() should follow symlink and get values. ( #1631 )
...
Previously parseDirents() ignores symbolic links. This patch fixes
the issue by following the symlink using os.Stat().
Fixes #1545
2016-05-13 11:39:48 -07:00
Krishnan Parthasarathi
9e45d138cc
Closed readCloser for each multipart in xl.GetObject ( #1629 )
...
* Closed readCloser for each multipart in xl.GetObject
2016-05-13 04:50:13 -07:00
karthic rao
ee8605e333
Make bucket failure fix with high concurrent load ( #1630 )
2016-05-13 04:03:38 -07:00
karthic rao
e4958f9757
Removing regexp check and adding string based check, regexp check was unnecessary here ( #1627 )
2016-05-13 03:33:53 -07:00
Harshavardhana
b62774d32f
storage/xl: Return errVolumeAccessDenied if disks cannot be accessed. ( #1621 )
...
Fixes #1614
2016-05-12 21:01:11 -07:00
koolhead17
d6e0f3ab33
added awscli commands & minor typo fix. ( #1578 )
2016-05-12 16:35:11 -07:00
Bala FA
3ff0a56e62
XL: Ignore errDiskNotFound in certain situations ( #1610 )
...
When a disk is removed while an operation is going on
(eg. single/multipart put object, list/multipart list objects etc),
its required to ignore errDiskNotFound error and continue the
operation.
Fixes #1552
2016-05-11 23:42:14 -07:00
Harshavardhana
50431e91a6
erasure: Handle failed disks so that we initialize properly if they are missing. ( #1607 )
...
Fixes #1592
Fixes #1579
2016-05-11 18:58:32 -07:00
Harshavardhana
d4745c7d6a
object: PutObjectHandler should set the md5Sum properly. ( #1604 )
...
Additionally add a test case as well for validating for us
to reply BadDigest properly.
Fixes #1603
2016-05-11 16:13:37 -07:00
Bala FA
adbcafefad
xl/CreateFile: handle errFileNameTooLong error properly ( #1523 )
...
When errFileNameTooLong error is returned from posix, xl.CreateFile()
treats the error specially by returning the same error immediately.
Fixes #1501
2016-05-11 12:55:02 -07:00
Harshavardhana
86e5d71519
erasure: MakeVol, DeleteVol and StatVol should hold locks. ( #1597 )
...
Since there is a good amount of overlap, each code has to lock
properly for the operation they are going to perform.
- MakeVol create vols in a routine on all disks, hold locks.
- DeleteVol delete vols in a routine on all disks, hold locks.
- StatVol stat vols in a routine on all disks, hold locks.
Fixes #1588
2016-05-11 12:54:21 -07:00
Harshavardhana
72748d2073
erasure: healVolume err should be different from shadowed version. ( #1590 )
...
Multiple go-routines updating the same shadowed variable can
cause a data race, avoid it by using its own err variable.
Fixes #1589
2016-05-11 01:36:09 -07:00
Harshavardhana
49141eb3e4
http: Remove minhttp package and use standard Golang http. ( #1587 )
...
The functionality provided by minhttp will be implemented
cleanly through our own APIs. Since we are not going to
send SIGUSR2 and manage configuration in that manner, it
doesn't make sense to use minhttp.
Fixes #1586
2016-05-10 18:03:00 -07:00
Harshavardhana
d1fa1d9352
Remove binary files from previous commit.
2016-05-10 15:49:17 -07:00
karthic rao
26e2c4bf4d
Replacing fastsha256 with crypto/sha256 package from golang standard package ( #1584 )
2016-05-10 14:20:11 -07:00
Krishna Srinivas
b044336329
XL/GetObject: If the offset does not fall in the first "dataBlock" it gives incorrect data. ( #1583 )
...
Fixes #1582
2016-05-10 11:38:49 -07:00
Krishna Srinivas
e99cb05516
XL/GetObject: offset should be reset to 0 after reading first part. ( #1580 ) ( #1581 )
2016-05-10 10:38:12 -07:00
Krishna Srinivas
409e09c1e5
XL/Selfheal: skip reading from disk if ReadFile had returned error. ( #1575 )
2016-05-10 01:24:58 -07:00
Krishna Srinivas
c314a98c1a
XL/list: fix panic on list when a disk is down. ( #1562 )
2016-05-10 00:35:29 -07:00
Harshavardhana
5f0ca64346
erasure: listOnlineDisks should return errFileNotFound for errReadQuorum. ( #1573 )
...
Fixes #1571
2016-05-10 00:10:34 -07:00
Harshavardhana
0db3218d5d
xl: getPartsMetadata fetch parts and decode in go-routine. ( #1569 )
...
Ref #1516
2016-05-09 23:51:05 -07:00
Harshavardhana
eec41c369c
posix: Return diskNotFound error rather than errVolumeNotFound ( #1568 )
...
Fixes #1559
2016-05-09 18:57:39 -07:00
Harshavardhana
b66c3bf35e
server: Enable server profiling as needed. ( #1565 )
2016-05-09 16:18:56 -07:00
Harshavardhana
f733120d3d
xl: CompleteMultipartUpload make sure to delete uploads.json ( #1539 )
...
Fixes #1537
Ref #1540 - for missing functionality in this patch.
2016-05-09 12:09:48 -07:00
Krishna Srinivas
6627388dc3
posix: remove dead code related to posix reserved suffixes. ( #1555 )
2016-05-09 11:40:30 -07:00
Harshavardhana
9d41414fb5
posix: reserved files should be filtered out at posix not object layer. ( #1554 )
2016-05-09 02:53:08 -07:00
Harshavardhana
722abe2d0f
xl/fs: pathJoin now takes variadic inputs. ( #1550 )
...
Retains slash for the last element.
Fixes #1546
2016-05-09 00:46:54 -07:00
Krishna Srinivas
04a5b25929
Multipart: Minimum part size limit does not apply to the last part during CompleteMultipartUpload. ( #1518 ) ( #1538 )
2016-05-08 23:49:49 -07:00
Harshavardhana
90ea494338
erasure: waitCloser should implement CloseWithError. ( #1543 )
...
This is needed so that the other end of the pipe receives
and error, cleanups temporary files.
2016-05-08 16:26:10 -07:00
Harshavardhana
a8fdd04e62
erasure: ReadFile should honor proper offsets. ( #1542 )
...
Fixes #1535
2016-05-08 15:39:24 -07:00
Harshavardhana
76c511c9fe
api: Extend S3 errors with Minio errors. ( #1533 )
...
Fixes #1530
2016-05-08 12:36:16 -07:00
Krishna Srinivas
75320f70d0
multipart: reject part upload if size is less than 5MB. ( #1518 )
2016-05-08 12:06:05 -07:00
Krishna Srinivas
88e1c04259
XL/ListDir: break out of loop if list on one disk is a success. ( #1534 )
2016-05-08 12:05:19 -07:00
Krishna Srinivas
a205aca6d2
init: Cleanup .minio/tmp directories recursively. Also takes care of cleaning up of parts directory during abortMultipartUpload. ( #1532 )
2016-05-08 10:15:34 -07:00
Harshavardhana
3f51dd4fd4
xl: CompleteMultipartUpload should rename files in a routine. ( #1527 )
...
This solves the client timeout while renaming 9000+ parts.
Fixes #1526
2016-05-08 02:38:35 -07:00
Harshavardhana
56b7df90e1
xl/fs: ListObjectParts should set nextPartNumberMarker properly. ( #1528 )
...
For list requests on parts more than 1000, would lead to an infinite
loop.
Fixes #1522
2016-05-08 02:21:12 -07:00
Harshavardhana
a56d5ef415
xl/fs: isFunctions should only return boolean. ( #1525 )
...
log the unrecognize errors.
2016-05-08 01:58:05 -07:00
Harshavardhana
937d68202d
server: Deadcode removal. ( #1517 )
2016-05-07 21:47:33 -07:00
Harshavardhana
bf563afb80
xl: DeleteObject regression over errChannel. ( #1521 )
...
DeleteObject would hang indefinitely - fix it.
2016-05-07 12:48:12 -07:00
Harshavardhana
091c1e8456
copyObject: No need to verify md5sum. ( #1520 )
...
Multipart objects are kept in non hex md5sum format.
This format doesn't comply with hex, so decoding
would fail invariably.
This is not necessary to validate and its not expected
error during a CopyObject operation.
Fixes #1519
2016-05-07 03:43:08 -07:00
Harshavardhana
751fa972f5
xl/fs: Multipart re-org introduce "uploads.json" ( #1505 )
...
Fixes #1457
2016-05-07 02:08:03 -07:00
Harshavardhana
434423de89
xl: Move format detection inside xl objects. ( #1515 )
...
Fixes #1449
2016-05-07 00:59:43 -07:00
Harshavardhana
a20ccb1e83
server: Print proper endpoint, along with https if configured. ( #1514 )
...
Fixes #1492
2016-05-06 21:18:29 -07:00
Harshavardhana
0625c050e6
xl/tests: Enable server handler tests over XL. ( #1512 )
...
Fixes #1513
2016-05-06 16:47:23 -07:00
Harshavardhana
0b74f5624e
xl: Fix how we deal with read offsets at erasure layer. ( #1511 )
...
Requires skipping necessary parts of dataBlocks during
decoding phase and requires us to properly skip the
entries as needed.
Thanks to Karthic for reproducing this important issue.
Fixes #1503
2016-05-06 16:25:08 -07:00
Krishna Srinivas
c06b9abc15
bucket-handlers: do not unescape marker as gorilla layer would have already done it. ( #1495 ) ( #1510 )
2016-05-06 16:04:46 -07:00
Krishna Srinivas
a5d31d4254
XL/ListObjects: use string.TrimSuffix instead of Trim. ( #1498 ) ( #1509 )
2016-05-06 13:28:55 -07:00
karthic rao
20ca65c793
Cleanup: mispell fixes
2016-05-06 12:32:44 -07:00
karthic rao
0b4bbe6d9e
Adding XL Object layer validation for existing unit tests of single node ( #1499 )
...
object layer.
Adding isBucketExist check for GetObjectInfo in the XL layer.
2016-05-06 11:57:04 -07:00
Krishna Srinivas
48d3be36da
XL/ListObjects: Fix ordering issue during listing if the files were uploaded as multipart uploads. ( #1498 ) ( #1506 )
...
i.e if two files "tmp" and "tmp.1" are uploaded as multipart we would list ""tmp.1"" before ""tmp"" as "tmp.1/" < "tmp/"
2016-05-06 10:19:09 -07:00
Harshavardhana
5133ea50bd
xl/fs: Make i/o operations atomic. ( #1496 )
2016-05-05 20:28:22 -07:00
Harshavardhana
17868ccd7f
handlers: overhaul entire writErrorResponse, simplify. ( #1472 )
2016-05-05 20:24:29 -07:00
Harshavardhana
ba5805e60a
bucketPolicy: Do not use regexes, just do prefix matches. ( #1497 )
...
AWS arn supports wildcards and this is flat namespace, simple
prefix matching is fine.
Fixes #1481
Fixes #1482
2016-05-05 19:58:48 -07:00
Harshavardhana
ca097de96c
xl/fs: Add initObjectLayer function. ( #1494 )
...
Fixes #1493
2016-05-05 15:00:03 -07:00
Bala FA
658a595d7a
xl-erasure: RenameFile should support quorum. ( #1487 )
...
Fixes #1463
2016-05-05 13:03:53 -07:00
Krishna Srinivas
247e835d7b
object: move go-routine listing from posix to objectLayer. ( #1491 )
2016-05-05 12:51:56 -07:00
Harshavardhana
46680788f9
xl/fs: cleanup '/.minio/tmp' directory on each initialization. ( #1490 )
2016-05-05 01:54:43 -07:00
Harshavardhana
ad40036cba
posix: filepath shouldn't be used anymore use path.Join ( #1486 )
2016-05-05 01:39:26 -07:00
Harshavardhana
82fbe908a3
object: DeleteBucket should return proper error for BucketNotEmpty. ( #1489 )
...
Fixes #1488
2016-05-05 01:35:21 -07:00
Harshavardhana
f145e1042f
quick/config: No need to use Data() with type assertion. ( #1480 )
...
Since input to quick.New() is a pointer the unmarshalled value
internally already has the value, subsequent type assertions
are not needed.
Thanks to Bala for finding this behavior.
Fixes #1475
2016-05-04 20:22:15 -07:00
Rajiv Makhijani
9dccbd6478
New Dockerfile for building & running minio inside Docker (inc. autobuild support) ( #1473 ) ( #1485 )
2016-05-04 17:07:19 -07:00
karthic rao
82113b747c
Resource matching fix to overcome issues with regular expression based match ( #1476 )
2016-05-04 16:56:57 -07:00
Rajiv Makhijani
a5959789d5
Make minimum file space percent a constant ( #1484 )
2016-05-04 15:39:06 -07:00
Harshavardhana
6988ed9257
xl/getObjectInfo: Returns back proper size, modTime and md5Sum. ( #1479 )
...
Fixes #1469
2016-05-04 15:28:58 -07:00
Rajiv Makhijani
321aefa026
Add Response for PostPolicyBucketHandler ( #1477 ) ( #1483 )
2016-05-04 15:24:10 -07:00
Harshavardhana
dd417e5476
fs: Handle cases of PutObject for an existing prefix. ( #1478 )
2016-05-04 12:18:40 -07:00
Bala FA
da3a53376c
server: save and compare multiple disks are used ( #1474 )
...
When server is run with multiple disks which uses xl interface where
order and count of disks are important, this patch saves such disks
configuration and compares in next run if there is a mismatch.
Fixes #1458
2016-05-04 12:18:20 -07:00
Harshavardhana
e4d89d8156
xl/deleteObject: Support deleting special multipart object. ( #1470 )
...
Fixes #1452
2016-05-03 17:52:54 -07:00
Harshavardhana
d0e854afb7
xl/fs: Bring in ".minio/tmp" directory support. ( #1464 )
...
All transactions happen through this directory inside ".minio/temp".
Adding this allows us to remove any temporary files which were not
committed before.
Fixes #1462
Fixes #1444
2016-05-03 16:10:24 -07:00
Harshavardhana
6f1811ee4d
config: Migration should save region properly. ( #1468 )
...
Fixes #1466
2016-05-03 15:17:58 -07:00
Yurii
bba5468368
minio: Replace 'bucket already exists' error by 'bucket already owned by you'. ( #1465 )
...
S3 API returns BucketAlreadyExists error when some another user has such bucket.
If user that creates the bucket already has it, s3 returns BucketAlreadyOwnedByYou.
As minio has only one user, it should behave accordingly.
Otherwise it causes failures in the applications that ignore creation of already existing bucket in the account, but fail when bucket name is used by someone else.
2016-05-03 03:19:04 -07:00
Harshavardhana
7ae40eb1bb
minhttp: Remove probe usage, move to golang error. ( #1459 )
...
Fixes #1454
2016-05-03 01:07:34 -07:00
Harshavardhana
ad8e27a963
xl: Rename 'xl.json' to 'file.json' ( #1461 )
...
Fixes #1460
2016-05-02 17:42:01 -07:00
Harshavardhana
ac7a7cec20
bucket-policy: Delete policy should remove policy properly. ( #1456 )
2016-05-02 16:58:10 -07:00
Harshavardhana
afd59c45dc
xl/fs: Move few functions into common code. ( #1453 )
...
- PutObject()
- PutObjectPart()
- NewMultipartUpload()
- AbortMultipartUpload()
Implementations across both FS and XL object layer
share common implementation.
2016-05-02 16:57:31 -07:00
Harshavardhana
3bf3d18f1f
rpc/client: Implement RenameFile properly. ( #1443 )
2016-05-02 03:12:18 -07:00
Harshavardhana
8102a4712a
xl/metadata: Keep the json erasure tag consistent. ( #1447 )
...
Currently the on-disk json has "Erasure" we should
keep it consistent name and move to lower case instead.
2016-05-02 01:13:07 -07:00
karthic rao
2393a3a0be
XL non-recursive fix ( #1450 )
2016-05-01 23:16:44 -07:00
Harshavardhana
d006129fb5
xl/vol: Add healing and quorum support for StatVol, MakeVol.
...
Fixes #1437
2016-05-01 18:42:00 -07:00
Harshavardhana
7caa82f32f
xl/fs: Rename minioMetaVolume to minioMetaBucket. ( #1442 )
2016-05-01 18:13:10 -07:00
Krishna Srinivas
286de4de2c
XL - fixes mostly related to multipart listing. ( #1441 )
...
* XL/Multipart: Use json.NewDecoder to decode read stream.
* XL/Multipart: fix recursive and non-recursive listing.
* XL/Multipart: Create object part with md5sum later using RenameFile.
* XL/Multipart: ListObjectParts should list parts in order.
previously: uploadID.10.md5sum < uploadID.2.md5sum
fix : uploadID.00010.md5sum > uploadID.00002.md5sum
* XL/Multipart: Keep the size of each part in the multipart metadata file to avoid stats on the parts.
* XL/Multipart: fix listing bug which was showing size of the multipart uploaded objects as 0.
2016-05-01 17:52:16 -07:00
Harshavardhana
ba7a55c321
xl: ReedSolomon code fix small file erasure bug. ( #1431 )
...
For files less than 'dataBlocks', erasure encoding would fail
with short data due to a bug in the implementation itself.
Relax the error return, even a single byte can be properly
erasure coded without issues.
Fixes #1413
2016-05-01 15:30:40 -07:00
Harshavardhana
e05aa762a9
fs: Create object part with md5sum later using RenameFile. ( #1440 )
...
Fixes #1340 .
2016-05-01 14:50:30 -07:00
Krishna Srinivas
0c27d8e5b1
XL/Multipart: maintain the parts info in multipart.json after complete-multipart-upload. ( #1436 )
2016-05-01 01:25:48 -07:00
Harshavardhana
443ec37765
xl: Add disk usages properly for ListVols() and StatVol(). ( #1435 )
2016-04-30 23:34:43 -07:00
Bala FA
d5df8b8b8d
xl: remove unused err return in listFileVersions() ( #1434 )
2016-04-30 03:21:54 -07:00
Harshavardhana
ac2933c799
windows: Enable erasure test for windows. ( #1432 )
...
Fixes #1363
2016-04-30 02:52:23 -07:00
Bala FA
84afec9ae0
xl: fix DeleteFile() removing meta data files without updating it ( #1433 )
...
Fixes #1428 #1427
2016-04-30 02:52:15 -07:00
Harshavardhana
27c50a70cc
obj: support object names with curly braces. ( #1429 )
...
Example files like
```
/usr/share/mozilla/extensions/{ec8030f7-c20a-464f-9b0e-13a3a9e97384}/ubufox@ubuntu.com.xpi
Song's Son.ogg
```
Should be supported.
2016-04-29 20:19:08 -07:00
Bala FA
a978975eea
xl: add quorum support for DeleteFile() ( #1426 )
...
Fixes #1396
2016-04-29 18:43:18 -07:00
Harshavardhana
dc45ea3946
log: Fix file logging, enable it properly. ( #1424 )
...
Fixes #1419
2016-04-29 17:54:02 -07:00
Harshavardhana
9eb56f0676
xl/healFile: Handle errors and continue ( #1425 )
...
Fixes #1354
2016-04-29 17:52:49 -07:00
Harshavardhana
10a010c1ad
xl/fs: Object layer - keep common functions into single place. ( #1423 )
2016-04-29 17:52:17 -07:00
Harshavardhana
a9935f886c
vendor: update reedsolomon package with new perm improvements. ( #1422 )
2016-04-29 14:59:03 -07:00
Harshavardhana
4e34e03dd4
xl/fs: Split object layer into interface. ( #1415 )
2016-04-29 14:24:10 -07:00
nomadlogic
4d1b3d5e9a
docs: FreeBSD minio source intructions ( #1421 )
...
* Modifications of documentation for using and building minio server on FreeBSD.
- update example of enabling compression to use lz4 vs gzip and provide
explanation of benefits of lz4
- provide walkthrough of building minio server on FreeBSD with binary
golang and gmake
* Fixing markdown syntax for code blocks so we render correctly.
* typo fix
* reword compression enablement docs for easier reading
2016-04-29 13:35:20 -07:00
Krishna Srinivas
7066ce5160
XL/Multipart: rename the parts instead of concatenating. ( #1416 )
2016-04-29 12:17:48 -07:00
Krishna Srinivas
39df425b2a
lock: bug fixes. ( #1420 )
...
* release Lock on map before trying to Lock NS
* delete NS lock from map if no more refs.
* refactor to avoid repetition of code.
2016-04-29 11:39:20 -07:00
Harshavardhana
984903cce1
server: Add global namespace lock. ( #1398 )
...
Fixes #1393
2016-04-29 01:29:09 -07:00
karthic rao
8deddb82f4
Cleanup: Moving IsValidLocationContraint to handler utils
2016-04-28 20:01:11 -07:00
Harshavardhana
a1a667ae5d
xl: Change fileMetadata to xlMetadata. ( #1404 )
...
Finalized backend format
```
{
"version": "1.0.0",
"stat": {
"size": 24256,
"modTime": "2016-04-28T00:11:37.843Z"
},
"erasure": {
"data": 5,
"parity": 5,
"blockSize": 4194304
],
"minio": {
"release": "RELEASE.2016-04-28T00-09-47Z"
}
}
```
2016-04-28 19:27:02 -07:00
Harshavardhana
41b35cff7b
xl: Fixes a bug in read quorum ListFiles() ( #1412 )
...
Fixes a bug in #1406
2016-04-28 17:32:46 -07:00
Harshavardhana
eed756777b
object: Allow '[' and ']' as part of object names.
2016-04-28 13:29:32 -07:00
Harshavardhana
2ac10209cc
xl: ListFiles - return sorted files. ( #1408 )
...
Fixes #1407
2016-04-28 01:48:57 -07:00
Bala FA
5bd6b0b510
xl: check read quorum for ListFiles() ( #1406 )
...
Fixes #1364
2016-04-27 21:09:26 -07:00
karthic rao
1813e9c070
Cleanup - Comments and readability fixes ( #1386 )
2016-04-27 19:28:13 -07:00
Scott McClellan
f87a19a15c
Minor changes to CONTIRBUTING.md instructions ( #1403 )
2016-04-27 16:57:16 -07:00
Harshavardhana
5fffd558d0
xl/heal: Make healFile non-blocking for StatFile and ReadFile. ( #1399 )
...
Fixes #1355
2016-04-27 15:10:19 -07:00
Harshavardhana
b51bef85e6
objectapi: ListMultipart now lists more than 1000 entries in non-recursive. ( #1397 )
...
Having keyMarker with "/" is a valid marker.
Fixes #1394
2016-04-27 13:19:48 -07:00
Krishna Srinivas
d0e5470050
ListMultipart fixes ( #1392 )
...
* ListMultipart: listLeafEntries() - return earlier if a directory is found.
* ListMultipart: do listLeafEntries() only for directories.
2016-04-27 00:15:40 -07:00
Harshavardhana
90987df9b4
objectapi: Simplify ListMultipart combine recursive and non-recursive. ( #1390 )
...
Fixes #1365
2016-04-26 17:57:16 -07:00
Harshavardhana
ad1abc4486
xl-v1/Cleanup: use listOnlineDisks instead of getReadableDisks. ( #1389 )
...
Remove usage of getFileVersionQuorumMap, instead use listFileVersions
to get the version list and extract higherVersion.
Fixes #1379
Fixes #1378
Fixes #1377
2016-04-26 13:03:37 -07:00
Krishna Srinivas
4333e529e6
xl/ListFiles: return as many objects as requested. ( #1383 )
...
* xl/ListFiles: return as many objects as requested and take care of eof (#1361 )
* xl/ListFiles: fix review comments.
* xl/ListFiles: Add windows filepath translation.
* xl/ListFiles: Use slashSeparator instead of "/". Remove filepath.FromSlash() as golang-windows takes care of it automatically.
2016-04-26 10:35:39 -07:00
koolhead17
9685f88b84
Added FreeBSD installation steps with ZFS. ( #1388 )
2016-04-26 10:17:02 -07:00
Harshavardhana
5f80edf232
routers: Fix a crash while initializing network fs. ( #1382 )
...
Crash happens when 'minio server filename' a file name is
provided instead of a directory on command line argument.
```
panic: runtime error: slice bounds out of range
goroutine 1 [running]:
panic(0x5eb460, 0xc82000e0b0)
/usr/local/opt/go/libexec/src/runtime/panic.go:464 +0x3e6
main.splitNetPath(0x7fff5fbff9bd, 0x7, 0x0, 0x0, 0x0, 0x0)
/Users/harsha/mygo/src/github.com/minio/minio/network-fs.go:49 +0xb7
main.newNetworkFS(0x7fff5fbff9bd, 0x7, 0x0, 0x0, 0x0, 0x0)
/Users/harsha/mygo/src/github.com/minio/minio/network-fs.go:90 +0x20a
main.configureServerHandler(0xc82024e1c8, 0x5, 0xc8200640e0, 0x1, 0x1, 0x0, 0x0)
/Users/harsha/mygo/src/github.com/minio/minio/routers.go:43 +0x6ce
main.configureServer(0xc82024e1c8, 0x5, 0xc8200640e0, 0x1, 0x1, 0x5)
/Users/harsha/mygo/src/github.com/minio/minio/server-main.go:86 +0x67
```
2016-04-25 18:10:40 -07:00
Harshavardhana
42254b5c4d
xl: Rename blockingWriteCloser to waitCloser. ( #1376 )
2016-04-25 16:00:58 -07:00
Harshavardhana
00c697393a
Merge pull request #1381 from minio/xl-layer
...
Implement XL layer
2016-04-25 15:59:30 -07:00
Harshavardhana
55032ffdf9
xl: Simplify blockingWriter and its usage. ( #1373 )
...
This removes odd races since we don't need to
track errors and avoids locking. All we need
is a Wait() and Done() waitgroup.
2016-04-25 12:47:31 -07:00
Harshavardhana
8bce699dae
xl: Add logging. ( #1372 )
2016-04-25 12:47:31 -07:00
Harshavardhana
57f35c2bcc
xl: Introduce new blocking writer to make CreateFile atomic. ( #1362 )
...
Creates a new write closer that must be released
by the read consumer. This is necessary so that
while commiting the underlying writers in erasure
coding we need to make sure we reply success only if
we have committed to disk.
This in turn also fixes plethora of bugs related to
subsequent PutObject() races with namespace locking.
This patch also enables most of the tests, other than
ListObjects paging which has some issues still.
Fixes #1358 , #1360
2016-04-25 12:47:31 -07:00
Harshavardhana
cab6805f09
xl: Enable a subset of tests for XL branch. ( #1359 )
2016-04-25 12:47:31 -07:00
Krishna Srinivas
8c85815106
xl: refactor functions to xl-v1-common.go xl-v1-utils.go. ( #1357 )
2016-04-25 12:47:31 -07:00
Krishna Srinivas
becc814531
Xl layer selfheal quorum2
...
* xl/selfheal: selfheal based on read quorum on GET
* xl: getReadableDisks() also returns whether self-heal is needed so that this info can be used by ReadFile/SelfHeal/StatFile.
* xl: trigger selfheal from StatFile.
2016-04-25 12:47:31 -07:00
Harshavardhana
9bd9441107
xl: Simplify reading metadata and add a new fileMetadata type. ( #1346 )
2016-04-25 12:47:31 -07:00
Harshavardhana
f3784d1087
xl: Handle read quorum for StatVol, ListVols
2016-04-25 12:47:31 -07:00
Harshavardhana
91588209fa
obj: Object api handle all errors in common location. ( #1343 )
2016-04-25 12:47:31 -07:00
Krishna Srinivas
5c33b68318
xl: code refactor, cleanup ReadFile and CreateFile.
2016-04-25 12:47:31 -07:00
Bala FA
45b3d3e21f
xl: add quorum support for create file
2016-04-25 12:47:31 -07:00
Harshavardhana
141a44bfbf
xl: Fix ReadFile to keep the order always for reading the data back. ( #1339 )
...
Also fixes a stackoverflow bug in namespace locking.
2016-04-25 12:47:31 -07:00
Harshavardhana
c7bf471c9e
list/xl: Fix the way marker is handled in leafDirectory verification.
2016-04-25 12:47:31 -07:00
Krishna Srinivas
c302875774
selfheal: implement self-heal. Heals the missing parts. ( #1335 )
2016-04-25 12:47:31 -07:00
Harshavardhana
b76f3f1d62
xl: Add more fixes and cleanup.
...
Simplify cleanup of temporary files during createFile operations.
2016-04-25 12:47:31 -07:00
Bala FA
ada0f82b9a
xl: add quorum support for read file and name space locking. ( #1333 )
2016-04-25 12:47:31 -07:00
Harshavardhana
a98a7fb1ad
Implement XL layer - preliminary work.
2016-04-25 12:47:31 -07:00
Harshavardhana
bf8a9702a4
tests: Fix a bug in TestObjectAPIIsUploadIDExists. ( #1375 )
...
The following code crashes when upload ID does not
exist, since we are setting err == nil when we find
err == errFileNotFound.
```
if e == nil {
t.Fatal(e.Error())
```
Fix it.
2016-04-25 12:47:08 -07:00
karthic rao
6e372f83b4
Tests: object api multipart tests and bug fixes.
2016-04-25 10:39:28 -07:00
Harshavardhana
e9fba04b36
logging: Enable logging across storage fs layer. ( #1367 )
...
Adds log.Debugf at all the layer - fixes #1074
2016-04-24 00:36:00 -07:00
Harshavardhana
d63d17012d
tests: Add API suite tests back for object api. ( #1352 )
2016-04-21 23:40:01 -07:00
Harshavardhana
444d1f8a65
miniobrowser: Vendorize to new changes in miniobrowser.
2016-04-21 20:35:48 -07:00
karthic rao
560c3bd153
Adding return statement after error response in the lastest commit to verify location constraint ( #1348 )
2016-04-21 20:08:08 -07:00
Harshavardhana
4cf73caf02
api: Add diskInfo as part of StatVol and ListVols. ( #1349 )
...
It is the bucket and volumes which needs to have this
value rather than the DiskInfo API itself. Eventually
this can be extended to show disk usage per
Buckets/Volumes whenever we have that functionality.
For now since buckets/volumes are thinly provisioned
this is the right approach.
2016-04-21 20:07:47 -07:00
Harshavardhana
1284ecc6f2
api: Fix verification of checkLeafDirectory. ( #1347 )
...
This fixes a problem where leaf directory has more than 1000
entries, also resulting in listing issues, leading to an infinite
loop.
Fixes #1334
2016-04-21 18:05:26 -07:00
karthic rao
cb1116725b
api: verify Location constraint for make bucket. ( #1342 )
2016-04-20 17:35:38 -07:00
koolhead17
c3d0a3d51e
Update README.md
...
Our community contributor bought this to our attention so we have to add region as well in s3cmd config file.
2016-04-19 10:22:40 -07:00
Harshavardhana
e0f8fed011
object: handle Error responses and handle errDiskFull. ( #1331 )
2016-04-19 02:42:10 -07:00
Harshavardhana
6bc17a3aea
server: Attempt to increase max open files. ( #1328 )
2016-04-18 22:05:32 -07:00
Dave Henderson
ff9a6b00cc
isContainerized should look for /.dockerenv not /.dockerinit
...
Signed-off-by: Dave Henderson <dhenderson@gmail.com >
2016-04-18 19:26:56 -07:00
matt robinson
af907a35a9
add environment var to explicitly indicate containerized and allow running as root ( #1327 )
2016-04-18 16:30:11 -07:00
Harshavardhana
b47d722d8e
fs: Fix filtering out valid paths from previous #1321 fix ( #1323 )
...
Fixes #1324
2016-04-17 12:00:23 -07:00
Harshavardhana
33633fd15d
fs: Filter out valid paths and volnames. ( #1321 )
2016-04-16 19:33:29 -07:00
Harshavardhana
b2ec7da9f9
Merge pull request #1314 from minio/split-fs
...
Split fs code into storage layer and object layer.
2016-04-16 18:09:31 -07:00
Harshavardhana
be002ac01e
fs/object: Fix issues from review comments.
2016-04-16 17:57:14 -07:00
Krishna Srinivas
149c6ca094
listMultipart: bugfixes. ( #1318 )
2016-04-16 16:25:53 -07:00
Harshavardhana
8457af5708
fs: Add proper volume and path validation.
2016-04-16 16:25:53 -07:00
Krishna Srinivas
caa35f68fa
listMultipart: implement support for marker. ( #1313 )
2016-04-16 16:25:53 -07:00
Harshavardhana
30b0b4deba
storage/server/client: Enable storage server, enable client storage.
2016-04-16 16:25:53 -07:00
Krishna Srinivas
01a439f95b
refactor: add multipart code to the object layer.
2016-04-16 16:25:53 -07:00
Krishna Srinivas
3c48537f20
refactor: refactor code to separate fs into object-layer and fs layer. ( #1305 )
2016-04-16 16:25:53 -07:00
karthic rao
188bb92d8a
bucket-policy parset tests, and bug fixes ( #1317 )
2016-04-15 18:23:19 -07:00
Harshavardhana
6b3fc03701
docker: Update docker document
2016-04-15 17:40:56 -07:00
Harshavardhana
8112291d43
Add FreeBSD binary link and make a release
2016-04-14 17:22:47 -07:00
Harshavardhana
93666827f4
release: Add freebsd/amd64 build and remove zip, tgz. ( #1316 )
2016-04-13 23:34:55 -07:00
GarimaKapoor
ac30bef72a
Revised Docker.md ( #1311 )
2016-04-11 23:59:12 -07:00
Bala FA
bea6f33b08
backend/fs: remove timer channel from scanMultipartDir() ( #1310 )
...
Previously scanMultipartDir() returns object info channel and timer
channel where timer channel is used to check whether object info
channel is alive or not. This causes a race condition that timeout
may occur while object info channel in use.
This patch fixes the issue by removing timer channel and uses object
info channel directly where each object info has End bool field
indicates whether received object info is end or not.
2016-04-11 19:00:08 -07:00
Michael Werle
9fb1c79456
Improved Docker examples ( #1308 )
...
- Fixed a bug in the persistent docker command ("server" in place of "export")
- Added example of how to set consistent keys with ephemeral data, particularly useful for testing.
2016-04-11 12:33:34 -07:00
Harshavardhana
6b5699b15f
config: console logging should be enabled by default. ( #1307 )
2016-04-09 14:19:59 -07:00
Harshavardhana
33cd910d3a
backend/fs: More cleanup and start using checkBuckeArg. ( #1306 )
...
backend/fs: More cleanup and start using checkBucketArg.
2016-04-08 17:13:16 -07:00
Bala FA
6af761c86c
enhance multipart functions to use fsDirent ( #1304 )
...
* backend/fs: scanMulitpartDir returns directories only for recursive listing
* backend/fs: enhance multipart functions to use fsDirent
2016-04-08 11:46:03 -07:00
Matthew Buckett
bedd867c0b
The fs command doesn't work any more, using server
...
Also there's no /export/data folder which stops minio starting so using /export instead.
2016-04-08 08:53:55 -07:00
Harshavardhana
37330bda98
Merge pull request #1302 from harshavardhana/web-api
...
web: Change /rpc to /webrpc
2016-04-08 02:04:06 -07:00
Harshavardhana
8603185f2f
browser: Add new ui-assets.go
2016-04-08 01:47:30 -07:00
Harshavardhana
fbd02d530d
web: Change /rpc to /webrpc
2016-04-08 01:46:46 -07:00
Harshavardhana
bad2f2afbb
storage: Add storage interface.
2016-04-07 19:33:06 -07:00
Harshavardhana
b182e94acc
signature: Handle presigned payload if set.
...
Validate payload with incoming content.
Fixes #1288
2016-04-07 03:04:18 -07:00
Anand Babu (AB) Periasamy
4e6c4da518
Update README.md
2016-04-06 20:54:06 -07:00
Donald Guy
e8cd1aad8d
accessPolicy: prevent backdoor ListBucket via brute-force 404s, per docs + small fixes
...
* accessPolicy: copy object should require PutObject
* accessPolicy: cite mpu perms doc only for relevant operations
* accessPolicy: prevent backdoor ListBucket via brute-force 404s, per docs
2016-04-06 18:31:40 -07:00
Donald Guy
8b4a5f07b4
accessPolicy: allow anonymous HEAD for Getable objects
...
* accessPolicy: allow anonymous HEAD for Getable objects
* accessPolicy: allow anonymous HEAD of Listable Buckets
2016-04-06 16:40:54 -07:00
Harshavardhana
ff4e04d942
atomic/fs: use safe package for atomic writes, even in multipart.
2016-04-06 16:05:30 -07:00
Bala FA
dfba4ff397
doc: add multipart documentation about staging files
2016-04-05 19:56:19 -07:00
Harshavardhana
06e3171076
Merge pull request #1290 from balamurugana/devel
...
Refactor multipart upload
2016-04-05 18:21:46 -07:00
Bala FA
2b3a118636
Merge pull request #1 from harshavardhana/devel
...
Fix list objects test and remove all the old unnecessary files.
2016-04-06 06:43:43 +05:30
Harshavardhana
8986a6802a
Fix ListMultipartUploads 'mc ls -I' now works properly.
2016-04-05 16:39:02 -07:00
Harshavardhana
3fcc60de91
Move the files and rename some functions.
...
- Rename dir.go as 'fs-multipart-dir.go'
- Move the push/pop to fs-multipart.go and rename them as save/lookup.
- Rename objectInfo instances in fs-multipart as multipartObjInfo.
2016-04-05 12:26:19 -07:00
Harshavardhana
9632c94e7a
Fix list objects test and remove all the old unnecessary files.
...
- Fix tests for new changes.
- Change Golang err as 'e' for the time being, before we bring in
probe removal change.
- Remove old structs and temporary files.
2016-04-05 12:07:19 -07:00
Bala.FA
083e4e9479
backend/fs: Refactor multipart upload
...
This patch modifies multipart upload related functions as below
* New multipart upload call creates file
EXPORT_DIR/.minio/BUCKET/PATH/TO/OBJECT/UPLOAD_ID.uploadid
* Put object part call creates file
EXPORT_DIR/.minio/BUCKET/PATH/TO/OBJECT/UPLOAD_ID.PART_NUMBER.MD5SUM_STRING
* Abort multipart call removes all files matching
EXPORT_DIR/.minio/BUCKET/PATH/TO/OBJECT/UPLOAD_ID.*
* Complete multipart call does
1. creates a staging file
EXPORT_DIR/.minio/BUCKET/PATH/TO/OBJECT/UPLOAD_ID.complete.TEMP_NAME
then renames to
EXPORT_DIR/.minio/BUCKET/PATH/TO/OBJECT/UPLOAD_ID.complete
2. rename staging file
EXPORT_DIR/.minio/BUCKET/PATH/TO/OBJECT/UPLOAD_ID.complete
to EXPORT_DIR/BUCKET/PATH/TO/OBJECT
2016-04-05 22:22:29 +05:30
Harshavardhana
c69fdf0cf2
listObjects: Cleanup and naming conventions.
...
- Marker should be escaped outside in handlers.
- Delimiter should be handled outside in handlers.
- Add missing comments and change the function names.
- Handle case of 'maxKeys' when its set to '0', its a valid
case and should be treated as such.
2016-04-04 19:55:07 -07:00
Krishna Srinivas
85ab1df5a8
listObjects: do not do stat during readdir()
...
* listObjects: improve response time by not doing stat during readDir() operation.
* listObjects: Add windows support.
* listObjects: Readdir() in batches to conserve memory. Add solaris build.
* listObjects: cleanup code.
2016-04-04 17:27:55 -07:00
Harshavardhana
7623e0f8e8
docker: Fix bug in start.sh arguments.
2016-04-04 11:30:18 -07:00
Anand Babu (AB) Periasamy
9843aa1f7a
Merge pull request #1284 from harshavardhana/list
...
objectAPI: Fix object API interface, remove unnecessary structs.
2016-04-03 21:13:48 -07:00
Harshavardhana
0479d4976b
objectAPI: Fix object API interface, remove unnecessary structs.
...
ObjectAPI changes.
```
ListObjects(bucket, prefix, marker, delimiter string, maxKeys int) (ListObjectsInfo, *probe.Error)
ListMultipartUploads(bucket, objectPrefix, keyMarker, uploadIDMarker, delimiter string, maxUploads int) (ListMultipartsInfo, *probe.Error)
ListObjectParts(bucket, object, uploadID string, partNumberMarker, maxParts int) (ListPartsInfo, *probe.Error)
CompleteMultipartUpload(bucket string, object string, uploadID string, parts []completePart) (ObjectInfo, *probe.Error)
```
2016-04-03 15:25:01 -07:00
Anand Babu (AB) Periasamy
12515eabe2
Merge pull request #1280 from harshavardhana/region
...
signature: No need to validate region for getBucketLocation and listBuckets
2016-04-03 01:01:05 -07:00
Harshavardhana
a6a4e7e297
signature: No need to validate region for getBucketLocation and listBuckets.
...
This type of check is added for making sure that we can support
custom regions.
ListBuckets and GetBucketLocation are always "us-east-1" rest
should look for the configured region.
Fixes #1278
2016-04-02 18:42:32 -07:00
Anand Babu (AB) Periasamy
2c793a2ea7
Merge pull request #1282 from harshavardhana/remove-old-code
...
cleanup: Remove old donut/xl code and erasure implementation.
2016-04-02 18:14:21 -07:00
Anand Babu (AB) Periasamy
2bb262cc56
Merge pull request #1279 from harshavardhana/backend
...
config: Migrate to the new version. Remove backend details.
2016-04-02 17:59:21 -07:00
Harshavardhana
379e0abf03
cleanup: Remove old donut/xl code and erasure implementation.
...
This is a change to bring in 'klauspost/reedsolomon' library
in #1270 patch.
2016-04-02 17:30:35 -07:00
Harshavardhana
484ba91b08
config: Migrate to the new version. Remove backend details.
...
Migrate to new config format v4.
```
{
"version": "4",
"credential": {
"accessKey": "WLGDGYAQYIGI833EV05A",
"secretKey": "BYvgJM101sHngl2uzjXS/OBF/aMxAN06JrJ3qJlF"
},
"region": "us-east-1",
"logger": {
"console": {
"enable": true,
"level": "fatal"
},
"file": {
"enable": false,
"fileName": "",
"level": "error"
},
"syslog": {
"enable": false,
"address": "",
"level": "debug"
}
}
}
```
This patch also updates [minio cli spec](./minio.md)
2016-04-02 17:29:31 -07:00
Harshavardhana
6037fe66e9
minio: Simplify for gosimple tool complaints.
2016-04-02 17:28:54 -07:00
Harshavardhana
ba3a5805c1
vendorize: Add updated ui-assets.go.
2016-04-02 17:27:36 -07:00
Anand Babu (AB) Periasamy
33830bfcae
Merge pull request #1273 from harshavardhana/fs-linux
...
fs: Break fs package to top-level and introduce ObjectAPI interface.
2016-04-01 17:03:23 -07:00
Harshavardhana
efc80343e3
fs: Break fs package to top-level and introduce ObjectAPI interface.
...
ObjectAPI interface brings in changes needed for XL ObjectAPI layer.
The new interface for any ObjectAPI layer is as below
```
// ObjectAPI interface.
type ObjectAPI interface {
// Bucket resource API.
DeleteBucket(bucket string) *probe.Error
ListBuckets() ([]BucketInfo, *probe.Error)
MakeBucket(bucket string) *probe.Error
GetBucketInfo(bucket string) (BucketInfo, *probe.Error)
// Bucket query API.
ListObjects(bucket, prefix, marker, delimiter string, maxKeys int) (ListObjectsResult, *probe.Error)
ListMultipartUploads(bucket string, resources BucketMultipartResourcesMetadata) (BucketMultipartResourcesMetadata, *probe.Error)
// Object resource API.
GetObject(bucket, object string, startOffset int64) (io.ReadCloser, *probe.Error)
GetObjectInfo(bucket, object string) (ObjectInfo, *probe.Error)
PutObject(bucket string, object string, size int64, data io.Reader, metadata map[string]string) (ObjectInfo, *probe.Error)
DeleteObject(bucket, object string) *probe.Error
// Object query API.
NewMultipartUpload(bucket, object string) (string, *probe.Error)
PutObjectPart(bucket, object, uploadID string, partID int, size int64, data io.Reader, md5Hex string) (string, *probe.Error)
ListObjectParts(bucket, object string, resources ObjectResourcesMetadata) (ObjectResourcesMetadata, *probe.Error)
CompleteMultipartUpload(bucket string, object string, uploadID string, parts []CompletePart) (ObjectInfo, *probe.Error)
AbortMultipartUpload(bucket, object, uploadID string) *probe.Error
}
```
2016-04-01 15:58:39 -07:00
Harshavardhana
272c5165aa
Merge pull request #1272 from krishnasrinivas/get-auth
...
GetAuth implementation. min/max check for accessKey and secretKey.
2016-04-01 09:38:58 -07:00
Krishna Srinivas
e318925f62
credentials: min/max length check for credentials.
2016-04-01 21:52:39 +05:30
Harshavardhana
2395c42fb5
Merge pull request #1277 from krishnasrinivas/remove-minio-go2
...
UI-handler: remove minio-go dependancy.
2016-04-01 08:56:16 -07:00
Harshavardhana
9333dc3294
Merge pull request #1204 from hackintoshrao/test-bucket
...
Test: Changes to TestPutBucket to catch the race
2016-04-01 08:53:34 -07:00
Karthic Rao
30fc970eab
Changes to TestPutBucket to catch the race
2016-04-01 15:21:16 +05:30
Krishna Srinivas
331890c4c8
UI-handler: remove minio-go dependancy.
2016-04-01 13:56:32 +05:30
Anand Babu (AB) Periasamy
ae5c65d3c6
Merge pull request #1275 from harshavardhana/signature
...
error: Signature errors should be returned with APIErrorCode.
2016-03-31 23:37:38 -07:00
Harshavardhana
02ad48466d
error: Signature errors should be returned with APIErrorCode.
...
The reasoning is that we can reply back with wide range of
S3 error responses, which would provide more richer context
to S3 client.
Fixes #1267
2016-03-31 23:28:40 -07:00
Harshavardhana
a84c466a40
Merge pull request #1251 from harshavardhana/release-fixes
...
release: gz doesn't preserve permissions use tar.gz
2016-03-30 14:34:19 -07:00
Harshavardhana
956142be37
Merge pull request #1271 from krishnasrinivas/set-auth2
...
UI: implement SetAuth/GenerateAuth handlers for changing credentials.
2016-03-29 09:38:18 -07:00
Krishna Srinivas
5201905ad0
UI: implement SetAuth/GenerateAuth handlers for changing credentials.
2016-03-29 21:08:36 +05:30
Anand Babu (AB) Periasamy
186998ad99
Merge pull request #1266 from harshavardhana/cleanup
...
routers: Move API and Web routers into their own files.
2016-03-27 14:25:02 -07:00
Harshavardhana
aa8c9bad54
routers: Move API and Web routers into their own files.
...
This is done to ensure we have a clean way to add new routers such as
- diskRouter
- configRouter
- lockRouter
2016-03-27 13:28:36 -07:00
Harshavardhana
59ee5a547c
release: gz doesn't preserve permissions use tar.gz
...
And fix various other issues with release script.
2016-03-26 23:44:32 -07:00
Harshavardhana
1502e2f29f
Merge pull request #1265 from vadmeste/add_fbsd_support
...
Add simple FreeBSD support, make the minio project compilable
2016-03-26 22:28:07 -07:00
Anand Babu (AB) Periasamy
90a46faf31
Merge pull request #1228 from harshavardhana/signature-cleanup
...
signature: Move signature outside, use a layered approach for signature verification
2016-03-26 15:46:52 -07:00
Harshavardhana
9dca46e156
signature: Use a layered approach for signature verification.
...
Signature calculation has now moved out from being a package to
top-level as a layered mechanism.
In case of payload calculation with body, go-routines are initiated
to simultaneously write and calculate shasum. Errors are sent
over the writer so that the lower layer removes the temporary files
properly.
2016-03-26 15:21:05 -07:00
Anis Elleuch
663f24064b
Add simple FreeBSD support, make the minio project compilable
2016-03-26 22:39:34 +01:00
Harshavardhana
1b0bc814c4
docker: Fix docker Makefile.
2016-03-24 22:53:13 -07:00
Anand Babu (AB) Periasamy
cd5992c6db
Merge pull request #1262 from harshavardhana/docker-file
...
docker: Fix docker command entry.
2016-03-24 22:44:09 -07:00
Harshavardhana
1ef5ab3c28
docker: Fix docker command entry.
2016-03-24 20:38:36 -07:00
Anand Babu (AB) Periasamy
5bd47861d6
Merge pull request #1261 from harshavardhana/update-message
...
minio: Server upon start displays a message if update is available.
2016-03-24 20:25:36 -07:00
Harshavardhana
3538c9f598
minio: Server upon start displays a message if update is available.
...
This code also handles to turn itself off when network is not
available and if request fails. Also prints only when the update
is available.
2016-03-24 20:03:51 -07:00
Anand Babu (AB) Periasamy
24ae5467c8
Merge pull request #1260 from harshavardhana/minio
...
server: Print a message if not backends are configured.
2016-03-24 17:23:03 -07:00
Harshavardhana
36267eb6e2
server: Print a message if not backends are configured.
2016-03-24 10:47:54 -07:00
Harshavardhana
8255590b3c
config/main: set the missing value.
2016-03-24 10:41:42 -07:00
Anand Babu (AB) Periasamy
4f6cf5a6b2
Merge pull request #1123 from harshavardhana/rewrite-v1
...
config/main: Re-write config files - add to new config v3
2016-03-24 08:52:35 -07:00
Harshavardhana
aaf97ea02c
config/main: Re-write config files - add to new config v3
...
- New config format.
```
{
"version": "3",
"address": ":9000",
"backend": {
"type": "fs",
"disk": "/path"
},
"credential": {
"accessKey": "WLGDGYAQYIGI833EV05A",
"secretKey": "BYvgJM101sHngl2uzjXS/OBF/aMxAN06JrJ3qJlF"
},
"region": "us-east-1",
"logger": {
"file": {
"enable": false,
"fileName": "",
"level": "error"
},
"syslog": {
"enable": false,
"address": "",
"level": "debug"
},
"console": {
"enable": true,
"level": "fatal"
}
}
}
```
New command lines in lieu of supporting XL.
Minio initialize filesystem backend.
~~~
$ minio init fs <path>
~~~
Minio initialize XL backend.
~~~
$ minio init xl <url1>...<url16>
~~~
For 'fs' backend it starts the server.
~~~
$ minio server
~~~
For 'xl' backend it waits for servers to join.
~~~
$ minio server
... [PROGRESS BAR] of servers connecting
~~~
Now on other servers execute 'join' and they connect.
~~~
....
minio join <url1> -- from <url2> && minio server
minio join <url1> -- from <url3> && minio server
...
...
minio join <url1> -- from <url16> && minio server
~~~
2016-03-23 19:16:09 -07:00
Harshavardhana
85e50f2bb9
Merge pull request #1258 from hackintoshrao/list-object-prefix-response
...
api: GetObjectInfo fix
2016-03-23 11:09:22 -07:00
Karthic Rao
c8570edaab
Issue of 'mc ls' when prefix is a directory fixed, tests added for GetObjectInfo
2016-03-23 19:42:04 +05:30
Anand Babu (AB) Periasamy
f1161d830f
Merge pull request #1257 from harshavardhana/content-md5
...
routers: Fix order of PostPolicyHandlers and headers.
2016-03-22 20:49:21 -07:00
Anand Babu (AB) Periasamy
4aa098ede9
Merge pull request #1256 from harshavardhana/resources
...
bucketpolicy: checkBucketPolicy should keep resources in map.
2016-03-22 20:47:48 -07:00
Harshavardhana
76bda0d8f1
routers: Fix order of PostPolicyHandlers and headers.
2016-03-22 17:54:44 -07:00
Harshavardhana
996d2e2a10
bucketpolicy: checkBucketPolicy should keep resources in map.
...
This is done to make sure to avoid appending duplicates for
resources for each actions.
2016-03-22 17:04:39 -07:00
Harshavardhana
2edf32adfa
Merge pull request #1253 from koolhead17/patch-5
...
Update README.md
2016-03-22 17:01:59 -07:00
Harshavardhana
e3a3283883
Merge pull request #1255 from hackintoshrao/list-object-prefix-response
...
api: ListObject - Changing to empty response when prefixDir doesn't exist
2016-03-22 17:01:48 -07:00
Karthic Rao
7be79b507b
Changing to empty response when prefixDir doesn't exist
2016-03-23 04:46:10 +05:30
Harshavardhana
600a932acb
Merge pull request #1254 from hackintoshrao/formatting-fix
...
Formating: Formating issues fixed
2016-03-22 04:34:57 -07:00
Karthic Rao
ff41c050d5
Formatting issues fixed.
2016-03-22 15:55:29 +05:30
koolhead17
7f993bb5e6
Update README.md
...
fixed missing spelling,
2016-03-22 13:37:04 +05:30
Harshavardhana
7a97622fed
Merge pull request #1252 from koolhead17/patch-4
...
docs: Add more s3cmd commands
2016-03-22 01:04:35 -07:00
koolhead17
da691dc100
Update README.md
...
Added more s3cmd commands associated & known to work well with Minio server
2016-03-22 12:39:44 +05:30
Harshavardhana
e2c515b334
Merge pull request #1245 from hackintoshrao/fs-bucket-tests
...
api: ListObject - tests, benchmark, optimization
2016-03-21 21:51:48 -07:00
Karthic Rao
b55922effe
Fix for Istruncated set to true under certain conditions.
...
Optimizing List Objects by using binary sort to discard entries in cases
where prefix or marker is set.
Adding test coverage to ListObjects.
Adding benchmark to ListObjects.
2016-03-22 10:09:16 +05:30
Anand Babu (AB) Periasamy
407316c631
Merge pull request #1249 from harshavardhana/remove-deadcode
...
main: Remove all the dead/unused code.
2016-03-21 07:55:07 -07:00
Harshavardhana
902aa05021
main: Remove all the dead/unused code.
...
This patch removes some dead and unused code.
2016-03-21 01:12:29 -07:00
Anand Babu (AB) Periasamy
d95aac4b36
Merge pull request #1246 from harshavardhana/list-response
...
api: ListMultipartUploads and ListParts responded more entries.
2016-03-20 00:07:06 -07:00
Harshavardhana
fc72a0362f
api: ListMultipartUploads and ListParts responded more entries.
...
Issue is empty entries were added since allocating an array was
followed by an append. Keep the index and copy the right entries
precisely.
Fixes an issue reported at - https://github.com/minio/mc/issues/1642
2016-03-19 23:52:40 -07:00
Anand Babu (AB) Periasamy
1900d16d22
Merge pull request #1244 from harshavardhana/zip-compress
...
buildscripts: compress release binaries.
2016-03-18 23:37:52 -07:00
Harshavardhana
41cba3a457
buildscripts: compress release binaries.
...
Fix update command as well to show compressed files in updates.
2016-03-18 23:30:54 -07:00
Harshavardhana
a90faf5996
Merge pull request #1216 from sreeram-boyapati/choose_arch
...
buildscripts: Enable user to choose an arch to build
2016-03-18 12:13:03 -07:00
Harshavardhana
e7e2fae156
Merge pull request #1238 from hackintoshrao/maxkeys-zero-fix
...
[GH-1237] api: Handling maxKeys=0 case with a empty response
2016-03-18 02:57:40 -07:00
Karthic Rao
99af0444b7
Handling maxKeys=0 case with a empty response
2016-03-18 15:16:30 +05:30
Harshavardhana
2357e00317
Fix s3cmd config
2016-03-18 02:07:17 -07:00
Anand Babu (AB) Periasamy
27c75d8be4
Merge pull request #1241 from harshavardhana/print
...
[GH-1240] main: Print keys after init and full server initialization.
2016-03-17 23:42:46 -07:00
Harshavardhana
72d364cbf2
[GH-1240] main: Print keys after init and full server initialization.
...
Setting MINIO_ACCESS_KEY and MINIO_SECRET_KEY re-writes the values
in config properly, but the init message is not updated.
Fix it by delay printing keys until everything is properly
initialized.
2016-03-17 20:01:52 -07:00
Harshavardhana
a729d02c31
Merge pull request #1227 from krishnasrinivas/docker-md
...
docker: simplify docker run instructions.
2016-03-16 18:38:37 -07:00
Harshavardhana
c80c06680d
Merge pull request #1235 from krishnasrinivas/handle-eaddrinuse
...
startup: do not fail for non-EADDRINUSE errors. Fixes #1234
2016-03-16 18:38:27 -07:00
Harshavardhana
facd5a9ffd
Merge pull request #1239 from awwalker/verify-headers
...
api: CopyObject - verify the additional headers before starting to write.
2016-03-16 18:37:56 -07:00
awwalker
34f2c5bcdf
verify before writing
...
merge
verify headers before writing
2016-03-16 18:03:23 -07:00
Krishna Srinivas
fd943c704d
docker: simplify docker run instructions.
2016-03-17 01:39:49 +05:30
Krishna Srinivas
e5b411caf4
startup: do not fail for non-EADDRINUSE errors. Fixes #1234
2016-03-17 01:36:08 +05:30
Anand Babu (AB) Periasamy
c784707ed8
Merge pull request #1233 from harshavardhana/bucket-policy
...
bucketpolicy: Improve bucket policy validation, avoid nested rules.
2016-03-15 18:57:02 -07:00
Harshavardhana
88714e7c8e
bucketpolicy: Improve bucket policy validation, avoid nested rules.
...
Bucket policy validation is more stricter now, to avoid nested
rules. The reason to do this is keep the rules simpler and more
meaningful avoiding conflicts.
This patch implements stricter checks.
Example policy to be generally avoided.
```
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"s3:GetObject",
"s3:DeleteObject"
],
"Effect": "Allow",
"Principal": {
"AWS": [
"*"
]
},
"Resource": [
"arn:aws:s3:::jarjarbing/*"
]
},
{
"Action": [
"s3:GetObject",
"s3:DeleteObject"
],
"Effect": "Deny",
"Principal": {
"AWS": [
"*"
]
},
"Resource": [
"arn:aws:s3:::jarjarbing/restic/key/*"
]
}
]
}
```
2016-03-15 17:50:23 -07:00
Harshavardhana
2e3e164f16
Update Docker.md
2016-03-15 10:01:58 -07:00
Harshavardhana
15b5f4d7a3
Update Caddy.md
2016-03-14 20:51:16 -07:00
Harshavardhana
83f9a50137
Merge pull request #1229 from brendanashworth/improve-fs-getobject
...
pkg/fs: optimize GetObject, add benchmark
2016-03-13 14:03:45 -07:00
Brendan Ashworth
583e4ecff6
pkg/fs: optimize GetObject syscalls for common case
...
In the common case, GetObject is called on a bucket that exists and an
object that exists and is not a directory. It should be optimized for
this case, thus error-related syscalls are pushed back until they are
necessary.
This should not impact performance negatively in the uncommon case, and
instead drops two otherwise unnecessary os.Stat's in the common case.
The race conditions around a proper error being returned were present
beforehand.
It also renames 'err' to 'e'.
2016-03-13 13:56:33 -07:00
Brendan Ashworth
b2257682e4
pkg/fs: add benchmark for GetObject
...
This commit adds a benchmark for GetObject. It doesn't leverage the I/O
as much because it uses short text for data, just 58 chars.
2016-03-13 11:13:06 -07:00
Harshavardhana
09789de586
Merge branch 'awwalker-new-copy-headers'
2016-03-12 10:56:11 -08:00
awwalker
9a5e3299fc
api/object: Add CopyObject to support match/modified copy headers
...
Adds support for the following request headers:
- x-amz-copy-source-if-match
- x-amz-copy-source-if-none-match
- x-amz-copy-source-if-unmodified-since
- x-amz-copy-source-if-modified-since
Fixes #1176
2016-03-12 10:54:23 -08:00
Harshavardhana
2c09df24ee
Merge pull request #1225 from hackintoshrao/fs-bucket-tests
...
tests: Add tests for ListBuckets
2016-03-12 01:26:56 -08:00
Karthic Rao
53a76439a2
test for GetBucketInfo
2016-03-12 14:31:30 +05:30
Anand Babu (AB) Periasamy
6b0af08885
Merge pull request #1224 from harshavardhana/simplify
...
cleanup: Remove unecessary packages and tests. Simplify.
2016-03-11 22:58:43 -08:00
Harshavardhana
5282a79eda
cleanup: Remove unecessary packages and tests. Simplify.
2016-03-11 19:53:55 -08:00
Anand Babu (AB) Periasamy
3d8b9afa8f
Merge pull request #1222 from harshavardhana/cleanup-fix
...
cleanup: Rename ObjectMetadata as ObjectInfo.
2016-03-11 17:14:22 -08:00
Harshavardhana
52751d81cb
cleanup: Rename ObjectMetadata as ObjectInfo.
...
Fixes #1215
2016-03-11 16:58:08 -08:00
Harshavardhana
c81f4b0228
Merge pull request #1218 from hackintoshrao/better-fs-util-test
...
Test: Better structuring of fs-utils test
2016-03-11 06:54:43 -08:00
Karthic Rao
ec8c1d4ef6
Better structuring of fs-utils test
2016-03-11 19:19:47 +05:30
Sreeram Boyapati
62bd44f873
buildscripts: Enable user to choose an arch to build
...
- Building minio for all architectures takes a lot of time.
Choose the one user needs
2016-03-11 10:17:56 +05:30
Anand Babu (AB) Periasamy
b5c77b641d
Merge pull request #1209 from harshavardhana/err-naming
...
error: Add proper prefixes for s3Error codes.
2016-03-10 18:59:16 -08:00
Harshavardhana
fdf3d64793
error: Add proper prefixes for s3Error codes.
...
This patch adds 'Err' prefix for all standard API
error codes and also adds a proper type for them.
2016-03-10 18:38:46 -08:00
Anand Babu (AB) Periasamy
373d335d94
Merge pull request #1214 from brendanashworth/improve-listbuckets
...
ListBuckets test & improvement, IsValid{Bucket,Object}Name fix, test, docs
2016-03-10 18:20:18 -08:00
Anand Babu (AB) Periasamy
b16025abf4
Merge pull request #1213 from harshavardhana/presigned
...
auth: Detect anonymous as the last resort.
2016-03-10 18:11:43 -08:00
Harshavardhana
166ef09c3d
auth: Detect anonymous as the last resort.
2016-03-10 17:55:36 -08:00
Harshavardhana
e54aa10201
Merge pull request #1212 from harshavardhana/venodr
...
vendor: Update ui-assets with new changes and release.
2016-03-10 17:55:25 -08:00
Harshavardhana
5606232567
vendor: Update ui-assets with new changes and release.
2016-03-10 17:36:32 -08:00
Harshavardhana
9352cb87c6
Merge pull request #1211 from harshavardhana/vendorize
...
vendor: Add minio-go vendor updates.
2016-03-10 15:50:14 -08:00
Harshavardhana
e781959d5b
vendor: Add minio-go vendor updates.
2016-03-10 14:33:15 -08:00
Harshavardhana
af295f3600
Merge pull request #1186 from balamurugana/devel
...
api: refactor list object handling in fs backend
2016-03-10 13:43:49 -08:00
Bala.FA
c70bc2209e
api: refactor list object handling in fs backend
...
When list object is invoked, it creates a goroutine if not available
for given parameters else uses existing goroutine. These goroutines
are alive for 15 seconds for further continuation list object request
else they exit.
Fixes #1076
2016-03-11 02:20:51 +05:30
Harshavardhana
5cb546d288
Merge pull request #1210 from krishnasrinivas/port-check-fix
...
startup: specify the network - tcp4/tcp6 for ListenTCP()
2016-03-10 08:54:02 -08:00
Krishna Srinivas
010e775b17
startup: specify the network - tcp4/tcp6 for ListenTCP()
2016-03-10 17:17:47 +05:30
Harshavardhana
2cba605514
Merge pull request #1206 from fwessels/notice-typo
...
Fix typo
2016-03-09 09:25:24 -08:00
Harshavardhana
d740d3aaae
Merge pull request #1208 from krishnasrinivas/port-check
...
startup: do not start minio server if port is not free. Fixes #1207
2016-03-09 09:25:05 -08:00
Anand Babu (AB) Periasamy
5ac4afa4d1
Merge pull request #1080 from harshavardhana/bucket-policy
...
accessPolicy: Implement Put, Get, Delete access policy.
2016-03-09 07:11:39 -08:00
Krishna Srinivas
ea7ea427ca
startup: do not start minio server if port is not free. Fixes #1207
2016-03-09 19:39:40 +05:30
frankw
027f7efbdb
Fix typo
2016-03-09 13:55:58 +01:00
Harshavardhana
d5057b3c51
accessPolicy: Implement Put, Get, Delete access policy.
...
This patch implements Get,Put,Delete bucket policies
Supporting - http://docs.aws.amazon.com/AmazonS3/latest/dev/access-policy-language-overview.html
Currently supports following actions.
"*": true,
"s3:*": true,
"s3:GetObject": true,
"s3:ListBucket": true,
"s3:PutObject": true,
"s3:CreateBucket": true,
"s3:GetBucketLocation": true,
"s3:DeleteBucket": true,
"s3:DeleteObject": true,
"s3:AbortMultipartUpload": true,
"s3:ListBucketMultipartUploads": true,
"s3:ListMultipartUploadParts": true,
following conditions for "StringEquals" and "StringNotEquals"
"s3:prefix", "s3:max-keys"
2016-03-08 17:44:50 -08:00
Harshavardhana
846410c563
Merge pull request #1205 from harshavardhana/patch-3
...
Fix docker documentation.
2016-03-08 11:45:45 -08:00
Harshavardhana
f268762ec7
Fix docker documentation.
2016-03-08 11:23:24 -08:00
Atul Jha
41739e0913
Fix doc as per community member suggestion.
...
Added suggestion mentioned by our community member - fixes #1116
2016-03-08 10:42:56 -08:00
Brendan Ashworth
cd3eb63c4a
pkg/fs: test, document, and fix IsValid{Bucket,Object}Name
...
This commit improves the docs for both functions (more Go-like) and
drops an unnecessary condition in IsValidBucketName. This also drops a
condition in IsValidObjectName where "" (empty string) was a valid
object name. This has been fixed and will no longer return true.
This commit also adds tests for both functions, including a regression
test for the bug fix.
2016-03-07 19:59:24 -08:00
Brendan Ashworth
a5d0bef4e2
pkg/fs: test, bench, and drop unnecessary check in ListBuckets
...
There is now a simple test and a benchmark for ListBuckets. I also
dropped an unnecessary check that was simply repeated from above,
guaranteed to be true.
2016-03-07 19:58:33 -08:00
Harshavardhana
114f9de5eb
Merge pull request #1201 from harshavardhana/time
...
handlers: Cleanup time handler helpers.
2016-03-07 12:05:26 -08:00
Harshavardhana
761cb2c740
handlers: Cleanup time handlers helpers.
2016-03-07 10:47:45 -08:00
Harshavardhana
59fbb6d081
Merge pull request #1199 from brendanashworth/improvements-fs
...
pkg/fs improvements
2016-03-07 00:31:57 -08:00
Brendan Ashworth
fab45aae40
pkg/fs: add bucket test and benchmarks
...
Lots of useful benchmarks and a simple test addition!
2016-03-07 00:07:11 -08:00
Brendan Ashworth
7399d8ceaa
pkg/fs: skip unnecessary os.Stat system call
2016-03-07 00:07:11 -08:00
Brendan Ashworth
0a0451a0fb
pkg/fs: DRY SetBucketMetadata
...
It had a lot of code that was the same as GetBucketMetadata, so instead
call GBM from SBM so as to reduce doing the same thing in two different
spots. Theoretically this will induce a small overhead as now at least
two calls of denormalizeBucket are made, although this shouldn't be
noticeable.
2016-03-07 00:07:11 -08:00
Brendan Ashworth
294ea814bf
pkg/fs: for locks, prefer defer and read-only ops
...
This commit prefers the use of 'defer' for fs.Unlock (and fs.RUnlock)
because it is more idiomatic Go and reduces repetition in the code,
lending to a cleaner code base.
It also switches a few uses of the lock to read-only locks, which should
improve performance of those functions dramatically in certain contexts.
2016-03-07 00:07:11 -08:00
Anand Babu (AB) Periasamy
d54a8a9c07
Merge pull request #1195 from harshavardhana/delete-objects
...
api: Implement multiple objects Delete api - fixes #956
2016-03-06 18:52:39 -08:00
Harshavardhana
aed62788d9
api: Implement multiple objects Delete api - fixes #956
...
This API takes input XML input in following form.
```
<?xml version="1.0" encoding="UTF-8"?>
<Delete>
<Quiet>true</Quiet>
<Object>
<Key>Key</Key>
</Object>
<Object>
<Key>Key</Key>
</Object>
...
</Delete>
```
and responds the list of successful deletes, list of errors
for all the deleted objects.
```
<?xml version="1.0" encoding="UTF-8"?>
<DeleteResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/ ">
<Deleted>
<Key>sample1.txt</Key>
</Deleted>
<Error>
<Key>sample2.txt</Key>
<Code>AccessDenied</Code>
<Message>Access Denied</Message>
</Error>
</DeleteResult>
```
2016-03-06 18:31:50 -08:00
Harshavardhana
6f842124ad
Merge pull request #1197 from brendanashworth/improve-DRY-1
...
api: DRY code and add new test
2016-03-06 16:30:48 -08:00
Brendan Ashworth
adf74ffdb0
api: DRY code and add new test
...
This commit makes code cleaner and reduces the repetitions in the code
base. Specifically, it reduces the clutter in setObjectHeaders. It also
merges encodeSuccessResponse and encodeErrorResponse together because
they served no purpose differently. Finally, it adds a simple test for
generateRequestID.
2016-03-06 13:26:27 -08:00
Harshavardhana
e883583804
Merge pull request #1192 from harshavardhana/color
...
console: Fix console color printing on windows.
2016-03-04 13:50:49 -08:00
Harshavardhana
164dfe2ec9
console: Fix console color printing on windows.
...
Print colored accessKeyID and secretAccessKey are unreadable on windows
on command prompts and powershell.
Use the console package from minio client.
2016-03-04 10:07:19 -08:00
Harshavardhana
91800cff53
Merge pull request #1189 from krishnasrinivas/assets-gzip-encoding
...
browser-assets: serve asset files with compression.
2016-03-04 01:12:11 -08:00
Krishna Srinivas
44b2037667
browser-assets: serve asset files with compression.
2016-03-04 07:47:26 +05:30
Harshavardhana
8bf882e5c0
Merge pull request #1188 from harshavardhana/vendorize
...
browser: vendorize to new browser update
2016-03-03 18:04:09 -08:00
Harshavardhana
95d3ecb9ce
browser: vendorize to new browser update
2016-03-03 17:46:09 -08:00
Harshavardhana
97472e76b0
Merge pull request #1187 from harshavardhana/support-newfiles
...
routers: Support special asset files.
2016-03-03 17:25:25 -08:00
Harshavardhana
df0bce374c
routers: Support special asset files.
...
- loader.css
- logo.svg
- {firefox,chrome,safari}.png
2016-03-03 17:13:15 -08:00
Harshavardhana
356b889e66
Merge pull request #1184 from harshavardhana/multipart
...
multipart: remove proper MD5, rather create MD5 based on parts to be s3 compatible.
2016-03-02 18:42:24 -08:00
Harshavardhana
f111997184
multipart: remove proper MD5, rather create MD5 based on parts to be s3 compatible.
...
This increases the performance phenominally.
2016-03-02 14:20:49 -08:00
Harshavardhana
5f2cfdfbe2
Merge pull request #1185 from harshavardhana/fix-signature
...
signature: Fix signature handling of parallel requests.
2016-03-02 14:20:18 -08:00
Harshavardhana
17d145df3a
signature: Fix signature handling of parallel requests.
...
Signature struct should be immutable, this fixes an issue
with AWS cli not being able to do multipart put operations.
2016-03-02 11:49:50 -08:00
GarimaKapoor
b37fbabe7f
Update README.md
2016-03-02 11:07:23 -08:00
Harshavardhana
d7767d6431
Merge pull request #1181 from hackintoshrao/vet-shadow-fix
...
build: Removing error initialization to solve variable shadowing
2016-03-01 21:26:15 -08:00
Karthic Rao
6651f5b368
go vet shadow error patch
2016-03-02 09:55:00 +05:30
Harshavardhana
dc622674e1
Merge pull request #1178 from harshavardhana/list-objects
...
list: Fix handling of maxKeys and prefixes.
2016-03-01 17:55:19 -08:00
Harshavardhana
c7021f6a95
list: Fix handling of maxKeys and prefixes.
...
This fixes a problem of requeuing the same request
and also fixes a major problem of sending truncated
for full key prefixes.
Fixes #1177
2016-03-01 17:34:31 -08:00
Harshavardhana
53ca192fe7
Merge pull request #1175 from harshavardhana/get-object
...
api: Implement support for additional request headers.
2016-02-28 19:42:40 -08:00
Harshavardhana
ee1b86e517
api: Implement support for additional request headers.
...
Now GetObject and HeadObject both support
- If-Modified-Since, If-Unmodified-Since
- If-Match, If-None-Match
request headers.
These headers are used to further handle the responses for GetObject
and HeadObject API.
Fixes #1098
2016-02-28 19:34:20 -08:00
Harshavardhana
0b2e449727
Merge pull request #1174 from harshavardhana/copy-object
...
api: Implement CopyObject s3 API, doing server side copy.
2016-02-27 20:44:56 -08:00
Harshavardhana
3ff8a1b719
api: Implement CopyObject s3 API, doing server side copy.
...
Fixes #1172
2016-02-27 19:51:59 -08:00
Harshavardhana
2520298734
Merge pull request #1170 from krishnasrinivas/caching
...
caching: disable caching of index.html and enable caching for other UI asset files.
2016-02-27 00:02:51 -08:00
Krishna Srinivas
af7170675d
caching: disable caching of index.html and enable caching for other UI asset files.
2016-02-27 12:22:26 +05:30
Harshavardhana
7780a1ce4c
Merge pull request #1171 from harshavardhana/disable-multi
...
api: Return NotImplemented for MultiDelete and CopyObject APIs
2016-02-26 16:34:37 -08:00
Harshavardhana
ae6e774377
api: Return NotImplemented for MultiDelete and CopyObject APIs
2016-02-26 15:57:30 -08:00
Harshavardhana
2ec211e52a
Merge pull request #1169 from minio/harshavardhana-patch-1
...
Update README.md
2016-02-24 20:30:29 -08:00
Harshavardhana
9122f06307
Update README.md
2016-02-24 19:57:11 -08:00
Harshavardhana
04d6408c31
Merge pull request #1167 from harshavardhana/fix-release-tag
...
build: Fix release tag.
2016-02-23 17:16:26 -08:00
Harshavardhana
024c00addd
build: Fix release tag.
2016-02-23 16:56:41 -08:00
Harshavardhana
2c26b98242
Merge pull request #1166 from harshavardhana/release
...
build: Add release builds, now generated with 'make release'
2016-02-23 16:02:51 -08:00
Harshavardhana
223245cc45
build: Add release builds, now generated with 'make release'
...
Currently supported platforms are
- linux{amd64,arm,386}
- winows{amd64,386}
- darwin{amd64}
2016-02-23 15:14:02 -08:00
Harshavardhana
781540081d
vendor: Update to new upstream changes from fatih/color
...
Brings in changes like support for Solaris/FreeBSD.
2016-02-23 15:01:17 -08:00
Harshavardhana
acb44e294c
Merge pull request #1140 from harshavardhana/go1.6
...
build/vet: Fix all the shadowing reports with go1.6
2016-02-23 14:50:46 -08:00
Harshavardhana
408aa72146
build/vet: Fix all the shadowing reports with go1.6
...
Golang 1.6 is default version for the build now.
Additionally set 'GODEBUG=cgocheck=0' for now, until
we fix the erasure coding package.
Readmore here https://tip.golang.org/doc/go1.6#cgo
2016-02-23 14:34:39 -08:00
Harshavardhana
04424ae9c2
Merge pull request #1163 from harshavardhana/minio-browser
...
web: Removing dependency for gpg and downloading assets.
2016-02-23 14:16:25 -08:00
Harshavardhana
2181003609
web: Removing dependency for gpg and downloading assets.
...
Assets are vendorized from now on and updated for each release.
2016-02-23 13:32:12 -08:00
Harshavardhana
997141d031
Merge pull request #1161 from krishnasrinivas/feb-23
...
UI: serve index.html if the requested file is not found in the assets bundle.
2016-02-23 12:14:52 -08:00
Krishna Srinivas
e509bcb2b9
UI: serve index.html if the requested file is not found in the assets bundle.
2016-02-24 00:36:30 +05:30
Anand Babu (AB) Periasamy
07da31f8b8
Merge pull request #1150 from harshavardhana/signature
...
signV4: Move pkg/signature to pkg/s3/signature4
2016-02-23 12:39:28 +05:30
Harshavardhana
653ceee9ee
signV4: Move pkg/signature to pkg/s3/signature4
...
Cleanup and move this to relevant path.
2016-02-22 22:47:09 -08:00
Anand Babu (AB) Periasamy
b31dac9162
Merge pull request #1144 from harshavardhana/definitions
...
cleanup: Remove definitions and move them to its relative places acco…
2016-02-23 01:50:17 +05:30
Harshavardhana
31c941d320
Merge branch 'osallou-feature_credentials_envvars'
2016-02-22 10:49:14 -08:00
Olivier Sallou
678585c513
use environment variables to set and override access and secret keys at server startup
2016-02-22 10:49:02 -08:00
Harshavardhana
800b19d8e5
cleanup: Remove definitions and move them to its relative places accordingly
...
- Move fs-definitions.go and break them into fs-datatypes.go, fs-bucket-acl.go
and fs-utils.go
- Move api-definitions.go to api-response.go, where they should be.
- Move web-definitions to its related handlers.
2016-02-22 10:41:27 -08:00
Harshavardhana
bd6850e79f
Merge pull request #1155 from harshavardhana/verify-yasm
...
build: Verify yasm version and complain - fixes #1154
2016-02-22 03:26:03 -08:00
Harshavardhana
18fd0a0f81
build: Verify yasm version and complain - fixes #1154
2016-02-22 03:17:17 -08:00
Harshavardhana
8824b18c29
Merge pull request #1151 from harshavardhana/bucket-fix
...
router: Fix "/minio" router for web.
2016-02-21 23:05:43 -08:00
Harshavardhana
5da1972d1f
router: Fix "/minio" router for web.
2016-02-21 22:59:01 -08:00
Harshavardhana
d3966d1dde
Merge pull request #1145 from harshavardhana/bucket
...
web: Handle private bucket match from prefix to exact match.
2016-02-21 12:14:09 -08:00
Harshavardhana
393636f74b
Merge pull request #1149 from alexex/master
...
Alpine & CA-Certs for SSL support
2016-02-20 02:24:03 -08:00
Harshavardhana
91a7b13529
web: Handle private bucket match from prefix to exact match.
...
Filter out 'privateBucket' if any from listBuckets output.
2016-02-20 02:22:56 -08:00
Harshavardhana
185d29a899
Merge pull request #1148 from harshavardhana/fix-bug
...
web: Add targetProto for putObjectURL,getObjectURL SSL requests.
2016-02-20 02:15:56 -08:00
Alexander Jung-Loddenkemper
342f8fbe5d
Alpine & CA-Certs for SSL support
...
Switch to alpine as a base image and install the ca certifcates to enable the minio docker container to run behind a SSL proxy, which will fix #1146
2016-02-20 11:15:19 +01:00
Harshavardhana
4db136c19c
web: Add targetProto for putObjectURL,getObjectURL SSL requests.
...
Currently the default request was based on the 'minio server'
SSL configuration. But inside a proxy this is invalid browser
needs to send which protocol it wishes the PresignedURL
should be generated for.
2016-02-20 01:59:18 -08:00
Harshavardhana
a18620fa86
Merge pull request #1141 from harshavardhana/target-host
...
web: PresignedGet/Put targetHost should always have a valid value.
2016-02-19 15:53:20 -08:00
Harshavardhana
d4c91426a7
web: PresignedGet/Put targetHost should always have a valid value.
...
PresignedURLs should always be generated for the targetHost, so that
we can avoid signature issues.
2016-02-19 15:31:38 -08:00
Harshavardhana
51e611a380
Merge pull request #1139 from harshavardhana/docker
...
docker: Make sure that we properly check for containers.
2016-02-18 13:59:22 -08:00
Harshavardhana
354229732b
docker: Make sure that we properly check for containers.
2016-02-18 13:39:44 -08:00
Harshavardhana
1c33926239
Merge pull request #1136 from harshavardhana/add-proxy
...
presigned: Fix a bug in presigned request verification.
2016-02-18 02:46:04 -08:00
Harshavardhana
91a092792a
presigned: Fix a bug in presigned request verification.
...
Additionally add Docker proxy configuration.
2016-02-18 02:23:12 -08:00
Harshavardhana
d561f0cc0b
Merge pull request #1135 from abperiasamy/vendor-update-go-homedir
...
vendor update for go-homedir
2016-02-18 00:14:58 -08:00
Anand Babu (AB) Periasamy
f53e9dd1b8
vendor update for go-homedir
2016-02-18 13:02:41 +05:30
Harshavardhana
2a6bc604db
Merge pull request #1132 from harshavardhana/merge-ports
...
web/rpc: Merge ports with API server.
2016-02-17 21:22:48 -08:00
Harshavardhana
dd9aaa855c
web/rpc: Merge ports with API server.
...
Fixes #1081 and #1130
2016-02-17 20:28:15 -08:00
Harshavardhana
5a9333a67b
signature: Rewrite signature handling and move it into a library.
2016-02-16 17:28:16 -08:00
Harshavardhana
b531bb31bb
Merge pull request #1129 from harshavardhana/cpu-klauspost
...
cpu: Remove pkg/cpu in favor of better klauspost/cpuid.
2016-02-15 14:06:05 -08:00
Harshavardhana
9e10ee7e47
cpu: Remove pkg/cpu in favor of better klauspost/cpuid.
...
Fixes #1128
2016-02-15 13:50:33 -08:00
Harshavardhana
a173313bc2
Merge pull request #1127 from abperiasamy/master
...
rewrite minio runtime checks
2016-02-15 10:39:01 -08:00
Anand Babu (AB) Periasamy
bbca70e13b
rewrite minio runtime checks
2016-02-15 17:56:56 +05:30
Harshavardhana
e73301944a
Merge pull request #1126 from harshavardhana/fix-multipart
...
fs/multipart: Handle un-ordered creation of multiparts.
2016-02-14 00:49:07 -08:00
Harshavardhana
fbab7128d5
fs/multipart: Handle un-ordered creation of multiparts.
...
Fixes #1125
2016-02-14 00:39:15 -08:00
Harshavardhana
d38bbf3562
Merge pull request #1124 from harshavardhana/json-rpc-fix
...
rpc: Fix json rpc to handle array and object request params.
2016-02-13 19:15:52 -08:00
Harshavardhana
e59ceba51b
rpc: Fix json rpc to handle array and object request params.
...
rpc/v2/json2 code has a bug where it treats all jsonrpc 2.0
request params like an 'object'. In accordance with the spec
it could be both 'object' or an 'array'.
Handle both cases.
2016-02-13 19:01:36 -08:00
Harshavardhana
40e8893fb9
Merge pull request #1122 from harshavardhana/osx-build
...
build: Simplify and build only with Makefiles.
2016-02-13 01:27:15 -08:00
Harshavardhana
ebdbe2db44
build: Simplify and build only with Makefiles.
...
Configure is not portable.
2016-02-13 01:19:13 -08:00
Harshavardhana
3fd637c1f4
Merge branch 'json-rpc'
2016-02-12 20:32:47 -08:00
Krishna Srinivas
56ffe79f25
jsonrpc: return json2.Error from the json-rpc layer to the client
2016-02-12 20:30:02 -08:00
Krishna Srinivas
6ad39cb386
WebUI: move from rpc/v2/json to rpc/v2/json2 which has better error response structure.
2016-02-12 20:29:56 -08:00
Harshavardhana
f98675660b
Merge pull request #1119 from harshavardhana/minio
...
xl: Moved to minio/minio
2016-02-11 15:54:02 -08:00
Harshavardhana
62f6ffb6db
xl: Moved to minio/minio - fixes #1112
2016-02-11 15:43:36 -08:00
Harshavardhana
f0e2edd632
Merge pull request #1120 from krishnasrinivas/version-check
...
UI: implement rpc call to return UI version
2016-02-11 11:07:57 -08:00
Krishna Srinivas
5e32dec4fb
UI: implement rpc call to return UI version
2016-02-12 00:26:27 +05:30
Harshavardhana
33bd97d581
Merge pull request #1118 from harshavardhana/make-file
...
build: Change UI assets location.
2016-02-10 16:21:58 -08:00
Harshavardhana
70bbf4c8ec
build: Change UI assets location.
2016-02-10 16:10:33 -08:00
Harshavardhana
785a1f0eb0
Merge pull request #1117 from harshavardhana/devel
...
pkg/ioutils: remove usage of os.Lstat() in FTW()
2016-02-10 13:52:16 -08:00
Harshavardhana
6e9d73426b
pkg/ioutils: True should be true
2016-02-10 13:33:36 -08:00
Bala.FA
5e4b13f4bd
remove unused functions
2016-02-10 13:32:53 -08:00
Bala.FA
255505a83b
pkg/ioutils: remove usage of os.Lstat() in FTW()
...
As os.Readdir() is used get file entries where statinfo is already
present. This patch fixes to use statinfo provided by os.Readdir().
2016-02-10 13:32:53 -08:00
Harshavardhana
31dbdb1787
Merge pull request #1113 from harshavardhana/list-objects-optimization
...
listObjects: list objects minor optimization.
2016-02-10 10:33:37 -08:00
Harshavardhana
7c38430710
Merge pull request #1115 from krishnasrinivas/return-error-type
...
jsonrpc: WrapError() makes jsonrpc return unnecessary details in the error message.
2016-02-10 10:18:12 -08:00
Krishna Srinivas
318265ecaf
jsonrpc: WrapError() makes jsonrpc return unnecessary details in the error message.
2016-02-10 21:22:37 +05:30
Harshavardhana
98ee5fcf55
build: Add spelling checks and check if curl is installed.
2016-02-10 00:18:05 -08:00
Harshavardhana
9b29af8bbe
listObjects: list objects minor optimization.
...
Minor optimization.
- Add 1000 entries buffered channel for walkerCh.
- Reset marker after the lexical order has reached and
compare only if the marker is set.
2016-02-09 21:45:19 -08:00
Harshavardhana
4ef01bc225
Merge pull request #1109 from krishnasrinivas/getuiversion
...
rpc: Add rpc uiVersion string
2016-02-09 10:29:28 -08:00
Krishna Srinivas
bf6078df13
webrpc: Implement GetUIVersion json-rpc API.
2016-02-09 18:09:32 +05:30
Harshavardhana
4e328d7b2c
Merge pull request #1106 from harshavardhana/support
...
pkg/user: Support 32bit darwin.
2016-02-08 01:59:02 -08:00
Harshavardhana
42fcb27308
pkg/user: Support 32bit darwin in user package.
2016-02-08 01:34:25 -08:00
Harshavardhana
e79a73a3f5
Merge pull request #1105 from harshavardhana/cleanup-assets
...
build: Cleanup assets file upon make clean.
2016-02-07 11:06:08 -08:00
Harshavardhana
2c6da82788
build: Cleanup assets file upon make clean.
2016-02-07 10:55:51 -08:00
Harshavardhana
012d7ae2c3
Merge pull request #1104 from harshavardhana/response-content
...
getObject: Add support for special response headers.
2016-02-07 04:03:13 -08:00
Harshavardhana
99fbc0fcb3
getObject: Add support for special response headers.
...
Supports now response-content-type, response-content-disposition,
response-cache-control, response-expires.
2016-02-07 03:55:16 -08:00
Harshavardhana
de79440de2
Merge pull request #1103 from harshavardhana/min-free-disk
...
server: Remove max-buckets option and now max buckets is unlimited.
2016-02-06 18:33:04 -08:00
Harshavardhana
f4c8120cf9
server: Remove max-buckets option and now max buckets is unlimited.
...
minio server max-buckets option removed. min-free-disk option is
now a flag.
2016-02-06 18:25:47 -08:00
Harshavardhana
e7fec22224
Merge pull request #1102 from harshavardhana/fs-multi
...
multipart: Increase locked critical for CompleteMultipart.
2016-02-06 02:02:41 -08:00
Harshavardhana
4e6e78598f
multipart: Increase locked critical for CompleteMultipart.
2016-02-06 01:46:05 -08:00
Harshavardhana
643ef30533
Merge pull request #1101 from harshavardhana/combine
...
multipart: Multipart session map now is based on uploadID.
2016-02-05 23:38:19 -08:00
Harshavardhana
8df201ef30
multipart: Multipart session map now is based on uploadID.
...
- Fixes initiating parallel uploads, and configs being quickly
re-written by another incoming request.
- Parallel uploads work smoothly now and return expected behavior.
2016-02-05 23:32:30 -08:00
Harshavardhana
3f5804f75a
Merge pull request #1100 from harshavardhana/multipart-resume
...
multipart: Multipart resume simplify further.
2016-02-05 17:49:37 -08:00
Harshavardhana
69bd001c8b
multipart: Multipart resume simplify further.
2016-02-05 17:40:08 -08:00
Harshavardhana
7f7697ca38
Merge pull request #1099 from harshavardhana/fix-lock
...
setBucketMetadata: Fix a deadlock
2016-02-05 16:47:05 -08:00
Harshavardhana
8bf1045645
setBucketMetadata: Fix a deadlock.
2016-02-05 15:48:08 -08:00
Harshavardhana
c922dd6fbd
Merge pull request #1097 from harshavardhana/mimedb
...
fs: Use mimedb now for contentType
2016-02-05 15:22:49 -08:00
Harshavardhana
6f80380497
fs: Use mimedb now.
2016-02-05 15:09:23 -08:00
Harshavardhana
35dcccb4cd
Merge remote-tracking branch 'abperiasamy/mimedb' into mimedb
2016-02-05 15:03:47 -08:00
Harshavardhana
fca425f156
Merge pull request #1092 from harshavardhana/more-fixes
...
multipart: Code cleanup
2016-02-05 14:58:43 -08:00
Harshavardhana
a4c005ce30
multipart: Code cleanup
...
- More locking cleanup. Fix naming convention.
- Simplify concatenation and blocking calls.
2016-02-05 14:42:09 -08:00
Harshavardhana
5b4c73e74d
Merge pull request #1096 from krishnasrinivas/web-remove-object
...
JSONrpc: implement removeObject RPC call
2016-02-05 10:22:24 -08:00
Krishna Srinivas
3a8fff46f9
JSONrpc: implement removeObject RPC call
2016-02-05 19:46:36 +05:30
Bala.FA
d79fcb1800
fix: handle Transfer-Encoding for make bucket
...
In case of make bucket, there is a chance of Transfer-Encoding is sent
where Content-Length is missing. This patch fixes the problem by
checking whether Transfer-Encoding: chunked is set along with
Content-Length.
2016-02-05 19:01:39 +05:30
Anand Babu (AB) Periasamy
d8abb36653
contentdb replaced by new mimedb
2016-02-05 03:49:24 -08:00
Harshavardhana
3a55d05eff
Merge pull request #1091 from harshavardhana/gpg
...
build: Add build dependency check for 'gpg'
2016-02-05 03:20:23 -08:00
Harshavardhana
ddc99e3112
build: Add build dependency check for 'gpg'
2016-02-05 03:10:23 -08:00
Harshavardhana
198a92c3c4
Merge pull request #1090 from harshavardhana/multipart
...
fs: Add granular locking.
2016-02-04 21:46:06 -08:00
Harshavardhana
8557cbc9b7
fs: Add granular locking.
2016-02-04 20:40:58 -08:00
Harshavardhana
f2113d35be
Merge pull request #1089 from harshavardhana/docker
...
docker: Fix docker build.
2016-02-04 19:13:14 -08:00
Harshavardhana
c9d2904e42
docker: Fix docker build.
2016-02-04 18:22:37 -08:00
Harshavardhana
1c75d35c26
Merge pull request #1088 from harshavardhana/enable-ui-assets
...
ui-assets: Integrate UI assets.
2016-02-04 18:17:27 -08:00
Harshavardhana
a066184bed
ui-assets: Integrate UI assets.
2016-02-04 18:07:05 -08:00
Harshavardhana
53a983659e
Merge pull request #1084 from krishnasrinivas/go-bindata-assetfs
...
UI: vendorize github.com/elazarl/go-bindata-assetfs which is needed by ui-assets.go
2016-02-04 18:06:14 -08:00
Harshavardhana
ef47255f5e
Merge pull request #1087 from harshavardhana/handlers
...
handlers: Fix the naming of all handlers.
2016-02-04 15:30:19 -08:00
Harshavardhana
012fbe756b
handlers: Fix the naming of all handlers.
2016-02-04 15:02:53 -08:00
Harshavardhana
4d97c042da
Merge pull request #1086 from krishnasrinivas/browser-cache
...
browser-caching: enable browser caching for webUI
2016-02-04 14:41:38 -08:00
Krishna Srinivas
a344e7713a
browser-caching: enable browser caching for WebUI
2016-02-05 03:54:05 +05:30
Harshavardhana
09a54f9032
Merge pull request #1085 from harshavardhana/fs-api
...
fs: Cleanup Golang errors to be called 'e' and probe to be called as …
2016-02-04 14:02:19 -08:00
Harshavardhana
7a3409c309
fs: Cleanup Golang errors to be called 'e' and probe to be called as 'err'
...
- Replace the ACL checks back, remove them when bucket
policy is implemented.
- Move FTW (File Tree Walk) into ioutils package.
2016-02-04 13:43:52 -08:00
Krishna Srinivas
d038393156
UI: vendorize github.com/elazarl/go-bindata-assetfs which is needed by ui-assets.go
2016-02-04 16:50:34 +05:30
Harshavardhana
b49f21ec82
Merge pull request #1082 from harshavardhana/about-api
...
api: Implement About API.
2016-02-03 23:19:40 -08:00
Harshavardhana
e63a982dee
api: Implement About API.
2016-02-03 22:46:45 -08:00
Harshavardhana
a1c6e4055b
Merge pull request #1073 from harshavardhana/createobject
...
fs: Fail createObject with appropriate message.
2016-02-03 21:56:44 -08:00
Harshavardhana
835b297ba7
fs: Fail createObject with appropriate message.
...
Fail createObject() if a file already exists and one attempts
to create a prefix/directory by same name.
Send an approriate error back to the client as 409 Conflict.
2016-02-03 21:49:36 -08:00
Harshavardhana
729e032a50
Merge pull request #1079 from harshavardhana/implement-stat
...
web: GetObjectURL should check if file exists before generating URL.
2016-02-03 00:15:54 -08:00
Harshavardhana
64b7da4686
web: GetObjectURL should check if file exists before generating URL.
...
Fixes - https://github.com/minio/miniobrowser/issues/20
2016-02-03 00:00:36 -08:00
Harshavardhana
ff0dd38957
Merge pull request #1078 from harshavardhana/auto-expiry
...
expiry: Remove auto-expiry.
2016-02-02 23:40:30 -08:00
Harshavardhana
454d71cafa
expiry: Remove auto-expiry.
...
Move the logic outside and use scripting, cronjob to delete files.
Fixes #1019
2016-02-02 19:35:51 -08:00
Harshavardhana
15924a8f05
Merge pull request #1077 from harshavardhana/flags
...
flags: Remove anonymous, ratelimit, json and web-address flags.
2016-02-02 19:01:52 -08:00
Harshavardhana
df91661ec6
flags: Remove anonymous, ratelimit, json and web-address flags.
...
- Web address now uses the port + 1 from the API address port directly.
- Remove ratelimiting, ratelimiting will be achieved if necessary through
iptables.
- Remove json flag, not needed anymore.
- Remove anonymous flag, server will be no more anonymous for play.minio.io
we will use demo credentials.
2016-02-02 18:37:09 -08:00
Harshavardhana
e39b6caada
Merge pull request #1075 from harshavardhana/content-type
...
statObject: Make sure to lowercase file extensions.
2016-02-02 18:02:59 -08:00
Harshavardhana
81fcbd2a54
statObject: Make sure to lowercase file extensions.
2016-02-02 17:54:59 -08:00
Harshavardhana
b01594ac33
Merge pull request #1072 from harshavardhana/query
...
vendor: Update minio-go library with fixes for object listing.
2016-02-02 14:56:49 -08:00
Harshavardhana
de9682a4e7
vendor: Update minio-go library with fixes for objectlisting.
2016-02-02 11:59:55 -08:00
Harshavardhana
9ddfa2529c
Merge pull request #1070 from harshavardhana/bug-fix
...
web: ListObjects is delimited, do not send a stat on prefix.
2016-02-01 13:33:23 -08:00
Harshavardhana
b3bde61396
web: ListObjects is delimited, do not send a stat on prefix.
2016-02-01 12:47:46 -08:00
Harshavardhana
9dfce111d9
Merge pull request #1069 from harshavardhana/list-objects
...
contentType: Reply back proper contentTypes based on the file extension.
2016-02-01 12:35:56 -08:00
Harshavardhana
0aedb67de0
contentType: Reply back proper contentTypes based on the file extension.
...
Currently the server would set 'application/octet-stream' for all
objects, set this value based on the file extension transparently.
This is useful in case of minio browser to facilitate displaying
proper icons for the different mime data types.
2016-02-01 12:19:58 -08:00
Harshavardhana
23ca11f75b
Merge pull request #1068 from harshavardhana/update-doc
...
Add doc change.
2016-02-01 11:11:39 -08:00
Harshavardhana
d55f72f09a
Add doc change.
2016-02-01 02:45:19 -08:00
Harshavardhana
141e8029bf
Merge pull request #1067 from harshavardhana/minio-message
...
server: Add new set of message.
2016-01-30 18:41:46 -08:00
Harshavardhana
a0c753b6eb
server: Add new set of message.
2016-01-30 18:33:33 -08:00
Harshavardhana
0784969767
Merge pull request #1065 from harshavardhana/read-me
...
doc: Update readme.md
2016-01-29 18:15:10 -08:00
Harshavardhana
cce2f41125
doc: Update readme.md
2016-01-29 18:09:34 -08:00
Harshavardhana
25cd37b209
Merge pull request #1063 from harshavardhana/trim-inputs
...
jwt: Trim username and password strings
2016-01-29 02:18:58 -08:00
Harshavardhana
716fde3b93
Merge pull request #1062 from notnoopci/no-empty-md5-header
...
Don't set empty ETag values
2016-01-28 20:51:59 -08:00
Harshavardhana
8af8889a36
jwt: Trim username and password strings
2016-01-28 20:48:41 -08:00
Mahmood Ali
43685788ab
Don't set empty ETag values
...
Currently, metadata.Md5 value isn't populated, yet the ETag is set to
`""`, causing AWS Java SDK to fail integrity checks with GetObject
api calls.
2016-01-28 22:57:23 -05:00
Harshavardhana
52f00042b4
Merge pull request #1061 from harshavardhana/order-presign
...
presign: Verify query params for presign individually
2016-01-28 12:27:13 -08:00
Harshavardhana
2469c9c591
presign: Verify query params for presign individually
...
Incoming request params in presigned can come in different order
for different implementations. Rather than verifying a full string
we should verify individual params instead.
This patch fixes an incompatibility issue with AWS SDK Java.
Fixes #1059 - Thanks to @notnoopci for reporting this problem.
2016-01-28 12:16:56 -08:00
Harshavardhana
c4588b3cb9
Merge pull request #1060 from krishnasrinivas/cors-fixes
...
CORS: cors handling should be before auth handling. cors should allow PUT.
2016-01-28 10:49:09 -08:00
Krishna Srinivas
81b255511f
CORS: cors handling should be before auth handling. cors should allow PUT.
2016-01-28 22:54:03 +05:30
Harshavardhana
ed69c58490
Merge pull request #1057 from harshavardhana/fix
...
listObjects: ListObjects should have idempotent behavior.
2016-01-28 03:24:16 -08:00
Harshavardhana
5934a00058
listObjects: ListObjects should have idempotent behavior.
...
listObjects was returning inconsistent results, i.e missing
entries during recursive and non-recursive listing. This led
to 'mc mirror' copying contents repeatedly consisdering
these files to be missing on the destination.
This patch addresses this problem - fixes #1056
2016-01-28 03:17:40 -08:00
Anand Babu (AB) Periasamy
2d8512465f
Merge pull request #1047 from fwessels/master
...
Updated mc and aws client doc strings
2016-01-27 12:48:08 -08:00
Harshavardhana
be5a865764
Merge pull request #1055 from harshavardhana/update
...
update: Minio fix update url.
2016-01-27 11:54:18 -08:00
Harshavardhana
0799a0bec5
update: Minio fix update url.
2016-01-27 11:43:26 -08:00
Harshavardhana
dfc84dd451
Merge pull request #1054 from harshavardhana/json-web
...
jwt: Deprecate RSA usage, use HMAC instead.
2016-01-27 03:48:49 -08:00
Harshavardhana
db387912f2
jwt: Deprecate RSA usage, use HMAC instead.
...
HMAC is a much simpler implementation, providing the same
benefits as RSA, avoids additional steps and keeps the code
simpler.
This patch also additionally
- Implements PutObjectURL API.
- GetObjectURL, PutObjectURL take TargetHost as another
argument for generating URL's for proper target destination.
- Adds experimental TLS support for JSON RPC calls.
2016-01-27 03:38:33 -08:00
Harshavardhana
0c96ace8ad
Merge pull request #1053 from harshavardhana/infinite-loop
...
listObjects: Marker should be unescaped before being used internally.
2016-01-26 23:38:57 -08:00
Harshavardhana
9ca3372870
listObjects: Marker should be unescaped before being used internally.
...
Without this change listObjects() goes into an infinite loop for
files which have special characters i.e "++" encoded with "%2B%2B".
We have to unescape and convert them to their native representation
before being used internally.
Fixes #1052
2016-01-26 23:32:59 -08:00
Harshavardhana
5d87fdb35c
Merge pull request #1051 from harshavardhana/fix-multipart
...
multipart: NewMultipartUpload shouldn't return empty UploadID
2016-01-26 15:15:44 -08:00
Harshavardhana
2e311168ee
multipart: NewMultipartUpload shouldn't return empty UploadID
...
Existing code
```
{
if os.IsNotExist(e) {
e = os.MkdirAll(objectDir, 0700)
if e != nil {
return "", probe.NewError(e)
}
}
return "", probe.NewError(e) ---> Error was here.
}
```
For a successful 'MkdirAll' it would still return an empty uploadID,
but the 'error' would be nil. This would succeed the request but
client would fail.
Fix is to re-arrange the logic. Thanks to Alexander Neumann @fd0, for
reporting this problem.
2016-01-26 15:00:34 -08:00
Harshavardhana
e6df2d639c
Merge pull request #1050 from fwessels/aws-cli
...
Updated 'aws s3 ls' example in README.md to include s3://
2016-01-26 14:28:33 -08:00
frankw
dd4c08cd79
Updated 'aws s3 ls' example in README.md to include s3:// (which is apparently needed for other 'aws s3' commands such as 'cp' or 'rm')
2016-01-26 23:09:06 +01:00
frankw
9a0cd354a3
Updated 'aws s3 ls' example in README.md to include s3:// (which is apparently needed for other aws s3 commands such as cp or rm)
2016-01-26 22:55:12 +01:00
frankw
5ad9167673
Merge remote-tracking branch 'upstream/master'
2016-01-26 22:50:10 +01:00
Harshavardhana
68a25aa425
Merge pull request #1039 from harshavardhana/channel-ftw
...
listObjects: Channel based file tree walk.
2016-01-26 12:41:21 -08:00
Harshavardhana
18375b7794
ioutils: Add tests
2016-01-26 12:34:04 -08:00
Harshavardhana
13feabefd5
diskInfo: Add DiskInfo API
2016-01-26 12:08:45 -08:00
Harshavardhana
1341fb79c3
listBuckets: Bump up the limit of max buckets to 1000.
2016-01-26 11:49:17 -08:00
Harshavardhana
f5d6be158e
listObjects: Simplify channel based changes.
2016-01-26 02:19:55 -08:00
frankw
0a5094c73a
Added Alias argument in configuring/using mc client for minio server
...
Both in README.md as well as sample commands when starting minio server
2016-01-26 10:42:21 +01:00
Harshavardhana
682020ef2f
listObjects: Channel based changes.
...
Supports:
- prefixes
- marker
2016-01-25 20:39:38 -08:00
Krishna Srinivas
9e18bfa60e
listObjects: Channel based ftw - initial implementation.
2016-01-25 18:58:07 -08:00
Harshavardhana
67a70eb6d6
Merge pull request #1046 from harshavardhana/bucket-not-empty
...
deleteBucket: Directory not empty error on windows is "directory is n…
2016-01-25 18:04:04 -08:00
Harshavardhana
2ec9b16667
deleteBucket: Directory not empty error on windows is "directory is not empty"
2016-01-25 17:58:43 -08:00
Harshavardhana
35d4521ece
Merge pull request #1045 from harshavardhana/get-object-cleanups
...
api: More cleanups at WebAPI.
2016-01-25 17:45:53 -08:00
Harshavardhana
ae2f15c6d0
api: More cleanups at WebAPI.
...
- Fixes a bug where bucketName was not denormalized.
- Remove unneeded functions from jwt.go
2016-01-25 17:30:08 -08:00
Harshavardhana
f1ea609175
Merge pull request #1042 from harshavardhana/jwt-api
...
jwt: Add JWT support for minio server.
2016-01-25 16:45:22 -08:00
Harshavardhana
497f13d733
api: Various fixes.
...
- limit list buckets to limit only 100 buckets, all uppercase buckets
are now lowercase and work transparently with all calls.
- Change disk.Stat to disk.GetInfo and return back disk.Info{} struct.
- Introduce new ioutils package which implements ReadDirN(path, n),
ReadDirNamesN(path, n)
2016-01-25 16:08:27 -08:00
Harshavardhana
432a073e6b
Add MakeBucket API.
2016-01-24 22:45:22 -08:00
Harshavardhana
3f1c4bb4b0
Bring in the list APIs implemented by Bala <bala@minio.io>
2016-01-24 16:39:48 -08:00
Harshavardhana
0a9496462a
jwt: Add JWT support for minio server.
...
Please read JWT.md before using this feature.
2016-01-22 17:38:05 -08:00
Harshavardhana
d8fa68ff7e
Merge pull request #1040 from hackintoshrao/make-file-edit
...
Minor changes to Makefile
2016-01-20 01:56:59 -08:00
Karthic Rao
b457a61cb2
Minor changes to Makefile to avoid the make failure when GOPATH/bin is not part of PATH
2016-01-20 14:46:12 +05:30
Anand Babu (AB) Periasamy
9cb590d800
Merge pull request #1038 from harshavardhana/shadow
...
server: Fix shadowing bug reported by go vet on go1.6beta2
2016-01-18 20:56:42 -08:00
Harshavardhana
1aec985c00
server: Fix shadowing bug reported by go vet on go1.6beta2
2016-01-18 18:45:24 -08:00
Harshavardhana
092ed972d0
Merge pull request #1037 from harshavardhana/add-config
...
serverConfig: Add a new region config entry.
2016-01-17 01:50:30 -08:00
Harshavardhana
cb7b2762f9
serverConfig: Add a new region config entry.
...
To change default region from 'us-east-1' to 'custom'.
Add a region value in your 'config.json'.
"version": "2",
"credentials": {
"accessKeyId": "****************",
"secretAccessKey": "***************",
"region": "my-region"
},
2016-01-17 01:39:11 -08:00
Harshavardhana
8a7bf0dde0
Merge pull request #1036 from harshavardhana/docker
...
build: Do not hardcode docker binary path
2016-01-15 10:42:13 -08:00
Harshavardhana
023f799820
build: Do not hardcode docker binary path
...
Fixes #1035
2016-01-15 10:36:45 -08:00
Harshavardhana
8ff43086fb
Merge pull request #1034 from harshavardhana/handle-cgo
...
build: Handle builds on env where CGO_ENABLED=0
2016-01-14 18:28:47 -08:00
Harshavardhana
88686dc6e3
build: Handle builds on env where CGO_ENABLED=0
...
Fixes #1033
2016-01-14 18:19:01 -08:00
Harshavardhana
3cb92bdf9c
Merge pull request #1032 from koolhead17/command-modification
...
Modified command for adding a host to minio.
2016-01-13 10:52:12 -08:00
koolhead17
d4dbd09a9c
Modified command for adding a host to minio.
2016-01-13 23:36:42 +05:30
Harshavardhana
a6941c0b44
Merge pull request #1030 from harshavardhana/expect-100
...
signature: Add aws-cli work-around for now.
2016-01-09 10:53:54 -08:00
Harshavardhana
8cdaf87c8f
signature: Add aws-cli work-around for now.
...
Golang http server strips off 'Expect' header, if the
client sent this as part of signed headers we need to
handle otherwise we would see a signature mismatch.
`aws-cli` sets this as part of signed headers which is
a bad idea since servers trying to implement AWS
Signature version '4' will all encounter this issue.
According to
http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.20
Expect header is always of form:
Expect = "Expect" ":" 1#expectation
expectation = "100-continue" | expectation-extension
So it safe to assume that '100-continue' is what would
be sent, for the time being keep this work around.
2016-01-09 10:47:59 -08:00
Harshavardhana
70bd51994c
Merge pull request #1026 from harshavardhana/add-doc
...
Add aws-cli documentation.
2016-01-08 20:01:38 -08:00
Harshavardhana
80d175b34c
Add aws-cli documentation.
2016-01-08 19:59:23 -08:00
Harshavardhana
c963fe244d
Merge pull request #1022 from koolhead17/patch-1
...
Update README.md
2016-01-08 10:33:10 -08:00
koolhead17
888111816d
Update README.md
...
minor change, adding '--' with help option as we introduced the same with mc.
2016-01-08 23:47:08 +05:30
Harshavardhana
3bf7685327
Merge pull request #1021 from harshavardhana/transfer-encoding
...
http: Enable Transfer-Encoding chunked transfer
2016-01-08 00:50:31 -08:00
Harshavardhana
0c6a6dc380
http: Enable Transfer-Encoding chunked transfer
...
Fixes #1020
2016-01-08 00:47:20 -08:00
Harshavardhana
5e06c15b44
Merge pull request #1017 from koolhead17/GOenv-change
...
Modified instructions for installing Go.
2015-12-29 23:52:22 -08:00
koolhead17
f13ad3e02e
Modifed instructions for installing Go.
2015-12-30 12:23:28 +05:30
Harshavardhana
82212f5fbc
Merge pull request #1016 from harshavardhana/fix-signautre
...
s3cmd: Fix signature issues related to s3cmd.
2015-12-28 18:12:21 -08:00
Harshavardhana
d955ce4123
s3cmd: Fix signature issues related to s3cmd.
...
Support regions both 'us-east-1' and 'US' (short hand for US Standard)
honored by S3.
2015-12-28 18:05:28 -08:00
Harshavardhana
7228bc9919
Merge pull request #1015 from harshavardhana/location
...
location: Return a set location properly on complete multipart upload…
2015-12-28 15:30:43 -08:00
Harshavardhana
e7474bed43
location: Return a set location properly on complete multipart upload request.
2015-12-28 15:20:06 -08:00
Harshavardhana
d548316343
Merge pull request #1014 from harshavardhana/fixes
...
handlers: read ContentLength directly from http.Request
2015-12-27 23:07:51 -08:00
Harshavardhana
2f67559684
handlers: read ContentLength value directly from http.Request.
...
Do not look for Content-Length in headers and try to convert them into
integer representations use ContentLength field from *http.Request*.
If Content-Length is understood to be as '-1' then treat it as an error
condition, since it could be a malformed body to crash the server.
Fixes #1011
2015-12-27 23:03:32 -08:00
Harshavardhana
7aab7ba946
Merge pull request #1013 from harshavardhana/bucket-location
...
bucket-location: Implement bucket location response.
2015-12-27 00:50:42 -07:00
Harshavardhana
0345c8fffb
bucket-location: Implement bucket location response.
2015-12-27 00:48:11 -07:00
Harshavardhana
155a4110b1
Merge pull request #1010 from harshavardhana/update-url
...
Fix update URL.
2015-12-22 17:31:41 -08:00
Harshavardhana
7385302578
Fix update URL.
2015-12-22 17:28:47 -08:00
Harshavardhana
a9c634cc56
Merge pull request #1006 from harshavardhana/fix-mc-download-url
...
minio: Fix mc and minio download URLs.
2015-12-22 08:21:20 -08:00
Harshavardhana
523e75759d
minio: Fix mc and minio download URLs.
...
Fixes #991
2015-12-13 13:57:56 -08:00
Anand Babu (AB) Periasamy
738c543720
Merge pull request #1004 from harshavardhana/version
...
s3cmd: Handle support for s3cmd.
2015-12-09 17:21:46 -08:00
Harshavardhana
3c71c5c80c
s3cmd: Handle support for s3cmd.
2015-12-09 17:16:53 -08:00
Harshavardhana
7187668828
Merge pull request #1002 from harshavardhana/access
...
accessPolicy: Access policy response is not correct.
2015-12-07 14:59:57 -08:00
Harshavardhana
b07eed56be
accessPolicy: Access policy response is not correct.
2015-12-07 14:52:27 -08:00
Harshavardhana
83b2cd9ceb
Merge pull request #1001 from harshavardhana/user
...
pkg/user: Add pending tests.
2015-12-07 14:18:48 -08:00
Harshavardhana
1bfb490f90
pkg/user: Add pending tests.
2015-12-07 14:13:54 -08:00
Harshavardhana
06879b3b44
Merge pull request #1000 from harshavardhana/fs-bucket
...
bucketName: relax bucket names, now allow numbers as starting charact…
2015-12-07 14:04:26 -08:00
Harshavardhana
4fc161ddb1
bucketName: relax bucket names, now allow numbers as starting characters.
2015-12-07 13:58:55 -08:00
Harshavardhana
1413e761cf
Merge pull request #998 from harshavardhana/config-folder
...
minio: Add config-folder option.
2015-12-07 13:58:33 -08:00
Harshavardhana
836f5204af
minio: Add config-folder option.
...
Fixes #997
2015-12-07 12:34:09 -08:00
Harshavardhana
a97c4ebce3
Merge pull request #999 from harshavardhana/acl
...
ignore-handlers: Enhance ignore handlers to cater for bucket resource…
2015-12-07 12:24:00 -08:00
Harshavardhana
57430fe183
ignore-handlers: Enhance ignore handlers to cater for bucket resources with or without separators
...
Fixes an issue which we saw with minio-py
2015-12-07 12:06:27 -08:00
Harshavardhana
16cf9d9055
Merge pull request #996 from minio/harshavardhana-patch-1
...
Add s3cmd support status
2015-12-06 13:45:25 -08:00
Harshavardhana
aac85334fd
Add s3cmd support status
...
Fixes #987
2015-12-06 13:40:48 -08:00
Harshavardhana
6255f186dd
Merge pull request #995 from minio/harshavardhana-patch-1
...
Update README.md
2015-12-04 16:57:30 -08:00
Harshavardhana
330ceddfd4
Update README.md
2015-12-04 16:26:09 -08:00
Anand Babu (AB) Periasamy
bed528d569
Merge pull request #994 from abperiasamy/contentdb-race
...
fixes race in Init
2015-12-03 01:11:14 -08:00
Anand Babu (AB) Periasamy
8e8538175b
fixes race in Init
2015-12-03 01:08:05 -08:00
Harshavardhana
c327c56a16
Merge pull request #992 from abperiasamy/contentdb
...
contentdb provides file.ext to content-type lookups.
2015-12-02 17:00:09 -08:00
Anand Babu (AB) Periasamy
25df427383
contentdb file.ext to content-type lookups
2015-12-02 16:55:26 -08:00
Harshavardhana
5b18c787f0
Merge pull request #990 from harshavardhana/update-readme
...
doc: update README.md
2015-12-02 11:31:59 -08:00
Harshavardhana
9c64a232e2
doc: update README.md
2015-12-02 11:28:41 -08:00
Harshavardhana
df94b4d3ca
Merge pull request #989 from harshavardhana/maintainers
...
doc: Add MAINTAINERS.md
2015-12-02 10:55:30 -08:00
Harshavardhana
661229d7f7
doc: Add MAINTAINERS.md
2015-12-02 10:50:54 -08:00
Harshavardhana
66b3918b04
Merge pull request #988 from harshavardhana/add-release
...
release: Add release scripts.
2015-12-02 10:46:15 -08:00
Harshavardhana
9722eaf02c
release: Add release scripts.
2015-12-02 10:43:41 -08:00
Harshavardhana
f7029fc6f1
Merge pull request #986 from harshavardhana/fix-makefile
...
makefile: Fix docker image
2015-11-28 12:13:35 -08:00
Harshavardhana
c22eb6d2c5
makefile: Fix docker image
2015-11-28 12:10:13 -08:00
Harshavardhana
62e4d0d74b
Merge pull request #985 from minio/harshavardhana-patch-1
...
Fix docker.md add data volume container
2015-11-28 11:38:18 -08:00
Harshavardhana
1111e53a12
Fix docker.md with data volume container
2015-11-28 11:33:23 -08:00
Harshavardhana
910e270c1c
Merge pull request #984 from minio/harshavardhana-patch-1
...
Update Docker.md
2015-11-27 20:54:24 -08:00
Harshavardhana
10b6a6a74c
Update Docker.md
2015-11-27 20:41:41 -08:00
Harshavardhana
f35128eec5
Merge pull request #983 from harshavardhana/improve-update
...
update: Re-write update command.
2015-11-25 21:23:56 -08:00
Harshavardhana
3f9e1c1e68
update: Re-write update command.
2015-11-25 21:17:57 -08:00
Harshavardhana
b429bd7ceb
Merge pull request #982 from harshavardhana/aws-sdk-go
...
doc: Cleanup aws-sdk-go.md
2015-11-23 13:59:30 -08:00
Harshavardhana
3478c8a9a2
doc: Cleanup aws-sdk-go.md
2015-11-23 13:53:55 -08:00
Harshavardhana
3986e3cf2b
Merge pull request #981 from harshavardhana/signature
...
signature/region: Remove 'milkyway' and use 'us-east-1' as default.
2015-11-23 13:43:02 -08:00
Harshavardhana
7c91a8495f
signature/region: Remove 'milkyway' and use 'us-east-1' as default.
...
Fixes #980
2015-11-23 13:40:23 -08:00
Harshavardhana
fe2cb11e76
Merge pull request #979 from harshavardhana/fix-docs
...
minio: Fix documentation.
2015-11-23 13:32:20 -08:00
Harshavardhana
b9aa6a03b8
minio: Fix documentation.
2015-11-23 13:29:08 -08:00
Harshavardhana
9206a6ade9
Merge pull request #978 from harshavardhana/fs-fix
...
fs: Filter out $multiparts properly.
2015-11-22 01:58:43 -08:00
Harshavardhana
a328120e4d
fs: Filter out $multiparts properly.
...
Relax md5 requirement during complete multipart upload - ref #977
2015-11-22 01:49:57 -08:00
Harshavardhana
2392252a7c
Merge pull request #975 from harshavardhana/cli
...
cli: vendorize to new CLI package updates.
2015-11-21 09:03:58 -08:00
Harshavardhana
ab3fd8ea7f
cli: vendorize to new CLI package updates.
...
- Fix a new line issue for minioHelpTemplate.
- Fixes #974
2015-11-21 09:01:18 -08:00
Harshavardhana
ffa22cf31e
Merge pull request #973 from harshavardhana/atomic-sync
...
atomic: do not sync by default, if needed use CloseAndSync()
2015-11-17 23:13:53 -08:00
Harshavardhana
da2a6066c7
atomic: do not sync by default, if needed use CloseAndSync()
2015-11-17 23:04:18 -08:00
Harshavardhana
3bdee8e4f6
Merge pull request #972 from harshavardhana/merge-atomic
...
fs: use new atomic package - use FileCreateWithPrefix() API
2015-11-17 20:50:35 -08:00
Harshavardhana
35b9f965f1
fs: use new atomic package - use FileCreateWithPrefix() API
2015-11-17 16:32:20 -08:00
Harshavardhana
26f83f108a
Merge pull request #971 from harshavardhana/update
...
doc: update download locations in Readme.md
2015-11-14 02:46:36 -08:00
Harshavardhana
3e4d69be87
doc: update download locations in Readme.md
2015-11-14 02:39:02 -08:00
Harshavardhana
1d123e479e
Merge pull request #970 from harshavardhana/update
...
vendorize: Add new changes for sha256, sha512 for 32bit support.
2015-11-14 00:58:19 -08:00
Harshavardhana
ff161a9943
vendorize: Add new changes for sha256, sha512 for 32bit support.
2015-11-14 00:55:24 -08:00
Harshavardhana
8046b24b97
Merge pull request #969 from harshavardhana/32bit
...
386: Support minio server on 32bit linux.
2015-11-14 00:35:16 -08:00
Harshavardhana
f8e59e8399
386: Support minio server on 32bit linux.
2015-11-14 00:29:18 -08:00
Harshavardhana
a384ae868d
Merge pull request #968 from harshavardhana/update
...
update: change update command reflecting new changes on dl.minio.io
2015-11-13 20:35:26 -08:00
Harshavardhana
d65f3422a9
update: change update command reflecting new changes on dl.minio.io
2015-11-13 20:31:19 -08:00
Harshavardhana
249c444e88
Merge pull request #967 from harshavardhana/handle-readonly-buckets
...
acl: Handle readonly buckets properly
2015-11-13 20:10:50 -08:00
Harshavardhana
e1a33deabf
acl: Handle readonly buckets properly
2015-11-13 20:07:39 -08:00
Harshavardhana
ff59181527
Merge pull request #966 from harshavardhana/readdir-optimize
...
fs: Improve upon proper lexical ordering for ListObjects()
2015-11-11 14:56:33 -08:00
Harshavardhana
d668117a99
fs: Improve upon proper lexical ordering for ListObjects()
...
Handle sorting properly making sure that we treat fs like a
flat namespace.
2015-11-11 14:52:03 -08:00
Harshavardhana
ae52fc682d
Merge pull request #965 from harshavardhana/handle-spaces
...
signature: Handle all corner cases with spaces.
2015-11-10 18:17:44 -08:00
Harshavardhana
b0a89b1f1b
signature: Handle all corner cases with spaces.
2015-11-10 18:11:49 -08:00
Harshavardhana
cefb9b1651
Merge pull request #964 from harshavardhana/attempt-windows
...
build: Attempt to enable windows compilation
2015-11-08 03:44:49 -08:00
Harshavardhana
c67a8cb6e5
build: Attempt to enable windows compilation
2015-11-08 03:40:53 -08:00
Harshavardhana
152c73e1ce
Merge pull request #961 from krishnasrinivas/docker-fixes2
...
docker: second --ldflags was overriding the first --ldflags option
2015-11-07 15:37:43 -08:00
Krishna Srinivas
f77851bee0
docker: second --ldflags was overriding the first --ldflags option
2015-11-07 15:21:01 -08:00
Harshavardhana
03f185db5f
Merge pull request #963 from harshavardhana/operations
...
doc: Add comments for router operations
2015-11-07 14:30:54 -08:00
Harshavardhana
c44754629e
doc: Add comments for router operations
2015-11-07 14:15:22 -08:00
Harshavardhana
53a464b5f1
Merge pull request #960 from masu-mi/fix-trailing-slash
...
modify registerCloudStorageAPI for improving compatibility with S3.
2015-11-07 03:24:46 -08:00
Kanai Masumi
84de2e33c4
Fix: permit trailing slash for compatible with S3.
...
ex.
s3cmd requests to path:`/<bucket>/` for PutBucket.
2015-11-07 20:22:13 +09:00
Harshavardhana
f00f674b69
Merge pull request #962 from harshavardhana/update-ldflags-windows
...
build: update LDFLAGS for windows
2015-11-07 00:14:44 -08:00
Harshavardhana
2f98fa0a14
build: update LDFLAGS for windows
2015-11-07 00:12:35 -08:00
Harshavardhana
fe1684d706
Merge pull request #958 from krishnasrinivas/docker-fixes
...
docker: the docker image will now contain just the static binary
2015-11-06 21:11:13 -08:00
Krishna Srinivas
440bec28d9
docker: the docker image will now contain just the static binary
2015-11-06 20:44:58 -08:00
Harshavardhana
8ef4ec24ca
Merge pull request #959 from minio/call-home
...
Add missing Jobs section
2015-11-06 19:02:08 -08:00
Harshavardhana
7c17035ce2
Update README.md
2015-11-06 18:58:28 -08:00
Harshavardhana
fc155f67d9
Merge pull request #957 from harshavardhana/improve-logging
...
logger: Improve logger input argument handling and colorize outputs
2015-11-03 22:54:53 -08:00
Harshavardhana
15909e5463
logger: Improve logger input argument handling and colorize outputs
2015-11-03 22:51:42 -08:00
Harshavardhana
02d843ef5d
Merge pull request #954 from harshavardhana/minio-commit
...
minio/probe: Add missing Commit-ID for probe appinfo
2015-11-02 15:15:51 -08:00
Harshavardhana
74b62f1798
minio/probe: Add missing Commit-ID for probe appinfo
2015-11-02 15:05:57 -08:00
Harshavardhana
2601b3c487
Merge pull request #953 from harshavardhana/ldflags-style
...
build: Versioning now populated through ldflags
2015-11-02 02:43:04 -08:00
Harshavardhana
7845515f36
build: Versioning now populated through ldflags
2015-11-02 02:37:26 -08:00
Harshavardhana
ef04507bc4
Merge pull request #952 from harshavardhana/server-pkg
...
fs/bucket: Move bucket metadata into buckets.json
2015-11-01 21:27:52 -08:00
Harshavardhana
ab15f56a61
fs/bucket: Move bucket metadata into buckets.json
2015-11-01 21:25:01 -08:00
Harshavardhana
38aa7e1678
Merge pull request #951 from harshavardhana/bucket-delete
...
Simplify bucket delete - remove only bucket directory, no need to rec…
2015-10-30 16:11:36 -07:00
Harshavardhana
baf66988e9
Simplify bucket delete - remove only bucket directory, no need to recursively traverse
2015-10-30 16:03:18 -07:00
Harshavardhana
7d38967f22
Merge pull request #950 from harshavardhana/enhance-cors
...
Change default options for cors to handle HEAD and allow all headers
2015-10-30 12:14:21 -07:00
Harshavardhana
bd0436bf98
Change default options for cors to handle HEAD and allow all headers
2015-10-30 11:56:50 -07:00
Harshavardhana
7df81f8707
Merge pull request #949 from krishnasrinivas/rm-empty-dirs
...
Remove empty directories while removing an oobject
2015-10-30 00:16:27 -07:00
Krishna Srinivas
0010f0ee10
Remove empty directories while removing an oobject
2015-10-29 23:30:16 -07:00
Harshavardhana
ff7fa3637c
Merge pull request #948 from minio/harshavardhana-patch-1
...
Update INSTALLGO.md
2015-10-28 17:16:29 -07:00
Harshavardhana
36854f266d
Update INSTALLGO.md
2015-10-28 15:43:45 -07:00
Harshavardhana
679d099b16
Merge pull request #947 from harshavardhana/through-typo
...
Fix description typo
2015-10-27 11:59:09 -07:00
Harshavardhana
49260abad4
Fix description typo
2015-10-27 11:38:28 -07:00
Anand Babu (AB) Periasamy
d0ff64c3c8
Merge pull request #946 from abperiasamy/build-constants
...
build time constants
2015-10-26 02:46:16 -07:00
Anand Babu (AB) Periasamy
588833d06f
build time constants
2015-10-26 02:41:04 -07:00
Harshavardhana
07839caf9b
Merge pull request #945 from harshavardhana/release-tag
...
Update new changes in probe and add setAppInfo
2015-10-25 11:16:35 -07:00
Harshavardhana
3566d08c52
Update new changes in probe and add setAppInfo
2015-10-25 11:11:29 -07:00
Harshavardhana
9b2cff14bb
Merge pull request #944 from abperiasamy/new-probe
...
updated probe
2015-10-23 20:12:47 -07:00
Anand Babu (AB) Periasamy
8e68591933
updated probe
2015-10-23 20:09:14 -07:00
Anand Babu (AB) Periasamy
e1a992908c
Merge pull request #943 from abperiasamy/pkg-update
...
added pkg-update to update a vendorized package
2015-10-23 19:33:35 -07:00
Anand Babu (AB) Periasamy
a8f75f5cc1
added pkg-update to update a vendorized package
2015-10-23 19:29:44 -07:00
Harshavardhana
19bcbf76f1
Merge pull request #942 from harshavardhana/sort-unique
...
Leverage sort Interface to provide sortUnique function
2015-10-23 15:58:54 -07:00
Harshavardhana
53adfb38f4
Leverage sort Interface to provide sortUnique function
2015-10-23 15:55:41 -07:00
Harshavardhana
9476e0b374
Merge pull request #941 from harshavardhana/add-fs
...
Move ListObjects into its own file
2015-10-22 15:42:59 -07:00
Harshavardhana
dbaa4d8643
Move ListObjects into its own file
2015-10-22 15:39:04 -07:00
Harshavardhana
9680ceb54e
Merge pull request #940 from harshavardhana/aws-sdk-go
...
Add documentation for aws-sdk-go
2015-10-22 01:53:54 -07:00
Harshavardhana
fc5a1d7408
Add documentation for aws-sdk-go
2015-10-22 01:50:31 -07:00
Harshavardhana
485da0cb91
Merge pull request #938 from minio/harshavardhana-patch-1
...
go get -u
2015-10-22 00:27:50 -07:00
Harshavardhana
16f6a7d370
Update README.md
2015-10-22 00:20:34 -07:00
Harshavardhana
daffe943bb
Merge pull request #937 from harshavardhana/bucket
...
Fix all remaining windows path issues.
2015-10-22 00:08:48 -07:00
Harshavardhana
1f66f4869b
Fix all remaining windows path issues.
2015-10-22 00:05:10 -07:00
Harshavardhana
8978c19db6
Merge pull request #936 from abperiasamy/description
...
move description above usage
2015-10-21 22:37:40 -07:00
Anand Babu (AB) Periasamy
b077bb9165
move description above usage
2015-10-21 22:27:20 -07:00
Harshavardhana
dbe3d9489e
Merge pull request #935 from harshavardhana/fs-bucket
...
On windows translate Prefix, Marker and Delimiter for paths
2015-10-21 22:08:52 -07:00
Harshavardhana
afa27b9847
On windows translate Prefix, Marker and Delimiter for paths
2015-10-21 22:00:03 -07:00
Harshavardhana
1157898244
Merge pull request #934 from harshavardhana/osx-build
...
Rename _linux to _nix for OS X build
2015-10-21 20:33:14 -07:00
Harshavardhana
1184cc5e8c
Rename _linux to _nix for OS X build
2015-10-21 20:18:53 -07:00
Harshavardhana
aa043ba1db
Merge pull request #933 from harshavardhana/enable-different-logger
...
Enable all config loggers
2015-10-21 17:40:19 -07:00
Harshavardhana
09e51002ed
Enable all config loggers
2015-10-21 17:35:07 -07:00
Harshavardhana
bf644aaba9
Merge pull request #932 from harshavardhana/logger-functionality
...
Add config command
2015-10-21 16:04:24 -07:00
Harshavardhana
2ccadc2b17
Add config command
2015-10-21 15:41:14 -07:00
Harshavardhana
7966ed46c8
Merge pull request #929 from harshavardhana/file-logger
...
Add logger command
2015-10-21 00:04:59 -07:00
Harshavardhana
56003fded7
Add logger command - also migrate from old config to newer config
2015-10-21 00:02:16 -07:00
Harshavardhana
e8892a9f3c
Merge pull request #931 from harshavardhana/add-arm-support
...
Add arm support for build scripts
2015-10-20 12:26:41 -07:00
Harshavardhana
32898c72fa
Add arm support for build scripts
2015-10-20 12:03:03 -07:00
Harshavardhana
ed9ef861e1
Merge pull request #930 from harshavardhana/fix-portability
...
Fix portability issues for arm on raspberry pi
2015-10-20 11:39:43 -07:00
Harshavardhana
b74852116a
Fix portability issues for arm on raspberry pi
2015-10-20 11:22:00 -07:00
Harshavardhana
27612b9aed
Merge pull request #928 from harshavardhana/server-main
...
Add new logging connectors
2015-10-19 23:14:40 -07:00
Harshavardhana
469a3475b6
Add new logging connectors
2015-10-19 23:11:32 -07:00
Harshavardhana
d20842fc52
Merge pull request #927 from harshavardhana/accesslog-handler
...
Implement accessLog handler
2015-10-19 13:30:20 -07:00
Harshavardhana
b9ea18b8b8
Implement accessLog handler
2015-10-19 13:07:09 -07:00
Harshavardhana
e9d5ec3d64
Merge pull request #926 from harshavardhana/available-disk-space
...
Add 5% cumulative reduction in total size of the disk
2015-10-19 10:47:51 -07:00
Harshavardhana
dddb1650de
Add 5% cumulative reduction in total size of the disk
...
This is done due to filesystem holding additional metadata and inode space
which is unaccounted for during min-free-disk calculation.
2015-10-19 10:33:34 -07:00
Harshavardhana
d5e5b8777f
Merge pull request #923 from harshavardhana/auto-expiry
...
Add initial cut of auto expiry of objects
2015-10-19 01:41:30 -07:00
Harshavardhana
179d2d7dac
Add initial cut of auto expiry of objects
2015-10-19 01:34:31 -07:00
Harshavardhana
c590acfc38
Merge pull request #925 from harshavardhana/min-free-disk
...
Implement min-free-disk as a subcommand, deprecate flag
2015-10-19 01:06:26 -07:00
Harshavardhana
c065be656c
Implement min-free-disk as a subcommand, deprecate flag
2015-10-19 00:59:20 -07:00
Harshavardhana
bcbd537d63
Merge pull request #922 from harshavardhana/min-free-disk
...
Implementing min-free-disk
2015-10-18 00:25:54 -07:00
Harshavardhana
5b2fa33bdb
Implementing min-free-disk
2015-10-18 00:23:14 -07:00
Harshavardhana
18a6d7ea5d
Merge pull request #921 from harshavardhana/disk-fs
...
Improve disk code to return back disk StatFS{} structure
2015-10-17 20:21:24 -07:00
Harshavardhana
a8a935f5fd
Improve disk code to return back disk StatFS{} structure
...
```
StatFS {
Total int64
Free int64
FSType string
}
```
Provides more information in a cross platform way.
2015-10-17 20:19:26 -07:00
Harshavardhana
4b3961e1df
Merge pull request #920 from harshavardhana/free-disk-space
...
Add disk package
2015-10-17 16:52:59 -07:00
Harshavardhana
aee0845b2e
Add disk package
...
Implements
- Stat returns total and free disk space supported across platforms
- Type returns type of the filesystem underneath
2015-10-17 16:48:24 -07:00
Harshavardhana
f490af85e8
Merge pull request #919 from harshavardhana/update-command
...
Implement update command
2015-10-17 15:08:43 -07:00
Harshavardhana
47f1ffa1f3
Implement update command
2015-10-17 15:04:54 -07:00
Harshavardhana
357f9d7a05
Merge pull request #917 from harshavardhana/fs-separator
...
Add fs separator
2015-10-17 12:12:07 -07:00
Harshavardhana
1256ca86d0
Add fs separator
2015-10-17 12:05:12 -07:00
Harshavardhana
c2859c1f7d
Merge pull request #916 from harshavardhana/fs-utils
...
If directory already removed, return nil and move on
2015-10-17 00:15:47 -07:00
Harshavardhana
2ec679a089
If directory already removed, return nil and move on
2015-10-17 00:13:46 -07:00
Harshavardhana
c24235df8b
Merge pull request #915 from harshavardhana/delete-bucket
...
Implement delete bucket properly with proper error handlings
2015-10-17 00:05:28 -07:00
Harshavardhana
d534fc5a4f
Implement delete bucket properly with proper error handlings
2015-10-17 00:01:12 -07:00
Harshavardhana
7a4ca0b9c2
Merge pull request #914 from harshavardhana/enhance-filtering-further
...
Enhance listing further, this time handle cases related to common pre…
2015-10-16 23:15:51 -07:00
Harshavardhana
c9af01d807
Enhance listing further, this time handle cases related to common prefixes
2015-10-16 23:11:41 -07:00
Harshavardhana
5fb46cf75c
Merge pull request #913 from harshavardhana/http-status
...
Reply back proper statuses for DeleteBucket/DeleteObject
2015-10-16 20:06:38 -07:00
Harshavardhana
704fa420a3
Reply back proper statuses for DeleteBucket/DeleteObject
2015-10-16 20:03:44 -07:00
Harshavardhana
f825a32b53
Merge pull request #908 from harshavardhana/bucket-acl-support
...
Implement Bucket ACL support
2015-10-16 19:49:37 -07:00
Harshavardhana
0eb7f078f9
Implement Bucket ACL support
2015-10-16 19:47:30 -07:00
Harshavardhana
8fb45e92f9
Merge pull request #907 from harshavardhana/anonymous
...
If anonymous mode is set avoid verifying signature at lower level
2015-10-16 14:10:26 -07:00
Harshavardhana
f367ac927f
Merge branch 'master' into anonymous
2015-10-16 13:55:53 -07:00
Harshavardhana
9a01026a78
If anonymous mode is set avoid verifying signature at lower level
2015-10-16 13:47:44 -07:00
Harshavardhana
76fbb16ea8
Merge pull request #906 from minio/harshavardhana-patch-1
...
Update README.md
2015-10-16 13:27:10 -07:00
Harshavardhana
97393fd2a2
Update README.md
2015-10-16 11:56:53 -07:00
Harshavardhana
97cc446af8
Merge pull request #905 from harshavardhana/update-description
...
Update minio micro services description
2015-10-16 11:45:59 -07:00
Harshavardhana
94b0243341
Update minio micro services description
2015-10-16 11:40:47 -07:00
Harshavardhana
762b798767
Migrate this project to minio micro services code
2015-10-16 11:26:08 -07:00
Harshavardhana
8c4119cbeb
Merge pull request #904 from harshavardhana/fix-bugs
...
Fix bugs in post policy and presigned signature handling
2015-10-14 15:47:55 -07:00
Harshavardhana
2d0cc80646
Fix bugs in post policy and presigned signature handling
2015-10-14 15:45:34 -07:00
Harshavardhana
189188fdf3
Merge pull request #903 from technosophos/fix/902-missing-dot-minio
...
During auth generation, create directory if it does not exist.
2015-10-12 13:53:38 -07:00
Matt Butcher
ac6abd608f
During auth generation, create directory if it does not exist.
...
Addresses issue #902
2015-10-12 14:46:44 -06:00
Harshavardhana
f1c099af5f
Merge pull request #901 from harshavardhana/add-windows
...
Add windows support for minhttp library
2015-10-11 23:20:50 -07:00
Harshavardhana
3318cba132
Add windows support for minhttp library
2015-10-11 01:08:16 -07:00
Harshavardhana
6df28d062b
Merge pull request #900 from harshavardhana/migrate-vendor
...
Migrate to new govendor
2015-10-10 22:31:40 -07:00
Harshavardhana
886d6ac4a7
Migrate to new govendor
2015-10-10 22:28:56 -07:00
Harshavardhana
2f5fa394ce
Merge pull request #899 from harshavardhana/fix-signature-v4-bugs
...
Fix some bugs in controller rpc
2015-10-10 19:07:28 -07:00
Harshavardhana
7cde4577d0
Fix some bugs in controller rpc
2015-10-10 19:03:59 -07:00
Harshavardhana
05de60a598
Merge pull request #898 from harshavardhana/merge-few-files
...
Restructure top level files a bit, merge code into common file
2015-10-09 10:12:36 -07:00
Harshavardhana
9b2d38d142
Restructure top level files a bit, merge code into common file
2015-10-09 10:05:49 -07:00
Harshavardhana
f48b699a8e
Merge pull request #897 from harshavardhana/add-rpc-signature-handler
...
Add rpc signature handler
2015-10-08 22:31:45 -07:00
Harshavardhana
7fea9cb550
Add rpc signature handler
2015-10-08 22:28:11 -07:00
Harshavardhana
312af12fd5
Merge pull request #896 from harshavardhana/get-bucket-acl
...
Implement GetBucketACL - fixes #893
2015-10-08 11:22:18 -07:00
Harshavardhana
11048708bb
Implement GetBucketACL - fixes #893
2015-10-08 11:12:44 -07:00
Anand Babu (AB) Periasamy
bf901d3b9a
Merge pull request #895 from abperiasamy/tasker
...
new task model minio server
2015-10-08 02:23:29 -07:00
Anand Babu (AB) Periasamy
b52697e6ad
new task model minio server
2015-10-08 02:20:24 -07:00
Harshavardhana
8c6204e35e
Merge pull request #894 from harshavardhana/putbucket-acl-handler
...
Add proper router for handling putBucketACLHandler
2015-10-07 20:39:52 -07:00
Harshavardhana
d18ca4b40d
Add proper router for handling putBucketACLHandler
2015-10-07 20:36:47 -07:00
Harshavardhana
e719adec8b
Merge pull request #892 from harshavardhana/add-quick-version
...
Add quick.CheckVersion() to verify config version quickly before unma…
2015-10-07 17:49:16 -07:00
Harshavardhana
a060b158c8
Add quick.CheckVersion() to verify config version quickly before unmarshalling the full struct
...
This is needed during migration where we would need to verify the underlying version number
in a quick way.
2015-10-07 17:44:33 -07:00
Harshavardhana
c2fdccade4
Merge pull request #891 from harshavardhana/server-signature-v4
...
Enforce signature v4 tests all the time, server defaults to only auth…
2015-10-07 11:08:31 -07:00
Harshavardhana
ee377c9bff
Enforce signature v4 tests all the time, server defaults to only authenticated requests.
...
All requests must be authenticated to minio server from now on by using keys generated at
``${HOME}/.minio/users.json`` - from ``minio controller`` during its first time run.
Add a new hidden option ``--anonymous`` for running server in unauthenticated mode.
2015-10-07 10:43:27 -07:00
Harshavardhana
00b0f2e0d4
Merge pull request #890 from harshavardhana/donut-erasure
...
Make erasure Encode and Decode atomic to avoid races
2015-10-06 23:07:56 -07:00
Harshavardhana
ab5ea997ab
Make erasure Encode and Decode atomic to avoid races
2015-10-06 23:05:01 -07:00
Harshavardhana
e6d935731a
Merge pull request #889 from harshavardhana/canonicalize
...
Canonicalize all the incoming input values, now PresignedPostPolicy w…
2015-10-06 10:28:34 -07:00
Harshavardhana
1b42398e8b
Canonicalize all the incoming input values, now PresignedPostPolicy works with minio-go
2015-10-06 10:21:28 -07:00
Harshavardhana
31007cd0fa
Merge pull request #888 from harshavardhana/erasure-simply
...
Make erasure matrix type not optional choose automatically
2015-10-05 22:41:50 -07:00
Harshavardhana
d5ce2f6944
Make erasure matrix type not optional choose automatically
...
Remove option of providing Technique and handling errors based on that
choose a matrix type automatically based on number of data blocks.
INTEL recommends on using cauchy for consistent invertible matrices,
while vandermonde is faster we should default to cauchy for large
data blocks.
2015-10-05 22:38:02 -07:00
Harshavardhana
cf0e1a156b
Merge pull request #887 from harshavardhana/fix-encoding-bug-donut
...
Fix encoding bug in donut during encoding phase
2015-10-05 22:15:22 -07:00
Harshavardhana
4ed50a8004
Fix encoding bug in donut during encoding phase
...
Stream reading needs to check for length parameter being non zero,
after Reading() a predefined set of buffer length an EOF might be returned
with length == 0.
Erasure taking this zeroed data in might wrongly encode it as part of existing
data blocks which leads to errors while decoding even when the other contents
are intact.
2015-10-05 22:12:53 -07:00
Harshavardhana
f9174632bb
Merge pull request #885 from harshavardhana/json-output
...
Add --json output formatter for server
2015-10-05 00:23:19 -07:00
Harshavardhana
f0a8dbecae
Add --json output formatter for server
2015-10-05 00:20:49 -07:00
Harshavardhana
cd489b71e2
Merge pull request #884 from harshavardhana/add-controller-command-line
...
First time mode for controller
2015-10-04 16:44:41 -07:00
Harshavardhana
c4faf47e64
First time mode for controller
...
- Upon first time invocation ``minio controller`` would create access keys and secret id
- Upon request passing 'keys' arg ``minio controller`` would provide the keys
- Add colorized notification
2015-10-04 16:42:16 -07:00
Harshavardhana
bdd8e5873a
Merge pull request #883 from harshavardhana/simplify-signature
...
Simplify signature handling
2015-10-04 13:19:19 -07:00
Harshavardhana
cfdb29cac0
Simplify signature handling
...
This change brings a new SignatureHandler where Presigned.
Requests without Payload are handled very early before even
going through the call.
This change simplifies Donut codebase to not have signature related
logic for all API's.
Simplification is still needed for Payload based signature eg. PUT/POST calls
, which are still part of the donut codebase, which will be done subsequently
after donut re-write.
2015-10-04 13:15:33 -07:00
Harshavardhana
3de10f9472
Merge pull request #882 from harshavardhana/remove-http-responses-injson
...
Remove using HTTP responses in json reply always in application/xml
2015-10-04 01:25:49 -07:00
Harshavardhana
2a9c37ba26
Remove using HTTP responses in json reply always in application/xml
2015-10-04 01:22:50 -07:00
Harshavardhana
6c7543d49b
Merge pull request #880 from harshavardhana/implement-presigned-policy
...
Implement presigned policy
2015-10-04 00:04:28 -07:00
Harshavardhana
c8de5bad2f
Implement presigned policy
2015-10-04 00:01:34 -07:00
Harshavardhana
09dc360e06
Merge branch 'vadmeste-print_json_syntax_error_line_number'
2015-10-03 12:26:04 -07:00
Anis ELLEUCH
b5ea05d839
A better way to print prettified json syntax error msg
2015-10-03 12:25:44 -07:00
Harshavardhana
db293aedb7
Merge pull request #881 from technosophos/feature/docker-go151
...
Change Dockerfile to use smaller distro, non-root user, and Go 1.5.1
2015-10-02 14:43:30 -07:00
Matt Butcher
c486dfbb7b
Add non-root minio user.
...
This adds a minio user and runs minio as that user instead of
as root.
2015-10-02 15:22:23 -06:00
Matt Butcher
37a02670f5
Use ubuntu-debootstrap and Go 1.5.1.
...
Currently, the Dockerfile is broken because it installs Go 1.5
while the minimum required version is 1.5.1.
Also, switch to a minimval version of Ubuntu (ubuntu-debootstrap)
and reduce the image size by 70M in unneeded dependencies.
2015-10-02 13:38:51 -06:00
Harshavardhana
62e31e7eb0
Merge pull request #879 from harshavardhana/presigned-signature-v4
...
Implement presigned signature v4 support
2015-10-01 10:21:05 -07:00
Harshavardhana
3b070dee16
Fix an important metadata getObject bug in donut
2015-10-01 10:18:03 -07:00
Harshavardhana
81cc017f91
Implement presigned signature v4 support
2015-10-01 10:17:47 -07:00
Harshavardhana
6012e18123
Merge pull request #878 from harshavardhana/multipart-donut
...
Reduce memory usage for memory multipart write by doing io.Pipe() str…
2015-09-30 20:56:20 -07:00
Harshavardhana
50750efb52
Reduce memory usage for memory multipart write by doing io.Pipe() streaming copy
2015-09-30 20:53:30 -07:00
Harshavardhana
daa089fb06
Merge pull request #876 from harshavardhana/probe
...
Probe stringer should avoid frivolous newlines
2015-09-29 10:17:19 -07:00
Harshavardhana
8c7c5df770
Prober stringer should avoid frivolous newlines
2015-09-29 10:13:11 -07:00
Anand Babu (AB) Periasamy
77cca1e648
Update README.md
2015-09-26 15:34:28 -07:00
Anand Babu (AB) Periasamy
a083733ce2
updated jobs section
2015-09-26 15:29:05 -07:00
Harshavardhana
da1293240c
Merge pull request #875 from harshavardhana/fetch-donut
...
Fetch donut stats properly, update assetfs with new changes
2015-09-26 00:44:14 -07:00
Harshavardhana
301ffe60a2
Fetch donut stats properly, update assetfs with new changes
2015-09-26 00:38:25 -07:00
Harshavardhana
83d8de05ce
Merge pull request #874 from harshavardhana/update-assetfs
...
Add new changes to WebUI with implementation of GenerateAuth(), AddServer()
2015-09-25 19:09:24 -07:00
Harshavardhana
5c7c1ade3f
Add new changes to WebUI with implementation of GenerateAuth(), AddServer()
2015-09-25 19:06:31 -07:00
Harshavardhana
7ac45ecddc
Merge pull request #873 from harshavardhana/controller-assetfs
...
Add WebUI assetfs initial version
2015-09-25 00:48:06 -07:00
Harshavardhana
3b278b7f67
Add WebUI assetfs initial version
2015-09-25 00:45:23 -07:00
Harshavardhana
43eac57c7e
Merge pull request #872 from harshavardhana/fix-decoding-failure
...
Fix Linux/Mac OS X erasure decoding failure with new Golang version 1…
2015-09-24 21:45:08 -07:00
Harshavardhana
cd52d7a11c
Fix Linux/Mac OS X erasure decoding failure with new Golang version 1.5.1
...
Fixes #871
2015-09-24 21:42:59 -07:00
Harshavardhana
e793431852
Merge pull request #870 from harshavardhana/disable-multipart
...
Disable multipart for donut backend from being used
2015-09-24 18:56:53 -07:00
Harshavardhana
3785489153
Disable multipart for donut backend from being used
...
Will enable it later after cleanup
2015-09-24 18:54:30 -07:00
Harshavardhana
18dd7fc346
Merge pull request #867 from krishnasrinivas/discoverservers
...
Add DiscoverServers and GetControllerNetInfo controller APIs
2015-09-24 10:41:30 -07:00
Krishna Srinivas
5ebbc6bb0e
Add DiscoverServers and GetControllerNetInfo controller APIs
2015-09-23 16:06:37 -07:00
Anand Babu (AB) Periasamy
7b934a7c6c
Merge pull request #869 from abperiasamy/donut-check
...
remove mount-point requirement
2015-09-23 12:33:57 -07:00
Anand Babu (AB) Periasamy
8c356d4f5a
remove mount-point requirement
2015-09-23 12:16:18 -07:00
Harshavardhana
3a52d9b207
Merge pull request #868 from harshavardhana/fmt
...
Do not use fmt.Println with formatting strings
2015-09-23 09:31:46 -07:00
Harshavardhana
39c8991e5f
Do not use fmt.Println with formatting strings
2015-09-23 09:21:11 -07:00
Harshavardhana
9d61b2d6db
Merge pull request #866 from harshavardhana/proxy-request-rebalance-storagestats
...
Implement a new Donut service on server side
2015-09-22 19:18:26 -07:00
Harshavardhana
cc223b5278
Implement a new Donut service on server side
2015-09-22 19:08:02 -07:00
Harshavardhana
c92145c3df
Merge pull request #865 from krishnasrinivas/dummydonutstats
...
Donut dummy services - StorageStats, RebalaceStats
2015-09-22 16:03:44 -07:00
Krishna Srinivas
2607ab559a
Donut dummy services - StorageStats, RebalaceStats
2015-09-22 15:57:22 -07:00
Harshavardhana
099b6fcf92
Merge pull request #864 from krishnasrinivas/serverrep-fix
...
Json tags to some structs were missing. Fix ServerRep reply on the se…
2015-09-22 01:36:04 -07:00
Krishna Srinivas
ca17408be0
Json tags to some structs were missing. Fix ServerRep reply on the server.
2015-09-22 01:31:20 -07:00
Harshavardhana
3b050a4299
Merge pull request #863 from krishnasrinivas/hosts-string
...
Change ControllerArgs Hosts array to Host string
2015-09-22 01:19:29 -07:00
Krishna Srinivas
53e87a0790
Change ControllerArgs Hosts array to Host string
2015-09-22 01:15:27 -07:00
Harshavardhana
b45c6020e4
Merge pull request #862 from krishnasrinivas/probe-fix
...
Fix logrus error message logging
2015-09-21 16:38:25 -07:00
Krishna Srinivas
bf8b8981bf
Fix logrus error message logging
2015-09-21 11:48:49 -07:00
Harshavardhana
c3aa35424e
Merge pull request #861 from harshavardhana/controller
...
Add list of servers, for controller args.
2015-09-20 17:47:49 -07:00
Harshavardhana
1f364483e3
Add list of servers, for controller args.
2015-09-20 17:29:25 -07:00
Anand Babu (AB) Periasamy
345521f34d
Merge pull request #860 from abperiasamy/rm-gomaxprocs
...
setting GOMAXPROCS is no longer
2015-09-20 16:08:33 -07:00
Anand Babu (AB) Periasamy
45146cc138
setting GOMAXPROCS is no longer
2015-09-20 16:06:16 -07:00
Harshavardhana
39e2209755
Merge pull request #859 from harshavardhana/atomic-test
...
Move atomic package to the top and simplify its tests
2015-09-20 14:50:01 -07:00
Harshavardhana
b938e40fb5
Move atomic package to the top and simplify its tests
2015-09-20 13:51:38 -07:00
Harshavardhana
fb84335010
Merge pull request #858 from harshavardhana/rename
...
Rename files accordingly - consolidating further
2015-09-20 12:51:18 -07:00
Harshavardhana
6df97ef00a
Rename files accordingly
2015-09-20 12:44:50 -07:00
Harshavardhana
8d204b38eb
Merge pull request #856 from harshavardhana/rpc-controller
...
Consolidate controller rpc into one single service
2015-09-20 12:32:08 -07:00
Harshavardhana
d1621691b7
Consolidate controller rpc into one single service
2015-09-20 12:27:42 -07:00
Harshavardhana
29673fed76
Merge pull request #855 from harshavardhana/simplification-toplevel-names
...
Improve code further - this time further simplification of names
2015-09-19 21:24:03 -07:00
Harshavardhana
674631f9d8
Improve code further - this time further simplification of names
2015-09-19 21:21:39 -07:00
Harshavardhana
2721bef8da
Add errorIf for all API handlers to print call trace upon errors
2015-09-19 19:55:12 -07:00
Harshavardhana
e510e97f28
Consolidating more codebase and cleanup in server / controller
2015-09-19 19:55:12 -07:00
Harshavardhana
d9328d25e9
Merge pull request #854 from krishnasrinivas/servicehandlers2
...
Controller Service proxies rpc calls to the corresponding servers
2015-09-19 19:40:20 -07:00
Krishna Srinivas
e600bd6b4f
Controller Service proxies rpc calls to the corresponding servers
2015-09-19 19:37:20 -07:00
Harshavardhana
c953b0dab3
Merge pull request #851 from harshavardhana/cleanup
...
Move all server and controller packages into top-level
2015-09-19 01:09:42 -07:00
Harshavardhana
d54488f144
Move all server and controller packages into top-level
2015-09-19 01:07:42 -07:00
Harshavardhana
d808c3685d
Merge pull request #850 from harshavardhana/logger-tests
...
Add tests for minio top level logger
2015-09-18 23:44:52 -07:00
Harshavardhana
7d8cfa0a77
Add tests for minio top level
2015-09-18 23:41:36 -07:00
Anand Babu (AB) Periasamy
ec0fdf95e5
Merge pull request #849 from abperiasamy/version-format
...
new version format and some cleanup
2015-09-18 23:35:26 -07:00
Anand Babu (AB) Periasamy
89a86948b5
new version format and some cleanup
2015-09-18 23:33:28 -07:00
Anand Babu (AB) Periasamy
d1f1b7ac31
new version format and some cleanup
2015-09-18 23:27:04 -07:00
Harshavardhana
2141c9f70f
Merge pull request #848 from harshavardhana/rpc-tests
...
Add new rpc tests for Server.Add and Server.List, improve Version.Get…
2015-09-18 17:47:15 -07:00
Harshavardhana
778f8cd222
Add new rpc tests for Server.Add and Server.List, improve Version.Get RPC to provide more details
2015-09-18 17:44:46 -07:00
Harshavardhana
b59d7882ef
Merge pull request #847 from harshavardhana/enhance-signature-handler
...
Enhance signature handler - throw back valid error messages
2015-09-18 15:17:08 -07:00
Harshavardhana
2a15dd5eab
Enhance signature handler - throw back valid error messages
2015-09-18 15:14:55 -07:00
Harshavardhana
ac93bbb41d
Merge pull request #846 from harshavardhana/new-changes
...
With new auth config changes, restructure the API code to use the new style
2015-09-18 03:45:03 -07:00
Harshavardhana
6a5e5c1826
With new auth config changes, restructure the API code to use the new style
2015-09-18 03:41:05 -07:00
Harshavardhana
7a61447ce5
Merge pull request #845 from harshavardhana/add-missing
...
Add missing reply.Name and add possible failure tests
2015-09-18 03:20:54 -07:00
Harshavardhana
b4ce1e8c1d
Add missing reply.Name and add possible failure tests
2015-09-18 03:15:19 -07:00
Harshavardhana
6803b64768
Merge pull request #844 from harshavardhana/enhance-auth
...
Enhance auth JSONRPC, now provides persistent output
2015-09-18 03:05:37 -07:00
Harshavardhana
f8bb85aeb7
Enhance auth JSONRPC, now provides persistent output
...
Implements
- Auth.Generate("user")
- Auth.Fetch("user")
- Auth.Reset("user")
This patch also adds testing for each of these cases
2015-09-18 03:02:39 -07:00
Anand Babu (AB) Periasamy
8be09ce1cf
Merge pull request #843 from abperiasamy/logger
...
logrus based logger
2015-09-17 23:59:06 -07:00
Anand Babu (AB) Periasamy
1394d8a7a8
logrus based logger
2015-09-17 23:49:30 -07:00
Harshavardhana
51652c38cb
Merge pull request #842 from harshavardhana/fix-minio
...
Fix minio header in accordance with rfc2616.txt
2015-09-17 23:48:08 -07:00
Harshavardhana
4bcd86408b
Fix minio header in accordance with rfc2616.txt
2015-09-17 23:46:10 -07:00
Harshavardhana
0079849938
Merge pull request #841 from harshavardhana/tests-travis
...
Run tests only on travis, local builds just do govet, golint and gofmt
2015-09-17 22:40:12 -07:00
Harshavardhana
bd33ccc3a2
Run tests only on travis, local builds just do govet, golint and gofmt
2015-09-17 22:31:11 -07:00
Harshavardhana
c223427cbf
Merge pull request #840 from harshavardhana/version
...
Version is a package now, will be re-used across codebase.
2015-09-17 20:20:02 -07:00
Harshavardhana
7093a05ab1
Version is a package now, will be re-used across codebase.
2015-09-17 20:17:33 -07:00
Harshavardhana
1c5454e007
Merge pull request #839 from harshavardhana/add-missing-golint
...
Fix all the golint complaints about newly added changes
2015-09-17 18:58:36 -07:00
Harshavardhana
1887114444
Fix all the golint complaints about newly added changes
...
Do not use func(this *server), such generic names should not be used
for writing struct methods.
2015-09-17 18:53:42 -07:00
Harshavardhana
9e7e00e270
Merge pull request #838 from harshavardhana/probe-error
...
Add more documentation for probe
2015-09-17 18:15:34 -07:00
Harshavardhana
03ef6533c8
Add more documentation for probe
2015-09-17 18:10:42 -07:00
Harshavardhana
f4bd7b151c
Merge pull request #835 from harshavardhana/commands-fatal
...
Add trie to verify wrong inputs, and provide meaningful messages
2015-09-17 16:53:22 -07:00
Harshavardhana
77c71bd596
Add trie to verify wrong inputs, and provide meaningful messages
2015-09-17 16:49:08 -07:00
Harshavardhana
d8656c80bc
Merge pull request #834 from harshavardhana/version-checks
...
Add version checks for golang 1.5.1
2015-09-17 16:36:56 -07:00
Harshavardhana
b5246dbd7d
Add version checks for golang 1.5.1
2015-09-17 16:34:06 -07:00
Harshavardhana
e821d4d0cc
Merge pull request #833 from krishnasrinivas/server-service-2
...
Add json rpc ServerService - provides only dummy data right now
2015-09-17 15:51:46 -07:00
Harshavardhana
156ebf6a60
Update README.md
2015-09-17 15:50:30 -07:00
Krishna Srinivas
c49407ced4
rename rpc/server.go -> rpc/rpc.go. rpc/server.go will accomodate ServerService
2015-09-17 15:46:51 -07:00
Harshavardhana
f6e3ad889d
Merge pull request #832 from minio/harshavardhana-patch-1
...
Update README.md
2015-09-17 15:18:39 -07:00
Harshavardhana
99ec05be3e
Update README.md
2015-09-17 15:16:28 -07:00
Harshavardhana
c7fe91de49
Merge pull request #828 from harshavardhana/controller-consolidation
...
Consolidate controller, move rpc package into controller - remove dan…
2015-09-15 19:41:21 -07:00
Harshavardhana
3f4b98ca4c
Consolidate controller, move rpc package into controller - remove dangling code in pkg/server
2015-09-15 19:38:36 -07:00
Harshavardhana
8d5f6e0b96
Merge pull request #826 from krishnasrinivas/middleware-cleanup
...
Remove unneeded functions in middleware init
2015-09-15 18:11:45 -07:00
Krishna Srinivas
b1b387b157
Remove unneeded functions in middleware init
2015-09-15 18:09:09 -07:00
Harshavardhana
87a20e3ba3
Merge pull request #827 from harshavardhana/krishnasrinivas-parallel-read2
...
Read from the disks in parallel during object read
2015-09-15 17:38:13 -07:00
Harshavardhana
45d8898019
Merge branch 'parallel-read2' of https://github.com/krishnasrinivas/minio into krishnasrinivas-parallel-read2
...
Make few more changes and rebased with current master
2015-09-15 17:33:33 -07:00
Harshavardhana
ba6ddd5924
Merge pull request #825 from harshavardhana/simplify
...
Simplify version parsing with newVersion() make it more robust
2015-09-10 00:52:31 -07:00
Harshavardhana
83c1c4982a
Simplify version parsing with newVersion() make it more robust
2015-09-10 00:48:46 -07:00
Anand Babu (AB) Periasamy
de633b0ff8
Merge pull request #824 from abperiasamy/logrus
...
vendor package logrus
2015-09-09 21:57:02 -07:00
Anand Babu (AB) Periasamy
93b406c986
vendor package logrus
2015-09-09 21:51:45 -07:00
Anand Babu (AB) Periasamy
28641dc525
Merge pull request #823 from abperiasamy/logrus
...
logrus logger
2015-09-09 20:35:55 -07:00
Anand Babu (AB) Periasamy
6930e4d668
logrus logger
2015-09-09 20:28:09 -07:00
Harshavardhana
4495c2d8d8
Update INSTALLGO.md
2015-09-09 16:02:13 -07:00
Harshavardhana
83bac2b08f
Merge pull request #822 from harshavardhana/verify-runtime
...
Verify golang runtime for 1.5.1 and above, also verify if runner is a root - disallow it
2015-09-09 15:39:33 -07:00
Harshavardhana
d3f9a9da0d
Verify golang runtime for 0.5.1 and above, also verify if runner is a root
2015-09-09 15:37:06 -07:00
Harshavardhana
17f5df689e
Merge pull request #821 from harshavardhana/shadow
...
Avoid shadowing variables and enable checks to avoid them during build
2015-09-09 15:18:21 -07:00
Harshavardhana
1e2c010174
Avoid shadowing variables and enable checks to avoid them during build
2015-09-09 15:14:55 -07:00
Anand Babu (AB) Periasamy
2c877acd48
Merge pull request #820 from abperiasamy/remove-globals
...
remove debug option
2015-09-09 12:53:52 -07:00
Anand Babu (AB) Periasamy
e387e5578d
remove debug option
2015-09-09 12:46:20 -07:00
Harshavardhana
0d4da7767b
Update donut-metadata.md
2015-09-08 22:27:20 -07:00
Harshavardhana
96bc95c952
Update donut-metadata.md formatting changes
2015-09-08 22:15:51 -07:00
Harshavardhana
bae43ccbc7
Merge pull request #819 from harshavardhana/donut-metadata
...
Add donut metadata configs
2015-09-08 21:35:34 -07:00
Harshavardhana
fcd290193d
Add donut metadata configs
2015-09-08 21:32:55 -07:00
Harshavardhana
2eb17eab89
Merge pull request #818 from harshavardhana/logging
...
Disable logging for now
2015-09-05 21:30:51 -07:00
Harshavardhana
b649eff3fb
Disable logging for now
2015-09-05 21:28:35 -07:00
Harshavardhana
95cb056fd5
Merge pull request #817 from harshavardhana/probe
...
Add probe to main
2015-09-05 21:23:43 -07:00
Harshavardhana
593d90c7de
Add probe to main
2015-09-05 21:20:54 -07:00
Harshavardhana
fa11112226
Merge pull request #816 from harshavardhana/erasure
...
Simplify erasure package for OSX
2015-09-05 20:58:03 -07:00
Harshavardhana
d0f945f8e7
Simplify erasure package for OSX
2015-09-05 20:19:43 -07:00
Harshavardhana
90a247b336
Merge pull request #815 from harshavardhana/remove-cors-controller
...
Remove cors controller
2015-08-31 17:20:37 -07:00
Harshavardhana
d999ce282a
Add cors deps
2015-08-31 17:16:27 -07:00
Harshavardhana
afff3f8885
Revert "Enable controller to have CORS"
...
This reverts commit f39ac24e99 .
2015-08-31 17:15:49 -07:00
Harshavardhana
b5907ec806
Merge pull request #813 from krishnasrinivas/server-cors-add
...
Add CORS support to minio s3 server
2015-08-31 17:06:26 -07:00
Krishna Srinivas
1e82ee1192
Add CORS support to minio s3 server
2015-08-31 16:59:52 -07:00
Harshavardhana
d5d4c7046f
Merge pull request #812 from harshavardhana/enable-cors
...
Enable controller to have CORS
2015-08-31 01:54:53 -07:00
Harshavardhana
f39ac24e99
Enable controller to have CORS
2015-08-31 01:47:05 -07:00
Harshavardhana
dcf0c71ca3
Merge pull request #810 from harshavardhana/restructure
...
Restructure server code, controller now runs in silo
2015-08-27 17:10:54 -07:00
Harshavardhana
025f95b1d6
Restructure server code, controller now runs in silo
2015-08-27 17:07:32 -07:00
Anand Babu (AB) Periasamy
2f260adc69
Merge pull request #809 from abperiasamy/ret-untrace
...
return *probe.Error for Untrace() as well.
2015-08-24 03:43:13 -07:00
Anand Babu (AB) Periasamy
c11aa1c892
return *probe.Error for Untrace() as well.
2015-08-24 03:35:24 -07:00
Harshavardhana
526c8b4d76
Merge pull request #808 from harshavardhana/pkg
...
Add package add and remove commands to Makefile
2015-08-22 22:26:02 -07:00
Harshavardhana
e6a072e0ad
Add package add and remove commands to Makefile
2015-08-22 22:23:36 -07:00
Harshavardhana
4d1f38d28c
Merge pull request #807 from harshavardhana/migrate
...
Migrate to golang1.5 release with GO15VENDOREXPERIMENT=1 enabled
2015-08-22 18:37:56 -07:00
Harshavardhana
988d39a5b6
Migrate to golang1.5 release with GO15VENDOREXPERIMENT=1 enabled
2015-08-22 18:35:37 -07:00
Harshavardhana
5721d85c9a
Merge pull request #806 from harshavardhana/tests
...
Tests were running 4 times due to multiple times the TestingT{} was b…
2015-08-20 22:35:37 -07:00
Harshavardhana
0e416ea699
Tests were running 4 times due to multiple times the TestingT{} was being called
...
Calling multiple times TestingT{} will hook up runner for Suites for that many times
which would lead to repeated running tests.
Fix it by only initializing it once for all the Suites
2015-08-20 22:32:50 -07:00
Harshavardhana
555c946670
Merge pull request #805 from harshavardhana/quick
...
Add a new quick.Load() function to load directly any config file prov…
2015-08-20 20:35:45 -07:00
Harshavardhana
068d1d1ba9
Add a new quick.Load() function to load directly any config file provided a quick compatible struct{} is also provided
2015-08-20 20:33:49 -07:00
Harshavardhana
2768c9752b
Merge pull request #804 from harshavardhana/update-contributors
...
Update AB's duplicate email address
2015-08-20 16:14:23 -07:00
Harshavardhana
d87d90a6ef
Update AB's duplicate email address
2015-08-20 16:10:01 -07:00
Harshavardhana
67f9df0926
Merge pull request #803 from harshavardhana/minio-server-changes
...
Refactoring minio server command and flags
2015-08-20 13:09:31 -07:00
Harshavardhana
74587886d2
Refactoring minio server command and flags
2015-08-20 13:07:33 -07:00
Anand Babu (AB) Periasamy
8463040cd1
Merge pull request #802 from abperiasamy/probe-reverse
...
return call stack in reverse
2015-08-19 22:44:29 -07:00
Anand Babu (AB) Periasamy
b49b8cdbe8
return call stack in reverse
2015-08-19 22:40:27 -07:00
Anand Babu (AB) Periasamy
f223c8b36d
Merge pull request #801 from abperiasamy/omitjson
...
skip Env in json printinf if empty
2015-08-19 01:23:59 -07:00
Anand Babu (AB) Periasamy
826202716f
skip Env in json printinf if empty
2015-08-19 01:21:28 -07:00
Anand Babu (AB) Periasamy
56c548c133
Merge pull request #800 from abperiasamy/tracepoint-public
...
make tracePoint public as well
2015-08-19 01:13:37 -07:00
Anand Babu (AB) Periasamy
76c40e075a
make tracePoint public as well
2015-08-19 01:02:39 -07:00
Harshavardhana
8789c5a49d
Merge pull request #799 from harshavardhana/pr_out_fix_typo_in_probe
...
Fix typo in probe
2015-08-18 23:52:04 -07:00
Harshavardhana
ac928b5092
Fix typo in probe
2015-08-18 23:42:41 -07:00
Anand Babu (AB) Periasamy
076354056b
Merge pull request #798 from abperiasamy/probe-simplification
...
simplify probe APIs
2015-08-18 19:34:04 -07:00
Anand Babu (AB) Periasamy
cdf93e534c
simplify probe APIs
2015-08-18 19:30:17 -07:00
Harshavardhana
76d11ed1e4
Merge pull request #797 from harshavardhana/migrate_quick
...
Migrate pkg/quick from mc
2015-08-13 16:35:03 -07:00
Harshavardhana
e9c5a51bc6
Migrate pkg/quick from mc
2015-08-13 16:29:55 -07:00
Harshavardhana
e4dc8ee7b4
Merge pull request #796 from harshavardhana/migrate-to-govendor
...
Migrate to govendor to avoid limitations of godep
2015-08-12 19:40:34 -07:00
Harshavardhana
61175ef091
Migrate to govendor to avoid limitations of godep
...
- over the course of a project history every maintainer needs to update
its dependency packages, the problem essentially with godep is manipulating
GOPATH - this manipulation leads to static objects created at different locations
which end up conflicting with the overall functionality of golang.
This also leads to broken builds. There is no easier way out of this other than
asking developers to do 'godep restore' all the time. Which perhaps as a practice
doesn't sound like a clean solution. On the other hand 'godep restore' has its own
set of problems.
- govendor is a right tool but a stop gap tool until we wait for golangs official
1.5 version which fixes this vendoring issue once and for all.
- govendor provides consistency in terms of how import paths should be handled unlike
manipulation GOPATH.
This has advantages
- no more compiled objects being referenced in GOPATH and build time GOPATH
manging which leads to conflicts.
- proper import paths referencing the exact package a project is dependent on.
govendor is simple and provides the minimal necessary tooling to achieve this.
For now this is the right solution.
2015-08-12 19:24:57 -07:00
Harshavardhana
b4c8b4877e
Merge pull request #795 from harshavardhana/pr_out_probe_revamped_to_provide_for_a_new_wrappederror_struct_to_wrap_probes_as_error_interface
...
Probe revamped to provide for a new WrappedError struct to wrap probes as error interface
2015-08-08 00:23:36 -07:00
Harshavardhana
45b59b8456
Probe revamped to provide for a new WrappedError struct to wrap probes as error interface
...
This convenience was necessary to be used for golang library functions like io.Copy and io.Pipe
where we shouldn't be writing proxies and alternatives returning *probe.Error
This change also brings more changes across code base for clear separation regarding where an error
interface should be passed encapsulating *probe.Error and where it should be used as is.
2015-08-08 00:16:38 -07:00
Harshavardhana
28d9565400
Merge pull request #793 from harshavardhana/pr_out_use_command_not_found_helper
...
use command not found helper
2015-08-03 18:25:57 -07:00
Harshavardhana
f8141493bd
use command not found helper
2015-08-03 18:06:19 -07:00
Harshavardhana
b8461980bb
Merge pull request #792 from harshavardhana/pr_out_migrate_from_iodine_to_probe
...
Migrate from iodine to probe
2015-08-03 16:36:04 -07:00
Harshavardhana
d09fd8b0a1
Migrate from iodine to probe
2015-08-03 16:33:44 -07:00
Harshavardhana
7f13095260
Merge pull request #791 from harshavardhana/pr_out_more_changes_to_probe_to_avoid_nil_dereferences
...
More changes to probe to avoid nil dereferences
2015-08-03 01:49:53 -07:00
Harshavardhana
884e9771b2
More changes to probe to avoid nil dereferences
2015-08-03 01:47:37 -07:00
Harshavardhana
a096913dde
Merge pull request #790 from harshavardhana/pr_out_minor_changes_to_probe
...
Minor changes to probe
2015-08-02 20:35:10 -07:00
Harshavardhana
65e4aede82
Minor changes to probe
2015-08-02 20:33:49 -07:00
Anand Babu (AB) Periasamy
574f2aaafa
Merge pull request #789 from abperiasamy/trace-on-new
...
trace on New and add read locks
2015-08-02 12:03:44 -07:00
Anand Babu (AB) Periasamy
697009c0a1
trace on New and add read locks
2015-08-02 11:58:28 -07:00
Harshavardhana
d9493909d8
Merge pull request #788 from krishnasrinivas/open-and-openfile
...
rename disk.OpenFile to Open which will do os.Open (which will be rea…
2015-08-02 11:05:04 -07:00
Krishna Srinivas
ee4432bc40
rename disk.OpenFile to Open which will do os.Open (which will be read-only). disk.OpenFile will do os.OpenFile (which can be rw, append)
2015-08-02 17:34:29 +05:30
Anand Babu (AB) Periasamy
aa8663f7cc
Merge pull request #787 from abperiasamy/probe
...
probe package to trace & return errors
2015-08-02 02:41:10 -07:00
Anand Babu (AB) Periasamy
a728ddc027
probe package to trace & return errors
2015-08-02 02:38:08 -07:00
Harshavardhana
0a1ba049ba
Merge pull request #786 from harshavardhana/pr_out_fix_command_template_typo_and_fix_others
...
fix command template typo and fix others.
2015-08-01 10:54:06 -07:00
Harshavardhana
81a7772fcd
fix command template typo and fix others.
2015-08-01 10:51:02 -07:00
Harshavardhana
ae685f0548
Merge pull request #785 from harshavardhana/pr_out_deprecate_make_go_go_back_to_makefile_make_go_is_not_genversion_go
2015-07-31 17:18:35 -07:00
Harshavardhana
5d3379ed7e
deprecate 'make.go', go back to Makefile - make.go is not genversion.go
2015-07-31 17:16:54 -07:00
Harshavardhana
1a27e5eb28
Merge pull request #784 from harshavardhana/pr_out_merge_cmd_donut_into_minio_cmd_deprecate_controller_rpc_request
...
Merge cmd/donut into minio cmd, deprecate controller RPC request
2015-07-31 13:06:53 -07:00
Harshavardhana
aabfd541e1
Merge cmd/donut into minio cmd, deprecate controller RPC request
2015-07-31 12:57:15 -07:00
Harshavardhana
6be5a2fb7e
Merge pull request #783 from harshavardhana/pr_out_crypto_cleanup_remove_unused_functions
...
crypto/cleanup: remove unused functions
2015-07-29 13:12:04 -07:00
Harshavardhana
2671b2dbf4
crypto/cleanup: remove unused functions
2015-07-29 13:09:55 -07:00
Harshavardhana
179c5441c3
Merge pull request #782 from krishnasrinivas/no-signature-no-sha256
...
when signature is not available there is no need to compute sha256
2015-07-29 11:21:40 -07:00
Krishna Srinivas
fdd2c22fa5
Do go fmt on bucket.go
2015-07-29 23:49:25 +05:30
Krishna Srinivas
ae8089c9b6
when signature is not available there is no need to compute sha256
2015-07-29 21:09:02 +05:30
Krishna Srinivas
bdc00624fd
get erros from buffered channel. Return error during object read only if we have readers < k
2015-07-29 19:36:41 +05:30
Harshavardhana
de74ae9d59
Merge pull request #781 from harshavardhana/pr_out_minor_add_commands_into_donut_template
...
minor: Add commands into donut template
2015-07-28 19:42:19 -07:00
Harshavardhana
f15375426a
minor: Add commands into donut template
2015-07-28 19:40:02 -07:00
Harshavardhana
d70695fc6f
Merge pull request #780 from harshavardhana/pr_out_collapse_getpartialobject_into_getobject_
...
Collapse GetPartialObject() into GetObject()
2015-07-28 19:35:50 -07:00
Harshavardhana
d346250f1c
Collapse GetPartialObject() into GetObject()
2015-07-28 19:33:56 -07:00
Harshavardhana
09aeabcf40
Merge pull request #779 from harshavardhana/pr_out_add_krishna_and_nate_into_contributors_list
...
Add krishna and nate into contributors list
2015-07-28 16:56:53 -07:00
Harshavardhana
1a47c47f11
Add krishna and nate into contributors list
2015-07-28 16:38:21 -07:00
Harshavardhana
6707e58bb3
Merge pull request #777 from krishnasrinivas/missingEncodedBlocksCount
...
Use missingEncodedBlocksCount directly instead of "-1" workaround in …
2015-07-28 11:14:29 -07:00
Krishna Srinivas
e1280779ed
Read from the disks in parallel during object read
2015-07-28 18:04:17 +05:30
Krishna Srinivas
1ea91d2fa2
Use missingEncodedBlocksCount directly instead of "-1" workaround in missingEncodedBlocks[]
...
Makes Code more readable
2015-07-28 16:56:06 +05:30
Harshavardhana
55d22fa8d6
Merge pull request #776 from harshavardhana/dep_branch
...
Fix dependency checking on osx
2015-07-27 18:26:39 -07:00
Harshavardhana
c503bf412f
Fix dependency checking on osx
2015-07-27 18:19:22 -07:00
Harshavardhana
7133513600
Merge pull request #775 from harshavardhana/pr_out_strip_off_quotes_from_etag_for_verifying_complete_multipart_upload
...
Strip off quotes from ETag for verifying complete multipart upload
2015-07-25 23:12:39 +00:00
Harshavardhana
b0ea64a04f
Strip off quotes from ETag for verifying complete multipart upload
2015-07-25 16:10:05 -07:00
Harshavardhana
e082f26e10
Improving EncoderStream to return error only upon non io.EOF.
...
io.EOF is okay since io.ReadFull will not have read any bytes at all.
Also making error channel receive only for go routine.
2015-07-25 15:57:30 -07:00
Harshavardhana
4ac23d747c
Merge pull request #773 from krishnasrinivas/put-object-stream
...
Encoder now directly reads from the object stream. Using split.Stream…
2015-07-25 22:39:18 +00:00
Krishna Srinivas
bcfaa12a4d
Encoder now directly reads from the object stream. Using split.Stream() was causing lot of redundant memory operations.
2015-07-26 03:54:39 +05:30
Harshavardhana
ed07310471
Merge pull request #772 from harshavardhana/pr_out_use_new_app_extrainfo_inside_minio_and_donut_commands_properly
...
use new app.ExtraInfo inside minio and donut commands properly
2015-07-25 06:56:54 +00:00
Harshavardhana
0eefbdef0c
use new app.ExtraInfo inside minio and donut commands properly
2015-07-24 23:55:18 -07:00
Harshavardhana
8346cc74db
Merge pull request #771 from harshavardhana/pr_out_rename_more
...
Rename more
2015-07-25 01:11:28 +00:00
Harshavardhana
d6a0e0cc55
Rename more
2015-07-24 18:09:53 -07:00
Harshavardhana
80b7bc7ccc
Merge pull request #770 from harshavardhana/pr_out_move_from_minimalist_object_storage_to_minio_cloud_storage
...
Move from Minimalist Object Storage to Minio Cloud Storage
2015-07-25 00:53:49 +00:00
Harshavardhana
63c9cf0c4b
Move from Minimalist Object Storage to Minio Cloud Storage
2015-07-24 17:51:40 -07:00
Harshavardhana
b9c8b04092
Merge pull request #769 from harshavardhana/pr_out_update_slack_secure_builds
...
Update slack secure builds
2015-07-23 21:43:03 +00:00
Harshavardhana
7aa4e70bb3
Update slack secure builds
2015-07-23 14:41:19 -07:00
Harshavardhana
3b9043c2ac
Merge pull request #768 from harshavardhana/pr_out_listobjects_now_considers_multipart_objects_also_move_to_upstream_check_v1
...
ListObjects now considers multipart objects, also move to upstream check.v1
2015-07-18 22:52:50 +00:00
Harshavardhana
43c908d5b9
ListObjects now considers multipart objects, also move to upstream check.v1
2015-07-18 15:49:41 -07:00
Harshavardhana
e397fa48c4
Merge pull request #767 from harshavardhana/pr_out_remove_unnecessary_updateat_
...
remove unnecessary updateAt()
2015-07-18 02:53:41 +00:00
Harshavardhana
966786c78e
remove unnecessary updateAt()
2015-07-17 19:48:09 -07:00
Harshavardhana
ebb2d4ec5f
Merge pull request #766 from harshavardhana/pr_out_donut_multipart_support_initial_cut_wip
...
Donut multipart support no get, listobjects() support yet
2015-07-17 23:26:50 +00:00
Harshavardhana
c1da10a207
Donut multipart support no get, listobjects() support yet
2015-07-17 16:23:51 -07:00
Harshavardhana
107ca4cbad
Update README.md
2015-07-16 23:57:02 -07:00
Harshavardhana
e81f44f623
Merge pull request #765 from harshavardhana/pr_out_add_corresponding_tests
2015-07-17 00:42:03 +00:00
Harshavardhana
86a887f9d4
Add corresponding tests
2015-07-16 17:40:19 -07:00
Harshavardhana
5eae32f2b0
Return proper InvalidArgument messages like s3 for invalid data for ListObjects(), ListObjectParts(), ListMultipartUploads()
2015-07-16 17:22:45 -07:00
Harshavardhana
9d525ecadc
Merge pull request #763 from harshavardhana/pr_out_method_not_allowed_is_right_response_for_delete_operations_and_add_tests
...
Method not allowed is right response for DELETE() operations and add tests
2015-07-16 21:36:05 +00:00
Harshavardhana
e605787e65
Method not allowed is right response for DELETE() operations and add tests
2015-07-16 14:15:24 -07:00
Harshavardhana
52f21e6696
Merge pull request #761 from harshavardhana/pr_out_use_updatedeps_script_to_update_godeps
2015-07-16 19:59:21 +00:00
Harshavardhana
e4543489fe
Use updatedeps script to update godeps
2015-07-16 12:57:32 -07:00
Harshavardhana
245ea26395
Merge pull request #760 from harshavardhana/pr_out_fix_an_issue_with_reusing_closed_network_connetion_changing_the_way_ratelimitedlistener_is_initialized
...
Fix an issue with reusing closed network connetion, changing the way rateLimitedListener is initialized
2015-07-16 17:59:50 +00:00
Harshavardhana
1f2e6a40a0
Fix an issue with reusing closed network connetion, changing the way rateLimitedListener is initialized
2015-07-16 10:56:14 -07:00
Harshavardhana
dc11a411c7
Merge pull request #759 from harshavardhana/pr_out_heal_buckets_upon_init_if_needed_adding_new_disks_and_hup_works
...
Heal buckets upon init if needed, adding new disks and HUP works
2015-07-16 17:00:36 +00:00
Harshavardhana
5507a39840
Heal buckets upon init if needed, adding new disks and HUP works
2015-07-16 09:59:05 -07:00
Harshavardhana
00a701a155
Merge pull request #758 from harshavardhana/pr_out_add_file_method_to_ratelimitlistener_for_extracting_underlying_fd_
...
Add File() method to Ratelimitlistener for extracting underlying fd()
2015-07-16 07:13:10 +00:00
Harshavardhana
e4574c7d6f
Add File() method to Ratelimitlistener for extracting underlying fd()
2015-07-16 00:11:17 -07:00
Harshavardhana
2e8f154f34
Iodine should indent with EmitJSON()
2015-07-16 00:10:55 -07:00
Harshavardhana
a23a61da11
Merge pull request #757 from harshavardhana/pr_out_remove_scsi_non_portable_code_instead_donut_make_implements_functionality_to_instantiate_a_donut
...
Remove scsi non portable code, instead "donut make" implements functionality to instantiate a donut
2015-07-15 18:59:14 +00:00
Harshavardhana
4498662c16
Remove scsi non portable code, instead "donut make" implements functionality to instantiate a donut
2015-07-15 11:55:57 -07:00
Harshavardhana
49c705da99
Merge pull request #756 from harshavardhana/pr_out_fix_a_crash_during_listobjects_populating_nextmarker
...
Fix a crash during listObjects() populating NextMarker
2015-07-15 17:06:15 +00:00
Harshavardhana
6baf45e360
Fix a crash during listObjects() populating NextMarker
2015-07-15 10:04:15 -07:00
Harshavardhana
a204f53eac
Merge pull request #755 from harshavardhana/pr_out_add_initial_version_of_heal_remove_rebalance
...
Add initial version of heal, remove rebalance
2015-07-15 05:59:20 +00:00
Harshavardhana
2553654e24
Add initial version of heal, remove rebalance
2015-07-14 22:56:41 -07:00
Harshavardhana
6a64019659
Merge pull request #754 from harshavardhana/pr_out_live_multiple_disk_removal_works_properly
...
Live multiple disk removal works properly
2015-07-15 03:50:05 +00:00
Harshavardhana
e37c5315ec
Live multiple disk removal works properly
2015-07-14 20:46:14 -07:00
Harshavardhana
48bb69701e
Merge pull request #753 from harshavardhana/pr_out_wire_up_sha512_matching_inside_donut_along_with_md5sum
...
Wire up sha512 matching inside donut along with md5sum
2015-07-15 02:49:39 +00:00
Harshavardhana
e1e4908515
Wire up sha512 matching inside donut along with md5sum
2015-07-14 19:47:50 -07:00
Harshavardhana
2e5e85d8ad
Merge pull request #752 from harshavardhana/pr_out_handle_removal_of_disks_getobject_now_reads_if_disks_are_missing_underneath_add_initial_stub_healing_code
...
Handle removal of disks - getObject() now reads if disks are missing underneath, add initial stub healing code
2015-07-15 01:54:41 +00:00
Harshavardhana
e885259584
Handle removal of disks - getObject() now reads if disks are missing underneath, add initial stub healing code
2015-07-14 18:53:00 -07:00
Harshavardhana
74853caf0c
Merge pull request #751 from harshavardhana/pr_out_return_x_amz_request_id_for_all_replies
...
Return x-amz-request-id for all replies
2015-07-14 21:52:09 +00:00
Harshavardhana
efbf3eabb7
Return x-amz-request-id for all replies
2015-07-14 14:44:03 -07:00
Harshavardhana
518ecae711
Merge pull request #750 from harshavardhana/pr_out_remove_global_custom_config_path_variabls_use_get_set_methods_instead
2015-07-14 21:04:15 +00:00
Anand Babu (AB) Periasamy
7e2df542f1
Remove command name from title.
2015-07-14 11:58:37 -07:00
Anand Babu (AB) Periasamy
352a2dd34b
Update README.md
2015-07-14 11:56:47 -07:00
Harshavardhana
da8b9fd112
Remove global custom config path variables, use get/set methods instead
2015-07-14 11:56:15 -07:00
Anand Babu (AB) Periasamy
893782ffab
Update README.md
2015-07-14 10:57:53 -07:00
Harshavardhana
326e1364be
Merge pull request #749 from harshavardhana/pr_out_read_req_body_for_putbucket_if_any
...
Read req.Body for PutBucket() if any
2015-07-14 16:33:37 +00:00
Harshavardhana
c4cf7635bf
Read req.Body for PutBucket() if any
2015-07-14 09:30:10 -07:00
Harshavardhana
e4dd49bdd9
Merge pull request #748 from harshavardhana/pr_out_fix_an_ugly_multipart_bug
...
Fix an ugly multipart bug
2015-07-14 04:40:24 +00:00
Harshavardhana
45ddec925c
Fix an ugly multipart bug
2015-07-13 21:38:01 -07:00
Harshavardhana
00acc47158
Merge pull request #747 from harshavardhana/pr_out_for_missing_parts_reply_back_as_invalidpart_
...
For missing parts reply back as InvalidPart{}
2015-07-14 02:32:56 +00:00
Harshavardhana
7ae60a6d10
For missing parts reply back as InvalidPart{}
2015-07-13 19:31:01 -07:00
Harshavardhana
fdee527a4d
Merge pull request #746 from harshavardhana/pr_out_writeobject_encoded_data_in_go_routine_fix_another_multipart_issue
...
WriteObject() encoded data in go routine, fix another multipart issue
2015-07-13 22:58:59 +00:00
Harshavardhana
634f70f3b0
WriteObject() encoded data in go routine, fix another multipart issue
2015-07-13 15:56:54 -07:00
Harshavardhana
70e56dc594
Merge pull request #745 from harshavardhana/pr_out_remove_dependency_on_minio_cli_for_make_go_fixes_738
...
Remove dependency on minio/cli for make.go fixes #738
2015-07-13 18:41:21 +00:00
Harshavardhana
1e80925ca7
Remove dependency on minio/cli for make.go fixes #738
2015-07-13 11:39:28 -07:00
Harshavardhana
dfa43f7982
Merge pull request #744 from harshavardhana/pr_out_add_a_ratelimited_listener_than_a_ratelimited_handler_more_precise
...
Add a ratelimited listener than a ratelimited handler - more precise
2015-07-13 18:16:27 +00:00
Harshavardhana
1bad92356d
Add a ratelimited listener than a ratelimited handler - more precise
2015-07-13 11:12:17 -07:00
Anand Babu (AB) Periasamy
8af5933b07
Merge pull request #743 from flandr/fix-osx
...
Fix OS X build
2015-07-13 10:15:53 -07:00
Nate Rosenblum
ec347f96fd
Fix OS X build
...
- Explicitly cast statfs_t members to int64 (this structure is
platform-specific)
- Add pass-through New methods to Darwin SHA package
- Move scsi pkg types to common translation unit (package was empty)
- Add stub implementations mount/disk ops for OS X
2015-07-13 10:06:55 -07:00
Harshavardhana
07eb9d9a4c
Merge pull request #742 from harshavardhana/pr_out_add_mkdonut_examples
...
Add mkdonut examples
2015-07-13 04:40:20 +00:00
Harshavardhana
f360ee0ab5
Add mkdonut examples
2015-07-12 21:37:57 -07:00
Harshavardhana
36904d5da9
Merge pull request #741 from harshavardhana/pr_out_mkdonut_now_creates_a_donut_processing_cli_args
...
mkdonut now creates a donut processing cli args
2015-07-13 04:26:45 +00:00
Harshavardhana
55e4d0c6a5
mkdonut now creates a donut processing cli args
2015-07-12 21:21:31 -07:00
Harshavardhana
10fd5b8092
Merge pull request #740 from harshavardhana/pr_out_add_mkdonut_command
...
Add mkdonut command
2015-07-13 02:29:53 +00:00
Harshavardhana
535bcc3eac
Add mkdonut command
2015-07-12 19:16:36 -07:00
Harshavardhana
4c95775864
Merge pull request #739 from harshavardhana/pr_out_renaming_nimble_to_minhttp
2015-07-13 01:05:29 +00:00
Harshavardhana
58a1d865a9
Renaming nimble to minhttp
2015-07-12 18:01:52 -07:00
Harshavardhana
65d97fa131
Merge pull request #737 from harshavardhana/pr_out_add_signature_v4_tests
...
Add signature v4 tests
2015-07-12 20:37:52 +00:00
Harshavardhana
847440196e
Add signature v4 tests
2015-07-12 13:36:02 -07:00
Harshavardhana
74e44cd0c4
Merge pull request #736 from harshavardhana/pr_out_head_shouldn_t_have_any_body_handle_it_in_writeerrorresponse_
...
HEAD shouldn't have any body, handle it in writeErrorResponse()
2015-07-11 17:36:55 +00:00
Harshavardhana
7615a6bfe5
HEAD shouldn't have any body, handle it in writeErrorResponse()
2015-07-11 10:34:55 -07:00
Harshavardhana
7fde241ee2
Merge pull request #735 from harshavardhana/pr_out_cached_api_test_should_also_have_a_custom_config_path_would_conflict_with_your_minio_local_path
2015-07-11 03:43:35 +00:00
Harshavardhana
8977f9a524
cached api test should also have a custom config path, would conflict with your .minio local path
2015-07-10 20:39:30 -07:00
Harshavardhana
0a8ea1aca5
Merge pull request #734 from harshavardhana/pr_out_do_not_reply_on_ignoredheaders_for_server_rely_on_signedheaders_sent_as_part_of_authorization_header
2015-07-11 02:40:33 +00:00
Harshavardhana
97d4a27c7e
Do not reply on ignoredHeaders for server, rely on SignedHeaders sent as part of Authorization header
2015-07-10 19:37:22 -07:00
Harshavardhana
538572ca91
Merge pull request #733 from harshavardhana/pr_out_nodejs_http_library_sends_connection_header_during_http_request_this_clobbers_up_the_signature_handling_ignore_it
2015-07-10 22:50:51 +00:00
Harshavardhana
53f5d2c32b
nodejs http library sends Connection header during HTTP request, this clobbers up the signature handling ignore it
2015-07-10 15:44:47 -07:00
Harshavardhana
d0eb4a2aea
Merge pull request #732 from harshavardhana/pr_out_cleanup_temporary_writers_upon_errors_during_putobject_all_metadata_write_operations
...
Cleanup temporary writers upon errors during putObject(), all metadata() write operations
2015-07-10 21:13:59 +00:00
Harshavardhana
29838bb851
Cleanup temporary writers upon errors during putObject(), all metadata() write operations
2015-07-10 14:11:04 -07:00
Harshavardhana
6860b310c9
Merge pull request #731 from harshavardhana/pr_out_support_signature_v4_at_rest
...
Support signature v4 at rest
2015-07-10 18:52:03 +00:00
Harshavardhana
15dd0df187
Support signature v4 at rest
2015-07-10 11:49:27 -07:00
Harshavardhana
39026cb64b
Merge pull request #730 from harshavardhana/pr_out_rename_definitions_to_log_go_add_valid_prefixes
2015-07-10 18:13:25 +00:00
Harshavardhana
7fa514351c
Rename definitions to log.go, add valid prefixes
2015-07-10 11:11:20 -07:00
Harshavardhana
d7acf90b81
Merge pull request #729 from harshavardhana/pr_out_add_abbreviated_close_response_to_avoid_any_leaks
2015-07-10 17:21:59 +00:00
Harshavardhana
d5ffc16f25
Add abbreviated close response, to avoid any leaks
2015-07-10 10:20:00 -07:00
Harshavardhana
f1d52df130
Merge pull request #728 from harshavardhana/pr_out_handle_both_space_and_non_space_characters_in_signature_v4
2015-07-10 04:52:01 +00:00
Harshavardhana
e5006c738d
Handle both space and non-space characters, in signature v4 - add errors for all API's
2015-07-09 21:49:58 -07:00
Harshavardhana
eac92d4647
Merge pull request #727 from harshavardhana/pr_out_all_other_api_s_now_support_signature_v4
...
All other API's now support signature v4
2015-07-10 02:47:32 +00:00
Harshavardhana
84f427f14a
All other API's now support signature v4
2015-07-09 19:45:56 -07:00
Harshavardhana
00890c254e
CompleteMultipartUpload and CreateObjectPart now fully support signature v4
2015-07-09 19:01:15 -07:00
Harshavardhana
89c1215194
PutObject handler gets initial support for signature v4, working
2015-07-09 16:44:38 -07:00
Harshavardhana
4f29dc9134
Merge pull request #724 from harshavardhana/pr_out_add_mountinfo_functions_for_detecting_mount_disks
2015-07-09 19:27:10 +00:00
Harshavardhana
d461fa5ab1
Add mountinfo functions for detecting mount disks, and other rpc changes
2015-07-09 12:25:29 -07:00
Harshavardhana
0f0d0c65e7
Merge pull request #723 from harshavardhana/pr_out_generate_auth_now_saves_in_home_minio_users_json_also_authhandler_verifies_request_validity
2015-07-09 04:54:42 +00:00
Harshavardhana
8654ddb566
Generate auth now saves in ${HOME}/.minio/users.json, also authHandler verifies request validity
2015-07-08 21:53:13 -07:00
Harshavardhana
51d2d8e221
Merge pull request #722 from harshavardhana/pr_out_http_header_content_length_signifies_body_length_of_the_request_if_its_smaller_reply_appropriately
...
HTTP header Content-Length signifies body length of the request, if its smaller reply appropriately
2015-07-09 03:59:08 +00:00
Harshavardhana
375860077d
HTTP header Content-Length signifies body length of the request, if its smaller reply appropriately
...
This patch also handles large individual part sizes > 5MB by using less memory copies.
2015-07-08 20:56:41 -07:00
Harshavardhana
cea093bf65
Merge pull request #721 from harshavardhana/pr_out_add_server_side_signaturev4_check_not_wired_up_to_the_readers_yet
2015-07-08 23:59:01 +00:00
Harshavardhana
ec33d79d57
Add server side signaturev4 check, not wired up to the readers yet.
2015-07-08 16:57:03 -07:00
Harshavardhana
a904cb5002
Merge pull request #720 from harshavardhana/pr_out_add_auth_rpc_service_to_generate_access_keys
2015-07-08 21:43:17 +00:00
Harshavardhana
396b728031
Add auth rpc service to generate access keys, add corresponding test
2015-07-08 14:40:39 -07:00
Harshavardhana
770fd23afa
Renaming keys as auth, working towards signature v4 support for all put objects
2015-07-08 14:17:16 -07:00
Harshavardhana
b71f15d32d
Merge pull request #719 from harshavardhana/pr_out_fix_ssl_support_pointer_indirection_caused_nil_buffers
2015-07-08 18:04:26 +00:00
Harshavardhana
2413a110e6
Fix SSL support, pointer indirection caused nil buffers
2015-07-08 11:02:15 -07:00
Harshavardhana
f737c0540e
Merge pull request #718 from harshavardhana/pr_out_add_api_tests_for_both_donut_on_disk_and_donut_cache
...
Add API tests for both donut on disk and donut cache
2015-07-08 02:41:44 +00:00
Harshavardhana
d1deda3a96
Add API tests for both donut on disk and donut cache
2015-07-07 19:39:46 -07:00
Harshavardhana
c76d4f6cdd
Merge pull request #717 from harshavardhana/pr_out_add_rpc_tests
2015-07-08 00:31:49 +00:00
Harshavardhana
ece797c16e
Add rpc tests
2015-07-07 17:27:34 -07:00
Harshavardhana
676b9058de
Separate out memory statistics and system information into two different services
2015-07-07 16:59:20 -07:00
Harshavardhana
8abb96c030
If NodeDisks are not empty do not impose cache maxSize restriction
2015-07-07 16:41:40 -07:00
Harshavardhana
efa91474e7
Merge pull request #716 from harshavardhana/pr_out_add_nimblenet_tests
...
Add nimbleNet tests
2015-07-07 23:17:20 +00:00
Harshavardhana
a50a44b0ca
Add nimbleNet tests
2015-07-07 16:15:47 -07:00
Harshavardhana
aea83a5fe4
Merge pull request #715 from harshavardhana/pr_out_add_net_addr_wrapper_with_isequal_and_use_it
2015-07-07 22:38:03 +00:00
Harshavardhana
317096a0c4
Add net.Addr wrapper with IsEqual() and use it.
2015-07-07 15:33:08 -07:00
Harshavardhana
11b893804c
Moving os.MkdirAll() inside atomic for auto parent directory creates
2015-07-07 12:35:57 -07:00
Harshavardhana
5fc377ae57
Merge pull request #714 from harshavardhana/pr_out_move_atomic_file_writes_into_its_own_package_use_them_inside_quick_and_disk_packages
2015-07-07 19:31:42 +00:00
Harshavardhana
52cd23ad9f
Move atomic file writes into its own package, use them inside quick and disk packages
2015-07-07 12:29:14 -07:00
Harshavardhana
fadadf0e1a
Merge pull request #713 from harshavardhana/pr_out_across_donut_split_nimble_some_code_cleanup
2015-07-07 18:04:33 +00:00
Harshavardhana
3622fbc87d
Across donut, split, nimble some code cleanup
2015-07-06 21:55:21 -07:00
Harshavardhana
7be35915bc
Merge pull request #712 from harshavardhana/pr_out_add_multi_thread_protection_and_also_allow_atomic_file_creates_rename_upon_close_
...
Add multi-thread protection and also allow atomic file creates, rename upon Close()
2015-07-07 01:58:42 +00:00
Harshavardhana
bbb89b5776
Add multi-thread protection and also allow atomic file creates, rename upon Close()
2015-07-06 18:54:32 -07:00
Harshavardhana
c2c7bdf0cd
Cleanup nimble http
2015-07-06 18:51:20 -07:00
Harshavardhana
5132fd84db
Merge pull request #711 from harshavardhana/pr_out_avoid_config_reload_all_the_time_reload_is_manually_triggerred_from_outside
2015-07-07 00:29:46 +00:00
Harshavardhana
b029d0a5f0
Avoid config reload all the time, reload is manually triggerred from outside
2015-07-06 17:26:35 -07:00
Harshavardhana
8b94c53345
Fix issues with multipart upload
2015-07-06 16:22:27 -07:00
Harshavardhana
474954022e
Add modified grace library from facebookgo, rename it as nimble
2015-07-06 15:40:12 -07:00
Harshavardhana
2c18c3be68
Merge pull request #710 from harshavardhana/pr_out_add_donut_rpc_service_for_sending_changes_to_configuration_files
2015-07-06 18:13:18 +00:00
Harshavardhana
1d64e4b6c1
Add Donut rpc service for sending changes to configuration files
2015-07-06 11:10:06 -07:00
Harshavardhana
57d634da25
Merge pull request #709 from harshavardhana/pr_out_add_updateconfig_code_to_load_config_changes_if_possible_for_every_function
...
Add updateConfig code to load config changes if possible for every function
2015-07-06 05:51:46 +00:00
Harshavardhana
10b082144e
Add updateConfig code to load config changes if possible for every function
2015-07-05 22:46:42 -07:00
Harshavardhana
36835befe6
Merge pull request #708 from harshavardhana/pr_out_add_sighup_sigusr2_into_trapping_code_to_trap_signals_for_reloading_configuration
...
Add sighup, sigusr2 into trapping code, to trap signals for reloading configuration.
2015-07-06 04:43:31 +00:00
Harshavardhana
ba0a5ed416
Add sighup, sigusr2 into trapping code, to trap signals for reloading configuration.
...
Need to still figure out a way of graceful restarts - gave facebookgo/httpdown a shot,
but it is not suitable.
2015-07-05 21:40:53 -07:00
Harshavardhana
a74a2db8f0
Merge pull request #707 from harshavardhana/pr_out_fix_another_deadlock_inside_createobjectpart_code_premature_return_without_unlocking_
2015-07-06 03:29:25 +00:00
Harshavardhana
4a27ab0e58
Fix another deadlock inside CreateObjectPart() code, premature return without Unlocking()
...
Also this patch changes the cache key element to be interface{} type not string.
2015-07-05 20:26:32 -07:00
Harshavardhana
d0386dbce0
Merge pull request #706 from harshavardhana/pr_out_fix_go_installation_check_on_amazon_instance
...
Fix go installation check on amazon instance
2015-07-06 01:14:44 +00:00
Harshavardhana
75788c7a1d
Fix go installation check on amazon instance
2015-07-05 18:12:58 -07:00
Harshavardhana
46ab20dcee
Merge pull request #705 from harshavardhana/pr_out_add_basic_controller_code_initiating_json_rpc_connection_getting_list_of_disks_and_memstats_for_now
...
Add basic controller code, initiating json rpc connection getting list of disks and memstats for now.
2015-07-06 00:20:41 +00:00
Harshavardhana
7f0c14f2b7
Add basic controller code, initiating json rpc connection getting list of disks and memstats for now.
2015-07-05 17:17:41 -07:00
Harshavardhana
75a32d1c01
Merge pull request #704 from harshavardhana/pr_out_rename_stuttered_service_names_and_make_them_appropriate
2015-07-05 17:23:22 +00:00
Harshavardhana
a3ccb9d405
Rename stuttered service names and make them appropriate
2015-07-05 10:19:23 -07:00
Harshavardhana
18a8891a15
Merge pull request #703 from harshavardhana/pr_out_minor_changes_to_command_templates
2015-07-05 17:15:20 +00:00
Harshavardhana
adc0a1063c
Minor changes to command templates
2015-07-05 10:13:41 -07:00
Harshavardhana
486b82e950
Merge pull request #702 from harshavardhana/pr_out_add_disk_detection_for_linux_add_new_rpc_service_getdiskinfoservice_remove_dummy_helloservice_
2015-07-05 09:10:03 +00:00
Harshavardhana
e66a84242a
Add disk detection for Linux, add new RPC service GetDiskInfoService(), remove dummy HelloService()
2015-07-05 02:08:33 -07:00
Harshavardhana
181727ab57
Merge pull request #701 from harshavardhana/pr_out_move_to_container_list_datastructure_from_map_string_byte
...
Move to container/list datastructure from map[string][]byte
2015-07-05 00:10:25 +00:00
Harshavardhana
bab4a47525
Move to container/list datastructure from map[string][]byte
2015-07-04 17:08:23 -07:00
Harshavardhana
d11dfe003c
Merge pull request #700 from harshavardhana/pr_out_implement_new_cpu_detection_using_cpuid_cpuidex_plan9_instructions_from_klauspost_cpuid_project_remove_c_code
...
Implement new CPU detection using cpuid, cpuidex plan9 instructions from klauspost/cpuid project, remove C code
2015-07-04 21:30:53 +00:00
Harshavardhana
aa67a19e99
Implement new CPU detection using cpuid, cpuidex plan9 instructions from klauspost/cpuid project, remove C code
2015-07-04 14:28:16 -07:00
Harshavardhana
9977888972
Merge pull request #698 from harshavardhana/pr_out_implement_metadata_cache_metadata_cache_is_used_by_top_level_donut_right_now_rename_trove_as_data_cache
...
Implement metadata cache, metadata cache is used by top level donut right now. Rename trove as data cache
2015-07-04 04:12:39 +00:00
Harshavardhana
0a827305ad
Implement metadata cache, metadata cache is used by top level donut right now. Rename trove as data cache
...
We should use it internally everywhere.
2015-07-03 21:09:57 -07:00
Harshavardhana
7d2609856e
Merge pull request #697 from harshavardhana/pr_out_make_donut_do_everything_as_an_atomic_operation_this_avoids_all_the_deadlocks_and_races
...
Make donut do everything as an atomic operation, this avoids all the deadlocks and races
2015-07-04 00:18:45 +00:00
Harshavardhana
14844f48dd
Make donut do everything as an atomic operation, this avoids all the deadlocks and races
2015-07-03 17:16:58 -07:00
Harshavardhana
86bcfed2da
Merge pull request #696 from minio/server-cleanup
...
Server cleanup
2015-07-03 22:58:08 +00:00
Harshavardhana
30fc14e703
Restructure codebase move crypto, checksum to top-level, move `split` into donut, move crypto/keys into api/auth
2015-07-03 15:24:51 -07:00
Harshavardhana
8a4e7bcdcf
Add full API tests, move storage/donut to donut, add disk tests as well
2015-07-03 14:36:29 -07:00
Harshavardhana
7c37e9d06a
Make donut fully integrated back into API handlers
2015-07-02 21:04:04 -07:00
Harshavardhana
12bde7df30
Add simple Ticket Master which pro-actively sends messages on proceedChannel
...
Handlers are going to wait on proceedChannel, this the initial step towards
providing priority for different set of API operations
2015-07-02 21:04:04 -07:00
Harshavardhana
5cfb05465e
Add cache, donut tests separately - fix behavior differences
...
Remove priority queue, implement it using a simpler channels
2015-07-02 21:04:04 -07:00
Harshavardhana
ebe61d99d9
Use cache Append() for saving objects in memory, GetObject() caches un-cached entries while reading
2015-07-02 21:04:04 -07:00
Harshavardhana
bce93c1b3a
Integrate cache with donut, add tests
2015-07-02 21:04:04 -07:00
Harshavardhana
0533abf6a8
Make priority queue lambda function return error over a channel
2015-07-02 21:04:04 -07:00
Harshavardhana
38a6ce36e5
Remove slow AppendUniq code, rolling through over a slice is in-efficient
...
Remove it and use map instead
2015-07-02 21:04:04 -07:00
Harshavardhana
84810162f5
Add simple Version and GetSysInfo services
2015-07-02 21:04:04 -07:00
Harshavardhana
14ec42d646
Add initial implementation of priority queue, uses container/heap
2015-07-02 21:04:04 -07:00
Harshavardhana
eb5aa19dfa
Remove custom Config, will use quick Config instead for user access keys
2015-07-02 21:04:04 -07:00
Harshavardhana
701c3e5242
Add new RPC helpers wrapping over regular rpc packages, add middleware chaining ability
2015-07-02 21:04:04 -07:00
Harshavardhana
188785a886
Add and remove dependencies
2015-07-02 21:04:04 -07:00
Harshavardhana
4addf7a996
Restructure API handlers, add JSON RPC simple HelloService right now.
2015-07-02 21:04:04 -07:00
Harshavardhana
335c7827eb
More donut, cache, api cleanup
2015-07-02 21:04:04 -07:00
Harshavardhana
dc0df3dc0e
Breakaway from driver model, move cache into donut
2015-07-02 21:04:03 -07:00
Harshavardhana
72572d6c71
Remove some api server code bringing in new cleanup
2015-07-02 21:04:03 -07:00
Harshavardhana
c2031ca066
Add server and control command
2015-07-02 21:04:03 -07:00
Frederick F. Kautz IV
101784bc44
Merge pull request #695 from fkautz/pr_out_fixing_api_definitions
2015-07-02 13:16:49 -07:00
Frederick F. Kautz IV
cfbc169034
Fixing API definitions
2015-07-02 13:14:21 -07:00
Harshavardhana
1d31c76dd6
Merge pull request #694 from harshavardhana/pr_out_move_memory_code_out_add_it_as_layer_on_top_of_existing_cache_code_wip
2015-06-30 17:12:29 +00:00
Harshavardhana
8f61d6b6be
Move memory code out, add it as layer on top of existing donut code
...
Just like how http.Handlers can be overlayed on top of each other
with each implementing ServeHTTP().
drivers.Driver can be overlayed on top of each other in similar manner
which would implement the drivers.Driver interface.
API <----> cache <----> donut <----> donut(format)
2015-06-30 10:09:12 -07:00
Harshavardhana
fe3c618cc7
Merge pull request #693 from harshavardhana/pr_out_add_dummy_driver_for_community_to_submit_new_drivers
2015-06-29 23:48:37 +00:00
Harshavardhana
ab6e16bb41
Add dummy driver for community to submit new drivers
2015-06-29 16:43:50 -07:00
Harshavardhana
12de98fb62
Rename memory driver as cache
2015-06-29 16:43:50 -07:00
Harshavardhana
2571342451
Filesystem goes the high road *again*
2015-06-29 16:43:42 -07:00
Harshavardhana
c4c67581dc
Merge pull request #692 from harshavardhana/pr_out_isvalidbucket_is_sufficient_we_don_t_need_to_verify_for_
2015-06-29 22:27:11 +00:00
Harshavardhana
f74d6138da
IsValidBucket() is sufficient we don't need to verify for "."
2015-06-29 15:15:54 -07:00
Harshavardhana
b5a5861c8f
Merge pull request #691 from harshavardhana/pr_out_handle_couple_of_cases_of_oom_conditions_move_caching_to_getobject_rather_than_putobject_
...
Handle couple of cases of OOM conditions, move caching to GetObject() rather than PutObject()
2015-06-29 19:33:55 +00:00
Harshavardhana
3109909355
Handle couple of cases of OOM conditions, move caching to GetObject() rather than PutObject()
2015-06-29 12:28:50 -07:00
Harshavardhana
d07d0c670a
Return back proper errors in writeObjectData(), rename few functions
2015-06-29 11:46:35 -07:00
Harshavardhana
be816145a9
Merge pull request #690 from harshavardhana/pr_out_put_object_on_successful_write_returns_full_metadata_to_avoid_subsequent_getobjectmetadata_calls_in_driver
2015-06-29 18:21:33 +00:00
Harshavardhana
10c807f233
Put object on successful write returns full metadata, to avoid subsequent GetObjectMetadata() calls in driver
2015-06-29 11:15:46 -07:00
Harshavardhana
6921328b93
Avoid frivolous GetObjectMetadata() calls at driver level, return back all the information in donut ListObjects()
2015-06-29 11:14:58 -07:00
Harshavardhana
f05ad062ee
Merge pull request #689 from harshavardhana/pr_out_expand_http_server_struct_to_store_more_values
2015-06-29 07:17:31 +00:00
Harshavardhana
d8f7896a43
Expand http server struct to store more values
2015-06-29 00:12:28 -07:00
Harshavardhana
63f9647c80
Merge pull request #688 from harshavardhana/pr_out_use_errorchannels_only_for_services_not_for_drivers_reduce_them_to_use_simple_functions
2015-06-29 07:02:24 +00:00
Harshavardhana
42c0287943
Use errorChannels only for services not for drivers, reduce them to use simple functions
2015-06-28 23:59:47 -07:00
Harshavardhana
b2bf90afbd
Merge pull request #687 from harshavardhana/pr_out_move_to_set_not_append_due_to_large_memory_reference_copy
2015-06-28 17:15:06 +00:00
Harshavardhana
91e5f648cb
Move to Set() not Append() due to large memory reference copy
2015-06-28 10:13:12 -07:00
Harshavardhana
22abe1b397
Merge pull request #686 from harshavardhana/pr_out_add_free_method_for_proxyreader_to_aggressively_de_allocate_read_data_to_handle_certain_out_of_memory_conditions
...
Add free() method for proxyReader to aggressively de-allocate Read data, to handle certain out of memory conditions
2015-06-28 03:45:48 +00:00
Harshavardhana
ac4d8fe478
Add free() method for proxyReader to aggressively de-allocate Read data, to handle certain out of memory conditions
...
There are still some more out there
2015-06-27 20:43:25 -07:00
Harshavardhana
a4d20d1e75
Merge pull request #685 from harshavardhana/pr_out_add_append_method_to_trove_cache_for_appending_data_to_an_existing_key
...
Add Append() method to trove cache for appending data to an existing key
2015-06-28 03:28:06 +00:00
Harshavardhana
05f8654e3d
Add Append() method to trove cache for appending data to an existing key
...
This largely avoids a large buffer copy which would accumulate inside proxyReader{}
This patch also implements "initialize()" function to init and populate data
on all the existing buckets, avoiding the redundant ListBuckets() invoked by
every API call.
2015-06-27 20:25:24 -07:00
Harshavardhana
762aae7c32
Merge pull request #684 from harshavardhana/pr_out_make_sure_to_populate_on_disk_data_into_memory_upon_first_api_requests
2015-06-28 01:28:29 +00:00
Harshavardhana
367772b988
Make sure to populate on disk data into memory upon first API requests
2015-06-27 18:25:21 -07:00
Harshavardhana
350e6eb5bb
Merge pull request #683 from harshavardhana/pr_out_add_proper_command_paramters_for_donut
2015-06-28 00:41:13 +00:00
Harshavardhana
07a6aafc94
Add proper command paramters for donut
2015-06-27 17:39:22 -07:00
Harshavardhana
c65969077d
Merge pull request #682 from harshavardhana/pr_out_an_attempt_to_bring_in_memory_layer_into_donut_driver
2015-06-28 00:25:27 +00:00
Harshavardhana
45a7eab804
An attempt to bring in memory layer into donut driver
2015-06-27 17:23:34 -07:00
Harshavardhana
7ab16b5b83
Merge pull request #681 from harshavardhana/pr_out_keeping_the_lexical_order_same_add_optimizations_provide_a_comprehensive_response_from_listobjects_
...
Keeping the lexical order same add optimizations, provide a comprehensive response from ListObjects()
2015-06-27 20:15:24 +00:00
Harshavardhana
f3c25bcfc4
Keeping the lexical order same add optimizations, provide a comprehensive response from ListObjects()
2015-06-27 13:12:44 -07:00
Harshavardhana
795e48d492
Merge pull request #680 from harshavardhana/pr_out_rename_functions_for_their_purpose
2015-06-27 19:42:04 +00:00
Harshavardhana
ae66ae42c4
Rename functions for their purpose
2015-06-27 12:39:11 -07:00
Harshavardhana
9c2e861470
Merge pull request #679 from harshavardhana/pr_out_object_metadata_was_wrongly_misconstrued_to_be_mutable_handle_it
2015-06-27 06:26:21 +00:00
Harshavardhana
9a4680475f
Object metadata was wrongly misconstrued to be mutable, handle it
2015-06-26 23:22:53 -07:00
Harshavardhana
39f26acbc9
Merge pull request #678 from harshavardhana/pr_out_handle_racy_map_updates_in_listobjects_on_a_bucket
...
Avoid racy maps, read from disk on success return quickly. Many more optimizations
2015-06-27 02:59:50 +00:00
Harshavardhana
3aa6d90c5e
Avoid racy maps, read from disk on success return quickly. Many more optimizations
2015-06-26 19:49:37 -07:00
Harshavardhana
aab4937084
Merge pull request #677 from harshavardhana/pr_out_moving_to_more_typed_responses_this_removes_all_the_necessity_for_strconv
...
Donut moves to typed metadata, removing the necessity for strconv, parsing and string converstions
2015-06-26 23:26:17 +00:00
Harshavardhana
68974918ac
Donut moves to typed metadata, removing the necessity for strconv, parsing and string converstions
2015-06-26 16:23:12 -07:00
Harshavardhana
e3d8a9e0f1
Merge pull request #676 from harshavardhana/pr_out_add_new_metadata_definitions_and_use_them_wip
2015-06-26 20:38:02 +00:00
Harshavardhana
767d3743ee
Add new metadata definitions and use them
2015-06-26 13:34:09 -07:00
Harshavardhana
0cb3f76a91
Merge pull request #675 from harshavardhana/pr_out_import_quick_key_value_store_from_minio_client_for_persistent_state_files_primarily_for_donut
...
Import quick key value store from Minio Client for persistent state files, primarily for donut
2015-06-26 02:59:28 +00:00
Harshavardhana
9958e34772
Import quick key value store from Minio Client for persistent state files, primarily for donut
2015-06-25 19:57:31 -07:00
Harshavardhana
bd0dccd8f1
Merge pull request #674 from harshavardhana/pr_out_donut_cleanup_another_set
...
Donut cleanup another set
2015-06-26 01:59:45 +00:00
Harshavardhana
fb9adb5524
Donut cleanup another set
...
- Make sure to close all readers
- Fix errors in api_testsuite c.Assert(err, IsNil) should be done right after each function call
2015-06-25 18:54:34 -07:00
Harshavardhana
eec66f195a
Take all the ListObjects into bucket handlers
...
Earlier the listing would wait for all the objects to be processed
this is essentially very time consuming considering even for 100,000
files.
2015-06-25 18:04:29 -07:00
Harshavardhana
8405c4d42f
Merge pull request #673 from harshavardhana/pr_out_remove_more_bloated_code_simplify
...
Remove more bloated code - simplify
2015-06-25 21:14:43 +00:00
Harshavardhana
45e9d25931
Remove more bloated code - simplify
2015-06-25 13:02:08 -07:00
Harshavardhana
7ade42165f
Merge pull request #672 from harshavardhana/pr_out_add_simple_locking_for_donut_api_for_now_fixes_671
2015-06-25 18:33:08 +00:00
Harshavardhana
82dcbf262d
Add simple locking for donut API for now - fixes #671
2015-06-25 11:29:11 -07:00
Harshavardhana
5abcb7f348
Merge pull request #670 from harshavardhana/pr_out_go_vet_fixes_for_donut
2015-06-25 04:08:49 +00:00
Harshavardhana
03b4d3b308
Go vet fixes for donut
2015-06-24 21:07:03 -07:00
Harshavardhana
57a2b53178
Removing further bloated code simplifying
2015-06-24 21:03:39 -07:00
Harshavardhana
a2c205ff2e
Use external package disk for donut.
2015-06-24 21:03:02 -07:00
Harshavardhana
841ff01412
Move disk into its own package, remove bloated code
2015-06-24 20:13:47 -07:00
Harshavardhana
1682c748ac
Remove unnecessary interfaces from donut, cleanup
2015-06-24 19:43:38 -07:00
Harshavardhana
b915cc3611
Merge pull request #669 from harshavardhana/pr_out_add_sha256_and_sha512_windows_compatibility_layer
2015-06-24 21:43:26 +00:00
Harshavardhana
3498872467
Add sha256 and sha512 windows compatibility layer
2015-06-24 14:39:41 -07:00
Harshavardhana
2af863cefc
Merge pull request #668 from harshavardhana/pr_out_fix_a_bug_on_windows_regarding_blocksse3_calculation
2015-06-24 21:29:34 +00:00
Harshavardhana
77d35b87d4
Fix a bug on windows regarding blockSSE3 calculation
2015-06-24 14:24:33 -07:00
Harshavardhana
1056e7e180
Merge pull request #667 from harshavardhana/pr_out_add_windows_code_for_sha1_and_crc32c
2015-06-24 21:19:54 +00:00
Harshavardhana
f1410731db
Add windows code for sha1 and crc32c
2015-06-24 14:16:32 -07:00
Harshavardhana
fbee8f8122
Merge pull request #665 from harshavardhana/pr_out_fix_wrong_tmpfs_listing_in_document_filesystem_map
2015-06-23 23:17:21 -07:00
Harshavardhana
ba2d3dea74
Fix wrong TMPFS listing in donut filesystem map
2015-06-23 23:16:06 -07:00
Harshavardhana
792e6c2d3a
Merge pull request #664 from harshavardhana/pr_out_trim_iodine_path_properly_so_that_now_errors_have_github_com_minio_minio_prefixed
2015-06-23 13:39:33 -07:00
Harshavardhana
e818bc7187
Trim iodine path properly, so that now errors have github.com/minio/minio prefixed
2015-06-23 13:36:54 -07:00
Harshavardhana
3e9141d22d
Merge pull request #663 from harshavardhana/pr_out_add_missing_strongly_typed_errors_for_donut
2015-06-23 11:59:17 -07:00
Harshavardhana
2fd52ca284
Add missing strongly typed errors for Donut
2015-06-23 11:54:44 -07:00
Harshavardhana
a4fda9fa9c
Merge pull request #662 from vadmeste/enhance_golang_env_check
...
Check if go binary belongs to the go installation pointed by GOROOT env
2015-06-23 10:35:01 -07:00
Anis Elleuch
4bed0aa526
Check if go binary belongs to the go installation pointed by GOROOT env
2015-06-23 17:44:05 +01:00
Harshavardhana
d0dd047bef
Merge pull request #661 from harshavardhana/pr_out_fix_builddeps_paths_for_golang_installation
2015-06-22 12:09:10 -07:00
Harshavardhana
7a060110ff
Fix BUILDDEPS paths for golang installation
2015-06-22 12:07:23 -07:00
Harshavardhana
c340a10e8f
Merge pull request #660 from harshavardhana/pr_out_verify_d_donut_to_be_non_nil_usually_happens_when_multiple_go_versions_compilations_are_linked_possible_cause_for_659
2015-06-20 11:04:58 -07:00
Harshavardhana
3bf64f5669
Verify d.donut to be non-nil, usually happens when multiple go versions compilations are linked - possible cause for #659
2015-06-20 11:03:17 -07:00
Harshavardhana
cdeadae167
Merge pull request #657 from harshavardhana/pr_out_use_filepath_everywhere_instead_of_path_functions_for_portability_fixes_656
2015-06-18 16:42:29 -07:00
Harshavardhana
641f07cecf
Use filepath everywhere instead of path.{} functions for portability - fixes #656
2015-06-18 16:02:45 -07:00
Harshavardhana
285b1cc5d8
Merge pull request #655 from harshavardhana/pr_out_remove_redundant_ok_for_map
...
Remove redundant !ok for map
2015-06-17 22:39:33 -07:00
Harshavardhana
573a6134b2
Remove redundant !ok for map
2015-06-17 22:36:46 -07:00
Harshavardhana
3842a57f52
Merge pull request #653 from harshavardhana/pr_out_hold_lock_on_getglobalstatekey_fixes_652
...
Hold lock on GetGlobalStateKey() - fixes #652
2015-06-17 20:44:03 -07:00
Harshavardhana
e9a3fd677a
Hold lock on GetGlobalStateKey() - fixes #652
2015-06-17 20:37:49 -07:00
Harshavardhana
b793f53d48
Minior change filter() to filterObjects()
2015-06-17 20:35:44 -07:00
Harshavardhana
45424cfe52
Update README.md
2015-06-17 13:58:40 -07:00
Harshavardhana
daa113d89d
Merge pull request #651 from harshavardhana/pr_out_migrating_minio_server_build_to_do_go_run_make_go_style_like_mc
2015-06-16 20:27:31 -07:00
Harshavardhana
8bde4d4e34
Migrating minio server build to do 'go run make.go' style like mc
2015-06-16 20:23:02 -07:00
Harshavardhana
4998907835
Update README.md
2015-06-16 16:09:10 -07:00
Harshavardhana
bbc6c056e8
Update README.md
2015-06-16 16:03:17 -07:00
Harshavardhana
db666a1f6e
Update README.md
2015-06-16 15:58:04 -07:00
Harshavardhana
6b8766790b
Merge pull request #650 from harshavardhana/pr_out_add_time_since_on_buckets_created_time_to_avoid_deleting_buckets_spuriously_during_eviction_of_objects
2015-06-13 11:30:06 -07:00
Harshavardhana
d4e71beb54
Add time.Since on buckets created time to avoid deleting buckets spuriously during eviction of objects
2015-06-13 11:28:04 -07:00
Harshavardhana
da593b498c
Merge pull request #649 from harshavardhana/pr_out_minor_optimization_avoiding_one_unnecessary_copy
...
Minor optimization avoiding one unnecessary copy
2015-06-11 14:26:18 -07:00
Harshavardhana
636788997b
Minor optimization avoiding one unnecessary copy
2015-06-11 14:23:05 -07:00
Harshavardhana
609c587a27
Merge pull request #648 from harshavardhana/pr_out_make_sure_to_reply_back_etag_with_quotes_for_s3_compliance
...
Make sure to reply back ETag with quotes for s3 compliance
2015-06-09 01:14:49 -07:00
Harshavardhana
6d44274db6
Make sure to reply back ETag with quotes for s3 compliance
2015-06-09 01:13:02 -07:00
Harshavardhana
0c5aa7a3d8
Merge pull request #647 from harshavardhana/pr_out_remove_unnecessary_tlshelpers_ssl_works_with_domain_certificate
2015-06-08 20:57:10 -07:00
Harshavardhana
a947767339
Remove unnecessary tlsHelpers, SSL works with domain certificate
2015-06-08 20:54:41 -07:00
Harshavardhana
666ed34059
Merge pull request #646 from harshavardhana/pr_out_enable_tls_and_use_it
2015-06-08 20:13:17 -07:00
Harshavardhana
c6ecd94420
Enable TLS and use it
2015-06-08 20:10:59 -07:00
Harshavardhana
dcc1609d2e
Merge pull request #645 from harshavardhana/pr_out_return_back_entity_too_large_for_objects_bigger_than_cache_size
...
Return back entity too large for objects bigger than cache size
2015-06-08 17:12:22 -07:00
Harshavardhana
64b014369c
Return back entity too large for createObject requests bigger than cache size
2015-06-08 17:10:07 -07:00
Harshavardhana
5edd8cce3d
Merge pull request #644 from harshavardhana/pr_out_load_from_persisted_multipart_session_now_multipart_uploads_can_resume_even_after_restarting_minio_fs_server
2015-06-08 15:17:43 -07:00
Harshavardhana
ab1976bc14
Load from persisted multipart session, now multipart uploads can resume even after restarting minio fs server
2015-06-08 15:15:37 -07:00
Harshavardhana
4d2eab78d8
Merge pull request #643 from harshavardhana/pr_out_more_multipart_fixes_filter_out_parts_and_multipart_metadata_from_listobjects
...
More multipart fixes, filter out parts and multipart metadata from listObjects
2015-06-08 14:15:07 -07:00
Harshavardhana
2976f109df
More multipart fixes, filter out parts and multipart metadata from listObjects
2015-06-08 14:08:36 -07:00
Harshavardhana
765fb08e20
Merge pull request #642 from harshavardhana/multipartfixes
...
fs multipart fixes
2015-06-08 12:31:41 -07:00
Harshavardhana
e08d59be00
Make sure to delete uploadid's from active session file properly
2015-06-08 12:25:51 -07:00
Harshavardhana
0d3710cb9b
Store in json file instead of gob for fs metadata
2015-06-08 12:08:01 -07:00
Harshavardhana
50994d7d71
Merge pull request #641 from harshavardhana/pr_out_fix_erroneous_new_line_in_helptemplate
...
Fix erroneous new line in helptemplate
2015-06-08 11:59:14 -07:00
Harshavardhana
0571ad0b83
Fix erroneous new line in helptemplate
2015-06-08 11:57:14 -07:00
Harshavardhana
8671057005
Merge pull request #640 from harshavardhana/pr_out_add_delete_handlers_and_reply_back_as_notimplemented_instead_of_404
...
Add delete handlers and reply back as 'NotImplemented' instead of 404
2015-06-08 11:08:59 -07:00
Harshavardhana
8efc842b59
Add delete handlers and reply back as 'NotImplemented' instead of 404
2015-06-08 11:06:06 -07:00
Harshavardhana
69366e20d3
Merge pull request #639 from harshavardhana/pr_out_enhance_logging_handler_to_print_in_humanize_form_for_contentlength
2015-06-08 02:17:01 -07:00
Harshavardhana
8a320c4447
Enhance logging handler to print in humanize form for ContentLength
2015-06-08 02:14:09 -07:00
Harshavardhana
2d5c48a973
Merge pull request #638 from harshavardhana/pr_out_update_gorilla_mux_and_go_humanize
...
Update gorilla mux and go-humanize
2015-06-08 01:38:48 -07:00
Harshavardhana
b45e23b958
Update gorilla mux and go-humanize
2015-06-08 01:36:07 -07:00
Harshavardhana
964355206d
Merge pull request #637 from harshavardhana/pr_out_put_back_the_older_conn_limit
2015-06-06 18:12:04 -07:00
Harshavardhana
cefd740002
Put back the older conn_limit
2015-06-06 18:11:38 -07:00
Harshavardhana
78c0f85eb0
Merge pull request #636 from harshavardhana/pr_out_an_attempt_to_make_connection_limit_bufferred_channel_wip
2015-06-06 18:08:09 -07:00
Harshavardhana
371651bde6
Add rate limiter instead of connection limit for now
2015-06-06 18:04:55 -07:00
Harshavardhana
996c1a5c28
Merge pull request #635 from harshavardhana/pr_out_fix_a_deadlock_in_trove_when_evicting_objects_when_we_reach_the_maxsize_used
...
Fix a deadlock in trove when evicting objects when we reach the maxSize used
2015-06-06 16:06:53 -07:00
Harshavardhana
21916dc0a1
Fix a deadlock in trove when evicting objects when we reach the maxSize used
2015-06-06 16:02:52 -07:00
Harshavardhana
d78832fb39
Merge pull request #634 from harshavardhana/pr_out_make_connection_limit_configurable
2015-06-06 14:33:56 -07:00
Harshavardhana
8b43655d9f
Make connection limit configurable
2015-06-06 14:31:50 -07:00
Anand Babu (AB) Periasamy
6263b99e5a
Merge pull request #633 from minio/reorganize-commands
...
reorganize commands
2015-06-05 18:32:22 -07:00
Anand Babu (AB) Periasamy
2604ffa42a
reorganize commands
2015-06-05 18:29:24 -07:00
Harshavardhana
391e03881d
Merge pull request #632 from harshavardhana/pr_out_do_not_attempt_to_restore_while_saving_new_deps
2015-06-04 16:35:49 -07:00
Harshavardhana
5101273ec7
Do not attempt to restore while saving new deps
2015-06-04 16:32:35 -07:00
Harshavardhana
962f5d5160
Merge pull request #631 from harshavardhana/pr_out_implement_custom_date_command_for_cross_platform_portability
2015-06-04 12:35:08 -07:00
Harshavardhana
5d484e2770
Implement custom date command for cross platform portability
2015-06-04 12:32:23 -07:00
Harshavardhana
23861fa6a7
Merge pull request #630 from harshavardhana/pr_out_misc_changes_before_capturing_proper_video
2015-06-03 18:31:16 -07:00
Harshavardhana
a344f5b34d
Misc changes before capturing proper video
...
- Disable management UI command option
- Remove featureflags option, filesystem and memory already support multipart
- Print informative messages after starting minio server
2015-06-03 18:27:15 -07:00
Harshavardhana
a5b9c4fd26
Merge pull request #629 from harshavardhana/pr_out_enable_filesystem_for_api_tests_this_patch_also_implements_listmultipartuploads_
2015-06-01 20:59:06 -07:00
Harshavardhana
097f70a3c5
Enable filesystem for API tests, this patch also implements ListMultipartUploads()
2015-06-01 20:57:13 -07:00
Harshavardhana
bf36fcb6f7
Merge pull request #628 from harshavardhana/pr_out_skip_files_and_weird_directories_from_listbuckets_allowing_this_so_that_minio_fs_mode_can_be_used_as_ftp_service
...
Skip files and weird directories from listBuckets(), allowing this so that minio fs mode can be used as ftp service
2015-06-01 16:46:51 -07:00
Harshavardhana
bfb9209689
Skip files and weird directories from listBuckets(), allowing this so that minio fs mode can be used as ftp service
2015-06-01 16:43:43 -07:00
Harshavardhana
b038c4bd7f
Merge pull request #627 from harshavardhana/pr_out_implement_multipart_support
2015-06-01 16:28:26 -07:00
Harshavardhana
d86ba5f336
Implement multipart support
...
- implemented NewMultipartUpload()
- implemented CreateObjectPart()
- implemented ListObjectParts()
- implemented AbortMultipartUpload()
- implemented CompleteMultipartUpload()
2015-06-01 16:25:11 -07:00
Harshavardhana
00a976adf1
Merge pull request #626 from harshavardhana/pr_out_errorresponse_from_minio_server_needed_proper_xml_translations
2015-06-01 12:22:48 -07:00
Harshavardhana
c07d48e7e8
ErrorResponse from minio server needed proper xml translations
2015-06-01 12:19:27 -07:00
Harshavardhana
e1270bcdf6
Merge pull request #625 from harshavardhana/pr_out_make_caching_a_package_trove_and_use_it_inside_memory_driver
2015-05-31 18:43:47 -07:00
Harshavardhana
a91e519692
Make caching a package trove, and use it inside memory driver
2015-05-31 18:41:56 -07:00
Harshavardhana
122d67625f
Merge pull request #624 from vadmeste/pr_out_protect_exported_memory_object_api_by_a_mutex
...
Protect the exported method Delete() by a lock
2015-05-31 12:34:02 -07:00
Anis Elleuch
7e5a26f42c
Protect the exported method Delete()
2015-05-31 20:29:42 +01:00
Harshavardhana
437c2b31dd
Merge pull request #622 from harshavardhana/pr_out_add_filesystem_factory_functions_and_add_related_cli_options
2015-05-30 03:26:58 -07:00
Harshavardhana
82a0eac659
Add filesystem factory functions and add related cli options
2015-05-30 03:23:06 -07:00
Harshavardhana
0cc63706bb
Resurrect back fileystem driver
2015-05-30 03:12:01 -07:00
Harshavardhana
56566d055e
Merge pull request #621 from vadmeste/pr_out_forbid_upload_of_file_bigger_than_memory_backend
2015-05-28 14:09:41 -07:00
Anis Elleuch
d360d0fec3
Forbid the upload of files bigger than the memory backend capacity
2015-05-28 22:07:02 +01:00
Harshavardhana
ab80057c5c
Update README.md add asciinema video for minio
2015-05-27 20:16:38 -07:00
Harshavardhana
cb2a0f774a
Update README.md
2015-05-27 18:54:03 -07:00
Harshavardhana
079615a104
Update CONTRIBUTING.md
2015-05-27 18:41:14 -07:00
Harshavardhana
0371ff7242
Merge pull request #619 from minio/harshavardhana-patch-1
...
Rename BUILDDEPS.md to INSTALLGO.md
2015-05-27 18:38:08 -07:00
Harshavardhana
a23f449734
Merge pull request #620 from minio/harshavardhana-patch-2
...
Update README.md
2015-05-27 18:37:25 -07:00
Harshavardhana
d89118e3b6
Update README.md
2015-05-27 18:34:42 -07:00
Harshavardhana
cfa4329d21
Rename BUILDDEPS.md to INSTALLGO.md
2015-05-27 17:56:54 -07:00
Harshavardhana
7506227a77
Update BUILDDEPS.md
2015-05-27 17:51:20 -07:00
Harshavardhana
82f267706b
Merge pull request #618 from minio/harshavardhana-build-dep
...
Update BUILDDEPS.md
2015-05-27 17:46:45 -07:00
Harshavardhana
2cbd4e00df
Update BUILDDEPS.md
2015-05-27 17:43:39 -07:00
Harshavardhana
66967b8ec0
Merge pull request #617 from harshavardhana/pr_out_fix_erroneous_build_date_formatting
2015-05-26 02:18:27 -07:00
Harshavardhana
ddfa964025
Fix erroneous build date formatting
2015-05-26 02:14:54 -07:00
Harshavardhana
554a244b47
Add donut OSX support
2015-05-26 02:14:34 -07:00
Harshavardhana
92a37e5113
Merge pull request #615 from harshavardhana/pr_out_use_http_timeformat_to_return_s3_expects_us_to_return_in_gmt_not_utc
2015-05-25 20:18:52 -07:00
Harshavardhana
e46660a966
Use http.TimeFormat to return, S3 expects us to return in GMT not UTC
2015-05-25 20:17:04 -07:00
Harshavardhana
4b22a90182
Merge pull request #614 from harshavardhana/pr_out_handle_authorization_header_better
...
Handle authorization header better
2015-05-24 21:02:47 -07:00
Harshavardhana
19250296c6
Handle authorization header better
2015-05-24 20:59:53 -07:00
Harshavardhana
2d5e1d3797
Merge pull request #613 from harshavardhana/pr_out_relax_minium_object_size_limit_one_can_really_upload_a_0byte_file
2015-05-23 12:22:27 -07:00
Harshavardhana
9666f2e5bf
Relax minium object size limit, one can really upload a 0byte file
...
For example
~~~
$ mc ls https://s3.amazonaws.com/ferenginar/test.go
[2015-05-23 12:02:06 PDT] 0B test.go
$ curl -i -X HEAD https://s3.amazonaws.com/ferenginar/test.go
HTTP/1.1 200 OK
x-amz-id-2: ZMWLriPH+uQJ8IsaMHmuNOU/FXvdSq+s6O7ugUI9hZ695XJTNAZ1utKxh03w5Jcf
x-amz-request-id: F5A11F533B74DD8F
Date: Sat, 23 May 2015 19:19:29 GMT
Last-Modified: Sat, 23 May 2015 19:02:06 GMT
ETag: "d41d8cd98f00b204e9800998ecf8427e"
Accept-Ranges: bytes
Content-Type: binary/octet-stream
Content-Length: 0
Server: AmazonS3
~~~
2015-05-23 12:19:16 -07:00
Harshavardhana
d89396e2f0
Merge pull request #612 from harshavardhana/pr_out_add_contributors_file
2015-05-23 11:55:55 -07:00
Harshavardhana
eca8087993
Add contributors file
2015-05-23 11:49:07 -07:00
Harshavardhana
c63ba4be26
Merge pull request #611 from harshavardhana/pr_out_add_missing_validate_access_keys_after_extracted_from_authheader
2015-05-21 19:29:19 -07:00
Harshavardhana
e11f9110b6
add missing validate access keys after being extracted from AuthHeader
2015-05-21 19:23:47 -07:00
Harshavardhana
138288f788
Merge pull request #610 from harshavardhana/pr_out_fix_some_donut_todo_s_regarding_bucket_created_date
2015-05-21 11:17:24 -07:00
Harshavardhana
ad60b97d52
Fix some donut TODO's regarding bucket created date
2015-05-21 11:13:49 -07:00
Harshavardhana
39dac2afac
Merge pull request #609 from harshavardhana/pr_out_remove_unnecessary_todo_for_httpserver
2015-05-21 02:59:12 -07:00
Harshavardhana
cd817d34df
Finish last bits of api testsuite on server
2015-05-21 02:57:03 -07:00
Harshavardhana
0fbd36e18e
Merge pull request #608 from harshavardhana/pr_out_remove_bitdell_and_analytics_badge_these_services_are_not_working_anymore
2015-05-20 17:20:54 -07:00
Harshavardhana
d4bb71f6fd
Remove bitdell and analytics badge, these services are not working anymore
2015-05-20 17:18:15 -07:00
Harshavardhana
f1fecd7510
Merge pull request #607 from harshavardhana/pr_out_bump_gocyclo_back_to_16
2015-05-19 18:34:38 -07:00
Harshavardhana
f00bfd38da
Bump gocyclo back to 16
2015-05-19 18:32:50 -07:00
Harshavardhana
b84d8fcda7
Merge pull request #606 from harshavardhana/pr_out_fix_duplicate_map_allocation
2015-05-18 13:18:04 -07:00
Harshavardhana
3d61ba5930
Fix duplicate map allocation
2015-05-18 13:14:01 -07:00
Harshavardhana
8ce44bfbd1
Merge pull request #605 from harshavardhana/pr_out_update_totalnumber_of_multiparts_inside_createobjectpart_also_introduce_support_for_uploadidmarker_and_prefix_for_listmultipartuploads
2015-05-18 12:27:05 -07:00
Harshavardhana
e4c0d574dc
Update totalnumber of multiparts inside CreateObjectPart(), also introduce support for UploadIDMarker and Prefix for listMultipartUploads
2015-05-18 12:24:36 -07:00
Harshavardhana
b3e0b3e4dc
Merge pull request #604 from harshavardhana/pr_out_head_request_should_have_content_length_for_only_successful_response_there_is_no_response_body_for_errors_just_header_is_sufficient_fixes_603
2015-05-15 21:58:55 -07:00
Harshavardhana
7ce47130fd
HEAD request should have Content-Length for only successful response, there is no response body for errors, just header is sufficient - fixes #603
...
http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.13
""" in the case of the HEAD method, the size of the entity-body that would have been sent had the request been a GET. """
2015-05-15 21:55:59 -07:00
Harshavardhana
b0925965a7
Merge pull request #602 from harshavardhana/pr_out_remove_redundant_writeheader_calls
2015-05-15 21:00:32 -07:00
Harshavardhana
317637fcb8
Remove redundant WriteHeader calls
2015-05-15 20:58:28 -07:00
Harshavardhana
5c5761ae16
Remove unnecessary GetObjectMetadata prefix argument
2015-05-15 19:25:17 -07:00
Harshavardhana
8ad52364da
Merge pull request #601 from harshavardhana/pr_out_for_multiparts_use_its_own_cache_avoids_deadlocks_during_eviction_wip
...
For multiparts use its own cache avoids deadlocks during expiration
2015-05-14 20:29:15 -07:00
Harshavardhana
6275238a47
For multiparts use its own cache avoids deadlocks during object expiration
...
Using same object cache as regular objects for multiparts causes deadlocks during object expiration routine.
As a side affect this also fixes the bug of listing multiparts during regular 'listing' of objects.
2015-05-14 20:26:59 -07:00
Harshavardhana
5751b0bf36
Merge pull request #600 from harshavardhana/pr_out_fixing_a_race_in_newmultipartupload_session_for_simultaneous_requests_observed_at_play_minio_io
2015-05-14 19:13:44 -07:00
Harshavardhana
c41ebf7535
Fixing a race in NewMultipartUpload() session for simultaneous requests, observed at play.minio.io
2015-05-14 19:11:19 -07:00
Harshavardhana
eb34634e81
Merge pull request #599 from harshavardhana/pr_out_add_more_tests_for_multipart_now_in_driver_api_testsuite
2015-05-14 17:31:29 -07:00
Harshavardhana
8fb427a892
Add more tests for multipart now in driver api_testsuite
2015-05-14 17:28:39 -07:00
Harshavardhana
06a9e468d6
Merge pull request #598 from harshavardhana/pr_out_add_tests_for_abortmultipartupload_listobjectparts_listmultipartuploads
2015-05-14 17:09:57 -07:00
Harshavardhana
dba7c94571
Add tests for abortMultipartUpload, listobjectparts, listmultipartuploads
2015-05-14 17:07:30 -07:00
Harshavardhana
5f381a8bee
Merge pull request #597 from harshavardhana/pr_out_an_attempt_to_implement_listmultipartuploads_
2015-05-14 16:25:37 -07:00
Harshavardhana
1bd94ec6ab
An attempt to implement ListMultipartUploads()
2015-05-14 16:22:47 -07:00
Harshavardhana
4f7ae8af92
Merge pull request #596 from harshavardhana/pr_out_avoid_one_last_memory_copy_wip
2015-05-14 11:29:59 -07:00
Harshavardhana
82b043005a
Avoid one last memory copy
2015-05-14 11:26:55 -07:00
Harshavardhana
55b9a5f5e4
Merge pull request #595 from harshavardhana/pr_out_add_a_new_validatecontenttypehandler_verify_accept_header_earlier
2015-05-13 12:26:30 -07:00
Harshavardhana
5498c90788
Add a new validateContentTypeHandler{}, verify Accept header earlier
2015-05-13 12:22:15 -07:00
Harshavardhana
fdbfa5070b
Merge pull request #593 from harshavardhana/pr_out_more_scripts_and_code_from_minio_io_to_minio
2015-05-11 16:43:54 -07:00
Harshavardhana
de56909d55
More scripts and code from Minio-io to minio
2015-05-11 16:43:38 -07:00
Harshavardhana
946341ac29
Merge pull request #592 from harshavardhana/pr_out_move_cli_check_to_minio_from_minio_io
2015-05-11 16:40:23 -07:00
Harshavardhana
50559cfd65
Move cli, check to minio from minio-io
2015-05-11 16:40:12 -07:00
Harshavardhana
f5fb701c5c
Merge pull request #591 from harshavardhana/pr_out_change_all_minio_io_path_to_minio
...
Change all minio-io path to minio
2015-05-11 16:31:10 -07:00
Harshavardhana
8d06504068
Change all minio-io path to minio
2015-05-11 16:23:10 -07:00
Harshavardhana
e39de4f603
Fix typo introduced in previous commit
2015-05-11 14:24:31 -07:00
Harshavardhana
a972e2c3d6
Merge pull request #590 from harshavardhana/pr_out_remove_signaturev2_support_bring_in_signaturev4_header_only_validation_for_now
2015-05-11 14:07:07 -07:00
Harshavardhana
f45635524f
Remove SignatureV2 support, bring in SignatureV4 header only validation for now
2015-05-11 13:38:40 -07:00
Harshavardhana
01c1bd6fc5
Merge pull request #589 from harshavardhana/pr_out_use_convenience_functions_for_url_values_over_looping_through_keys_fix_a_bug_in_listobjectparts_
2015-05-10 11:38:02 -07:00
Harshavardhana
b48b7f51ef
Use convenience functions for url.Values over looping through keys, fix a bug in listObjectParts()
2015-05-10 11:32:41 -07:00
Harshavardhana
62f20c0f79
Merge pull request #588 from harshavardhana/pr_out_handle_partnumbermarker_with_listobjectparts_now_and_other_fixes
2015-05-09 19:41:25 -07:00
Harshavardhana
14b5828505
Handle partNumberMarker with listObjectParts now and other fixes
2015-05-09 19:39:00 -07:00
Frederick F. Kautz IV
729447794a
Merge pull request #587 from fkautz/pr_out_limiting_upload_id_size
2015-05-09 18:31:09 -07:00
Frederick F. Kautz IV
cf559fa48e
Limiting upload id size
2015-05-09 18:28:50 -07:00
Frederick F. Kautz IV
55bda21881
Merge pull request #586 from fkautz/pr_out_adding_support_for_memory_driver_test
2015-05-09 18:28:45 -07:00
Frederick F. Kautz IV
61a9ea0f4b
Adding support for memory driver test
2015-05-09 18:15:27 -07:00
Frederick F. Kautz IV
cfe425e948
Merge pull request #585 from fkautz/pr_out_adding_tests_and_fixes_for_multipart_uploads_uncovered_from_tests
2015-05-09 17:44:25 -07:00
Frederick F. Kautz IV
058604aa21
Adding tests and fixes for multipart uploads uncovered from tests
2015-05-09 17:42:14 -07:00
Harshavardhana
0c54f99674
Merge pull request #584 from harshavardhana/pr_out_implement_abortmultipart
2015-05-09 16:12:17 -07:00
Harshavardhana
8860aa0c8c
Implement AbortMultipart
2015-05-09 16:09:50 -07:00
Harshavardhana
9d407f6ee1
Merge pull request #583 from harshavardhana/pr_out_add_listparts_support_wip_do_not_merge
2015-05-09 13:59:35 -07:00
Harshavardhana
18c7f620cc
Add listparts support
2015-05-09 13:55:55 -07:00
Harshavardhana
ef793dcf03
Merge pull request #582 from harshavardhana/pr_out_handle_two_different_styles_of_time_format_s3cmd_now_compatible
2015-05-08 16:33:34 -07:00
Harshavardhana
04e388d620
Handle two different styles of time format, s3cmd now compatible
2015-05-08 16:01:59 -07:00
Harshavardhana
aaf205b1fe
Merge pull request #581 from harshavardhana/pr_out_move_expiration_code_into_its_own_and_add_ticker_select_case
2015-05-08 13:00:16 -07:00
Harshavardhana
01b474fca3
Move expiration code into its own and add ticker select case
2015-05-08 12:37:49 -07:00
Harshavardhana
8a65b90783
Merge pull request #580 from harshavardhana/pr_out_verify_incoming_md5sum_for_invidual_parts_in_complete_multipart_request_and_some_simplification_of_the_code
2015-05-08 04:01:37 -07:00
Harshavardhana
23a545673d
Verify incoming md5sum for invidual parts in complete multipart request and some simplification of the code
...
Add two new functions for session cleanup
2015-05-08 03:59:05 -07:00
Harshavardhana
a16a10afa9
Merge pull request #579 from harshavardhana/pr_out_re_enable_bucket_deletion_this_time_with_uploadid_sitting_there
2015-05-08 02:31:40 -07:00
Harshavardhana
616241a2c1
Re-enable bucket deletion, this time with uploadID sitting there
2015-05-08 02:28:38 -07:00
Harshavardhana
2ea10c798b
Implement proper errors for Multipart
2015-05-08 02:02:51 -07:00
Harshavardhana
39e0875699
Merge pull request #578 from harshavardhana/pr_out_now_objectstorage_go_works_properly_with_multipart_upload
...
Now objectstorage-go works properly with Multipart upload.
2015-05-07 23:03:18 -07:00
Harshavardhana
82c3656f79
Reply back CompleteMultipartUploadResult properly with final ETag computed
...
- Now s3 libraries and also objectstorage-go work properly
2015-05-07 23:01:20 -07:00
Harshavardhana
5a372f4dd7
Merge pull request #577 from fkautz/pr_out_router_should_work_with_uploads_unescaped_now
2015-05-07 21:43:28 -07:00
Frederick F. Kautz IV
ae5c99d7bb
Router should work with ?uploads unescaped now
2015-05-07 21:37:34 -07:00
Frederick F. Kautz IV
9dc7b82b39
Merge pull request #576 from fkautz/pr_out_adding_multipart_support
2015-05-07 20:00:20 -07:00
Frederick F. Kautz IV
0266ec4bf6
Merge pull request #575 from fkautz/pr_out_fixing_panic_in_intelligent_attempted_to_delete_nonexistent_key
2015-05-07 19:55:56 -07:00
Frederick F. Kautz IV
f050d5e974
Adding multipart support
2015-05-07 19:55:30 -07:00
Frederick F. Kautz IV
ed71c28c37
Fixing panic in intelligent, attempted to delete nonexistent key
2015-05-07 19:12:49 -07:00
Frederick F. Kautz IV
a4b4e9c148
Merge pull request #574 from fkautz/pr_out_adding_feature_flags
2015-05-07 13:59:23 -07:00
Frederick F. Kautz IV
d0c4334834
Adding feature flags
2015-05-07 13:56:54 -07:00
Harshavardhana
f1dbdbd234
Merge pull request #573 from harshavardhana/pr_out_get_rid_of_sub_domain_code_and_some_other_formatting_changes
2015-05-07 00:23:14 -07:00
Harshavardhana
1df28f3060
Get rid of sub-domain code and some other formatting changes
2015-05-07 00:21:40 -07:00
Harshavardhana
8c5a83ddcf
Merge pull request #572 from harshavardhana/pr_out_add_marker_support_for_donut_also_add_a_testing_code
2015-05-05 22:20:18 -07:00
Harshavardhana
6d3d92694f
Add Marker support for donut, also add a testing code
2015-05-05 22:18:24 -07:00
Harshavardhana
142f9263e1
Merge pull request #571 from harshavardhana/pr_out_add_lexicographic_marker_nextmarker_support_for_recursive_listing_of_objects_wip_do_not_merge
2015-05-05 18:12:18 -07:00
Harshavardhana
75028c2ad1
Add lexicographic Marker/NextMarker support for recursive listing of objects.
...
Also update times when an object is accessed logic
2015-05-05 18:05:34 -07:00
Frederick F. Kautz IV
7a87f89604
Merge pull request #570 from fkautz/pr_out_adding_xmlns
2015-05-05 13:32:31 -07:00
Frederick F. Kautz IV
24af2222c9
Adding xmlns
2015-05-05 12:54:34 -07:00
Harshavardhana
9d484719ae
Merge pull request #568 from harshavardhana/pr_out_fix_a_potential_race_between_expireobjects_and_set_over_items_map
2015-05-04 22:38:59 -07:00
Harshavardhana
193a6606db
Fix a potential race between ExpireObjects() and Set() over items map
2015-05-04 22:36:47 -07:00
Harshavardhana
3fc9b4554f
Merge pull request #567 from harshavardhana/pr_out_convert_groupcache_lru_to_be_memory_size_based_rather_than_entry_based_calling_it_lru_memory
...
Convert groupcache lru to be memory size based rather than entry based calling it lru-memory
2015-05-04 22:04:26 -07:00
Harshavardhana
ed1259d6f0
Deprecate LRU use just map[string]interface{} and call it intelligent 'cache'
2015-05-04 22:01:56 -07:00
Harshavardhana
63edb1e9a0
Nullify list memory to nil as we remove the element
2015-05-04 17:22:08 -07:00
Harshavardhana
d63064b8af
Optimize memory usage in GetPartialObject() for memory driver
2015-05-04 15:26:56 -07:00
Harshavardhana
c8f31d97a8
Modify LRU further to add object expiration
2015-05-04 04:32:44 -07:00
Harshavardhana
670f997b07
Bring blockingWriter from client and use it here
2015-05-04 03:55:43 -07:00
Harshavardhana
d0df548eb5
Use new LRU inside memory driver
2015-05-04 03:55:43 -07:00
Harshavardhana
f7caef2d26
Change CreateObject() to take size argument from content-length
2015-05-04 03:55:43 -07:00
Harshavardhana
c342ce1588
New LRU based on GroupCache which keeps track of memory rather than entries
2015-05-04 03:55:19 -07:00
Frederick F. Kautz IV
66e0fe23be
Merge pull request #566 from fkautz/pr_out_deleting_key_from_lastaccessedobjects_on_eviction
2015-05-01 22:21:11 -07:00
Frederick F. Kautz IV
3cf045fd1b
Deleting key from lastAccessedObjects on eviction
2015-05-01 22:01:07 -07:00
Frederick F. Kautz IV
ad2037f446
Merge pull request #565 from fkautz/pr_out_removing_groupcache_lru_in_favor_of_imported_local_version
2015-05-01 21:02:17 -07:00
Frederick F. Kautz IV
c80bbc86d5
Removing groupcache lru in favor of imported local version
2015-05-01 20:58:36 -07:00
Harshavardhana
600a75a786
Merge pull request #564 from harshavardhana/pr_out_rename_isvalidobject_to_isvalidobjectname_
2015-05-01 20:33:05 -07:00
Harshavardhana
9ec8925f1c
Rename IsValidObject() to IsValidObjectName()
2015-05-01 20:31:26 -07:00
Frederick F. Kautz IV
944d88c8ae
Merge pull request #563 from fkautz/pr_out_fixing_documentation_issues_with_expire
2015-05-01 18:22:03 -07:00
Frederick F. Kautz IV
ac595b1f2e
Fixing documentation issues with expire
2015-05-01 18:20:22 -07:00
Frederick F. Kautz IV
faaa647b0d
Merge pull request #562 from fkautz/pr_out_reimagining_memory_settings_cli_input
2015-05-01 18:17:02 -07:00
Frederick F. Kautz IV
33ecba8e49
Reimagining memory settings cli input
2015-05-01 18:12:58 -07:00
Frederick F. Kautz IV
b624072dbb
Merge pull request #561 from fkautz/pr_out_attempt_to_fix_memory_lock_on_get_preventing_write
2015-05-01 17:52:24 -07:00
Frederick F. Kautz IV
d158284cd5
Merge pull request #560 from fkautz/pr_out_removing_log_message_from_test
2015-05-01 17:47:43 -07:00
Frederick F. Kautz IV
25c268ac88
Attempt to fix memory lock on Get preventing write
2015-05-01 17:45:40 -07:00
Frederick F. Kautz IV
85f1b46492
Removing log message from test
2015-05-01 17:32:18 -07:00
Frederick F. Kautz IV
8e093eb6f6
Merge pull request #559 from fkautz/pr_out_fix_docker_support
2015-05-01 15:26:51 -07:00
Frederick F. Kautz IV
7cf86e6114
Fix docker support
2015-05-01 15:23:39 -07:00
Harshavardhana
1f3dd2374c
Merge pull request #558 from harshavardhana/pr_out_even_mux_vars_don_t_help_handle_it_by_looking_at_req_url_path
2015-05-01 12:43:51 -07:00
Harshavardhana
2c7b30e5e1
Even mux vars don't help, handle it by looking at req.URL.Path
2015-05-01 12:41:17 -07:00
Frederick F. Kautz IV
b69327a617
Merge pull request #556 from fkautz/pr_out_exposing_expiration_for_memory_driver
2015-04-30 21:28:12 -07:00
Frederick F. Kautz IV
b86b26e7e5
Exposing expiration for memory driver
2015-04-30 21:20:54 -07:00
Harshavardhana
c01fb92eca
Merge pull request #557 from harshavardhana/pr_out_make_builddate_universal
2015-04-30 21:19:04 -07:00
Harshavardhana
bbace9ac2e
Make buildDate universal
2015-04-30 21:16:49 -07:00
Harshavardhana
bbc32d6ac0
Disable domain based routing for now
2015-04-30 21:15:53 -07:00
Harshavardhana
f3078d047c
Get bucketName from mux.Vars instead of heuristic code, resolves subdomain issues
2015-04-30 21:11:32 -07:00
Frederick F. Kautz IV
e911cc332e
Merge pull request #554 from fkautz/pr_out_disable_eviction_on_0_expiration_time
2015-04-30 19:40:46 -07:00
Harshavardhana
9167312170
Merge pull request #555 from harshavardhana/pr_out_use_utc_everywhere
2015-04-30 19:40:37 -07:00
Harshavardhana
8e3d48bf35
Use UTC() everywhere
...
ref - https://github.com/golang/go/issues/1988 , no standard function to set timeZone
2015-04-30 19:38:32 -07:00
Frederick F. Kautz IV
94e24f1a05
Disable eviction on 0 expiration time
2015-04-30 19:34:35 -07:00
Frederick F. Kautz IV
b6d96335e7
Merge pull request #553 from fkautz/pr_out_setting_number_of_processors_to_use_to_max_available
2015-04-30 19:25:31 -07:00
Frederick F. Kautz IV
f59535f052
Setting number of processors to use to max available
2015-04-30 19:23:07 -07:00
Harshavardhana
72395798c8
Merge pull request #552 from harshavardhana/pr_out_minio_config_directory_should_be_0700_not_os_modedir
2015-04-30 17:40:15 -07:00
Harshavardhana
2de81c8a3c
Minio config directory should be 0700 not os.ModeDir
2015-04-30 15:42:49 -07:00
Frederick F. Kautz IV
e76337d620
Merge pull request #551 from fkautz/pr_out_wrapping_functions_that_require_gc_so_that_references_are_released_before_running
2015-04-30 13:11:26 -07:00
Frederick F. Kautz IV
8e2f7f9493
Wrapping functions that require gc so that references are released before running
2015-04-30 12:38:12 -07:00
Harshavardhana
4ea186729a
Merge pull request #550 from harshavardhana/pr_out_changes_to_createobject_now_returns_back_md5_along_with_any_error
2015-04-30 03:43:14 -07:00
Harshavardhana
d815e6adfd
Changes to CreateObject() now returns back md5 along with any error
...
- This change is necessary to avoid the racy calls to GetObjectMetadata()
- This change is also necessary since one has to reply back md5sum with
PUT object response header
2015-04-30 03:38:14 -07:00
Harshavardhana
13cae94191
Merge pull request #545 from harshavardhana/pr_out_conn_limit_changes
2015-04-30 01:26:24 -07:00
Harshavardhana
fd2203b1b7
Some more improvements to connection limit
2015-04-30 01:23:44 -07:00
Harshavardhana
58491d22fc
Merge pull request #548 from harshavardhana/pr_out_memory_now_evicts_bucket_if_no_more_objects_in_memory_struct
2015-04-30 00:33:21 -07:00
Harshavardhana
f8a16dd22b
Memory now evicts bucket if no more objects in memory struct
...
- To avoid race in expiration while accessing memory driver structs with
in two competing write locks. Use lru Len() instead to know exact length
to schedule for eviction.
- squash both bucket and object structs, instead use a separate map to keep
mutable info of lastAccessTime which can be independently used inside the
expiration routine.
2015-04-30 00:31:52 -07:00
Frederick F. Kautz IV
63ba97fd8e
Merge pull request #547 from fkautz/pr_out_removing_entity_too_large_errors_from_memory_driver
2015-04-30 00:12:01 -07:00
Frederick F. Kautz IV
ed0a9a9e06
Removing entity too large errors from memory driver
2015-04-30 00:09:54 -07:00
Frederick F. Kautz IV
446a137904
Merge pull request #546 from fkautz/pr_out_adding_debug_statements_for_testing
2015-04-29 21:15:12 -07:00
Frederick F. Kautz IV
aff2b3b06a
Adding debug statements for testing
2015-04-29 21:13:32 -07:00
Frederick F. Kautz IV
a521309b78
Merge pull request #542 from fkautz/pr_out_make_minio_work_with_curl_and_browsers_again
2015-04-29 20:21:38 -07:00
Frederick F. Kautz IV
82a16b8848
Make minio work with curl and browsers again
2015-04-29 20:17:26 -07:00
Frederick F. Kautz IV
73de05feb6
Merge pull request #544 from fkautz/pr_out_fixing_race_condition
2015-04-29 18:57:05 -07:00
Frederick F. Kautz IV
3a1386165f
Fixing race condition
2015-04-29 18:54:59 -07:00
Harshavardhana
710e732cf0
Merge pull request #543 from harshavardhana/pr_out_fix_out_of_memory_issue
2015-04-29 18:50:19 -07:00
Harshavardhana
03bca9a001
Fix out of memory issue
2015-04-29 18:33:19 -07:00
Harshavardhana
50d871c2db
Merge pull request #540 from harshavardhana/pr_out_remove_connection_limit_handle_throttling_outisde_in_iptables
2015-04-29 17:32:01 -07:00
Harshavardhana
25d4c0f6fa
A small logical change in messaging and logging
2015-04-29 17:30:54 -07:00
Harshavardhana
8be3b92ab7
Merge pull request #541 from fkautz/pr_out_adding_error_message_on_connection_limit_met
2015-04-29 17:24:04 -07:00
Frederick F. Kautz IV
f596c744ed
Adding error message on connection limit met
2015-04-29 17:21:23 -07:00
Harshavardhana
8d1d309630
Merge pull request #539 from harshavardhana/pr_out_return_resources_as_its_available_not_a_new_value
2015-04-29 16:50:07 -07:00
Harshavardhana
a89c4a57d3
Return resources as its available, not a new value
2015-04-29 16:47:14 -07:00
Harshavardhana
edf838df74
Merge pull request #538 from harshavardhana/pr_out_bump_up_gocyclo_to_18
2015-04-29 15:34:56 -07:00
Harshavardhana
a60d279817
bump up gocyclo to 18
2015-04-29 15:34:00 -07:00
Harshavardhana
251c99176a
Merge pull request #537 from harshavardhana/pr_out_add_proper_content_length_for_error_and_success_responses
2015-04-29 15:32:12 -07:00
Frederick F. Kautz IV
c718910300
Merge pull request #536 from fkautz/pr_out_expirations_are_now_based_on_last_accessed_instead_of_time_created
2015-04-29 15:30:54 -07:00
Frederick F. Kautz IV
0da04c6f17
Expirations are now based on last accessed instead of time created
2015-04-29 15:28:45 -07:00
Harshavardhana
92e4301414
Add proper content-length for error and success responses
...
- All compliance issues with S3 API for Put,Get,List (Bucket,Object) respectively
- Encodes and returns back proper HTTP headers
2015-04-29 15:28:28 -07:00
Frederick F. Kautz IV
c8db3e1c3b
Merge pull request #535 from fkautz/pr_out_adding_lru
2015-04-29 15:26:01 -07:00
Frederick F. Kautz IV
5fc5f79ab7
Adding lru
2015-04-29 15:08:26 -07:00
Frederick F. Kautz IV
b6daf99dcd
Merge pull request #533 from fkautz/pr_out_simplifying_memory_sleep_logic
2015-04-29 13:55:24 -07:00
Frederick F. Kautz IV
8fcb697844
Simplifying memory sleep logic
2015-04-29 13:53:09 -07:00
Frederick F. Kautz IV
362305d898
Merge pull request #534 from fkautz/pr_out_experiment_running_gofmt_on_godeps
2015-04-29 13:33:08 -07:00
Frederick F. Kautz IV
9001d87d20
Experiment running gofmt on Godeps
2015-04-29 13:30:14 -07:00
Frederick F. Kautz IV
8689ec9a54
Merge pull request #532 from fkautz/pr_out_adding_expiration
2015-04-29 13:11:35 -07:00
Frederick F. Kautz IV
5cc9418ca7
Adding expiration
2015-04-29 13:02:21 -07:00
Frederick F. Kautz IV
2ab8988a41
Merge pull request #531 from fkautz/pr_out_api_logging_is_now_in_its_own_package
2015-04-29 12:40:27 -07:00
Frederick F. Kautz IV
a608524ebf
api logging is now in its own package
2015-04-29 12:24:53 -07:00
Harshavardhana
9a969ea59b
Merge pull request #530 from harshavardhana/pr_out_collate_success_response_into_writesuccessresponse_add_docs
2015-04-29 10:58:37 -07:00
Harshavardhana
d1d5f7a78d
Collate success response into writeSuccessResponse(), add docs
2015-04-29 10:52:04 -07:00
Harshavardhana
45a0fb21fa
Merge pull request #529 from harshavardhana/pr_out_maxobjectsize_and_minobjectsize_limitation_added_at_putobjecthandler_
2015-04-29 02:23:34 -07:00
Harshavardhana
b18bb230da
maxObjectSize and minObjectSize limitation added at putObjectHandler()
...
Put() replies back with
- EntityTooLarge with > 5GB in single PUT operation
- EntityTooSmall with < 1B in single PUT operation
- IncompleteBody with ho Content-Length found in HTTP request header
2015-04-29 02:19:53 -07:00
Anand Babu (AB) Periasamy
c14496f928
Merge pull request #528 from abperiasamy/clean-clean
...
cleaning make:clean
2015-04-28 21:29:37 -07:00
Harshavardhana
b01fc9d125
Merge pull request #527 from harshavardhana/pr_out_add_minio_cli_changes_and_add_customapphelptemplate
2015-04-28 21:26:26 -07:00
Anand Babu (AB) Periasamy
b8eb34aded
cleaning make:clean
2015-04-28 21:24:59 -07:00
Harshavardhana
76701187e3
Add minio cli changes and add CustomAppHelpTemplate
2015-04-28 21:24:56 -07:00
Anand Babu (AB) Periasamy
a3d03184ca
Merge pull request #526 from abperiasamy/messages
...
print using console functions
2015-04-28 21:15:32 -07:00
Anand Babu (AB) Periasamy
43e9cb203d
print using console functions
2015-04-28 21:12:23 -07:00
Frederick F. Kautz IV
b0a4d07736
Merge pull request #525 from fkautz/pr_out_adding_initial_logging_framework_to_api_router
2015-04-28 20:44:39 -07:00
Frederick F. Kautz IV
172cd18455
Adding initial logging framework to api_router
2015-04-28 20:37:08 -07:00
Frederick F. Kautz IV
e7f2fc9e4e
Merge pull request #524 from fkautz/pr_out_adding_json_logging_to_stderr
2015-04-28 18:15:18 -07:00
Frederick F. Kautz IV
88cca647ea
Adding json logging to stderr
2015-04-28 18:11:53 -07:00
Frederick F. Kautz IV
83cd299b23
Merge pull request #523 from fkautz/pr_out_setting_gimme_to_use_go1_4_2
2015-04-28 12:36:16 -07:00
Frederick F. Kautz IV
926b1e81e7
Setting gimme to use go1.4.2
2015-04-28 12:35:49 -07:00
Frederick F. Kautz IV
b1b6fd5da9
Merge pull request #521 from fkautz/pr_out_removing_quotas_will_handle_with_iptables
2015-04-28 12:29:20 -07:00
Frederick F. Kautz IV
b8ea1c234b
Setting gimme to use go1.4.2
2015-04-28 12:22:12 -07:00
Frederick F. Kautz IV
de091068e8
Removing quotas, will handle with iptables
2015-04-28 12:14:34 -07:00
Anand Babu (AB) Periasamy
6c0d95a98c
Merge pull request #522 from abperiasamy/console
...
simple console
2015-04-27 22:03:25 -07:00
Anand Babu (AB) Periasamy
5127dfb4fc
simple console
2015-04-27 22:01:07 -07:00
Frederick F. Kautz IV
f22390ccef
Merge pull request #520 from fkautz/pr_out_updating_godep_deps
2015-04-27 21:11:42 -07:00
Frederick F. Kautz IV
adaa0e1821
Merge pull request #519 from fkautz/pr_out_removing_short_flags
2015-04-27 21:11:23 -07:00
Frederick F. Kautz IV
0cca734bc9
Updating godep deps
2015-04-27 21:02:36 -07:00
Frederick F. Kautz IV
fc59401ac5
Removing short flags
2015-04-27 20:20:03 -07:00
Frederick F. Kautz IV
2fcaa63db0
Revert "remove single char args"
...
This reverts commit 1e307d2888 .
2015-04-27 20:18:43 -07:00
Frederick F. Kautz IV
2c3329a519
Merge pull request #518 from fkautz/pr_out_critical_sections_are_now_protected_by_lock
2015-04-27 20:01:51 -07:00
Frederick F. Kautz IV
f2257a643c
Critical sections are now protected by lock
2015-04-27 19:59:58 -07:00
Harshavardhana
b85595addf
Merge pull request #516 from harshavardhana/pr_out_add_some_logging_with_debug
2015-04-27 19:44:34 -07:00
Anand Babu (AB) Periasamy
e9b146c1d8
Merge pull request #517 from abperiasamy/janitor-is-back
...
remove single char args
2015-04-27 19:34:37 -07:00
Anand Babu (AB) Periasamy
1e307d2888
remove single char args
2015-04-27 19:30:44 -07:00
Harshavardhana
605edbc307
Add some logging with debug
2015-04-27 19:26:24 -07:00
Harshavardhana
7bdcf4bbf3
Merge pull request #515 from fkautz/pr_out_fix_multiple_response_writeheader_calls
...
Fix multiple response.WriteHeader calls
2015-04-27 19:21:47 -07:00
Frederick F. Kautz IV
db5870be28
Fix multiple response.WriteHeader calls
2015-04-27 18:30:21 -07:00
Harshavardhana
7e72cefd74
Merge pull request #514 from harshavardhana/pr_out_fix_build_date_typo
2015-04-27 16:08:23 -07:00
Harshavardhana
740e12e81d
Fix build date typo
2015-04-27 16:08:11 -07:00
Frederick F. Kautz IV
fc2537eef9
Merge pull request #513 from fkautz/pr_out_adding_software_fallback
2015-04-27 14:41:41 -07:00
Frederick F. Kautz IV
289c859675
Adding software fallback
2015-04-27 13:55:40 -07:00
Harshavardhana
943192f684
Merge pull request #512 from harshavardhana/pr_out_add_build_date_using_ldflags
2015-04-27 13:11:12 -07:00
Harshavardhana
096a994c7b
Add build date using ldflags
2015-04-27 13:08:51 -07:00
Harshavardhana
66747399fd
Merge pull request #511 from harshavardhana/pr_out_verify_if_request_date_is_5minutes_late_reject_such_a_request_as_it_could_be_a_possible_replay_attack
2015-04-27 03:58:17 -07:00
Harshavardhana
ade803a923
Verify if request date is 5minutes late, reject such a request as
...
it could be a possible replay attack.
This commit also fixes #505 , by returning MethodNotAllowed instead of NotImplemented
2015-04-27 03:55:10 -07:00
Harshavardhana
1531802b73
Merge pull request #510 from harshavardhana/pr_out_now_client_requests_for_acl_changes_are_honored_through_putbucketacl_api
2015-04-27 03:06:12 -07:00
Harshavardhana
9232ce3b4e
Now client requests for ACL changes are honored through PutBucketACL API
2015-04-27 03:04:29 -07:00
Harshavardhana
1b411f9e86
Merge pull request #509 from harshavardhana/pr_out_now_donut_supports_bucket_acl_s_bringing_in_this_support_for_dl_minio_io
2015-04-27 02:11:49 -07:00
Harshavardhana
c99d96dbc2
Now donut supports bucket ACL's, bringing in this support for dl.minio.io
2015-04-27 02:09:32 -07:00
Harshavardhana
f5a51ae1bb
Merge pull request #507 from harshavardhana/pr_out_limit_memory_at_100_buckets_and_return_error_appropriately
2015-04-26 22:17:03 -07:00
Harshavardhana
8beca83090
Limit memory mode at 100 buckets and return error appropriately
...
~~~
mc: <ERROR> Failed to create bucket for URL [http://localhost:9000/newbucket-101 ]. Reason: [You have attempted to create more buckets than allowed.].
~~~
Response from server
2015-04-26 22:15:39 -07:00
Harshavardhana
f20515b4ed
Merge pull request #506 from harshavardhana/pr_out_enhance_error_responses_for_request_limit_and_bring_some_code_from_api_errors
2015-04-26 22:12:21 -07:00
Harshavardhana
2d96d5ad57
Enhance error responses for request limit and bring some code from api errors
...
~~~
mc: <ERROR> Failed to create bucket for URL [http://localhost:9000/newbucket-101 ]. Reason: [Reduce your request rate.].
~~~
Client sees proper errors now.
2015-04-26 22:10:59 -07:00
Frederick F. Kautz IV
8a57006b36
Merge pull request #504 from fkautz/pr_out_making_lock_much_less_aggressive_fixes_multiple_uploads_blocking
2015-04-26 20:11:50 -07:00
Frederick F. Kautz IV
e2807a167f
Merge pull request #503 from fkautz/pr_out_bandwidth_quota_now_supports_100_continue
2015-04-26 20:09:13 -07:00
Frederick F. Kautz IV
a974e4d7c7
Making lock much less aggressive, fixes multiple uploads blocking
2015-04-26 20:08:37 -07:00
Frederick F. Kautz IV
258bf55e31
Bandwidth quota now supports 100-Continue
2015-04-26 20:07:05 -07:00
Frederick F. Kautz IV
9431a54a27
finish me
2015-04-26 19:25:48 -07:00
Frederick F. Kautz IV
d11803b125
Merge pull request #502 from fkautz/pr_out_revert_exporting_writeerrorresponse
2015-04-26 17:04:28 -07:00
Frederick F. Kautz IV
202fc26719
Revert "Exporting WriteErrorResponse"
...
This reverts commit e05d46a0b5 .
2015-04-26 17:02:49 -07:00
Frederick F. Kautz IV
c29677b245
Merge pull request #501 from fkautz/pr_out_exporting_writeerrorresponse
2015-04-26 16:30:45 -07:00
Frederick F. Kautz IV
e05d46a0b5
Exporting WriteErrorResponse
2015-04-26 16:28:54 -07:00
Frederick F. Kautz IV
3fb4f2894d
Merge pull request #500 from fkautz/pr_out_adding_connection_limits
2015-04-26 16:28:50 -07:00
Frederick F. Kautz IV
64f7d04cd1
Adding connection limits
2015-04-26 16:14:51 -07:00
Frederick F. Kautz IV
e68e0c28f7
Merge pull request #499 from fkautz/pr_out_bandwidth_quotas_now_work_on_data_returned_from_server
2015-04-26 15:51:38 -07:00
Frederick F. Kautz IV
7867ee89fa
Bandwidth quotas now work on data returned from server
2015-04-26 15:49:55 -07:00
Harshavardhana
5fe173edf1
Merge pull request #498 from harshavardhana/pr_out_bring_in_changes_from_minio_io_cli_properly_
...
Bring in changes from minio-io/cli properly :-)
2015-04-26 15:43:29 -07:00
Harshavardhana
f7d2711d60
Merge pull request #494 from abperiasamy/md5sum-path-bug
...
Version is now based on MD5SUM of its binary
2015-04-26 15:42:48 -07:00
Harshavardhana
6332a8d108
Bring in changes from minio-io/cli properly :-)
2015-04-26 15:41:32 -07:00
Harshavardhana
9bc0f61191
Merge pull request #497 from minio-io/revert-496-pr_out_bring_in_changes_from_minio_io_cli
...
Revert "Bring in changes from minio-io/cli"
2015-04-26 15:39:47 -07:00
Harshavardhana
d7a8f5a5a4
Revert "Bring in changes from minio-io/cli"
2015-04-26 15:39:26 -07:00
Harshavardhana
40c26cf9a3
Merge pull request #496 from harshavardhana/pr_out_bring_in_changes_from_minio_io_cli
2015-04-26 15:36:24 -07:00
root
c2d8053787
Bring in changes from minio-io/cli
2015-04-26 15:34:13 -07:00
Frederick F. Kautz IV
a3af3514ca
Merge pull request #495 from fkautz/pr_out_quotas_are_more_accurate_occur_on_read_
2015-04-26 15:13:15 -07:00
Frederick F. Kautz IV
e2403cbc56
Quotas are more accurate, occur on Read()
2015-04-26 15:07:57 -07:00
Anand Babu (AB) Periasamy
0d97a98594
Version is now based on MD5SUM of its binary
2015-04-26 02:58:10 -07:00
Frederick F. Kautz IV
f94d9db4f7
Merge pull request #493 from fkautz/pr_out_switching_back_to_bigendian
2015-04-25 18:18:02 -07:00
Frederick F. Kautz IV
bbb493080f
Switching back to BigEndian
2015-04-25 18:16:21 -07:00
Frederick F. Kautz IV
6ac5fb78d8
Merge pull request #492 from fkautz/pr_out_adding_quotas_based_upon_type
2015-04-25 18:08:36 -07:00
Frederick F. Kautz IV
3a48f9fe4d
Adding quotas based upon type
2015-04-25 18:07:02 -07:00
Harshavardhana
5a2fb9741a
Merge pull request #491 from harshavardhana/pr_out_use_standard_functions_to_achieve_byteorder_converstion
2015-04-25 17:08:49 -07:00
Harshavardhana
b75f46aaa0
Use standard functions to achieve byteorder converstion
2015-04-25 13:18:15 -07:00
Anand Babu (AB) Periasamy
2eb1b8ad76
Merge pull request #490 from abperiasamy/hashbin
...
Version is now based on MD5SUM of its binary
2015-04-24 21:55:29 -07:00
Anand Babu (AB) Periasamy
b010fd0ff3
Version is now based on MD5SUM of its binary
2015-04-24 21:51:52 -07:00
Frederick F. Kautz IV
5b0591ffad
Merge pull request #489 from fkautz/pr_out_quota_handlers_now_log_against_ip_properly_against_a_duration
2015-04-24 20:53:26 -07:00
Frederick F. Kautz IV
e2475925ea
quota handlers now log against ip properly against a duration
2015-04-24 20:49:33 -07:00
Harshavardhana
b3a9d8b058
Merge pull request #483 from harshavardhana/pr_out_memory_now_handles_submitting_large_files_fixes_482
2015-04-24 15:25:38 -07:00
Harshavardhana
c5d8ca245d
Memory now handles submitting large files - fixes #482
2015-04-24 15:22:22 -07:00
Frederick F. Kautz IV
4b45b82711
Merge pull request #481 from fkautz/pr_out_add_initial_framework_for_quota
2015-04-24 00:19:25 -07:00
Frederick F. Kautz IV
fda50d52c8
Add initial framework for quota
2015-04-24 00:16:29 -07:00
Frederick F. Kautz IV
b3f51d53c4
Merge pull request #480 from fkautz/pr_out_moving_factories_and_start_minio_method_to_server_package
2015-04-23 16:15:55 -07:00
Frederick F. Kautz IV
5a607a6762
Moving factories and start minio method to server package
2015-04-23 16:13:50 -07:00
Frederick F. Kautz IV
c75b9d58f5
Merge pull request #479 from fkautz/pr_out_simplifying_server_config_handling
2015-04-23 16:13:39 -07:00
Frederick F. Kautz IV
5c515d3138
Simplifying server config handling
2015-04-23 12:44:22 -07:00
Harshavardhana
a216bf836a
Merge pull request #478 from harshavardhana/pr_out_donut_mode_will_now_take_multiple_paths_as_argument_updated_docs_as_well
2015-04-23 11:38:31 -07:00
Harshavardhana
5ae8d34f76
Donut mode will now take multiple paths as argument, updated docs as well
2015-04-23 11:36:14 -07:00
Harshavardhana
7c17a60d1f
Merge pull request #477 from harshavardhana/pr_out_update_command_docs_further
2015-04-23 03:37:05 -07:00
Harshavardhana
c7be1a1355
Update command docs further
2015-04-23 03:34:20 -07:00
Harshavardhana
d447ca9af1
Merge pull request #476 from harshavardhana/pr_out_disable_verifying_acl_until_we_get_webcli_up_and_running
2015-04-23 03:24:55 -07:00
Harshavardhana
a5729e8c02
Disable verifying ACL until we get webcli up and running
2015-04-23 03:21:08 -07:00
Harshavardhana
5ddd624a95
Merge pull request #475 from harshavardhana/pr_out_add_proper_help_and_several_other_cleanup
2015-04-23 02:47:49 -07:00
Harshavardhana
bcc4449de7
Add proper help and several other cleanup
2015-04-23 02:42:50 -07:00
Harshavardhana
b153834d4d
Merge pull request #474 from harshavardhana/pr_out_finishing_all_the_test_case_support_for_acl_and_other_fixes
2015-04-23 01:27:26 -07:00
Harshavardhana
e95604ff86
Finishing all the test case support for ACL and other fixes
2015-04-23 01:24:46 -07:00
Frederick F. Kautz IV
de710962c0
Merge pull request #473 from fkautz/pr_out_refactoring_cli_to_take_commands
2015-04-23 01:10:18 -07:00
Frederick F. Kautz IV
0f8626ab1a
Refactoring cli to take commands
2015-04-23 01:05:44 -07:00
Harshavardhana
4ed03881ff
Merge pull request #472 from harshavardhana/pr_out_further_fixes_for_acl_support_currently_code_is_disabled_in_all_the_handlers
...
Further fixes for ACL support, currently code is disabled in all the handlers
2015-04-22 19:35:07 -07:00
Harshavardhana
848c4ee31c
Further fixes for ACL support, currently code is disabled in all the handlers
...
Disabled because due to lack of testing support. Once we get that in we can
uncomment them back.
2015-04-22 19:31:45 -07:00
Harshavardhana
1c0ff2c758
ACL driver/storage layer support
2015-04-22 19:31:45 -07:00
Harshavardhana
c8713fd650
Bring in full fledged acl support
2015-04-22 18:26:35 -07:00
Harshavardhana
2c1455af1b
Implement x-amz-acl tests
2015-04-22 18:26:35 -07:00
Harshavardhana
107e077ec0
Implement x-amz-acl handling
2015-04-22 18:26:35 -07:00
Frederick F. Kautz IV
28785421cd
Merge pull request #471 from fkautz/pr_out_memory_driver_now_limits_by_size
2015-04-22 18:26:28 -07:00
Frederick F. Kautz IV
c56f7380d6
Memory driver now limits by size
2015-04-22 17:50:26 -07:00
Harshavardhana
276c02af62
Merge pull request #470 from harshavardhana/pr_out_remove_bucketpolicy_handlers_and_all_references
2015-04-22 14:49:36 -07:00
Harshavardhana
b121c8588f
Remove bucketpolicy handlers and all its references
2015-04-22 14:46:51 -07:00
Harshavardhana
ed67a1269d
Merge pull request #469 from harshavardhana/pr_out_file_driver_to_take_the_high_road
2015-04-16 02:21:44 -07:00
Harshavardhana
3f33643d39
File driver to take the high road
2015-04-16 02:16:39 -07:00
Frederick F. Kautz IV
7d01300d82
Merge pull request #468 from fkautz/pr_out_adding_lru_to_memory_driver_not_wired_to_command_line_opts
2015-04-15 12:58:30 -07:00
Frederick F. Kautz IV
59a81a73c6
Adding lru to memory driver, not wired to command line opts
2015-04-15 12:49:41 -07:00
Frederick F. Kautz IV
b464ddbf31
Merge pull request #467 from fkautz/pr_out_better_locking_in_memory_driver
2015-04-15 12:13:38 -07:00
Frederick F. Kautz IV
0cab3fd151
Better locking in memory driver
2015-04-15 11:48:56 -07:00
Frederick F. Kautz IV
e9930c49db
Merge pull request #466 from fkautz/pr_out_adding_slack_notifications
2015-04-12 11:48:11 -07:00
Frederick F. Kautz IV
a5b0547c1a
Adding slack notifications
2015-04-12 11:45:43 -07:00
Harshavardhana
c566b8dabd
Update README.md
2015-04-11 18:12:19 -07:00
Harshavardhana
9dd97b823d
Merge pull request #465 from harshavardhana/pr_out_adding_travis_and_godoc_badge
2015-04-11 18:10:24 -07:00
Harshavardhana
f6964b5753
Adding travis and godoc badge
2015-04-11 18:10:02 -07:00
Frederick F. Kautz IV
77060fec44
Merge pull request #463 from fkautz/pr_out_adding_travis_yml_to_minio
...
Adding travis.yml to minio
2015-04-11 17:52:06 -07:00
Frederick F. Kautz IV
bcbf1a2390
Adding travis.yml to minio
2015-04-11 17:49:50 -07:00
Harshavardhana
87e4755282
Merge pull request #464 from harshavardhana/pr_out_fix_build_for_travis
2015-04-11 17:31:37 -07:00
Harshavardhana
97d1f03a87
Fix build for travis
2015-04-11 17:31:01 -07:00
Harshavardhana
0128481f45
Merge pull request #462 from harshavardhana/pr_out_remove_docs_minimal_docs_going_to_be_at_minio_io_instead_and_rest_amazon_docs
2015-04-10 21:22:34 -07:00
Harshavardhana
c63bdf79fa
Remove docs, minimal docs going to be at minio.io instead and rest amazon docs
2015-04-10 21:21:59 -07:00
Harshavardhana
74506bb48b
Merge pull request #461 from harshavardhana/pr_out_fix_golint_complaint
2015-04-08 23:43:43 -07:00
Harshavardhana
148935392d
Fix golint complaint
2015-04-08 23:42:59 -07:00
Frederick F. Kautz IV
b2d54f323b
Merge pull request #460 from fkautz/pr_out_removing_unneeded_files
2015-04-08 20:58:53 -07:00
Frederick F. Kautz IV
26d8deee40
Removing unneeded files
2015-04-08 20:58:42 -07:00
Frederick F. Kautz IV
1d4bc0c34e
Merge pull request #459 from fkautz/pr_out_adding_comments_to_iodine
2015-04-08 19:44:52 -07:00
Frederick F. Kautz IV
7212b3406a
Adding comments to iodine
2015-04-08 19:44:44 -07:00
Frederick F. Kautz IV
0e89facf7d
Merge pull request #458 from fkautz/pr_out_adding_iodine_error_conversion
2015-04-08 19:32:48 -07:00
Frederick F. Kautz IV
c12d5ea8a9
Adding iodine error conversion
2015-04-08 19:32:37 -07:00
Harshavardhana
e2c54a0904
Merge pull request #457 from harshavardhana/pr_out_iodine_is_part_of_minio_now
2015-04-08 18:14:48 -07:00
Harshavardhana
38155e86ac
iodine is part of minio now
2015-04-08 18:05:48 -07:00
Harshavardhana
879e428596
Merge pull request #456 from harshavardhana/pr_out_add_everything_back_into_one_project_hood_still_missing_iodine
2015-04-08 17:56:14 -07:00
Harshavardhana
7fd1cc073c
Add everything back into one project hood, still missing iodine
2015-04-08 17:47:56 -07:00
Frederick F. Kautz IV
60b30bdd0a
Merge pull request #455 from fkautz/pr_out_moving_object_storage_drivers_back_to_minio
2015-04-08 16:28:30 -07:00
Frederick F. Kautz IV
6241c9880b
Moving object storage drivers back to minio
2015-04-08 16:28:14 -07:00
Harshavardhana
d3caf2b733
Merge pull request #454 from harshavardhana/pr_out_update_minio_io_donut_minio_io_objectdriver
2015-04-07 16:20:30 -07:00
Harshavardhana
ec39155089
Update minio-io/donut, minio-io/objectdriver
2015-04-07 16:14:44 -07:00
Harshavardhana
34d06101cc
Merge pull request #453 from harshavardhana/pr_out_implement_headbuckethandler_and_its_related_tests
2015-04-07 01:17:24 -07:00
Harshavardhana
2a21b7d639
Implement headBucketHandler() and its related tests
...
Remove internal testify/mock and use upstream - update all godeps
2015-04-07 01:16:32 -07:00
Harshavardhana
2ddde59b46
Merge pull request #452 from harshavardhana/pr_out_merge_new_changes_from_minio_io_donut_minio_io_objectdriver
2015-04-07 00:01:32 -07:00
Harshavardhana
b63c7fd84f
Merge new changes from minio-io/donut; minio-io/objectdriver
2015-04-07 00:01:19 -07:00
Harshavardhana
35b8bb368d
Merge pull request #451 from harshavardhana/pr_out_pull_in_new_changes_from_minio_io_objectdriver_and_minio_io_donut
2015-04-06 22:29:23 -07:00
Harshavardhana
649f1104dd
Pull in new changes from minio-io/objectdriver and minio-io/donut
2015-04-06 22:29:10 -07:00
Harshavardhana
81e2dafe36
Merge pull request #450 from harshavardhana/pr_out_update_to_new_changes_at_minio_io_objectdriver_and_minio_io_donut
2015-04-06 21:45:30 -07:00
Harshavardhana
b93b6785c2
Update to new changes at minio-io/objectdriver and minio-io/donut
...
New ObjectStorage interface is now fully written and working as
expected.
2015-04-06 21:44:32 -07:00
Frederick F. Kautz IV
81e4a3332c
Merge pull request #449 from fkautz/pr_out_updating_to_use_objectdriver
2015-04-06 18:00:44 -07:00
Frederick F. Kautz IV
ea961ef16d
Updating to use objectdriver
2015-04-06 18:00:18 -07:00
Frederick F. Kautz IV
1ebc692ea2
Merge pull request #447 from fkautz/pr_out_adding_debug_logger_to_log_package
2015-04-06 12:15:39 -07:00
Frederick F. Kautz IV
e71cc8d833
Adding Debug logger to log package
2015-04-06 12:15:19 -07:00
Harshavardhana
70c6258440
Merge pull request #446 from harshavardhana/pr_out_erasure_is_now_external_pull_it_and_update_from_minio_io_erasure
2015-04-05 23:53:19 -07:00
Harshavardhana
319feb52f1
erasure is now external pull it and update from minio-io/erasure
2015-04-05 23:51:24 -07:00
Frederick F. Kautz IV
603b851226
Merge pull request #445 from fkautz/pr_out_fix_fd_leak
2015-04-05 21:34:07 -07:00
Frederick F. Kautz IV
052817ff91
Fix fd leak
2015-04-05 21:33:24 -07:00
Harshavardhana
2452393088
Merge pull request #444 from harshavardhana/pr_out_return_proper_return_for_all_drivers_when_bucketnotfound
2015-04-05 15:56:06 -07:00
Harshavardhana
d324c4b061
Return proper return for all drivers when BucketNotFound
2015-04-05 15:55:10 -07:00
Harshavardhana
0475d7d056
Migrate to external Donut package
2015-04-05 14:59:24 -07:00
Frederick F. Kautz IV
183df976f3
Merge pull request #443 from fkautz/pr_out_reformatting_imports
2015-04-04 14:00:34 -07:00
Frederick F. Kautz IV
2f44438843
Reformatting imports
2015-04-04 13:59:59 -07:00
Frederick F. Kautz IV
55a6d1797f
Merge pull request #442 from fkautz/pr_out_replacing_gopkg_in_check_v1_with_minio_io_check
2015-04-04 13:38:10 -07:00
Frederick F. Kautz IV
be4bc44b4d
Replacing gopkg.in/check.v1 with minio-io/check
2015-04-04 13:37:30 -07:00
Frederick F. Kautz IV
e6abca1a63
Adding iodine to object HEAD verb
2015-04-04 12:46:34 -07:00
Frederick F. Kautz IV
8198259dda
Merge pull request #441 from fkautz/pr_out_updating_godeps_json
2015-04-03 20:45:24 -07:00
Frederick F. Kautz IV
54a5e80ea5
Updating Godeps.json
2015-04-03 20:45:17 -07:00
Frederick F. Kautz IV
2abb4bba3a
Merge pull request #440 from fkautz/pr_out_removing_godeps_from_pkg_api
2015-04-03 20:40:37 -07:00
Frederick F. Kautz IV
46badca07e
Removing Godeps from pkg/api
2015-04-03 20:40:21 -07:00
Frederick F. Kautz IV
6ce3133f1e
Merge pull request #439 from fkautz/pr_out_adding_test_for_invalid_range_in_api
2015-04-03 20:39:43 -07:00
Frederick F. Kautz IV
4171ad8b69
Adding test for invalid range in api
2015-04-03 20:35:39 -07:00
Harshavardhana
9e5911cc62
Merge pull request #438 from harshavardhana/pr_out_remove_mxj_update_gopkg_in_check_v1
2015-04-03 19:04:00 -07:00
Harshavardhana
717913bc8a
Remove mxj, Update gopkg.in/check.v1
2015-04-03 19:03:16 -07:00
Harshavardhana
4079dd1d12
Commit testify,objx, gorilla/mux, minio-io/erasure gopkg.in/check.v1
2015-04-03 18:57:47 -07:00
Harshavardhana
4b25d4e8db
Use verifyError instead of parsResponse, also get rid of mxj dependency
2015-04-03 18:56:47 -07:00
Frederick F. Kautz IV
321a1a2d36
Merge pull request #437 from fkautz/pr_out_better_error_handling_tests_for_get_object
2015-04-03 18:53:50 -07:00
Frederick F. Kautz IV
9954e5c280
Better error handling + tests for get object
2015-04-03 18:53:27 -07:00
Frederick F. Kautz IV
407251d51f
Merge pull request #436 from fkautz/pr_out_adding_tests_for_put_bucket_errors_fixed_mismatching_error
2015-04-03 18:38:05 -07:00
Frederick F. Kautz IV
db1e6e5988
Adding tests for put bucket errors, fixed mismatching error
2015-04-03 18:37:39 -07:00
Frederick F. Kautz IV
a6fc618f91
Merge pull request #435 from fkautz/pr_out_adding_iodine_and_tests_for_for_list_buckets_errors
2015-04-03 18:29:39 -07:00
Frederick F. Kautz IV
ebf1e784b3
Adding iodine and tests for for list buckets errors
2015-04-03 18:29:31 -07:00
Frederick F. Kautz IV
37c601c21c
Merge pull request #434 from fkautz/pr_out_adding_better_error_support_to_api
2015-04-03 18:06:43 -07:00
Frederick F. Kautz IV
5865295f5d
Adding better error support to api
2015-04-03 18:06:30 -07:00
Harshavardhana
a29990e4bc
Merge pull request #433 from harshavardhana/pr_out_add_full_rebuild_for_minio_command_to_avoid_godep_stale_issues
2015-04-02 16:12:17 -07:00
Harshavardhana
637d2d687a
Add full rebuild for minio command to avoid godep stale issues
2015-04-02 16:11:52 -07:00
Harshavardhana
138542ec65
Merge pull request #432 from harshavardhana/pr_out_build_everything_by_default
2015-04-02 14:38:45 -07:00
Harshavardhana
9c0b17c239
Build everything by default
2015-04-02 14:34:05 -07:00
Harshavardhana
3bfc29b3c1
Merge pull request #431 from harshavardhana/pr_out_update_cli_to_minio_io_cli
2015-04-02 12:50:33 -07:00
Harshavardhana
92136d49fd
Update cli to minio-io/cli
2015-04-02 12:49:40 -07:00
Frederick F. Kautz IV
7a117bb6c5
Merge pull request #430 from fkautz/pr_out_updating_godep_to_minio_io_godep
2015-04-01 20:57:38 -07:00
Frederick F. Kautz IV
a689769245
Updating godep to minio-io/godep
2015-04-01 20:57:27 -07:00
Frederick F. Kautz IV
1daee667b7
Merge pull request #429 from fkautz/pr_out_fixing_race_condition_in_api_test_go
2015-04-01 20:32:16 -07:00
Frederick F. Kautz IV
8757ccf9a3
Fixing race condition in api_test.go
2015-04-01 20:30:57 -07:00
Harshavardhana
315a03583e
Merge pull request #428 from harshavardhana/pr_out_add_missing_mxj_package
2015-04-01 20:08:51 -07:00
Harshavardhana
c5b927c9b5
Add missing mxj package
2015-04-01 20:08:39 -07:00
Harshavardhana
f848c4095f
Merge pull request #427 from harshavardhana/pr_out_parse_response_headers_in_tests
2015-04-01 20:08:09 -07:00
Harshavardhana
85e547d854
Parse response headers in tests
2015-04-01 20:07:51 -07:00
Harshavardhana
f5226737e5
Merge pull request #426 from harshavardhana/pr_out_add_erasure_to_godep
2015-04-01 16:58:54 -07:00
Harshavardhana
92f408af5d
Add erasure to godep
2015-04-01 16:56:43 -07:00
Frederick F. Kautz IV
07235399a1
Merge pull request #425 from fkautz/pr_out_minor_cleanup_of_memory_get_partial_object
2015-04-01 15:51:08 -07:00
Frederick F. Kautz IV
818b6d2e33
Minor cleanup of memory get partial object
2015-04-01 15:50:59 -07:00
Frederick F. Kautz IV
e73c3b379f
Merge pull request #424 from fkautz/pr_out_add_partial_object_support_to_donut_driver
2015-04-01 15:48:27 -07:00
Frederick F. Kautz IV
0c5ed6f3ba
Add partial object support to donut driver
2015-04-01 15:48:12 -07:00
Frederick F. Kautz IV
27d4893a7e
Merge pull request #423 from fkautz/pr_out_fix_whitespace
2015-04-01 12:31:11 -07:00
Frederick F. Kautz IV
563a914046
Fix whitespace
2015-04-01 12:30:58 -07:00
Frederick F. Kautz IV
994d7841c3
Merge pull request #422 from fkautz/pr_out_adding_comments_on_why_we_subtract_a_second_from_date1_in_an_api_test
2015-04-01 12:29:28 -07:00
Frederick F. Kautz IV
83f0f0971f
Adding comments on why we subtract a second from date1 in an api test
2015-04-01 12:29:08 -07:00
Frederick F. Kautz IV
646ce59bc5
Merge pull request #421 from fkautz/pr_out_minor_modification_for_creating_temp_dir_in_api_test
2015-04-01 12:07:46 -07:00
Frederick F. Kautz IV
1f699a094b
Minor modification for creating temp dir in api test
2015-04-01 12:07:33 -07:00
Frederick F. Kautz IV
224d53b3eb
Merge pull request #420 from fkautz/pr_out_api_test_now_states_which_driver_is_currently_running
2015-04-01 12:06:35 -07:00
Frederick F. Kautz IV
5b1350b4bf
api test now states which driver is currently running
2015-04-01 12:06:22 -07:00
Harshavardhana
44b4ef66e1
Merge pull request #419 from harshavardhana/pr_out_make_sure_in_api_we_handle_all_error_type_exclusively_and_return_certain_values_exclusively
2015-03-31 20:27:24 -07:00
Harshavardhana
a56098b8c9
Make sure in API we handle all error type exclusively, and return certain values exclusively
2015-03-31 20:26:16 -07:00
Frederick F. Kautz IV
0491e1bbf1
Merge pull request #418 from fkautz/pr_out_moving_api_test_to_fixtures_for_drivers
2015-03-31 19:53:39 -07:00
Frederick F. Kautz IV
6136960cfd
Moving api_test to fixtures for drivers
2015-03-31 19:53:25 -07:00
Frederick F. Kautz IV
252a8b54f5
Merge pull request #417 from fkautz/pr_out_adding_test_for_partial_objects
2015-03-31 19:07:52 -07:00
Frederick F. Kautz IV
959d90fcf7
Adding test for partial objects
2015-03-31 19:07:45 -07:00
Frederick F. Kautz IV
376dc30428
Merge pull request #416 from fkautz/pr_out_simplifying_error_handling_in_api_bucket_handlers
2015-03-31 19:07:38 -07:00
Frederick F. Kautz IV
62c4c997c2
Simplifying error handling in api bucket handlers
2015-03-30 14:27:32 -07:00
Frederick F. Kautz IV
06c329d26b
Merge pull request #415 from fkautz/pr_out_api_bucket_handlers_errors_simplified
2015-03-30 11:40:40 -07:00
Frederick F. Kautz IV
318d9d5aa8
API Bucket Handlers errors simplified
2015-03-30 11:40:28 -07:00
Frederick F. Kautz IV
33414915a1
Merge pull request #414 from fkautz/pr_out_simplifying_error_response_writing_for_object_handlers
2015-03-29 22:03:24 -07:00
Frederick F. Kautz IV
7d657eb55c
Simplifying error response writing for object handlers
2015-03-29 22:03:13 -07:00
Frederick F. Kautz IV
c3cda5d9a8
Adding comments to errors
2015-03-29 22:03:13 -07:00
Frederick F. Kautz IV
53f116616f
Merge pull request #413 from fkautz/pr_out_normalizing_imports_in_api
2015-03-29 21:41:47 -07:00
Frederick F. Kautz IV
ded090a8b0
Normalizing imports in api
2015-03-29 21:41:38 -07:00
Frederick F. Kautz IV
a99e60db84
Merge pull request #412 from fkautz/pr_out_more_api_cleanup
2015-03-29 21:38:44 -07:00
Frederick F. Kautz IV
8cbc1c2346
More api cleanup
2015-03-29 21:38:23 -07:00
Frederick F. Kautz IV
936520e6e0
Merge pull request #411 from fkautz/pr_out_cleaning_up_more_api_code
2015-03-29 21:07:00 -07:00
Frederick F. Kautz IV
452f0f8335
Cleaning up more api code
2015-03-29 21:06:51 -07:00
Frederick F. Kautz IV
0f6abe9ee3
Merge pull request #410 from fkautz/pr_out_rename_writeobjectheaders_to_setobjectheaders_fix_writing_multiple_headers_bug
2015-03-29 18:50:29 -07:00
Frederick F. Kautz IV
d644233054
Rename writeObjectHeaders to setObjectHeaders, fix writing multiple headers bug
2015-03-29 18:49:32 -07:00
Harshavardhana
6fd78dba42
Merge pull request #409 from harshavardhana/pr_out_more_simplification_to_erasure_also_add_two_more_test_functions
2015-03-29 14:53:00 -07:00
Harshavardhana
b748cf359a
More simplification to erasure also add two more test functions
2015-03-29 14:52:13 -07:00
Harshavardhana
4c6677fd2c
Merge pull request #408 from harshavardhana/pr_out_update_cli_with_new_changes
2015-03-29 13:44:43 -07:00
Harshavardhana
5defd8ffdd
Update cli with new changes
2015-03-29 13:38:30 -07:00
Frederick F. Kautz IV
3c7d606571
Merge pull request #407 from fkautz/pr_out_minor_change_from_trace_to_error_log
2015-03-28 18:47:52 -07:00
Frederick F. Kautz IV
86b32c2147
Minor change from trace to error log
2015-03-28 18:47:36 -07:00
Frederick F. Kautz IV
c728798137
Merge pull request #406 from fkautz/pr_out_simplfying_code_in_api_round_1_decoupling_functions_which_both_write_headers_and_data
2015-03-28 18:01:23 -07:00
Frederick F. Kautz IV
d5db891094
Simplfying code in api, round 1. Decoupling functions which both write headers and data.
2015-03-28 18:01:00 -07:00
Frederick F. Kautz IV
83a4a95e9d
Merge pull request #405 from fkautz/pr_out_refactoring_pkg_utils_log_copying_from_golang_s_log_and_modifying
2015-03-28 15:59:47 -07:00
Frederick F. Kautz IV
b165efdfcb
Refactoring pkg/utils/log, copying from golang's log and modifying
2015-03-28 15:58:48 -07:00
Frederick F. Kautz IV
7f753a3965
Merge pull request #404 from fkautz/pr_out_minor_style_fix
2015-03-28 13:24:27 -07:00
Frederick F. Kautz IV
fd16de3675
Minor style fix
2015-03-28 13:23:54 -07:00
Harshavardhana
f3f0a3a5f7
Merge pull request #403 from harshavardhana/pr_out_let_donut_driver_use_log_errorln_wrapper
2015-03-28 12:43:42 -07:00
Harshavardhana
767f4d1d8f
Let donut driver use log.Errorln wrapper
2015-03-28 12:43:26 -07:00
Frederick F. Kautz IV
fa6ee236f7
Merge pull request #402 from fkautz/pr_out_fix_for_readers_closing_in_erasure_early
2015-03-27 23:50:02 -07:00
Frederick F. Kautz IV
c901d0fb32
Fix for readers closing in erasure early
2015-03-27 23:49:14 -07:00
Harshavardhana
e8d0d4f4d8
Merge pull request #401 from harshavardhana/pr_out_add_missing_testify_mock
2015-03-27 23:27:33 -07:00
Harshavardhana
fc3ee4d459
Add missing testify/mock
2015-03-27 23:27:17 -07:00
Frederick F. Kautz IV
8e8281e210
Merge pull request #400 from fkautz/pr_out_adding_tests_and_a_fix_for_get_object_where_bucket_is_not_found
2015-03-27 23:23:42 -07:00
Frederick F. Kautz IV
96e26335e2
Adding tests and a fix for get object where bucket is not found
2015-03-27 23:23:22 -07:00
Frederick F. Kautz IV
166c620263
Merge pull request #399 from fkautz/pr_out_fixing_donut_location
2015-03-27 12:44:29 -07:00
Frederick F. Kautz IV
ad370f341b
Fixing donut location
2015-03-27 12:44:13 -07:00
Frederick F. Kautz IV
c486d5c3f7
Merge pull request #398 from fkautz/pr_out_iodine_now_serializes_error_type_as_string
2015-03-26 22:48:26 -07:00
Frederick F. Kautz IV
17d432e7f1
Iodine now serializes error type as string
2015-03-26 22:48:09 -07:00
Frederick F. Kautz IV
9883af2337
Merge pull request #397 from fkautz/pr_out_updating_iodine_to_match_new_api
2015-03-26 15:55:19 -07:00
Frederick F. Kautz IV
53439d7768
Updating iodine to match new api
2015-03-26 15:55:06 -07:00
Frederick F. Kautz IV
ebc1e062a9
Merge pull request #396 from fkautz/pr_out_removing_todo
2015-03-26 11:33:55 -07:00
Frederick F. Kautz IV
c5f10df957
Removing TODO
2015-03-26 11:33:50 -07:00
Frederick F. Kautz IV
f4790fc440
Merge pull request #395 from fkautz/pr_out_refactoring_erasure_each_block_is_encoded_by_a_function_iodine_included
2015-03-26 11:32:55 -07:00
Frederick F. Kautz IV
eaeeb4bef0
Refactoring erasure, each block is encoded by a function. Iodine included.
2015-03-26 11:32:39 -07:00
Frederick F. Kautz IV
31746fdeb2
Adding iodine to donutObjectWriter.Write
2015-03-26 10:49:03 -07:00
Frederick F. Kautz IV
bfbb97c98f
Merge pull request #393 from fkautz/pr_out_adding_iodine_to_object_writer
2015-03-26 10:42:57 -07:00
Frederick F. Kautz IV
f7a300400e
Merge pull request #392 from fkautz/pr_out_adding_iodine_to_node_local
2015-03-26 10:42:50 -07:00
Frederick F. Kautz IV
3c37218d7d
Merge pull request #391 from fkautz/pr_out_addnode_fails_with_proper_error_code_when_bucketid_is_malformed
2015-03-26 10:42:42 -07:00
Frederick F. Kautz IV
caf7e41499
Adding iodine to object_writer
2015-03-26 10:41:40 -07:00
Frederick F. Kautz IV
5cdf2564a5
Adding iodine to node_local
2015-03-26 10:21:24 -07:00
Frederick F. Kautz IV
70a2217eb2
AddNode fails with proper error code when bucketid is malformed
2015-03-26 10:02:40 -07:00
Frederick F. Kautz IV
8cf66ede66
Merge pull request #390 from fkautz/pr_out_adding_iodine_to_donut
2015-03-26 09:57:49 -07:00
Frederick F. Kautz IV
cc64417481
Adding iodine to donut
2015-03-26 09:57:21 -07:00
Harshavardhana
58bf15e899
Merge pull request #389 from harshavardhana/pr_out_update_makefile
2015-03-25 23:10:18 -07:00
Harshavardhana
4d57ca7c44
Update makefile
2015-03-25 23:10:03 -07:00
Anand Babu (AB) Periasamy
4cf85ccc71
Merge pull request #388 from abperiasamy/clean-build-constants
...
make clean build-constants.go
2015-03-25 23:05:21 -07:00
Anand Babu (AB) Periasamy
e14ba846b5
make clean build-constants.go
2015-03-25 23:04:27 -07:00
Anand Babu (AB) Periasamy
7d0fb1b9c6
Merge pull request #387 from abperiasamy/git-commit-id-generator
...
git commit hash generator
2015-03-25 22:58:54 -07:00
Anand Babu (AB) Periasamy
3c4aa85c69
git commit hash generator
2015-03-25 22:57:30 -07:00
Harshavardhana
aa44548a67
Merge pull request #386 from fkautz/pr_out_updating_iodine_to_support_new_iodine_error_error_map_string_string_error_method
2015-03-25 21:17:29 -07:00
Frederick F. Kautz IV
40e311a919
Updating iodine to support new iodine.Error(error, map[string]string) error method
2015-03-25 19:12:55 -07:00
Harshavardhana
68f36f5e08
Merge pull request #385 from abperiasamy/erasure_cache_tbls_matrix
2015-03-25 19:11:07 -07:00
Anand Babu (AB) Periasamy
b44d2100db
cache decode matrix tables and matrix
2015-03-25 19:09:34 -07:00
Harshavardhana
7503b7f9d1
Merge pull request #384 from harshavardhana/pr_out_fix_golint_complaints
2015-03-25 19:01:52 -07:00
Harshavardhana
2d0be3e3b2
fix golint complaints
2015-03-25 19:01:36 -07:00
Harshavardhana
638ed36cc6
Merge pull request #382 from abperiasamy/janitor-erasure-decode
2015-03-25 18:13:46 -07:00
Anand Babu (AB) Periasamy
bd39768de2
janitor duty on erasure-decode
2015-03-25 18:02:03 -07:00
Frederick F. Kautz IV
db2ac35da5
Merge pull request #381 from fkautz/pr_out_upgrading_iodine
2015-03-25 17:32:36 -07:00
Frederick F. Kautz IV
9b9d5e9d39
Upgrading iodine
2015-03-25 17:32:25 -07:00
Frederick F. Kautz IV
28355c7aa6
Merge pull request #380 from fkautz/pr_out_adding_iodine_to_donut_bucket
2015-03-25 17:05:36 -07:00
Frederick F. Kautz IV
eff48971ad
Adding iodine to donut bucket
2015-03-25 17:05:26 -07:00
Frederick F. Kautz IV
d81472f836
Merge pull request #379 from fkautz/pr_out_adding_iodine_to_pkg_server_server_go
2015-03-25 17:04:41 -07:00
Frederick F. Kautz IV
d52dd5ed5b
Adding iodine to pkg/server/server.go
2015-03-25 17:04:26 -07:00
Frederick F. Kautz IV
673072579d
Merge pull request #378 from fkautz/pr_out_adding_iodine_to_main_go
2015-03-25 17:03:45 -07:00
Frederick F. Kautz IV
d54999c4d0
Adding iodine to main.go
2015-03-25 17:03:34 -07:00
Harshavardhana
765d7da890
Merge pull request #377 from harshavardhana/pr_out_add_gocyclo_into_source_build_choosing_cyclomatic_complexity_tolerance_of_15_and_below
2015-03-25 15:59:19 -07:00
Harshavardhana
58082cd8dc
Add gocyclo into source build, choosing cyclomatic complexity tolerance of 15 and below
2015-03-25 15:57:17 -07:00
Harshavardhana
c366fdd2d8
loop only on readers once
2015-03-25 14:48:46 -07:00
Harshavardhana
d9bd5e2e10
Merge pull request #376 from harshavardhana/pr_out_comment_exported_functions_getencodedblockslen_getencodedblocklen
2015-03-25 13:32:59 -07:00
Harshavardhana
e1f4ed4650
Comment exported functions GetEncodedBlocksLen, GetEncodedBlockLen
2015-03-25 13:32:35 -07:00
Harshavardhana
fab6892bc3
Merge pull request #375 from harshavardhana/pr_out_golint_fixes
2015-03-25 13:25:38 -07:00
Harshavardhana
2cbd15b690
Golint fixes
2015-03-25 13:25:25 -07:00
Harshavardhana
9503a4df33
Merge pull request #374 from harshavardhana/pr_out_remove_unnecessary_if_block
2015-03-25 12:58:29 -07:00
Harshavardhana
d967babbaa
Remove unnecessary if <block>
2015-03-25 12:58:15 -07:00
Frederick F. Kautz IV
5461ab2605
Merge pull request #373 from fkautz/pr_out_fix_build_break
2015-03-25 11:50:13 -07:00
Frederick F. Kautz IV
5c0d92eb39
Fix build break
2015-03-25 11:50:05 -07:00
Harshavardhana
2f611e61ea
Merge pull request #372 from harshavardhana/pr_out_build_fix_remove_errors_not_used
2015-03-25 11:49:19 -07:00
Harshavardhana
54519bc0b9
Build fix, remove 'errors' not used
2015-03-25 11:49:04 -07:00
Frederick F. Kautz IV
63194c321d
Merge pull request #371 from fkautz/pr_out_adding_global_state_to_iodine
2015-03-25 11:45:52 -07:00
Frederick F. Kautz IV
337df7f7ec
Adding global state to iodine
2015-03-25 11:45:14 -07:00
Harshavardhana
0662f9e019
Merge pull request #207 from vadmeste/add_docker_file
...
Add Dockerfile to build a docker image with embarked minio server
2015-03-25 11:35:32 -07:00
Harshavardhana
d8361f8bb4
Merge pull request #370 from harshavardhana/pr_out_use_commitid_as_version
2015-03-25 11:20:47 -07:00
Harshavardhana
47cd2c94c8
Use commitID as version
2015-03-25 11:20:34 -07:00
Frederick F. Kautz IV
da89132def
Merge pull request #369 from fkautz/pr_out_updating_iodine_now_has_global_state
2015-03-25 10:40:08 -07:00
Frederick F. Kautz IV
abe49919ed
Updating iodine, now has global state.
2015-03-25 10:39:44 -07:00
Anand Babu (AB) Periasamy
31482659be
Merge pull request #368 from abperiasamy/commit-id
...
commid_id support
2015-03-25 01:51:13 -07:00
Anand Babu (AB) Periasamy
62af4c0b52
commid_id support
2015-03-25 01:50:01 -07:00
Frederick F. Kautz IV
d6507305ac
Merge pull request #367 from fkautz/pr_out_updating_iodine
2015-03-25 00:46:10 -07:00
Frederick F. Kautz IV
be229473b5
Updating iodine
2015-03-25 00:43:48 -07:00
Harshavardhana
d909a0e297
Merge pull request #366 from harshavardhana/pr_out_fix_nil_pointer_dereference_when_err_is_nil
2015-03-25 00:37:38 -07:00
Harshavardhana
0f8021abd1
Fix nil pointer dereference when err is nil
2015-03-25 00:37:25 -07:00
Frederick F. Kautz IV
e8d35258f1
Merge pull request #365 from fkautz/pr_out_adding_iodine_annotation_to_donut_start_path
2015-03-25 00:27:58 -07:00
Frederick F. Kautz IV
1be3766ae6
Adding iodine annotation to donut.Start(path)
2015-03-25 00:27:45 -07:00
Frederick F. Kautz IV
12a17f1ed7
Merge pull request #364 from fkautz/pr_out_using_iodine_in_donut_start_
2015-03-25 00:25:41 -07:00
Frederick F. Kautz IV
d06f5afdfc
Using iodine in donut.Start()
2015-03-25 00:25:27 -07:00
Harshavardhana
4d165dbcbb
Merge pull request #363 from harshavardhana/pr_out_avoid_empty_if_else_body_
2015-03-25 00:24:59 -07:00
Harshavardhana
b3c3a8d6f2
Avoid empty if else body..
2015-03-25 00:24:41 -07:00
Frederick F. Kautz IV
029e49a7d2
Merge pull request #362 from fkautz/pr_out_updating_check_v1
2015-03-25 00:21:55 -07:00
Frederick F. Kautz IV
2e857a6f94
Updating check.v1
2015-03-25 00:21:43 -07:00
Frederick F. Kautz IV
2aa1da16d7
Merge pull request #361 from fkautz/pr_out_updating_iodine
2015-03-25 00:18:00 -07:00
Frederick F. Kautz IV
2acd66ae96
Updating iodine
2015-03-25 00:17:50 -07:00
Harshavardhana
bbaa72f67d
Merge pull request #360 from harshavardhana/pr_out_fix_minio_hang_in_donut_driver
2015-03-25 00:16:22 -07:00
Harshavardhana
ada98c56c0
Fix minio hang in donut driver.
...
remove sending err to errorChannel until we figure out the right fix.
2015-03-25 00:15:36 -07:00
Frederick F. Kautz IV
938a0fa4a9
Merge pull request #359 from fkautz/pr_out_adding_iodine
2015-03-25 00:04:56 -07:00
Frederick F. Kautz IV
1ae74591f6
Adding iodine
2015-03-24 23:54:49 -07:00
Frederick F. Kautz IV
3fc54eea3a
Merge pull request #357 from fkautz/pr_out_populate_buckets_at_start
2015-03-24 21:04:48 -07:00
Frederick F. Kautz IV
dd5ba865ff
Merge remote-tracking branch 'origin/master' into HEAD
...
Conflicts:
pkg/drivers/donut/donut.go
pkg/storage/donut/bucket.go
pkg/storage/donut/donut.go
pkg/storage/donut/donut_test.go
pkg/storage/donut/donutdriver.go
2015-03-24 21:04:08 -07:00
Harshavardhana
a311a9c4b8
Merge pull request #358 from harshavardhana/pr_out_move_from_codegangsta_cli_to_minio_io_cli_fork
2015-03-24 20:41:04 -07:00
Harshavardhana
d0d585efc1
Move from codegangsta/cli to minio-io/cli fork
2015-03-24 20:40:19 -07:00
Frederick F. Kautz IV
cf2550b267
Populate buckets at start
2015-03-24 19:58:03 -07:00
Harshavardhana
4c7a61a68d
Merge pull request #356 from harshavardhana/pr_out_erasure_technique_is_now_uint8_handle_it_appropriately
2015-03-24 19:13:09 -07:00
Harshavardhana
653ea50a59
Erasure technique is now uint8, handle it appropriately
2015-03-24 19:12:52 -07:00
Harshavardhana
7d25490a2e
Merge pull request #355 from abperiasamy/janitor_erasure
2015-03-24 19:09:20 -07:00
Anand Babu (AB) Periasamy
fa9b1f341b
erasure encode janitor duty
2015-03-24 18:47:53 -07:00
Harshavardhana
62b69048ec
Merge pull request #354 from harshavardhana/pr_out_handle_errors_properly_during_erasure_decoding_also_populate_technique_and_verify
2015-03-24 18:14:40 -07:00
Harshavardhana
6a87ed001c
Handle errors properly during erasure Decoding, also populate technique and verify
2015-03-24 18:14:13 -07:00
Harshavardhana
d58dbe3fce
Merge pull request #353 from harshavardhana/pr_out_code_restructuring_assigning_proper_subsystems_to_each
2015-03-24 06:48:28 -07:00
Harshavardhana
0474439b43
Code restructuring, assigning proper subsystems to each
...
- Drivers contain
* donut/*
* file/*
* memory/*
- Storage format contains
* donut/*
- GetObject() --> renamed to GetObjectReader()
- Deleted stale objectwriter.go, renamed donutwriter.go to object_writer.go
Simplifying, and documenting codebase further
2015-03-24 06:47:10 -07:00
Frederick F. Kautz IV
6dd7a9d654
Merge pull request #352 from fkautz/pr_out_adding_donut_backend_and_setting_as_default
2015-03-23 20:07:52 -07:00
Frederick F. Kautz IV
3c4012f1e7
Adding donut backend and setting as default
2015-03-23 20:07:22 -07:00
Harshavardhana
7c1c4b8a02
Merge pull request #351 from harshavardhana/pr_out_implement_log_package_as_drop_in_replacement_for_handling_debug_log_level
2015-03-23 18:55:14 -07:00
Harshavardhana
914962bd93
Implement log package as drop-in replacement for handling Debug log-level
2015-03-23 18:18:11 -07:00
Frederick F. Kautz IV
2191fb739f
Merge pull request #350 from fkautz/pr_out_running_gofmt_manually_auto_gofmt_didn_t_simplify_code
2015-03-23 12:26:52 -07:00
Frederick F. Kautz IV
f91f5e59ee
Running gofmt manually, auto gofmt didn't simplify code
2015-03-23 12:26:41 -07:00
Frederick F. Kautz IV
56a298fd6a
Refactoring method variable names for erasureWriter struct
2015-03-23 12:25:20 -07:00
Frederick F. Kautz IV
742a186d10
Merge pull request #348 from fkautz/pr_out_adding_sys_as_reserved_in_erasurewriter
2015-03-23 11:56:37 -07:00
Frederick F. Kautz IV
d57e9d6c18
Adding sys. as reserved in erasureWriter
2015-03-23 11:56:24 -07:00
Frederick F. Kautz IV
920c734b2f
Merge pull request #347 from fkautz/pr_out_switching_content_hashing_from_sha512_to_md5
2015-03-23 11:46:28 -07:00
Frederick F. Kautz IV
b2f47ef446
Switching content hashing from sha512 to md5
2015-03-23 11:46:19 -07:00
Harshavardhana
80892c5c9b
Merge pull request #346 from harshavardhana/pr_out_rename_parms_as_params_for_brevity_and_misc_cleanup
2015-03-22 20:27:11 -07:00
Harshavardhana
b8e9834faa
Rename parms as --> params for brevity and misc cleanup
2015-03-22 20:26:01 -07:00
Frederick F. Kautz IV
9d5a6329ba
Merge pull request #345 from fkautz/pr_out_adding_sha512_support_for_pre_encoded_data_and_verification_on_decode
2015-03-22 20:11:52 -07:00
Frederick F. Kautz IV
b2c3172095
Adding sha512 support for pre-encoded data and verification on decode
2015-03-22 20:11:25 -07:00
Frederick F. Kautz IV
f471915646
Merge pull request #344 from fkautz/pr_out_exorcising_donutbox
2015-03-22 19:55:50 -07:00
Frederick F. Kautz IV
befac7d047
Exorcising donutbox
2015-03-22 19:55:15 -07:00
Frederick F. Kautz IV
331a3e80a3
Merge pull request #343 from fkautz/pr_out_reading_correct_block_size_in_erasure_coded_data
2015-03-22 18:49:41 -07:00
Frederick F. Kautz IV
eb8005cc65
Reading correct block size in erasure coded data
2015-03-22 18:49:30 -07:00
Frederick F. Kautz IV
601f5c77fe
Merge pull request #342 from abperiasamy/janitor_remove_k_m
2015-03-22 18:48:51 -07:00
Anand Babu (AB) Periasamy
b9b0b35edf
removed private k and m and some more cleanups
2015-03-22 18:22:09 -07:00
Frederick F. Kautz IV
293e438830
Merge pull request #341 from abperiasamy/janitor
2015-03-22 17:34:25 -07:00
Anand Babu (AB) Periasamy
3a76229551
encoded size function is now public + some janitor duty
2015-03-22 17:17:53 -07:00
Harshavardhana
562a57be84
Merge pull request #340 from harshavardhana/pr_out_make_sure_we_use_o_excl_with_o_create_to_make_sure_we_don_t_trip_over_existing_file
2015-03-22 15:56:31 -07:00
Harshavardhana
3929135c34
Make sure we use O_EXCL with O_CREATE to make sure we don't trip over existing file
2015-03-22 15:55:40 -07:00
Harshavardhana
b2ee7d989d
Merge pull request #339 from harshavardhana/pr_out_de_couple_donut_into_smaller_files_useful_for_ease_in_external_integration
2015-03-22 15:53:26 -07:00
Harshavardhana
2fa9320df2
De-couple donut into smaller files, useful for ease in external integration
2015-03-22 15:43:37 -07:00
Frederick F. Kautz IV
38bfc7a949
Merge pull request #338 from fkautz/pr_out_removing_log_tracers
2015-03-22 15:39:58 -07:00
Frederick F. Kautz IV
55f3659bb3
Removing log tracers
2015-03-22 15:39:50 -07:00
Frederick F. Kautz IV
4433142813
Merge pull request #337 from fkautz/pr_out_adding_list_objects_to_donut
2015-03-22 15:34:12 -07:00
Frederick F. Kautz IV
03f51eac29
Adding list objects to donut
2015-03-22 15:33:52 -07:00
Anis Elleuch
64fed14aac
Add Dockerfile to build a docker image with embarked minio server
2015-03-22 14:52:27 -07:00
Harshavardhana
dcd9a8ae12
Merge pull request #336 from harshavardhana/pr_out_donutfilewriter_should_return_value_of_file_close_
2015-03-22 14:51:47 -07:00
Harshavardhana
4b9fbd3b3a
DonutfileWriter should return value of file.Close()
2015-03-22 14:51:18 -07:00
Frederick F. Kautz IV
ab89298250
Merge pull request #335 from fkautz/pr_out_minor_change_removing_unnecessary_commented_method
2015-03-22 12:44:54 -07:00
Frederick F. Kautz IV
ced6edc9fe
Minor change, removing unnecessary commented method
2015-03-22 12:44:45 -07:00
Frederick F. Kautz IV
9fde4daa29
Merge pull request #334 from fkautz/pr_out_adding_getobjectmetadata_to_donut
2015-03-22 12:36:59 -07:00
Frederick F. Kautz IV
99cf484488
Adding GetObjectMetadata to donut
2015-03-22 12:36:51 -07:00
Frederick F. Kautz IV
7d6534088e
Merge pull request #333 from fkautz/pr_out_removing_gob_from_donut_driver
2015-03-22 12:17:36 -07:00
Frederick F. Kautz IV
703af6c3ce
Removing gob from donut driver
2015-03-22 12:17:12 -07:00
Harshavardhana
537870ec4f
Merge pull request #332 from harshavardhana/pr_out_fix_build_issues_for_new_donutdriver_
2015-03-22 02:53:39 -07:00
Harshavardhana
95affcb119
Fix build issues for new DonutDriver()
2015-03-22 02:53:21 -07:00
Frederick F. Kautz IV
15b205c439
Merge pull request #331 from fkautz/pr_out_new_donut_driver_with_local_file_encoding
2015-03-22 00:40:30 -07:00
Frederick F. Kautz IV
933d44c656
New donut driver with local file encoding
2015-03-22 00:40:21 -07:00
Harshavardhana
d92a179cbc
Merge pull request #330 from harshavardhana/pr_out_return_error_during_joinfiles_back_to_the_caller_for_inspection
2015-03-20 03:17:18 -07:00
Harshavardhana
a8056c4c12
Return error during JoinFiles back to the caller for inspection
2015-03-20 03:17:08 -07:00
Harshavardhana
8859232f93
Merge pull request #329 from harshavardhana/pr_out_update_license_header_mini_should_be_minimalist_really
2015-03-19 14:36:08 -07:00
Harshavardhana
2b0b5f014a
Update license header Mini should be Minimalist really
2015-03-19 14:35:50 -07:00
Harshavardhana
321c6dbe09
Merge pull request #328 from harshavardhana/pr_out_fix_bucketpolicy_to_handle_s3_and_minio_policy_names_separately
2015-03-19 12:23:02 -07:00
Harshavardhana
c3602967ab
Fix bucketpolicy to handle s3 and minio policy names separately.
...
Also fix an apparent bug in GetBucketPolicy{}
2015-03-19 12:22:14 -07:00
Harshavardhana
7496d89f8f
Merge pull request #327 from harshavardhana/pr_out_resource_list_updates
2015-03-19 00:49:17 -07:00
Harshavardhana
6eb5315249
Resource list updates
2015-03-19 00:47:34 -07:00
Harshavardhana
8293c787bf
Merge pull request #326 from harshavardhana/pr_out_refactor_to_move_config_handling_into_api
2015-03-18 22:49:19 -07:00
Harshavardhana
a1baed3bbc
Refactor to move config handling into api/
2015-03-18 22:49:02 -07:00
Harshavardhana
b61daa0f87
Merge pull request #325 from harshavardhana/pr_out_restructure_minio_api_move_signature_checks_from_utils_to_api
2015-03-18 18:17:31 -07:00
Harshavardhana
7ce3ab3193
Restructure minio api, move signature checks from utils to Api.
2015-03-18 18:16:19 -07:00
Frederick F. Kautz IV
7ddf6a9f8f
Merge pull request #324 from fkautz/pr_out_adding_interfaces_for_donut_driver
2015-03-18 14:27:20 -07:00
Frederick F. Kautz IV
2c69433588
Adding interfaces for donut driver
2015-03-18 14:27:10 -07:00
Frederick F. Kautz IV
4ab19943e8
Merge pull request #323 from fkautz/pr_out_renaming_files_to_match
2015-03-18 12:26:53 -07:00
Frederick F. Kautz IV
bf86e78714
Renaming files to match
2015-03-18 12:26:44 -07:00
Frederick F. Kautz IV
e625bf2bb5
Merge pull request #322 from fkautz/pr_out_renaming_encoded_driver_to_donutstorage
2015-03-18 12:25:53 -07:00
Frederick F. Kautz IV
1c9e550ee6
Renaming encoded driver to donutstorage
2015-03-18 12:25:37 -07:00
Frederick F. Kautz IV
0fdd9f6fc0
Merge pull request #321 from fkautz/pr_out_adding_interfaces_for_implementing_against
2015-03-17 22:34:18 -07:00
Frederick F. Kautz IV
d7f00194b5
Adding interfaces for implementing against
2015-03-17 22:33:59 -07:00
Frederick F. Kautz IV
982a52e0d4
Merge pull request #320 from fkautz/pr_out_removing_block_size_from_donut_writer_no_longer_necessary
2015-03-17 14:25:08 -07:00
Frederick F. Kautz IV
d6be8d3517
Removing block size from donut writer, no longer necessary
2015-03-17 14:24:59 -07:00
Harshavardhana
c551757415
Merge pull request #319 from harshavardhana/pr_out_now_minioapi_handles_content_md5_set_during_putobject_
2015-03-17 13:33:31 -07:00
Harshavardhana
74b3d092f2
Now MinioAPI handles Content-MD5 set during putObject()
...
- This change also facilitates proper error response
in case of md5 mismatch or corruption
- TODO a test function needs to be implemented
2015-03-17 13:32:13 -07:00
Harshavardhana
42006c2ab0
Merge pull request #318 from harshavardhana/pr_out_remove_contenttype_pseudo_func_
2015-03-17 00:47:46 -07:00
Frederick F. Kautz IV
1dcb8eceb7
Merge pull request #317 from fkautz/pr_out_fixing_order_of_input_and_layout_in_time_parse
2015-03-17 00:47:34 -07:00
Harshavardhana
dccf32e5c4
Remove contenttype pseudo func()
2015-03-17 00:47:22 -07:00
Frederick F. Kautz IV
90b337a3e8
Fixing order of input and layout in time.Parse
2015-03-17 00:47:17 -07:00
Frederick F. Kautz IV
4770f611e0
Merge pull request #316 from fkautz/pr_out_moving_to_more_accurate_rfc3339
2015-03-17 00:46:01 -07:00
Frederick F. Kautz IV
25725c442a
Moving to more accurate RFC3339
2015-03-17 00:45:50 -07:00
Frederick F. Kautz IV
639f542ac9
Merge pull request #315 from fkautz/pr_out_moving_from_iso8601_to_rfc3339
2015-03-17 00:42:25 -07:00
Frederick F. Kautz IV
e36fa95f1f
Moving from iso8601 to RFC3339
2015-03-17 00:42:08 -07:00
Frederick F. Kautz IV
6070d18217
Merge pull request #314 from fkautz/pr_out_encoded_now_passes_tests_enabling
2015-03-17 00:34:18 -07:00
Frederick F. Kautz IV
598eb5192e
Encoded now passes tests, enabling.
2015-03-17 00:33:49 -07:00
Harshavardhana
e5e6e4cfa7
Merge pull request #313 from harshavardhana/pr_out_encoded_storage_type_now_feature_complete
2015-03-16 23:46:26 -07:00
Harshavardhana
6d71935ea5
Encoded storage type now feature complete
2015-03-16 23:42:45 -07:00
Frederick F. Kautz IV
1766b588fb
Merge pull request #312 from fkautz/pr_out_list_objects_in_buckets
2015-03-16 20:43:50 -07:00
Frederick F. Kautz IV
34cf6682fe
list objects in buckets
2015-03-16 20:43:05 -07:00
Frederick F. Kautz IV
d99435963b
Merge pull request #311 from fkautz/pr_out_switching_from_base64_to_hex_for_etag
2015-03-16 18:39:07 -07:00
Frederick F. Kautz IV
1a88b36c55
Switching from base64 to hex for etag
2015-03-16 18:38:42 -07:00
Harshavardhana
81f1d3c195
Merge pull request #310 from harshavardhana/pr_out_make_blocksize_as_constant
2015-03-16 18:20:22 -07:00
Harshavardhana
b495f15178
Make blockSize as constant
2015-03-16 18:20:01 -07:00
Frederick F. Kautz IV
cda13750f7
Merge pull request #309 from fkautz/pr_out_performing_etag_off_of_bytes_rather_than_md5_string
2015-03-16 18:15:44 -07:00
Frederick F. Kautz IV
211b084e50
Performing etag off of bytes rather than md5 string
2015-03-16 18:15:37 -07:00
Frederick F. Kautz IV
a8c8e4d335
Merge pull request #308 from fkautz/pr_out_metadata_now_stores_hex_md5_not_base64_minio_api_now_converts_md5_hex_to_base64
2015-03-16 18:03:46 -07:00
Frederick F. Kautz IV
583c32c774
Metadata now stores hex md5, not base64. Minio API now converts md5 hex to base64.
2015-03-16 18:03:22 -07:00
Frederick F. Kautz IV
ed5051a795
Merge pull request #307 from fkautz/pr_out_better_handling_of_metadata_in_encoded
2015-03-16 17:41:28 -07:00
Frederick F. Kautz IV
10ae089c46
Better handling of metadata in encoded
2015-03-16 17:40:59 -07:00
Harshavardhana
dc1589bb5f
Merge pull request #306 from harshavardhana/pr_out_some_necessary_cleanup
2015-03-16 12:06:23 -07:00
Harshavardhana
15f68972a5
Some necessary cleanup
...
- Rename 'fs' as 'file' for brevity
- Rename 'inmemory' as 'memory' for brevity
- Change everywhere else
2015-03-16 11:46:16 -07:00
Harshavardhana
772d3bc3d4
Merge pull request #305 from harshavardhana/pr_out_remove_x509_package_janitor_continuation_monad
2015-03-15 19:45:09 -07:00
Harshavardhana
765fde13f1
Remove x509 package, janitor continuation monad
2015-03-15 19:44:55 -07:00
Harshavardhana
bf04d125fd
Merge pull request #304 from harshavardhana/pr_out_move_from_fmt_errorf_to_errors_new
2015-03-15 19:42:23 -07:00
Harshavardhana
596eeb097e
Move from fmt.Errorf to errors.New
...
In many cases fmt.Errorf is good enough, but since
error is an interface, you can use arbitrary data
structures as error values, to allow callers to
inspect the details of the error.
2015-03-15 19:41:16 -07:00
Frederick F. Kautz IV
7936b05451
Merge pull request #303 from fkautz/pr_out_decoding_and_returning_objects_now_works
2015-03-15 19:02:12 -07:00
Frederick F. Kautz IV
a589ddaddf
Decoding and returning objects now works
2015-03-15 19:02:00 -07:00
Frederick F. Kautz IV
a737246f92
Merge pull request #302 from fkautz/pr_out_metadata_and_writing_the_object_are_now_both_committed_together
2015-03-15 17:43:21 -07:00
Frederick F. Kautz IV
682bf085c4
Metadata and writing the object are now both committed together.
2015-03-15 17:42:33 -07:00
Frederick F. Kautz IV
3a2419d20f
Merge pull request #300 from fkautz/pr_out_adding_create_bucket_to_encoded
2015-03-15 12:28:39 -07:00
Frederick F. Kautz IV
96d0a089b8
Adding create bucket to encoded
2015-03-15 12:28:24 -07:00
Harshavardhana
4744149ec4
Merge pull request #299 from harshavardhana/pr_out_skip_uinimplemented_tests_and_some_cleanup
2015-03-15 01:24:23 -07:00
Harshavardhana
3f95c55067
Skip uinimplemented tests and some cleanup
2015-03-15 01:23:57 -07:00
Frederick F. Kautz IV
6990a64d01
Merge pull request #298 from fkautz/pr_out_working_in_memory_donut_driver
2015-03-15 00:41:14 -07:00
Frederick F. Kautz IV
3e453a394a
Working in memory donut driver
2015-03-15 00:41:06 -07:00
Frederick F. Kautz IV
e601d997f3
Merge pull request #297 from fkautz/pr_out_more_testing
2015-03-14 19:30:02 -07:00
Frederick F. Kautz IV
15d5e74837
More testing
2015-03-14 19:29:54 -07:00
Frederick F. Kautz IV
d52ca9a238
Merge pull request #296 from fkautz/pr_out_inmemory_donutbox_is_now_thread_safe
2015-03-14 19:21:44 -07:00
Frederick F. Kautz IV
7e61696901
Inmemory donutbox is now thread safe
2015-03-14 19:21:34 -07:00
Frederick F. Kautz IV
74eb6067f6
Merge pull request #295 from fkautz/pr_out_adding_start_of_inmem_donut
2015-03-14 18:02:43 -07:00
Frederick F. Kautz IV
85da9fc627
Adding start of inmem donut
2015-03-14 18:02:27 -07:00
Frederick F. Kautz IV
b441a27c67
Merge pull request #294 from fkautz/pr_out_renaming_singledisk_to_encoded
2015-03-14 11:39:03 -07:00
Frederick F. Kautz IV
54472c18fe
Renaming singledisk to encoded
2015-03-14 11:37:43 -07:00
Frederick F. Kautz IV
348c4d9a88
Removing old donut format
2015-03-14 11:36:19 -07:00
Frederick F. Kautz IV
2f4b92eb8d
Minor refactor to prepare for file version of donut
2015-03-14 11:34:45 -07:00
Harshavardhana
dffee21cde
Merge pull request #292 from harshavardhana/pr_out_update_erasure_readme_with_supported_architecture_list
2015-03-12 16:37:40 -07:00
Harshavardhana
f7b62caccc
Update erasure README with supported architecture list
2015-03-12 16:37:08 -07:00
Harshavardhana
959d482614
Merge pull request #291 from harshavardhana/pr_out_cosmetic_changes
2015-03-12 02:15:51 -07:00
Harshavardhana
f7fd526b28
Cosmetic changes
2015-03-12 02:11:04 -07:00
Harshavardhana
1e4a47ac5f
Merge pull request #290 from harshavardhana/pr_out_add_crc32c_license_headers_golang_and_minio
2015-03-12 02:09:06 -07:00
Harshavardhana
4c86e594d6
Add crc32c license headers Golang and Minio
2015-03-12 02:05:05 -07:00
Harshavardhana
a049024643
Merge pull request #289 from harshavardhana/pr_out_remove_minio_calc_chunk_size_c_code_to_write_one_in_golang
2015-03-12 02:04:08 -07:00
Harshavardhana
e27ba44d94
Remove minio_calc_chunk_size() C code to write one in Golang
2015-03-12 02:03:04 -07:00
Frederick F. Kautz IV
1047383bcf
Merge pull request #288 from fkautz/pr_out_more_refactoring_to_meet_new_donut_spec
2015-03-12 01:52:35 -07:00
Frederick F. Kautz IV
94fe03cfc1
More refactoring to meet new donut spec
2015-03-12 01:52:14 -07:00
Harshavardhana
9889796c9c
Merge pull request #287 from harshavardhana/pr_out_restructure_and_use_human_readable_names
2015-03-12 00:38:19 -07:00
Harshavardhana
67888cbaa9
Restructure and use human readable names
2015-03-12 00:36:33 -07:00
Harshavardhana
e21020b337
Merge pull request #280 from harshavardhana/pr_out_http_range_support_wip
...
Get object range request added
2015-03-12 00:11:38 -07:00
Harshavardhana
a42a48a455
Get object range request added
2015-03-12 00:07:27 -07:00
Frederick F. Kautz IV
1cfa2062ea
Merge pull request #285 from fkautz/pr_out_rename_storebucket_to_createbucket
2015-03-11 20:14:49 -07:00
Frederick F. Kautz IV
24d5018037
Rename StoreBucket to CreateBucket
2015-03-11 20:14:40 -07:00
Frederick F. Kautz IV
e0ef70e11b
Merge pull request #284 from fkautz/pr_out_adding_copyright
2015-03-11 19:59:32 -07:00
Frederick F. Kautz IV
8c6aa45458
Adding copyright
2015-03-11 19:59:20 -07:00
Frederick F. Kautz IV
a749f659aa
Merge pull request #283 from fkautz/pr_out_adding_object_donut_type
2015-03-11 19:57:03 -07:00
Frederick F. Kautz IV
7971b66e60
Adding object donut type
2015-03-11 19:56:18 -07:00
Frederick F. Kautz IV
3af2894a9e
Merge pull request #282 from fkautz/pr_out_adding_stubes_for_new_single_disk_driver
2015-03-11 13:36:35 -07:00
Frederick F. Kautz IV
ec9b20a94f
Adding stubes for new single disk driver
2015-03-11 13:36:21 -07:00
Frederick F. Kautz IV
a831757064
Merge pull request #281 from fkautz/pr_out_adding_idea_files_to_gitignore
2015-03-11 12:05:27 -07:00
Frederick F. Kautz IV
9da26de696
Adding IDEA files to gitignore
2015-03-11 12:05:12 -07:00
Harshavardhana
cd6a68400d
Merge pull request #279 from harshavardhana/pr_out_rename_dateformat_with_understandable_name_iso8601format_
2015-03-10 01:52:23 -07:00
Harshavardhana
a24bfb007d
Rename dateFormat with understandable name `iso8601Format`
2015-03-10 01:51:34 -07:00
Harshavardhana
dd0469542b
Merge pull request #278 from harshavardhana/pr_out_add_strict_checks_with_vet_golint_and_gofmt
2015-03-09 16:19:09 -07:00
Harshavardhana
b952855779
Add strict checks with vet, golint and gofmt
2015-03-09 16:17:38 -07:00
Harshavardhana
d02fb75dfd
Merge pull request #277 from harshavardhana/pr_out_simplify_fs_codebase_split_them_into_separate_files_and_more_commenting
2015-03-08 18:05:52 -07:00
Harshavardhana
41653774fc
Simplify fs codebase, split them into separate files and more commenting
2015-03-08 18:01:48 -07:00
Harshavardhana
f4271e1114
Merge pull request #276 from harshavardhana/pr_out_make_bucketpolicy_strictly_typed
2015-03-08 15:58:57 -07:00
Harshavardhana
01e98eb326
Make BucketPolicy strictly typed
2015-03-08 15:57:20 -07:00
Harshavardhana
7b57797455
Merge pull request #275 from harshavardhana/pr_out_add_delimiter_and_prefix_tests_also_add_inmemory_to_support_delimiters
2015-03-08 14:53:12 -07:00
Harshavardhana
8866b2cc5f
Add delimiter and prefix tests, also add inmemory to support delimiters
2015-03-08 14:51:31 -07:00
Harshavardhana
f907068798
Merge pull request #273 from harshavardhana/pr_out_strict_checks_for_delimiter_and_prefix_if_absent_or_present
2015-03-08 13:02:39 -07:00
Harshavardhana
7a01db8005
Strict checks for delimiter and prefix if absent or present
2015-03-08 13:01:39 -07:00
Harshavardhana
44f02058ea
Merge pull request #272 from harshavardhana/pr_out_revert_rfc1123_is_wrong_protocol_format_for_aws
2015-03-08 03:08:02 -07:00
Harshavardhana
ba85ed7566
Revert "RFC1123 is wrong protocol format for AWS"
...
This reverts commit c7b1f8ded8 .
2015-03-08 03:06:23 -07:00
Harshavardhana
54b87b483a
Merge pull request #271 from harshavardhana/pr_out_rfc1123_is_wrong_protocol_format_for_aws
2015-03-08 01:56:44 -08:00
Harshavardhana
c7b1f8ded8
RFC1123 is wrong protocol format for AWS
2015-03-08 01:55:25 -08:00
Harshavardhana
af3037457c
Merge pull request #270 from harshavardhana/pr_out_remove_redundant_uint32
2015-03-07 19:16:47 -08:00
Harshavardhana
9e89af0e3c
Remove redundant uint32
2015-03-07 19:15:51 -08:00
Harshavardhana
9553eb6b4a
Merge pull request #269 from harshavardhana/pr_out_get_object_metadata_needs_to_filter_based_of_prefix_when_delimiter_is_specified
2015-03-07 19:14:41 -08:00
Harshavardhana
f8e2b75e23
Get object metadata needs to filter based of Prefix when delimiter is specified
2015-03-07 19:13:15 -08:00
Frederick F. Kautz IV
b39eac36ea
Merge pull request #268 from fkautz/pr_out_fragment_reader_implemented
2015-03-07 18:04:29 -08:00
Frederick F. Kautz IV
679c6ace3a
Fragment reader implemented
2015-03-07 18:04:20 -08:00
Harshavardhana
df69038b04
Merge pull request #267 from harshavardhana/pr_out_add_md5_support_for_fsstorage_also_re_purpose_getobjectmetadata_for_listobjects
2015-03-07 17:16:00 -08:00
Harshavardhana
4751ed0e30
Add MD5 support for fsStorage, also re-purpose GetObjectMetadata for ListObjects
2015-03-07 17:15:00 -08:00
Frederick F. Kautz IV
1ec0cd6683
Merge pull request #266 from fkautz/pr_out_adding_read_header_for_erasure
2015-03-07 15:54:17 -08:00
Frederick F. Kautz IV
ca1a4b616c
Adding read header for erasure
2015-03-07 15:53:41 -08:00
Frederick F. Kautz IV
00381c887a
Adding erasure package to wrap versioned erasure. Ensures we always write latest version.
2015-03-07 13:04:02 -08:00
Harshavardhana
0658889d33
Merge pull request #264 from harshavardhana/pr_out_clang_lacks_proper_gas_support_implement_stubs_for_sha256_sha512_for_darwin
2015-03-07 02:56:42 -08:00
Harshavardhana
bf5a314144
clang lacks proper gas support, implement stubs for sha256,sha512 for darwin.
...
REF: http://llvm.org/bugs/show_bug.cgi?id=18918
2015-03-07 02:55:48 -08:00
Harshavardhana
43f3866842
Merge pull request #263 from harshavardhana/pr_out_if_incoming_request_ip_bucketfromhostname_needs_to_be_empty_string
2015-03-07 02:53:46 -08:00
Harshavardhana
1637138f86
If incoming request IP bucketfromHostname needs to be empty string
2015-03-07 02:53:29 -08:00
Harshavardhana
15a911c4e1
Merge pull request #262 from harshavardhana/pr_out_rename_test_data_to_testdata_go_will_ignore_it_automatically
2015-03-07 01:33:21 -08:00
Harshavardhana
a18bfa2fda
Rename test-data to testdata, go will ignore it automatically
2015-03-07 01:32:35 -08:00
Harshavardhana
c081b7c011
Merge pull request #261 from harshavardhana/pr_out_move_pkg_storage_erasure_to_pkg_encoding_erasure
2015-03-07 00:18:00 -08:00
Harshavardhana
d992bccd9a
Move pkg/storage/erasure to pkg/encoding/erasure - and other cleanups
2015-03-07 00:16:48 -08:00
Frederick F. Kautz IV
80fa38a41f
Merge pull request #260 from fkautz/pr_out_erasure_layer_now_writes_using_new_technique
2015-03-06 16:37:59 -08:00
Frederick F. Kautz IV
856e0100c0
Erasure layer now writes using new technique
2015-03-06 16:37:44 -08:00
Harshavardhana
15552a59f9
Merge pull request #259 from harshavardhana/pr_out_now_able_to_use_crc32c_and_sha1_on_darwin_with_osx_specific_code
2015-03-06 02:39:14 -08:00
Harshavardhana
856b7680df
Now able to use crc32c and sha1 on Darwin, with OSX specific code
2015-03-06 02:38:29 -08:00
Harshavardhana
46eb2eaeb9
Merge pull request #258 from harshavardhana/pr_out_further_fixes_
2015-03-06 02:06:32 -08:00
Harshavardhana
e5af8a3f5d
Further fixes -
...
- All test files have been renamed to their respective <package>_test name,
this is done in accordance with
- https://github.com/golang/go/wiki/CodeReviewComments#import-dot
imports are largely used in testing, but to avoid namespace collision
and circular dependencies
- Never use _* in package names other than "_test" change fragment_v1 to expose
fragment just like 'gopkg.in/check.v1'
2015-03-06 02:04:21 -08:00
Harshavardhana
02ccf123c9
Merge pull request #257 from harshavardhana/pr_out_update_contributing_guide
2015-03-05 23:39:25 -08:00
Harshavardhana
c00d1461b9
Golint cleanup from top level
2015-03-05 23:37:37 -08:00
Harshavardhana
540723d7ae
Golint cleanup pkg/api
2015-03-05 21:09:19 -08:00
Harshavardhana
76e601b26b
Golint cleanup pkg/storage
2015-03-05 21:09:19 -08:00
Harshavardhana
256faddab5
Golint cleanup utils/crypto/sha*,cpu,md5
2015-03-05 21:09:19 -08:00
Harshavardhana
66e31445ff
Golint cleanup pkg/utils/crypto/keys
2015-03-05 21:09:19 -08:00
Harshavardhana
b33e2d2f9b
Golint cleanup pkg/utils/config
2015-03-05 21:09:19 -08:00
Harshavardhana
861f13d5bd
Golint cleanup pkg/utils/cpu
2015-03-05 21:09:19 -08:00
Harshavardhana
c36450a83a
Golint cleanup pkg/utils/policy
2015-03-05 21:09:19 -08:00
Harshavardhana
3e321b6631
Golint cleanup pkg/utils/split
2015-03-05 21:09:19 -08:00
Frederick F. Kautz IV
237921338a
Merge pull request #256 from fkautz/pr_out_fragment_now_implements_new_version
2015-03-05 20:56:13 -08:00
Frederick F. Kautz IV
1186c47603
Fragment now implements new version
2015-03-05 20:56:02 -08:00
Frederick F. Kautz IV
e6db550fdc
Merge pull request #255 from fkautz/pr_out_renaming_donut_packages_to_match_layout
2015-03-05 19:44:56 -08:00
Frederick F. Kautz IV
665af44c70
Renaming donut packages to match layout
2015-03-05 19:44:01 -08:00
Frederick F. Kautz IV
15b26c5134
Merge pull request #254 from fkautz/pr_out_removing_innerxml
2015-03-04 14:25:18 -08:00
Frederick F. Kautz IV
6ec7894312
Removing innerxml
2015-03-04 14:24:56 -08:00
Frederick F. Kautz IV
862fc31d56
Merge pull request #253 from fkautz/pr_out_updating_godeps_goversion_and_packages
2015-03-04 12:52:25 -08:00
Frederick F. Kautz IV
8d40f60e7d
Updating godeps GoVersion and Packages
2015-03-04 12:52:02 -08:00
Frederick F. Kautz IV
5155b2f124
Merge pull request #252 from fkautz/pr_out_fixing_xml_tags
...
Fixing xml tags
2015-03-04 12:51:22 -08:00
Frederick F. Kautz IV
6f7840677a
Fixing xml tags
2015-03-04 11:02:37 -08:00
Harshavardhana
4bdfe61701
Merge pull request #251 from harshavardhana/pr_out_handle_individual_objects_too
2015-03-04 01:54:44 -08:00
Harshavardhana
f2e238199b
Handle individual objects too
2015-03-04 01:54:35 -08:00
Harshavardhana
cb46c61be4
Merge pull request #250 from harshavardhana/pr_out_add_proper_path_prefix_to_be_delimited
2015-03-04 00:57:00 -08:00
Harshavardhana
e400e85639
Add proper path prefix to be delimited
2015-03-04 00:54:39 -08:00
Harshavardhana
df135d7307
Merge pull request #249 from harshavardhana/pr_out_pathprefix_is_filtered_out_properly_and_delimited_as_well
2015-03-03 23:08:30 -08:00
Harshavardhana
cc46475c86
Pathprefix is filtered out properly and delimited as well
2015-03-03 19:45:54 -08:00
Frederick F. Kautz IV
ae9a0ea6fb
Merge pull request #248 from fkautz/pr_out_renaming_package_to_gopackage_to_fix_golint_error
2015-03-03 12:57:06 -08:00
Frederick F. Kautz IV
34c1e58332
Renaming Package to goPackage to fix golint error
2015-03-03 12:56:19 -08:00
Frederick F. Kautz IV
3e4509efeb
Merge pull request #247 from fkautz/pr_out_renaming_variable_to_pass_golint
2015-03-03 12:01:41 -08:00
Frederick F. Kautz IV
f4eed73d4e
Renaming variable to pass golint
2015-03-03 12:01:14 -08:00
Harshavardhana
a2bdc9df7e
Merge pull request #246 from harshavardhana/pr_out_janitor_duty_v2
2015-03-03 02:44:42 -08:00
Harshavardhana
0ebaac3594
Janitor duty v2
2015-03-03 02:43:24 -08:00
Harshavardhana
3db32f4935
Merge pull request #245 from harshavardhana/pr_out_force_exported_functions_to_have_comments
2015-03-03 02:41:24 -08:00
Harshavardhana
3a3c8645fc
Update comments across the codebase
2015-03-03 02:39:38 -08:00
Harshavardhana
137584d658
Add comments
2015-03-03 01:25:45 -08:00
Harshavardhana
721ec9ef5a
Force exported functions to have comments
2015-03-03 01:01:38 -08:00
Harshavardhana
3d158b1240
Merge pull request #244 from harshavardhana/pr_out_add_deadcode_code_which_recursivley_goes_into_all_directories_and_verifies_dangling_variables
2015-03-02 18:44:34 -08:00
Harshavardhana
c3ad0906e0
Add deadcode code which recursivley goes into all directories and verifies dangling variables.
2015-03-02 18:44:20 -08:00
Frederick F. Kautz IV
022b89dd9b
Merge pull request #243 from fkautz/pr_out_renaming_appendustr_to_appenduniqstr
2015-03-01 17:06:19 -08:00
Frederick F. Kautz IV
32f8c17174
Renaming AppendUstr to AppendUniqStr
2015-03-01 17:06:00 -08:00
Anand Babu (AB) Periasamy
81ede6c3fe
Merge pull request #241 from abperiasamy/janitor
...
On janitor duty.
2015-03-01 16:43:04 -08:00
Frederick F. Kautz IV
7ee3017a1c
Merge pull request #242 from fkautz/pr_out_renaming_appenduint_to_appenduniqint
2015-03-01 16:39:55 -08:00
Frederick F. Kautz IV
de6131cc7c
Renaming AppendUInt to AppendUniqInt
2015-03-01 16:39:45 -08:00
Anand Babu (AB) Periasamy
d9c9634138
removed custom Assert
2015-03-01 15:28:42 -08:00
Frederick F. Kautz IV
60454c53a2
Merge pull request #239 from fkautz/pr_out_adding_test_at_byte_level
2015-03-01 15:15:21 -08:00
Frederick F. Kautz IV
6f300c9ef0
Adding test at byte level
2015-03-01 15:13:09 -08:00
Anand Babu (AB) Periasamy
177121868b
removed HomeDir
2015-03-01 15:13:08 -08:00
Frederick F. Kautz IV
cd22304afb
Merge pull request #238 from fkautz/pr_out_specifying_more_exact_types_for_data
2015-03-01 14:43:20 -08:00
Frederick F. Kautz IV
0b08312ddd
Specifying more exact types for data
2015-03-01 14:43:11 -08:00
Harshavardhana
ed4bca240f
Merge pull request #237 from harshavardhana/pr_out_add_tests_add_newheader_for_populating_header_and_validateheader_
2015-03-01 13:21:26 -08:00
Harshavardhana
9f43659135
Add tests, add NewHeader() for populating Header and ValidateHeader()
2015-03-01 13:19:06 -08:00
Anand Babu (AB) Periasamy
6515678e87
Merge pull request #236 from fkautz/pr_out_adding_data_writer
...
Adding data writer
2015-03-01 11:36:35 -08:00
Frederick F. Kautz IV
7414dc9b75
Adding data writer
2015-03-01 11:32:30 -08:00
Harshavardhana
359af56397
Merge pull request #235 from harshavardhana/pr_out_move_appendustr_appenduint_into_helpers
2015-03-01 11:19:40 -08:00
Harshavardhana
eb9a4a83ac
Move AppendUstr(), AppendUint() into helpers
2015-03-01 11:19:03 -08:00
Frederick F. Kautz IV
9fb67c4195
Merge pull request #234 from fkautz/pr_out_changing_k_and_m_in_data_go_to_match_new_datatype_in_erasure_package
2015-03-01 01:28:43 -08:00
Frederick F. Kautz IV
be12a6b3d9
Changing K and M in data.go to match new datatype in erasure package
2015-03-01 01:28:33 -08:00
Harshavardhana
33601ff567
Merge pull request #233 from harshavardhana/pr_out_make_k_m_to_be_uint8_and_technique_becomes_its_own_type
2015-03-01 01:26:42 -08:00
Harshavardhana
b77c62e25d
Make K,M to be uint8 and Technique becomes its own type
2015-03-01 01:25:28 -08:00
Frederick F. Kautz IV
c290b360e4
Merge pull request #232 from fkautz/pr_out_making_changes_to_data_based_on_discussion
2015-03-01 01:18:11 -08:00
Frederick F. Kautz IV
bb0441917c
Making changes to data based on discussion
2015-03-01 01:16:22 -08:00
Harshavardhana
1f574d19fc
Merge pull request #231 from harshavardhana/pr_out_avoid_allocating_1gig_memory_avoid_such_mistakes
2015-03-01 00:44:05 -08:00
Harshavardhana
2f04a1299e
Avoid allocating 1gig memory, avoid such mistakes
2015-03-01 00:43:12 -08:00
Frederick F. Kautz IV
91c48e8d73
Merge pull request #230 from fkautz/pr_out_adding_data_go_does_nothing_for_now
2015-03-01 00:28:15 -08:00
Frederick F. Kautz IV
2916b1278e
Adding data.go, does nothing for now.
2015-03-01 00:27:52 -08:00
Frederick F. Kautz IV
9d30475f00
Merge pull request #229 from fkautz/pr_out_donut_frame_is_now_its_own_package
2015-03-01 00:02:35 -08:00
Frederick F. Kautz IV
b4ef522a9b
Donut frame is now its own package
2015-03-01 00:02:14 -08:00
Frederick F. Kautz IV
97e572f318
Merge pull request #228 from fkautz/pr_out_renaming_write_to_writeframe
2015-02-28 23:53:06 -08:00
Frederick F. Kautz IV
6c2428833a
Renaming Write to WriteFrame
2015-02-28 23:52:56 -08:00
Harshavardhana
8ab0cc29f2
Merge pull request #227 from harshavardhana/pr_out_add_donut_benchmark_for_64mb_128mb_256mb_512mb
2015-02-28 17:06:40 -08:00
Harshavardhana
6b397844b9
Add donut benchmark for 64MB, 128MB, 256MB, 512MB
...
Current average values
~~~
> minio/pkg/storage/donut/v1 *> go test -bench .
OK: 2 passed
PASS
BenchmarkDonut64M 5 266249634 ns/op 252.05 MB/s
BenchmarkDonut128M 3 436507394 ns/op 307.48 MB/s
BenchmarkDonut256M 2 836116359 ns/op 321.05 MB/s
BenchmarkDonut512M 1 1676738951 ns/op 320.19 MB/s
ok github.com/minio-io/minio/pkg/storage/donut/v1 8.430s
> minio/pkg/storage/donut/v1 *>
~~~
2015-02-28 17:05:46 -08:00
Harshavardhana
24c5ab70be
Merge pull request #226 from harshavardhana/pr_out_donut_gen_works_with_new_donut_frame
2015-02-28 16:30:34 -08:00
Harshavardhana
65f22083ca
donut_gen works with new Donut Frame
2015-02-28 16:30:18 -08:00
Frederick F. Kautz IV
4d98c355c9
Merge pull request #224 from abperiasamy/units
2015-02-28 16:12:10 -08:00
Frederick F. Kautz IV
a067fdb59d
Merge pull request #225 from fkautz/pr_out_donut_now_tests_amount_written_should_match_expected_amount_to_write
2015-02-28 16:10:09 -08:00
Frederick F. Kautz IV
ddc7cf835e
Donut now tests amount written should match expected amount to write
2015-02-28 16:09:52 -08:00
Frederick F. Kautz IV
a86e5b5586
Merge pull request #223 from fkautz/pr_out_updating_comment_describing_data_type
2015-02-28 16:00:12 -08:00
Frederick F. Kautz IV
d3d1205eb1
Updating comment describing data type
2015-02-28 15:59:52 -08:00
Frederick F. Kautz IV
c2ae974c94
Merge pull request #222 from fkautz/pr_out_new_donut_frame_implemented
2015-02-28 15:54:45 -08:00
Frederick F. Kautz IV
58a04ee831
New donut frame implemented
2015-02-28 15:54:13 -08:00
Anand Babu (AB) Periasamy
317a1141c0
removed unitconv
2015-02-28 15:53:44 -08:00
Frederick F. Kautz IV
9744efc725
Merge pull request #221 from fkautz/pr_out_adding_streaming_sha512
2015-02-28 15:52:10 -08:00
Frederick F. Kautz IV
90cd8b7b47
Adding streaming sha512
2015-02-28 15:51:33 -08:00
Harshavardhana
56bd413a16
Merge pull request #219 from harshavardhana/pr_out_add_1mb_block_benchmarks
2015-02-28 15:25:24 -08:00
Harshavardhana
7feca3f0d1
Add 1MB block benchmarks
2015-02-28 15:00:24 -08:00
Harshavardhana
20dc738b67
Merge pull request #218 from harshavardhana/pr_out_implement_proper_delimiter_and_prefix_handling
2015-02-28 14:48:46 -08:00
Harshavardhana
0c2d58bc6d
Implement proper delimiter and prefix handling
...
With this change Minio server now responds with, delimited
'object names' in conjunction with prefix filtering
~~~
<ListBucketResult>
<Name>example-bucket</Name>
<Prefix></Prefix>
<Marker></Marker>
<MaxKeys>1000</MaxKeys>
<Delimiter>/</Delimiter>
<IsTruncated>false</IsTruncated>
<Contents>
<Key>sample.html</Key>
<LastModified>2011-02-26T01:56:20.000Z</LastModified>
<ETag>example-bucket#sample.html</ETag>
<Size>142863</Size>
<Owner>
<ID>minio</ID>
<DisplayName>minio</DisplayName>
</Owner>
<StorageClass>STANDARD</StorageClass>
</Contents>
<CommonPrefixes>
<Prefix>photos/</Prefix>
</CommonPrefixes>
</ListBucketResult>
~~~
~~~
<ListBucketResult>
<Name>example-bucket</Name>
<Prefix>photos/2006/</Prefix>
<Marker></Marker>
<MaxKeys>1000</MaxKeys>
<Delimiter>/</Delimiter>
<IsTruncated>false</IsTruncated>
<CommonPrefixes>
<Prefix>photos/2006/feb/</Prefix>
</CommonPrefixes>
<CommonPrefixes>
<Prefix>photos/2006/jan/</Prefix>
</CommonPrefixes>
</ListBucketResult>
~~~
2015-02-28 14:47:19 -08:00
Harshavardhana
acee4d16fd
Merge pull request #217 from harshavardhana/pr_out_use_validstring
2015-02-27 19:49:33 -08:00
Harshavardhana
d5e5a3b3e6
Use validString
2015-02-27 19:49:18 -08:00
Harshavardhana
27fab98bf9
Merge pull request #216 from harshavardhana/pr_out_add_object_name_validation
2015-02-27 19:42:14 -08:00
Harshavardhana
6ebb48b4ea
Add object name validation
2015-02-27 19:42:04 -08:00
Harshavardhana
e7992320c1
Merge pull request #215 from harshavardhana/pr_out_sort_list_of_objects
2015-02-27 17:53:19 -08:00
Harshavardhana
9acb1f728c
Sort list of objects
2015-02-27 17:52:55 -08:00
Harshavardhana
3a048dc676
Merge pull request #208 from harshavardhana/pr_out_implement_delimiter_and_path_prefix
2015-02-27 16:24:15 -08:00
Harshavardhana
53669a0854
Implement delimiter, path prefix
2015-02-27 16:23:05 -08:00
Harshavardhana
01e84c596c
Merge pull request #214 from harshavardhana/pr_out_remove_donut_gen_binary_and_update_crc32c_function
2015-02-27 14:42:22 -08:00
Harshavardhana
3f8b3462e6
Remove donut_gen binary and update crc32c function
2015-02-27 14:41:53 -08:00
Harshavardhana
9aea7fd97b
Merge pull request #213 from harshavardhana/pr_out_crc32c_re_implementation_with_convenience_functions
2015-02-27 14:35:11 -08:00
Harshavardhana
8ce9b84b69
Crc32c re-implementation with Convenience functions
...
- Sum32([]byte) --> uint32
- Sum(io.Reader) --> uint32, error
2015-02-27 13:39:17 -08:00
Frederick F. Kautz IV
2e37791681
Merge pull request #212 from fkautz/pr_out_adding_comparison_for_footer_s_inim
2015-02-27 12:40:10 -08:00
Frederick F. Kautz IV
a42872d557
Merge pull request #211 from fkautz/pr_out_moving_donut_gen_to_its_own_dir_and_renaming_it_as_donut_gen_v1
2015-02-27 12:40:07 -08:00
Frederick F. Kautz IV
1f7a3c13a5
Adding comparison for footer's INIM
2015-02-27 12:39:52 -08:00
Frederick F. Kautz IV
2ca62f13a5
Moving donut_gen to its own dir and renaming it as donut_gen_v1
2015-02-27 12:39:14 -08:00
Frederick F. Kautz IV
22ddc3dcbb
Merge pull request #210 from fkautz/pr_out_seek_to_beginning_for_tempbuffer
2015-02-27 12:18:58 -08:00
Frederick F. Kautz IV
be768d01a2
Seek to beginning for tempbuffer
2015-02-27 12:18:27 -08:00
Frederick F. Kautz IV
da10fd5a83
Merge pull request #209 from fkautz/pr_out_adding_crc32c_to_write
2015-02-26 23:04:57 -08:00
Frederick F. Kautz IV
5c52da6ebe
Adding crc32c to write
2015-02-26 23:04:07 -08:00
Anis Elleuch
e4b6239277
Add Dockerfile to build a docker image with embarked minio server
2015-02-26 22:53:30 +01:00
Harshavardhana
1786408bcc
Merge pull request #206 from harshavardhana/pr_out_remove_http_server_timeouts_write_and_read
2015-02-24 13:56:56 -08:00
Harshavardhana
a9aaff0ee0
Remove HTTP server timeouts write and read.
...
Implement it later with a proper timer thread. Large file
transfers from 'mc' fail with i/o timeout without this change.
2015-02-24 13:55:24 -08:00
Frederick F. Kautz IV
e0d78e7381
Merge pull request #205 from fkautz/pr_out_making_donut_use_io_readwriteseek_instead_of_io_writer
2015-02-23 20:27:37 -08:00
Frederick F. Kautz IV
ef442289da
Making donut use io.ReadWriteSeek instead of io.Writer
2015-02-23 20:27:25 -08:00
Harshavardhana
0f45e17505
Merge pull request #204 from harshavardhana/pr_out_add_bitdeli_badge_and_expose_analytics_beacon
2015-02-23 18:06:19 -08:00
Harshavardhana
3348b79eb8
Add bitdeli badge and expose Analytics beacon
2015-02-23 18:05:58 -08:00
Harshavardhana
182d614bdd
Merge pull request #203 from harshavardhana/pr_out_more_updates_on_documentation
2015-02-23 17:45:16 -08:00
Harshavardhana
5e1e5ad786
More updates on documentation
2015-02-23 17:44:55 -08:00
Harshavardhana
2b8adef454
Merge pull request #202 from harshavardhana/pr_out_update_minioapi_documentation
2015-02-23 16:46:57 -08:00
Harshavardhana
b7868d23ac
Update minioapi documentation
2015-02-23 16:46:48 -08:00
Harshavardhana
0c36dc24f5
Merge pull request #201 from harshavardhana/pr_out_update_erasure_documentation
2015-02-23 13:39:30 -08:00
Harshavardhana
f4b5519451
Update erasure documentation
2015-02-23 13:39:16 -08:00
Harshavardhana
3b3e446102
Merge pull request #200 from abperiasamy/reorg
...
Reorg
2015-02-23 13:12:13 -08:00
Anand Babu (AB) Periasamy
6fc11932ec
make test -> test-all
2015-02-23 11:03:40 -08:00
Anand Babu (AB) Periasamy
de41e465c8
move httpserver under server
2015-02-23 11:03:01 -08:00
Harshavardhana
7c608ec857
Merge pull request #199 from harshavardhana/pr_out_fix_a_typo
2015-02-23 02:30:43 -08:00
Harshavardhana
baa1719ed2
Fix a typo
2015-02-23 02:30:36 -08:00
Harshavardhana
40014b8391
Merge pull request #198 from harshavardhana/pr_out_add_domain_and_subdomain_support_for_minioapi
2015-02-23 02:28:18 -08:00
Harshavardhana
51e80eaa6d
Add domain and subdomain support for MinioAPI
...
This change brings in domain and subdomain support
- ./minio --domain "yourminiodomain.com"
This change brings in a much needed feature by keeping
bucketnames as part of your 'DNS' name.
All your existing applications can be migrated off from s3 to
Minio without little to no modifications.
NOTE: Setting up DNS for your `buckets` is out of scope of this feature
2015-02-23 02:25:01 -08:00
Harshavardhana
2d3b00b831
Merge pull request #197 from harshavardhana/pr_out_purge_tiedot_wrapper
2015-02-22 20:04:42 -08:00
Harshavardhana
a92c6cb2b2
Purge tiedot wrapper
2015-02-22 20:02:09 -08:00
Harshavardhana
8d9613646f
Merge pull request #196 from harshavardhana/pr_out_remove_setup_development_environment_script
2015-02-21 21:39:37 -08:00
Harshavardhana
a3fe948c8b
Remove setup development environment script
2015-02-21 21:38:04 -08:00
Harshavardhana
87bca64173
Merge pull request #195 from harshavardhana/pr_out_add_license_header_for_minio_modifications_improvements_for_sha1_sha256_sha512_implementations
2015-02-21 15:54:02 -08:00
Harshavardhana
49a714d1a4
Add license header for Minio modifications, improvements for sha1,sha256,sha512 implementations
...
Also bring in SSE3 optimized public domain implementation of SHA1 from Intel ``FIPS PUB 180-1``
2015-02-21 15:50:47 -08:00
Harshavardhana
cfb1fe7f9d
Merge pull request #194 from harshavardhana/pr_out_implement_sha1_as_intel_optimized_set
2015-02-21 11:27:07 -08:00
Harshavardhana
12cff1be58
Implement sha1 as intel optimized set
2015-02-21 11:25:09 -08:00
Harshavardhana
e2bc6c8b27
Merge pull request #193 from vadmeste/fix_sha256_faulty_computation
...
Add missing asm instruction to correct sha256 calculation in avx mode
2015-02-21 10:49:49 -08:00
Anis Elleuch
1d4d7ff274
Add missing asm instruction to correct sha256 calculation in avx mode
2015-02-21 19:34:36 +01:00
Harshavardhana
121379c992
Merge pull request #192 from vadmeste/add_intel_sha256_optimized_algo
...
Use Intel optimized code (ssse3, avx, avx2) to calculate sha256
2015-02-21 08:35:59 -08:00
Anis Elleuch
e89271b8f5
Use Intel optimized code (ssse3, avx, avx2) to calculate sha256
2015-02-21 13:11:15 +01:00
Harshavardhana
3d70b96ae9
Merge pull request #191 from harshavardhana/pr_out_add_proper_entries_for_gobheader_data_copy
2015-02-21 01:45:41 -08:00
Harshavardhana
3151f9b6c3
Add proper entries for GobHeader, Data copy.
...
This commit also adds 'donut_gen.go' for generating sample donut file
2015-02-21 01:44:27 -08:00
Frederick F. Kautz IV
4db9b5a348
Merge pull request #189 from fkautz/pr_out_writing_data_out_with_error_handling
2015-02-20 23:24:11 -08:00
Frederick F. Kautz IV
8129d8fd0b
Writing data out with error handling
2015-02-20 23:24:01 -08:00
Frederick F. Kautz IV
97b2e7fb3b
Merge pull request #188 from fkautz/pr_out_moving_donut_code_to_v1_in_donut_package
2015-02-20 23:07:50 -08:00
Frederick F. Kautz IV
98ddbd840e
Moving donut code to v1 in donut package
2015-02-20 23:07:43 -08:00
Frederick F. Kautz IV
005ad1a9e1
Merge pull request #187 from fkautz/pr_out_fixing_magic_number_for_inim
2015-02-20 22:43:19 -08:00
Frederick F. Kautz IV
9833647fb7
Fixing magic number for INIM
2015-02-20 22:42:58 -08:00
Frederick F. Kautz IV
339fa40b21
Merge pull request #186 from abperiasamy/donut-magic
2015-02-20 19:15:11 -08:00
Anand Babu (AB) Periasamy
ec36a598c9
Magic as numbers are faster to encode and compare
2015-02-20 18:20:21 -08:00
Harshavardhana
eab19e3ce4
Merge pull request #185 from harshavardhana/pr_out_remove_go_cover
2015-02-20 17:44:46 -08:00
Harshavardhana
741e14f749
Remove go cover and remove unncessary Makefiles
2015-02-20 17:44:26 -08:00
Harshavardhana
afe7293aef
Merge pull request #184 from harshavardhana/pr_out_sha512_implemention_with_intel_assembly_code
2015-02-20 16:34:01 -08:00
Harshavardhana
19a4998fcb
SHA512 Implemention with Intel assembly code
2015-02-20 16:32:19 -08:00
Harshavardhana
a552800dcc
Merge pull request #183 from harshavardhana/pr_out_osx_compatibility_update_for_checkdeps_sh
2015-02-20 00:30:31 -08:00
Harshavardhana
cdd43a11be
OSX compatibility update for checkdeps.sh
2015-02-20 00:28:35 -08:00
Harshavardhana
83c636c80c
Merge pull request #182 from harshavardhana/pr_out_add_donut_tests_just_for_verification
2015-02-18 19:46:39 -08:00
Harshavardhana
c4f0aad9b5
Add donut tests, just for verification
2015-02-18 19:46:14 -08:00
Frederick F. Kautz IV
aa2c622f52
Merge pull request #181 from fkautz/pr_out_donut_structure_refined
2015-02-18 17:18:44 -08:00
Frederick F. Kautz IV
f7f34d08e8
Donut structure refined
2015-02-18 17:18:33 -08:00
Frederick F. Kautz IV
ab80c0ce12
Merge pull request #180 from abperiasamy/donut
2015-02-18 15:57:16 -08:00
Anand Babu (AB) Periasamy
9e304b53de
donut spec v1 prototype
2015-02-18 15:52:12 -08:00
Harshavardhana
49de6e0605
Merge pull request #179 from harshavardhana/pr_out_remove_govet_dependency_for_now
2015-02-18 15:51:42 -08:00
Harshavardhana
0a1a47da1b
remove govet dependency for now
2015-02-18 15:51:31 -08:00
Harshavardhana
1dc14bccfe
Merge pull request #178 from harshavardhana/pr_out_run_govet_and_fix
2015-02-18 15:34:26 -08:00
Harshavardhana
fab954f13f
Run govet and fix
2015-02-18 15:33:55 -08:00
Harshavardhana
1b1371ffec
Merge pull request #177 from harshavardhana/pr_out_no_symlink_fail_when_not_in_gopath
2015-02-18 14:36:33 -08:00
Harshavardhana
79455c3f9d
No symlink, fail when not in GOPATH
2015-02-18 14:35:49 -08:00
Harshavardhana
2bc7ded828
Merge pull request #176 from harshavardhana/pr_out_some_more_cleanup
2015-02-18 12:16:34 -08:00
Harshavardhana
f7a06a5d1e
Some more cleanup
2015-02-18 12:15:33 -08:00
Frederick F. Kautz IV
784ab9a551
Merge pull request #175 from fkautz/pr_out_adding_header_version
2015-02-18 10:21:40 -08:00
Frederick F. Kautz IV
05ac09bf69
Adding header version
2015-02-18 10:21:02 -08:00
Harshavardhana
c212ae18bc
Merge pull request #174 from harshavardhana/pr_out_no_need_to_check_pip_fixes_172
2015-02-18 01:24:10 -08:00
Harshavardhana
2a5183ff8a
No need to check pip fixes #172
2015-02-18 01:23:07 -08:00
Harshavardhana
d8e0be7575
Merge pull request #173 from harshavardhana/pr_out_make_channels_more_aware_of_errors_and_shutdown_on_error_some_cleanup
2015-02-18 00:24:35 -08:00
Harshavardhana
a3e5121f7b
Make channels more aware of errors and shutdown on error - some cleanup
2015-02-18 00:24:19 -08:00
Harshavardhana
4bdbfe47a2
Merge pull request #171 from harshavardhana/pr_out_expose_policy_code_for_api_router_usage
2015-02-17 19:06:14 -08:00
Harshavardhana
cf6d03b907
Expose policy code, for api router usage
2015-02-17 19:06:01 -08:00
Harshavardhana
5235badec9
Merge pull request #169 from harshavardhana/pr_out_add_policy_validation_code_more_rigorous
2015-02-17 18:10:00 -08:00
Harshavardhana
650b3bac9b
Merge pull request #168 from vadmeste/fix_devscripts_headers
2015-02-17 18:09:51 -08:00
Frederick F. Kautz IV
2af664eeaf
Merge pull request #170 from fkautz/pr_out_adding_scaffolding_for_writing_donut
2015-02-17 17:51:54 -08:00
Frederick F. Kautz IV
fe9a5c14a0
Adding scaffolding for writing donut
2015-02-17 17:47:10 -08:00
Harshavardhana
980d229272
Add policy validation code more rigorous
2015-02-17 13:23:16 -08:00
Anis Elleuch
04075b15c9
Fix Shebang position
...
Add licence to setup_development_environment.sh
2015-02-17 19:17:31 +01:00
Harshavardhana
7882c8c746
Merge pull request #167 from harshavardhana/pr_out_add_new_error_response
2015-02-16 20:44:41 -08:00
Harshavardhana
2696d2f4e7
Add new error response
2015-02-16 20:44:25 -08:00
Harshavardhana
45866ff3a7
Merge pull request #166 from harshavardhana/pr_out_skip_policy_files
2015-02-16 18:37:18 -08:00
Harshavardhana
f04beab7c2
Skip policy files
2015-02-16 18:37:09 -08:00
Harshavardhana
47a5dd7e72
Merge pull request #165 from vadmeste/more_check_for_call_results
2015-02-16 18:30:28 -08:00
Harshavardhana
4ce180bb72
Merge pull request #164 from harshavardhana/pr_out_implement_bucket_policy_handler
2015-02-16 18:28:48 -08:00
Harshavardhana
eeae64935e
Implement bucket policy handler and with galore of cleanup
2015-02-16 18:20:40 -08:00
Anis Elleuch
dc551e6c2e
Check function return values for downloading operations
...
Fix two regressions after changing the behavior of check_version
call function returns the status code of the first part of the pipe command
Fix indentation
Add more error msgs when git fails
MINIO_DEV takes the directory path where the env.sh is found
2015-02-16 22:29:49 +01:00
Harshavardhana
7d8c34e055
Merge pull request #163 from harshavardhana/pr_out_add_osx_deps_check_and_development_environment
2015-02-15 15:20:34 -08:00
Harshavardhana
574ee2a271
Add osx deps check and development environment
2015-02-15 14:48:11 -08:00
Harshavardhana
6af37d5ca1
Merge pull request #162 from vadmeste/installer_script_enhancements
2015-02-15 14:19:37 -08:00
Anis Elleuch
f95b954b9a
Add INT signal handler to accelerate quitting when asked by the user
...
Print the required version of the missing packages
Check the status code of curl download and abort when failed
2015-02-15 23:05:10 +01:00
Harshavardhana
be9f59c555
Merge pull request #161 from harshavardhana/pr_out_add_license_and_fix_development_scripts
2015-02-15 02:01:35 -08:00
Harshavardhana
ba3e2d41f7
Add license and fix development scripts
2015-02-15 01:59:48 -08:00
Harshavardhana
78755aa1e0
Merge pull request #160 from harshavardhana/pr_out_move_to_devscripts_to_avoid_top_level_dir_clutter
2015-02-14 17:29:38 -08:00
Harshavardhana
0d91e7d540
Move to devscripts to avoid top level dir clutter
2015-02-14 17:28:55 -08:00
Harshavardhana
5a2616ca47
Merge pull request #159 from harshavardhana/pr_out_further_changes_to_minio_dev_script
2015-02-14 17:23:09 -08:00
Harshavardhana
6b85cb1168
Further changes to minio dev script
2015-02-14 17:21:29 -08:00
Harshavardhana
58dcd5751e
Merge pull request #156 from vadmeste/add_minio_env_installer
2015-02-14 17:12:15 -08:00
Anis Elleuch
6e30ccd612
* Add a development environnment installer for minio project with an automatic dependency installation
...
* Install minio in /home/vadmeste/minio-dev by default
* Check for host architecutre before installing
* Check the minio dev directory before start
* env.sh is always in the root of minio-dev directory, use base dirname instead of hardcoded path
* Explain why we use main function in the script
* env.sh should not mess with GOROOT variable if already defined
* Force the user to set GOROOT var if golang is already installed
* Prettify the script output
* Hide pushd/popd output
2015-02-14 22:06:29 +01:00
Harshavardhana
685077eaa5
Merge pull request #158 from harshavardhana/pr_out_there_is_no_golang_version_1_4_0
2015-02-13 19:10:25 -08:00
Harshavardhana
cce10456d5
There is no golang version 1.4.0
2015-02-13 19:10:15 -08:00
Harshavardhana
be1b50247a
Merge pull request #157 from harshavardhana/pr_out_reorg_docs_and_cleanup
2015-02-13 19:07:06 -08:00
Harshavardhana
f07ccb035d
Reorg docs and cleanup
2015-02-13 19:06:47 -08:00
Harshavardhana
a527251865
Merge pull request #155 from harshavardhana/pr_out_update_docs
2015-02-13 15:26:48 -08:00
Harshavardhana
bd56b6e571
Update docs
2015-02-13 15:26:26 -08:00
Harshavardhana
16c520639f
Merge pull request #154 from harshavardhana/pr_out_write_error_responses_documentation
2015-02-13 15:21:16 -08:00
Harshavardhana
5a0a700a45
Write Error Responses documentation
2015-02-13 15:20:58 -08:00
Harshavardhana
08c6717f88
Merge pull request #153 from vadmeste/check_for_golang_version
2015-02-13 02:59:16 -08:00
Anis Elleuch
262d8cc23a
Check if the installed golang version is greater than 1.4.0
2015-02-13 10:09:15 +01:00
Anis Elleuch
2ef1df016b
Add a generic version checker function
2015-02-13 10:08:52 +01:00
Harshavardhana
e0b9a2f9ef
Merge pull request #152 from harshavardhana/pr_out_reply_in_xml_from_validate_and_ignore_handlers
2015-02-11 22:00:53 -08:00
Harshavardhana
966cdd4f05
Reply in xml from validate and ignore handlers
2015-02-11 22:00:45 -08:00
Harshavardhana
8a562ebbef
Merge pull request #151 from harshavardhana/pr_out_implement_s3_style_errorcodes_and_response
2015-02-11 21:28:22 -08:00
Harshavardhana
ac4f07906c
Implement S3 Style ErrorCodes and Response
...
This patchset also brings in lot of cleanup in terms of minioapi codebase
2015-02-11 21:27:28 -08:00
Harshavardhana
6e73ccec75
Merge pull request #149 from harshavardhana/pr_out_update_contributors_document
2015-02-09 13:34:34 -08:00
Harshavardhana
0ac611e452
Update contributors document
2015-02-09 13:34:19 -08:00
Harshavardhana
b92ce76197
Merge pull request #148 from harshavardhana/pr_out_update_docs
2015-02-08 17:51:00 -08:00
Harshavardhana
b12a493db8
Update Docs
2015-02-08 17:50:18 -08:00
Harshavardhana
4f43e37a73
Merge pull request #147 from harshavardhana/pr_out_restructure_docs_and_move_mkdocs_yml_to_top_level
2015-02-08 17:41:22 -08:00
Harshavardhana
d036bf3a2a
Restructure docs and move mkdocs.yml to top-level
2015-02-08 17:40:39 -08:00
Harshavardhana
6911310661
Merge pull request #146 from harshavardhana/pr_out_change_ports_to_9000_and_9001
2015-02-08 14:11:50 -08:00
Harshavardhana
d9a97b6c86
Change ports to 9000 and 9001
2015-02-08 14:10:59 -08:00
Harshavardhana
2b1b4f2713
Merge pull request #145 from harshavardhana/pr_out_add_license_header
2015-02-08 03:00:55 -08:00
Harshavardhana
680848bdcb
Add license header
2015-02-08 03:00:32 -08:00
Harshavardhana
b2724b80c1
Merge pull request #144 from harshavardhana/pr_out_verify_both_x_amz_date_and_date_header_before_discarding_request
2015-02-08 02:39:03 -08:00
Harshavardhana
8087ca2450
Verify both 'x-amz-date' and 'date' header before discarding request
2015-02-08 02:37:19 -08:00
Harshavardhana
62648c8636
Merge pull request #143 from harshavardhana/pr_out_changes_to_tests_pending_from_142
2015-02-06 20:47:32 -08:00
Harshavardhana
92f83982db
Changes to tests pending from #142
2015-02-06 20:47:21 -08:00
Harshavardhana
58f2c73f42
Merge pull request #142 from harshavardhana/pr_out_avoid_trailing_for_buckets_and_also_re_directs_in_accordance_with_aws_clients
2015-02-06 20:42:26 -08:00
Harshavardhana
ecb70a4c19
Avoid trailing '/' for buckets and also re-directs - in accordance with aws clients
2015-02-06 20:42:09 -08:00
Harshavardhana
a5bf6b7f14
Merge pull request #139 from harshavardhana/pr_out_implement_authorization_support
2015-02-06 02:09:21 -08:00
Harshavardhana
81fc11ee5d
Implement authorization support
2015-02-06 02:08:52 -08:00
Harshavardhana
eba86c07e9
Merge pull request #138 from harshavardhana/pr_out_ignore_eof_in_readconfig_
2015-02-05 16:12:00 -08:00
Harshavardhana
0e8b16a55d
Ignore EOF in ReadConfig()
2015-02-05 16:10:49 -08:00
Harshavardhana
829be023d2
Merge pull request #137 from harshavardhana/pr_out_reply_back_accesskey_secretkey_through_json
2015-02-05 16:00:00 -08:00
Harshavardhana
72d0999dcb
Reply back AccessKey, Secretkey through json
2015-02-05 15:59:44 -08:00
Harshavardhana
95f214c99b
Append to Users list
2015-02-05 15:40:51 -08:00
Harshavardhana
b3b73ff4cb
Merge pull request #136 from harshavardhana/pr_out_store_credentials_through_webui
2015-02-05 14:20:06 -08:00
Harshavardhana
4c90017b02
Store credentials through webui
2015-02-05 14:19:48 -08:00
Frederick F. Kautz IV
71fab1d91a
Merge pull request #135 from fkautz/pr_out_content_type_is_now_passed_from_minioapi_to_storage
2015-02-05 14:05:59 -08:00
Frederick F. Kautz IV
620435cea6
Content type is now passed from minioapi to storage
2015-02-05 14:05:43 -08:00
Harshavardhana
833756138d
Merge pull request #134 from harshavardhana/pr_out_create_the_non_existent_directory
2015-02-05 10:57:55 -08:00
Harshavardhana
19267fa29a
Create the non-existent directory
2015-02-05 10:57:26 -08:00
Frederick F. Kautz IV
549cf6b854
Merge pull request #133 from fkautz/pr_out_adding_content_type_to_backend_storage_drivers
2015-02-04 17:37:13 -08:00
Frederick F. Kautz IV
288bc3fcc7
Adding content type to backend storage drivers
2015-02-04 17:32:40 -08:00
Frederick F. Kautz IV
05665a9111
Merge pull request #132 from fkautz/pr_out_removing_xml_header_from_response_in_docs
2015-02-04 16:40:22 -08:00
Frederick F. Kautz IV
d44e42f809
Removing xml header from response in docs
2015-02-04 16:40:02 -08:00
Frederick F. Kautz IV
e5cd3dcb7c
Merge pull request #131 from fkautz/pr_out_removing_xmlname_from_json_request
2015-02-04 14:22:05 -08:00
Frederick F. Kautz IV
816310825e
Removing XMLName from json request
2015-02-04 14:21:47 -08:00
Frederick F. Kautz IV
ba4f8b3f56
Merge pull request #130 from fkautz/pr_out_adding_json_example_for_get_
2015-02-04 14:04:54 -08:00
Frederick F. Kautz IV
f1bf2cb5d7
Adding JSON Example for GET /
2015-02-04 14:04:41 -08:00
Frederick F. Kautz IV
2a6128a0b8
Merge pull request #129 from fkautz/pr_out_adding_additional_entry_into_bucket_list_in_docs
2015-02-03 17:36:23 -08:00
Frederick F. Kautz IV
eded653fd2
Adding additional entry into bucket list in docs
2015-02-03 17:36:14 -08:00
Frederick F. Kautz IV
833dde6106
Merge pull request #128 from fkautz/pr_out_fixing_symlink_test_to_not_link_if_file_exists
2015-02-03 17:16:16 -08:00
Frederick F. Kautz IV
0646e7ae40
Fixing symlink test to not link if file exists
2015-02-03 17:15:57 -08:00
Frederick F. Kautz IV
e4074f1580
Merge pull request #127 from fkautz/pr_out_accessing_dir_as_object_should_return_objectnotfound
2015-02-03 16:46:55 -08:00
Frederick F. Kautz IV
7152712f34
Accessing dir as object should return ObjectNotFound
2015-02-03 16:45:00 -08:00
Harshavardhana
297552e302
Merge pull request #126 from harshavardhana/pr_out_add_mkdocs_as_build_dependency_in_checkdeps_sh_and_builddeps_md
2015-02-03 00:42:28 -08:00
Harshavardhana
b5d92aaa14
Add mkdocs as build dependency in checkdeps.sh and BUILDDEPS.md
2015-02-03 00:42:16 -08:00
Harshavardhana
3b0869e984
Merge pull request #125 from harshavardhana/pr_out_make_sure_config_testing_is_done_in_tempdir
2015-02-03 00:25:47 -08:00
Harshavardhana
582ce3df36
Make sure config testing is done in tempdir
2015-02-03 00:25:33 -08:00
Harshavardhana
f4cd0aa978
Merge pull request #124 from harshavardhana/pr_out_add_json_config_reader_writer
2015-02-03 00:13:18 -08:00
Harshavardhana
b1db70c807
Add json config reader/writer
2015-02-03 00:12:46 -08:00
Harshavardhana
c26b6f3d76
Merge pull request #123 from harshavardhana/pr_out_use_helper_homedir_
2015-02-02 21:45:37 -08:00
Harshavardhana
aff4c5826b
Use helper HomeDir()
2015-02-02 21:44:17 -08:00
Frederick F. Kautz IV
189e1b5302
Merge pull request #122 from fkautz/pr_out_minor_header_update_renaming_to_get_
2015-02-02 16:23:34 -08:00
Frederick F. Kautz IV
60551dffca
Minor header update, renaming '/' to 'GET /'
2015-02-02 16:23:18 -08:00
Frederick F. Kautz IV
5b5c798dee
Merge pull request #121 from fkautz/pr_out_adding_mkdocs_to_gogenerate
2015-02-02 16:15:24 -08:00
Frederick F. Kautz IV
a79a5c10e0
Adding mkdocs to gogenerate.
2015-02-02 16:14:55 -08:00
Frederick F. Kautz IV
41688e491c
Merge pull request #120 from fkautz/pr_out_adding_minio_api_documentation
2015-02-02 15:42:07 -08:00
Frederick F. Kautz IV
9d4e7a50fc
Adding minio api documentation
2015-02-02 15:41:21 -08:00
Frederick F. Kautz IV
246793e30e
Merge pull request #119 from fkautz/pr_out_initial_mkdocs_configuration
2015-02-02 13:50:34 -08:00
Frederick F. Kautz IV
ba1b7f79ba
Initial mkdocs configuration
2015-02-02 13:50:07 -08:00
Frederick F. Kautz IV
42f5b3dc1d
Removing docs
2015-02-02 13:50:07 -08:00
Harshavardhana
8bedf43a27
Merge pull request #118 from harshavardhana/pr_out_consolidate_developers_docs_and_makefile_create_symlink_automatically
2015-02-01 21:19:07 -08:00
Harshavardhana
88af9ba666
Consolidate developers docs and Makefile, create symlink automatically
2015-02-01 21:18:46 -08:00
Harshavardhana
dd5380fc86
Merge pull request #117 from harshavardhana/pr_out_fix_build_issue
2015-02-01 17:21:55 -08:00
Harshavardhana
dc47ca1431
Fix build issue
2015-02-01 17:21:17 -08:00
Frederick F. Kautz IV
4d5e4cd46f
Merge pull request #116 from fkautz/pr_out_minioapi_testputobject_properly_creates_bucket_at_start_of_test
2015-02-01 14:46:59 -08:00
Frederick F. Kautz IV
a8eef13835
Minioapi TestPutObject properly creates bucket at start of test
2015-02-01 14:46:45 -08:00
Frederick F. Kautz IV
afd574af20
Merge pull request #115 from fkautz/pr_out_adding_test_to_ensure_list_objects_returns_an_error_for_non_existant_buckets_to_match_previous_bug_fix_and_fix_for_inmemory
2015-02-01 10:48:46 -08:00
Frederick F. Kautz IV
a092b44cf1
Adding test to ensure list objects returns an error for non existant buckets to match previous bug fix and fix for inmemory
2015-02-01 10:48:31 -08:00
Harshavardhana
be7c9aacf8
Merge pull request #114 from harshavardhana/pr_out_reply_back_errors_properly
2015-02-01 03:11:39 -08:00
Harshavardhana
5d197dd063
Reply back errors properly
2015-02-01 03:10:28 -08:00
Harshavardhana
862ce975d5
Merge pull request #113 from harshavardhana/pr_out_add_strictslash_for_path_re_direction
2015-02-01 02:49:27 -08:00
Harshavardhana
b635501fb7
Add StrictSlash for path re-direction
2015-02-01 02:49:09 -08:00
Harshavardhana
3ed884e495
Merge pull request #112 from harshavardhana/pr_out_service_a_new_webui_server_at_port_8081
2015-02-01 02:45:21 -08:00
Harshavardhana
e1b3836c60
Service a new WebUI server at port 8081.
...
It is provided to avoid the router clutter on the Object API
end. This is just a first cut sample we are going to build on
top of this the new management console UI.
General interest comes from subsequently writing a small UI to
generate `AccessKey` and `SecretKey` to build authorization
layer.
2015-02-01 02:44:53 -08:00
Frederick F. Kautz IV
392ca7451f
Merge pull request #111 from fkautz/pr_out_setting_order_when_retrieving_buckets
2015-02-01 00:17:21 -08:00
Frederick F. Kautz IV
54026dbc78
Setting order when retrieving buckets
2015-02-01 00:16:25 -08:00
Harshavardhana
c3287b7ad0
Merge pull request #110 from harshavardhana/pr_out_add_bucket_resources_to_ignore_handler
2015-01-31 22:47:37 -08:00
Harshavardhana
26aac9a439
Add bucket resources to ignore handler
2015-01-31 22:45:52 -08:00
Frederick F. Kautz IV
39250b7c49
Merge pull request #109 from fkautz/pr_out_converting_unimplemented_methods_list_into_an_http_handler
2015-01-31 19:50:48 -08:00
Frederick F. Kautz IV
94e124470c
Converting unimplemented methods list into an http handler
2015-01-31 19:50:32 -08:00
Harshavardhana
9e5b689dc5
Merge pull request #103 from harshavardhana/pr_out_add_database_query_function
2015-01-31 15:52:16 -08:00
Harshavardhana
46751f0b28
Add database query function, and some re-factoring
2015-01-31 15:51:18 -08:00
Frederick F. Kautz IV
bc779fcdea
Merge pull request #102 from fkautz/pr_out_adding_sublime_workspace_to_gitignore
2015-01-30 17:39:22 -08:00
Frederick F. Kautz IV
68205c8c6c
Adding sublime workspace to gitignore
2015-01-30 17:39:08 -08:00
Frederick F. Kautz IV
d3e3bfee97
Merge pull request #101 from fkautz/pr_out_adding_list_api_test
2015-01-30 17:37:46 -08:00
Frederick F. Kautz IV
0cd54031a7
Adding list api test
2015-01-30 17:36:53 -08:00
Harshavardhana
0903b7c46d
Merge pull request #100 from harshavardhana/pr_out_force_clients_to_have_certs
2015-01-30 16:52:46 -08:00
Harshavardhana
8a4128d304
Force clients to have certs
2015-01-30 16:51:33 -08:00
Frederick F. Kautz IV
fec80fb1b3
Merge pull request #99 from fkautz/pr_out_using_generic_make_test_and_make_build_disabled_signers_code_for_now_to_fix_build
2015-01-30 10:59:06 -08:00
Frederick F. Kautz IV
84c5df8441
Using generic make test ./... and make build ./..., disabled signers code for now to fix build.
2015-01-30 10:58:47 -08:00
Frederick F. Kautz IV
c3c5497d24
Merge pull request #98 from fkautz/pr_out_certificate_and_key_must_both_be_specified_or_empty_together
2015-01-29 11:29:54 -08:00
Frederick F. Kautz IV
91384e73cf
Certificate and key must both be specified or empty together
2015-01-29 11:29:42 -08:00
Frederick F. Kautz IV
58da5c0f87
Merge pull request #97 from fkautz/pr_out_adding_list_bucket_tests_and_fixing_inmemory_behavior
2015-01-29 11:23:50 -08:00
Frederick F. Kautz IV
7ca87e06aa
Adding list bucket tests and fixing inmemory behavior.
2015-01-29 11:23:14 -08:00
Harshavardhana
1306543b76
Merge pull request #96 from harshavardhana/pr_out_remove_redundant_tls_flag
2015-01-29 01:18:12 -08:00
Harshavardhana
7dd5f0ab8d
Remove redundant --tls flag
2015-01-29 01:18:00 -08:00
Harshavardhana
e3378ee448
Merge pull request #95 from harshavardhana/pr_out_getstoragetype_should_return_valid_results
2015-01-29 01:09:11 -08:00
Harshavardhana
8391a6e17a
getStorageType should return valid results
2015-01-29 01:08:54 -08:00
Harshavardhana
966707a7ed
Merge pull request #94 from harshavardhana/pr_out_prefix_if_empty_skip_it_before_isvalidbucket
2015-01-29 01:07:49 -08:00
Harshavardhana
82baccb8bf
Prefix if empty skip it before IsValidBucket
2015-01-29 01:07:37 -08:00
Frederick F. Kautz IV
93b47bc097
Merge pull request #93 from fkautz/pr_out_reducing_code_duplication_relating_to_content_type
2015-01-28 20:47:56 -08:00
Frederick F. Kautz IV
c6d537e66e
Reducing code duplication relating to content type
2015-01-28 20:47:23 -08:00
Frederick F. Kautz IV
de154550a3
Merge pull request #92 from fkautz/pr_out_updating_godep
2015-01-28 20:27:07 -08:00
Frederick F. Kautz IV
96a5fc0ee7
Updating godep
2015-01-28 20:26:52 -08:00
Frederick F. Kautz IV
092b8749cf
Merge pull request #91 from fkautz/pr_out_removing_unnecessary_comments
2015-01-28 19:54:35 -08:00
Frederick F. Kautz IV
3e8765bb27
Removing unnecessary comments
2015-01-28 19:54:08 -08:00
Frederick F. Kautz IV
391468c80e
Merge pull request #90 from fkautz/pr_out_reverting_back_to_codegansta_cli
2015-01-28 19:52:42 -08:00
Frederick F. Kautz IV
7b0942980b
Reverting back to codegansta/cli
2015-01-28 19:52:30 -08:00
Harshavardhana
57d764778d
Merge pull request #88 from harshavardhana/pr_out_add_x509_tests_and_should_be_part_of_the_build
2015-01-28 17:13:10 -08:00
Harshavardhana
ac99968796
Add x509 tests and should be part of the build
2015-01-28 17:12:59 -08:00
Frederick F. Kautz IV
be56e1cdad
Merge pull request #87 from fkautz/pr_out_reordering_minioapi_functions
2015-01-28 16:51:59 -08:00
Frederick F. Kautz IV
b170f3f5fc
Reordering minioapi functions
2015-01-28 16:51:44 -08:00
Frederick F. Kautz IV
452d46bfa6
Merge pull request #86 from fkautz/pr_out_adding_comments_to_explain_design_decisions_in_server_go
2015-01-28 16:33:28 -08:00
Frederick F. Kautz IV
66cc370e8b
Adding comments to explain design decisions in server.go
2015-01-28 16:33:20 -08:00
Frederick F. Kautz IV
698f0b815f
Merge pull request #85 from fkautz/pr_out_http_server_package_now_follows_convention
2015-01-28 16:30:02 -08:00
Frederick F. Kautz IV
f15e2c4e74
HTTP Server package now follows convention
2015-01-28 16:29:47 -08:00
Harshavardhana
1fdd62165e
Merge pull request #83 from harshavardhana/pr_out_authorization_validation_in_accordance_with_s3_signer_awsv2
2015-01-28 16:14:39 -08:00
Harshavardhana
d78cd581c5
Authorization validation in accordance with S3 signer AWSv2
2015-01-28 16:14:08 -08:00
Frederick F. Kautz IV
0225c51238
Merge pull request #84 from fkautz/pr_out_adding_server_config_structure
2015-01-28 16:08:03 -08:00
Frederick F. Kautz IV
e8399a6d05
Adding server config structure
2015-01-28 16:07:53 -08:00
Frederick F. Kautz IV
e25c39e20f
Merge pull request #82 from fkautz/pr_out_fixing_build_break
2015-01-28 15:28:11 -08:00
Frederick F. Kautz IV
f3e31fe795
Fixing build break
2015-01-28 15:27:59 -08:00
Frederick F. Kautz IV
801d3bf95c
Merge pull request #81 from fkautz/pr_out_migrating_from_codegangsta_cli_to_spf13_cobra
2015-01-28 15:12:31 -08:00
Frederick F. Kautz IV
e007a2db66
Migrating from codegangsta/cli to spf13/cobra
2015-01-28 15:12:02 -08:00
Frederick F. Kautz IV
910aa176b4
Merge pull request #80 from fkautz/pr_out_fix_build
2015-01-28 14:00:41 -08:00
Frederick F. Kautz IV
ca7b1a7fc6
Fix build
2015-01-28 14:00:08 -08:00
Harshavardhana
faf6835871
Merge pull request #79 from harshavardhana/pr_out_add_a_new_pkg_crypto_keys_first_cut
2015-01-28 12:03:28 -08:00
Harshavardhana
fbafc98edb
Add a new pkg/crypto/keys, first cut
...
- provides three functions
- GetRandomAlphaNumeric()
- GetRandomAlphaNumericFull()
- GetRandomBase64()
- ValidAccessKey()
2015-01-28 12:02:13 -08:00
Harshavardhana
cc54730eee
Merge pull request #78 from harshavardhana/pr_out_add_list_buckets_for_fs
2015-01-27 18:44:05 -08:00
Harshavardhana
2ccecd3f32
Add list buckets for 'fs'
2015-01-27 18:43:55 -08:00
Harshavardhana
3781734903
Merge pull request #77 from harshavardhana/pr_out_add_tiedot_integration_package_first_cut_more_improvements_needed
2015-01-27 18:15:50 -08:00
Harshavardhana
c63a6dba2f
Add 'tiedot' integration package, first cut - more improvements needed
2015-01-27 18:15:23 -08:00
Frederick F. Kautz IV
cbf83f2d38
Merge pull request #76 from fkautz/pr_out_creating_subdirectories_in_fs_now_works
2015-01-27 17:48:05 -08:00
Frederick F. Kautz IV
5daa408e2c
Creating subdirectories in fs now works
2015-01-27 17:47:11 -08:00
Harshavardhana
25ad24997e
Merge pull request #75 from harshavardhana/pr_out_remote_syso_files_in_make_clean
2015-01-27 13:13:39 -08:00
Harshavardhana
07a1ad8b2d
Remote syso files in 'make clean'
2015-01-27 13:13:25 -08:00
Frederick F. Kautz IV
e1e8bf8b26
Merge pull request #74 from fkautz/pr_out_adding_more_tests_including_ordering_of_objects_in_prefix
2015-01-27 13:12:55 -08:00
Frederick F. Kautz IV
044dc9af80
Adding more tests, including ordering of objects in prefix
2015-01-27 13:11:22 -08:00
Harshavardhana
af52f88187
Merge pull request #73 from harshavardhana/pr_out_fix_asm_path_for_yasm_version_check
2015-01-27 12:59:32 -08:00
Harshavardhana
0c87af7c64
Fix asm path for yasm version check
2015-01-27 12:59:21 -08:00
Harshavardhana
aa4bc2549e
Merge pull request #72 from harshavardhana/pr_out_add_erasure_package_in_its_full_form_v1_0
2015-01-27 12:57:51 -08:00
Harshavardhana
2547163cb2
Add erasure package in its full form v1.0
2015-01-27 12:55:17 -08:00
Frederick F. Kautz IV
77784ca884
Merge pull request #71 from fkautz/pr_out_removing_log_from_fs_test_go
2015-01-27 12:02:27 -08:00
Frederick F. Kautz IV
74ef65daac
Removing log from fs_test.go
2015-01-27 12:02:08 -08:00
Frederick F. Kautz IV
027d1b0f70
Merge pull request #70 from fkautz/pr_out_adding_fs_storage
2015-01-27 11:11:25 -08:00
Frederick F. Kautz IV
163a6c35db
Adding fs storage
2015-01-27 11:09:55 -08:00
Frederick F. Kautz IV
3461e7e300
Merge pull request #65 from fkautz/pr_out_adding_storage_modifications_for_upcoming_fs_system
2015-01-26 15:46:16 -08:00
Frederick F. Kautz IV
49641e3410
Adding storage modifications for upcoming fs system
2015-01-26 15:45:10 -08:00
Harshavardhana
7005d979bd
Merge pull request #68 from harshavardhana/pr_out_avoid_clang_optimization_keep_it_o0
2015-01-26 12:23:26 -08:00
Harshavardhana
f3a27169fc
Avoid 'clang' optimization, keep it -O0
2015-01-26 12:22:08 -08:00
Harshavardhana
156c35e60a
Merge pull request #67 from harshavardhana/pr_out_consolidate_reply_headers_simplify_contenttype
2015-01-25 18:24:08 -08:00
Harshavardhana
b6627b4d6a
Consolidate reply headers, simplify contentType
2015-01-25 18:23:39 -08:00
Harshavardhana
e073295fb9
Merge pull request #66 from harshavardhana/pr_out_cosmetic_change_minio_to_be_installed_with_makefile
2015-01-25 17:46:05 -08:00
Harshavardhana
4c0804c5bc
Cosmetic change 'minio' to be installed with Makefile
2015-01-25 17:45:00 -08:00
Harshavardhana
b5deb198d3
Merge pull request #64 from harshavardhana/pr_out_add_server_connection_into_http_header_for_all_api_replies
2015-01-25 17:41:55 -08:00
Harshavardhana
dfc3367e25
Add Server, Connection into HTTP header for all API replies
2015-01-25 17:40:22 -08:00
Harshavardhana
9e6eedfbbc
Merge pull request #63 from harshavardhana/pr_out_implement_tls_server
2015-01-25 17:23:21 -08:00
Harshavardhana
063832baaf
Implement TLS server
...
$ ./minio --tls --cert <your_cert> --key <your_private_key>
This patchset also provides crypto/x509 - which is a wrapper package
to generate X509 certificates.
This is necessary to provide certificates later through management console
2015-01-25 17:20:00 -08:00
Frederick F. Kautz IV
18ffa44c52
Merge pull request #62 from fkautz/pr_out_adding_error_support_to_all_commands_in_storage_interface
2015-01-25 15:35:22 -08:00
Frederick F. Kautz IV
2368e7c936
Adding error support to all commands in storage interface
2015-01-25 15:35:08 -08:00
Frederick F. Kautz IV
b181ca7395
Merge pull request #61 from fkautz/pr_out_adding_istruncated_to_minioapi
2015-01-25 13:33:19 -08:00
Frederick F. Kautz IV
d37bbfec41
Adding IsTruncated to minioapi
2015-01-25 13:32:39 -08:00
Frederick F. Kautz IV
aef9e60694
Merge pull request #60 from fkautz/pr_out_adding_more_api_suite_tests
2015-01-25 13:07:11 -08:00
Frederick F. Kautz IV
39208601dc
Adding more api suite tests
2015-01-25 13:06:36 -08:00
Frederick F. Kautz IV
8faaa2d21f
Merge pull request #59 from fkautz/pr_out_adding_paging_count_tests
2015-01-25 12:58:22 -08:00
Frederick F. Kautz IV
0da193e6d6
Adding paging count tests
2015-01-25 12:57:58 -08:00
Frederick F. Kautz IV
ef9968160e
Merge pull request #58 from fkautz/pr_out_adding_initial_test_suites
2015-01-25 11:22:56 -08:00
Frederick F. Kautz IV
82a69d3ab9
Adding initial test suites
2015-01-25 11:22:38 -08:00
Harshavardhana
4b586a51cf
Merge pull request #57 from harshavardhana/pr_out_for_listobjects_ignore_resources_should_be_for_buckets
2015-01-24 18:48:11 -08:00
Harshavardhana
15e832de64
For listobjects ignore resources should be for buckets
2015-01-24 18:46:23 -08:00
Harshavardhana
44f43809d7
Merge pull request #56 from harshavardhana/pr_out_return_http_statusnotimplemented_for_range_of_bucket_and_object_resources
2015-01-24 16:39:06 -08:00
Harshavardhana
c751811dcb
Return http.StatusNotImplemented for range of Bucket and Object resources
...
Not Implemented bucket resources
================================
"acl": true
"lifecycle": true
"policy": true
"location": true
"logging": true
"notification": true
"versions": true
"requestPayment": true
"versioning": true
"website": true
"uploads": true
Not Implemented object resources
================================
"uploadId": true
"acl": true
"torrent": true
"uploads": true
2015-01-24 16:37:25 -08:00
Harshavardhana
be9690f3e9
Merge pull request #55 from harshavardhana/pr_out_putbucket_before_putobject_in_testputobject
2015-01-24 15:43:17 -08:00
Harshavardhana
704d6d8b0a
Putbucket before PutObject in TestPutObject
2015-01-24 15:42:24 -08:00
Harshavardhana
77210fead3
Merge pull request #54 from harshavardhana/pr_out_add_bucket_to_be_part_of_object_struct_reply_back_with_bucketnotfound_for_erroneous_object_put_requests
2015-01-24 15:38:56 -08:00
Harshavardhana
357a81e879
Add bucket to be part of object struct, reply back with BucketNotFound
...
for erroneous Object Put requests.
Other minor cleanups, to follow
- https://github.com/golang/go/wiki/CodeReviewComments
2015-01-24 15:35:05 -08:00
Frederick F. Kautz IV
b92acf386f
Merge pull request #53 from fkautz/pr_out_adding_more_tests
2015-01-24 13:14:26 -08:00
Frederick F. Kautz IV
af1ec94925
Adding more tests
2015-01-24 13:13:03 -08:00
Harshavardhana
72179fbc84
Merge pull request #52 from harshavardhana/pr_out_implement_crc32c_for_darwin_update_documentation
2015-01-23 19:44:16 -08:00
Harshavardhana
01d15ca3b2
Implement crc32c for Darwin, update documentation
2015-01-23 19:43:51 -08:00
Frederick F. Kautz IV
727a8669fa
Merge pull request #51 from fkautz/pr_out_adding_tests_for_get_object_in_minioapi
2015-01-23 19:34:51 -08:00
Frederick F. Kautz IV
dd0c833718
Adding tests for GET object in minioapi.
2015-01-23 19:33:56 -08:00
Harshavardhana
4c07aef9e5
Merge pull request #50 from harshavardhana/pr_out_add_osx_builddeps_installation_instructions_macosx_support_works_under_gcc_as_clang_lacks_some_extended_syntax_gas_support_issue_48
2015-01-23 17:31:26 -08:00
Frederick F. Kautz IV
caa80f201e
Merge pull request #49 from fkautz/pr_out_cleaning_up_more_code
2015-01-23 17:29:47 -08:00
Harshavardhana
e890199cac
Add OSX builddeps installation instructions, MacOSX support
...
works under ``GCC`` as ``Clang`` lacks some extended syntax
GAS support issue#48
2015-01-23 17:29:29 -08:00
Frederick F. Kautz IV
71833b4513
Cleaning up more code.
2015-01-23 17:29:27 -08:00
Frederick F. Kautz IV
a3c0440b4a
Merge pull request #47 from fkautz/pr_out_removing_date_format_method
2015-01-23 17:21:24 -08:00
Frederick F. Kautz IV
9224095420
Removing date format method
2015-01-23 17:19:38 -08:00
Harshavardhana
ef3022a6c9
Merge pull request #46 from harshavardhana/pr_out_fix_content_type_in_getobjects
2015-01-23 13:16:48 -08:00
Harshavardhana
c516c74c51
Fix content-type in GetObjects
2015-01-23 13:16:31 -08:00
Frederick F. Kautz IV
aa2a714d50
Merge pull request #45 from fkautz/pr_out_cleaning_up_minioapi_code
2015-01-23 12:27:09 -08:00
Frederick F. Kautz IV
574742ec14
Cleaning up minioapi code
2015-01-23 12:26:37 -08:00
Harshavardhana
32abe927d0
Merge pull request #44 from harshavardhana/pr_out_get_bucket_list_objects_compliance_maxkeys_to_1000_rename_content_struct_to_item_for_cosmetic_reasons_for_xml_decoding_on_client_side
2015-01-23 12:16:37 -08:00
Harshavardhana
d9d80f7c22
Get Bucket List objects compliance MaxKeys to 1000, Rename
...
Content struct to Item for cosmetic reasons for xml decoding
on client side
2015-01-23 12:15:01 -08:00
Harshavardhana
3e11e9a879
Merge pull request #43 from harshavardhana/pr_out_list_buckets_response_should_be_nested_xml_buckets
2015-01-23 01:57:44 -08:00
Harshavardhana
132e661136
List buckets response should be nested xml buckets
...
Additionally check for 'Header' content-type before
referencing it, as it may be empty value
2015-01-23 01:57:26 -08:00
Harshavardhana
d482c39341
Merge pull request #42 from harshavardhana/pr_out_make_gcc_mandatory_on_osx_since_clang_lacks_certain_gas_features
2015-01-22 18:33:49 -08:00
Harshavardhana
3d43305b1f
Make gcc mandatory on OSX, since clang lacks certain GAS features
2015-01-22 18:33:35 -08:00
Frederick F. Kautz IV
0de3dea797
Merge pull request #41 from fkautz/pr_out_cosmetic_change_sorting_routes
2015-01-22 16:20:11 -08:00
Frederick F. Kautz IV
8636e189e6
Cosmetic change, sorting routes
2015-01-22 16:19:55 -08:00
Frederick F. Kautz IV
7a5e95f12d
Merge pull request #40 from fkautz/pr_out_adding_head_verb_to_minioapi
2015-01-22 14:26:09 -08:00
Frederick F. Kautz IV
8d1c447aac
Adding HEAD verb to minioapi
2015-01-22 14:25:53 -08:00
Frederick F. Kautz IV
3a3d9989d1
Merge pull request #39 from fkautz/pr_out_making_in_memory_storage_data_type_private
2015-01-21 17:15:29 -08:00
Frederick F. Kautz IV
079cdcd909
Making in memory storage data type private
2015-01-21 17:15:05 -08:00
Frederick F. Kautz IV
3f6ddada56
Merge pull request #38 from fkautz/pr_out_migrating_from_unix_ts_to_time_time
2015-01-21 17:13:03 -08:00
Frederick F. Kautz IV
8efd23b755
Migrating from unix ts to time.Time
2015-01-21 17:12:47 -08:00
Frederick F. Kautz IV
db32cb3e75
Merge pull request #37 from fkautz/pr_out_simplifying_bucket_name_convention_and_making_convention_public
2015-01-21 15:30:40 -08:00
Frederick F. Kautz IV
53a8536984
Fixing regex
2015-01-21 15:29:31 -08:00
Frederick F. Kautz IV
856781b2a4
Simplifying bucket name convention and making convention public
2015-01-21 15:22:15 -08:00
Frederick F. Kautz IV
ae0b88f319
Simplfying bucket naming convention
2015-01-21 15:20:35 -08:00
Harshavardhana
06d06b8251
Merge pull request #36 from harshavardhana/pr_out_remove_http_unused_package
2015-01-21 15:15:07 -08:00
Harshavardhana
546e762abf
Remove http unused package
2015-01-21 15:14:49 -08:00
Harshavardhana
bd1df87bd3
Merge pull request #34 from fkautz/pr_out_adding_etag_to_headers
2015-01-21 15:07:16 -08:00
Harshavardhana
832319d06b
Merge pull request #35 from harshavardhana/pr_out_add_arch_osx_for_isal_to_compile_properly_on_macosx
2015-01-21 15:05:34 -08:00
Harshavardhana
8582391ef6
Add arch=osx for isal to compile properly on MacOSX
2015-01-21 15:04:54 -08:00
Frederick F. Kautz IV
9ca1e37235
Adding etag to headers
2015-01-21 15:02:08 -08:00
Harshavardhana
3ecb0e82a1
Merge pull request #33 from harshavardhana/pr_out_add_arch_flags
2015-01-21 13:52:53 -08:00
Harshavardhana
e7c74fdde2
Add ARCH flags
2015-01-21 13:52:42 -08:00
Harshavardhana
5cb49175b1
Merge pull request #32 from harshavardhana/pr_out_remove_linux_specific_code_for_now_bring_them_in_later
2015-01-21 13:48:04 -08:00
Harshavardhana
9401262f75
Remove linux specific code for now, bring them in later
2015-01-21 13:47:50 -08:00
Frederick F. Kautz IV
e410337d26
Merge pull request #31 from fkautz/pr_out_extracting_storage_api_to_interface
2015-01-21 12:58:49 -08:00
Frederick F. Kautz IV
c23fa26830
Extracting storage api to interface
2015-01-21 12:58:33 -08:00
Harshavardhana
63d9689214
Merge pull request #30 from harshavardhana/pr_out_provide_etag_with_sha256sum_of_input_object_data
2015-01-21 10:57:22 -08:00
Harshavardhana
d44404dd81
Provide ETag with sha256Sum of input object data
2015-01-21 10:55:33 -08:00
Harshavardhana
5b67da7d96
Add PutBucket and ListBuckets service
2015-01-21 00:52:59 -08:00
Harshavardhana
e22ae2475a
Add license headers and other cleanup
2015-01-20 23:16:24 -08:00
Harshavardhana
9949a02e3e
Merge pull request #2 from harshavardhana/pr_out_provide_package_http_as_a_http_server_wrapper
...
Provide package http as a http.Server wrapper
2015-01-20 23:15:16 -08:00
Frederick F. Kautz IV
761249934b
Merge pull request #27 from fkautz/pr_out_adding_error_when_object_exists
2015-01-20 19:16:51 -08:00
Frederick F. Kautz IV
9a4c8bfaac
Merge pull request #26 from fkautz/pr_out_adding_handler_for_listing_objects_to_both_trailing_and_non_trailing_url
2015-01-20 19:16:50 -08:00
Frederick F. Kautz IV
9260e6404a
Adding error when object exists
2015-01-20 19:13:57 -08:00
Frederick F. Kautz IV
3026f77cfc
Adding handler for listing objects to both trailing and non-trailing url
2015-01-20 19:10:52 -08:00
Frederick F. Kautz IV
a7029206ff
Merge pull request #25 from fkautz/pr_out_list_objects_in_a_bucket
2015-01-20 18:40:46 -08:00
Frederick F. Kautz IV
cc4329fb12
List objects in a bucket.
2015-01-20 18:40:22 -08:00
Frederick F. Kautz IV
d6fd1b0a38
Merge pull request #24 from fkautz/pr_out_initial_work_for_xml_list_objects
2015-01-20 16:08:46 -08:00
Frederick F. Kautz IV
53190e1210
Initial work for xml list objects
2015-01-20 16:08:14 -08:00
Frederick F. Kautz IV
5a4f472638
Merge pull request #23 from fkautz/pr_out_simplifying_code
2015-01-20 14:07:53 -08:00
Frederick F. Kautz IV
3c1580fcd9
Simplifying code
2015-01-20 14:07:39 -08:00
Harshavardhana
b921c5b459
Merge pull request #22 from fkautz/pr_out_moving_scsi_pkg_away_from_errors_new
...
Moving scsi pkg away from errors.New
2015-01-20 11:35:43 -08:00
Frederick F. Kautz IV
2fedef04ac
Moving scsi pkg away from errors.New
2015-01-20 11:34:05 -08:00
Frederick F. Kautz IV
50703cf4d6
Merge pull request #21 from fkautz/pr_out_adding_error_handling_to_storage_get_call
2015-01-19 17:39:35 -08:00
Frederick F. Kautz IV
36268f07ea
Adding error handling to storage get call
2015-01-19 17:39:20 -08:00
Frederick F. Kautz IV
cc288a5fd7
Merge pull request #20 from fkautz/pr_out_moving_web_logic_to_minioapi_package
2015-01-19 16:04:35 -08:00
Frederick F. Kautz IV
f3705c5d61
Moving web logic to minioapi package
2015-01-19 15:59:01 -08:00
Frederick F. Kautz IV
7b91f2405c
Merge pull request #19 from fkautz/pr_out_store_objects_in_memory_map
2015-01-19 10:33:50 -08:00
Frederick F. Kautz IV
c791c7bca4
Store objects in memory map
2015-01-19 10:30:40 -08:00
Frederick F. Kautz IV
718794205f
wip
2015-01-18 17:05:50 -08:00
Frederick F. Kautz IV
7db1d5afef
Merge pull request #18 from fkautz/pr_out_closing_channel_when_http_server_dies
2015-01-18 17:05:44 -08:00
Frederick F. Kautz IV
2ede0bff52
Closing channel when http server dies
...
Fixing deadlock on error
2015-01-18 17:04:09 -08:00
Harshavardhana
3f03ce7334
Merge pull request #14 from fkautz/pr_out_handler_now_only_listens_to_get_requests
...
handler now only listens to GET requests
2015-01-18 16:57:26 -08:00
Frederick F. Kautz IV
1176d38e84
handler now only listens to GET requests
2015-01-18 16:23:17 -08:00
Frederick F. Kautz IV
e1cd8d7412
Merge pull request #13 from fkautz/pr_out_populating_http_handler_with_test_scaffolding
2015-01-18 16:20:10 -08:00
Frederick F. Kautz IV
c766f3617b
Populating http handler with test scaffolding
2015-01-18 16:16:02 -08:00
Frederick F. Kautz IV
8d20e81295
Merge pull request #12 from fkautz/pr_out_minor_refactoring_moving_http_request_logic_from_storage_to_server
2015-01-18 15:51:04 -08:00
Frederick F. Kautz IV
d3d1c1afb3
Minor refactoring, moving http request logic from storage to server
2015-01-18 15:50:53 -08:00
Harshavardhana
4e644a1f41
Merge pull request #11 from fkautz/pr_out_adding_initial_web_server_and_storage_server_module_infrastructure
2015-01-18 15:33:46 -08:00
Frederick F. Kautz IV
f356599e22
Adding initial web server and storage server module infrastructure
2015-01-18 15:06:24 -08:00
Harshavardhana
34b3cb7f9f
Merge pull request #9 from harshavardhana/pr_out_change_developer_md_to_contributing_md_in_accordance_with
2015-01-18 14:58:43 -08:00
Harshavardhana
511d415fc5
Change DEVELOPER.md to CONTRIBUTING.md in accordance with
...
- https://github.com/blog/1184-contributing-guidelines
2015-01-18 14:57:28 -08:00
Frederick F. Kautz IV
530e609618
Merge pull request #10 from fkautz/pr_out_removing_old_server
2015-01-18 14:56:05 -08:00
Frederick F. Kautz IV
f550e84cf4
Removing old server
...
Removing storage drivers
2015-01-18 14:54:46 -08:00
Harshavardhana
7c861eec12
Merge pull request #7 from fkautz/pr_out_removing_old_server
...
Removing old server
2015-01-17 20:25:02 -08:00
Frederick F. Kautz IV
b8981ce946
Removing old server
2015-01-17 10:39:35 -08:00
Harshavardhana
ad8010f2a4
Merge pull request #6 from harshavardhana/pr_out_consolidate_asm_s_from_crc32c_sha1_sha256_and_sha512
2015-01-15 23:11:08 -08:00
Harshavardhana
b1266de212
Consolidate asm.S from crc32c, sha1, sha256 and sha512
2015-01-15 23:09:03 -08:00
Anand Babu (AB) Periasamy
c8a65caf8c
Merge pull request #5 from harshavardhana/pr_out_remove_assembly_code_bring_back_c_code
...
Remove assembly code bring back C code
2015-01-15 18:53:44 -08:00
Harshavardhana
3bf73642fd
Remove assembly code bring back C code
2015-01-15 18:53:15 -08:00
Harshavardhana
6e84b1df97
Provide package http as a http.Server wrapper
2015-01-14 12:43:41 -08:00
Harshavardhana
0f66f0f5de
Merge pull request #4 from harshavardhana/pr_out_further_restructure
2015-01-14 12:42:18 -08:00
Harshavardhana
68de9ac19e
Further restructure
2015-01-14 12:40:43 -08:00
Harshavardhana
3178879b03
Merge pull request #3 from abperiasamy/make-clean
2015-01-14 12:21:24 -08:00
Anand Babu (AB) Periasamy
d4d43fa018
clean isal files as well
2015-01-14 11:53:46 -08:00
Harshavardhana
fb665c85d5
Merge pull request #1 from harshavardhana/pr_out_full_restructure_in_accordance_with
2015-01-14 11:34:53 -08:00
Harshavardhana
432275e966
Full restructure in accordance with
...
- pkg/{subsystem}/{package} style
- modify Makefile to reflect the new style,
consolidate various entries
- add a dummy ``main.go`` at top level
2015-01-14 11:29:04 -08:00
Harshavardhana
033ad56a61
Merge pull request #167 from harshavardhana/pr_out_merge_with_intel_isal_changes_from_github_com_minio_io_isal
2015-01-11 00:43:46 -08:00
Harshavardhana
f347a1e590
Merge with Intel ISAL changes from github.com/minio-io/isal
...
- These changes bring in a much needed Mac OSX port for
Intel ISAL library
- At the current stage this MacOSX part of code is
considered beta
- pkg/cpu now supports OSX
- pkg/checksum/crc32c - is still WIP, rest of the packages
have been validated
2015-01-11 00:39:39 -08:00
Harshavardhana
1dca91cd75
Update README.md
2015-01-08 19:10:42 -08:00
Harshavardhana
1c24353754
Update README.md
2015-01-08 19:05:21 -08:00
Harshavardhana
c82d2b95d7
Merge pull request #166 from harshavardhana/pr_out_intel_isal_requires_yasm_version_1_2_0_with_avx2_instruction_support_document_it
2015-01-06 20:52:04 -08:00
Harshavardhana
35fa667164
Intel ISAL requires yasm version 1.2.0 > with AVX2 instruction support, document it!
2015-01-06 20:51:07 -08:00
Harshavardhana
43b1e178c2
Merge pull request #165 from harshavardhana/pr_out_disable_sha1_sha256_sha512_avx_avx2_sse3_crypto_implementations
2015-01-06 17:02:56 -08:00
Harshavardhana
462808b87a
Disable sha1,sha256,sha512 avx,avx2,sse3 crypto implementations.
...
Re-implement them later, once stable
2015-01-06 17:02:25 -08:00
Harshavardhana
d58f68ccaa
Merge pull request #164 from harshavardhana/pr_out_add_checks_for_proper_yasm_installed_fixes_163
2015-01-06 15:30:33 -08:00
Harshavardhana
c97ed4f0a4
Add checks for proper yasm installed - fixes #163
2015-01-06 15:30:02 -08:00
Harshavardhana
ed7e58235d
Merge pull request #162 from harshavardhana/pr_out_remove_stale_docs
...
Remove stale docs
2015-01-06 02:02:35 -08:00
Harshavardhana
fcd768a88f
Remove stale docs
2015-01-06 02:01:48 -08:00
Harshavardhana
9078743e6e
Merge pull request #161 from harshavardhana/pr_out_add_supported_platforms_table
...
Add supported platforms table
2015-01-06 01:50:52 -08:00
Harshavardhana
9b016c900b
Add supported platforms table
2015-01-06 01:49:42 -08:00
Harshavardhana
9e6a9657d1
Merge pull request #160 from fkautz/pr_out_adding_index_command
...
Adding index command
2015-01-06 01:31:58 -08:00
Frederick F. Kautz IV
992dea1bf3
Adding index command
2015-01-06 22:26:09 +13:00
Frederick F. Kautz IV
d93e14e288
Merge pull request #159 from harshavardhana/pr_out_rename_split_file_split
2015-01-04 15:14:02 +13:00
Harshavardhana
fff6b589cf
Rename split-file --> split
2015-01-03 18:03:21 -08:00
Harshavardhana
f02d6a2b2b
Merge pull request #158 from fkautz/pr_out_adding_initial_split_file_command
2015-01-03 17:59:04 -08:00
Frederick F. Kautz IV
f5010e1ee2
Adding initial split-file command
2015-01-04 14:47:37 +13:00
Frederick F. Kautz IV
b5d84790a2
Further simplifying merging files. Eliminated a structure, switched to PipeWriter
2015-01-04 14:46:51 +13:00
Harshavardhana
c3bc7176dc
Merge pull request #157 from harshavardhana/pr_out_remove_template_files_make_them_const_strings_instead
2015-01-01 17:36:41 -08:00
Harshavardhana
0212079cd5
Remove template files, make them const strings instead
2015-01-01 17:31:51 -08:00
Harshavardhana
024b3a29d3
Update DEVELOPER.md
2014-12-30 17:45:17 -08:00
Harshavardhana
62d56f4a62
Update DEVELOPER.md
2014-12-30 17:42:09 -08:00
Harshavardhana
8088539c79
Update README.md
2014-12-30 17:34:37 -08:00
Harshavardhana
c6d94292dd
Update README.md
2014-12-30 04:33:47 -08:00
Harshavardhana
fa401bf020
Update README.md
2014-12-30 04:27:45 -08:00
Harshavardhana
2ef22862d0
Add quora follow button
2014-12-30 04:22:24 -08:00
Harshavardhana
4915eee00f
Merge pull request #156 from harshavardhana/pr_out_remove_freenode_moving_off_to_gitter
2014-12-30 03:14:12 -08:00
Harshavardhana
92636ef097
Remove freenode, moving off to gitter
2014-12-30 03:13:10 -08:00
Harshavardhana
8ee4d62ace
Merge pull request #153 from harshavardhana/pr_out_fix_grammar
2014-12-30 02:44:46 -08:00
Harshavardhana
c9dca1a395
Fix grammar
2014-12-30 02:44:08 -08:00
Harshavardhana
080b0d7c08
Merge pull request #152 from harshavardhana/pr_out_add_diskattrmap_scsiattrmap_for_probed_scsi_devices
2014-12-30 02:39:22 -08:00
Harshavardhana
0e4a26e3b4
Add Diskattrmap, Scsiattrmap for probed scsi devices
...
Additional changes
- Use ``iota`` for constants
- Remove unncessary C header files
- ``new-cmd`` now depends on codegangsta cli
2014-12-30 02:37:47 -08:00
Frederick F. Kautz IV
460fd63f05
Merge pull request #151 from harshavardhana/pr_out_a_full_restructure
...
A full restructure
2014-12-30 18:24:40 +13:00
Harshavardhana
6b36b5c551
A full restructure
2014-12-29 21:22:26 -08:00
Harshavardhana
bdc4c3ebe7
Merge pull request #149 from harshavardhana/pr_out_implement_sysctl_map
2014-12-23 23:35:07 -08:00
Harshavardhana
df6e89407e
Implement sysctl map
2014-12-23 23:34:50 -08:00
Harshavardhana
c65b971e97
Merge pull request #148 from harshavardhana/pr_out_mercurial_not_needed_anymore_export_goroot
2014-12-23 20:41:58 -08:00
Harshavardhana
db49ad5c91
Mercurial not needed anymore, export GOROOT
2014-12-23 20:35:05 -08:00
Harshavardhana
6ad9e4e0d4
Merge pull request #147 from gitter-badger/gitter-badge
...
Add a Gitter chat badge to README.md
2014-12-22 16:39:16 -08:00
The Gitter Badger
3a7b6e3b8d
Added Gitter badge
2014-12-23 00:38:08 +00:00
Harshavardhana
d82066dcab
Merge pull request #145 from harshavardhana/pr_out_implement_md5c_function_slower_than_golang_s_implementation_keeping_it_in_repo_to_make_further_improvements_and_also_rename
2014-12-21 03:52:00 -08:00
Harshavardhana
13650e088c
Implement md5c function, slower than Golang's implementation
...
keeping it in repo to make further improvements and also rename
minio-hash ---> crypto
2014-12-21 03:50:33 -08:00
Frederick F. Kautz IV
958890276d
Merge pull request #144 from fkautz/pr_out_adding_minio_hash_to_install_via_makefile
2014-12-21 23:17:02 +13:00
Frederick F. Kautz IV
25c8bfee24
Adding minio-hash to install via Makefile
2014-12-21 23:16:29 +13:00
Frederick F. Kautz IV
5966f8b86b
Merge pull request #142 from fkautz/pr_out_adding_minio_hash_with_streaming_crypto_hashes
2014-12-21 16:42:00 +13:00
Frederick F. Kautz IV
ff55d40432
Adding md5, sha256, sha512 to Makefile
2014-12-21 13:40:09 +13:00
Frederick F. Kautz IV
2278df9910
Adding minio-hash with streaming crypto hashes
2014-12-21 13:04:41 +13:00
Harshavardhana
aa6cd015fa
Update README.md
2014-12-20 09:23:06 -08:00
Harshavardhana
d7117b97e6
Merge pull request #141 from harshavardhana/pr_out_update_documentation
2014-12-20 09:09:48 -08:00
Harshavardhana
d4c05f11c7
Update documentation
2014-12-20 09:09:35 -08:00
Harshavardhana
d2784ba73d
Merge pull request #140 from harshavardhana/pr_out_update_developer_docs_to_update_to_go1_4
2014-12-20 08:56:59 -08:00
Harshavardhana
2a54b9279a
Update developer docs to update to Go1.4
2014-12-20 08:56:21 -08:00
Harshavardhana
a3fc096aa7
Merge pull request #139 from fkautz/pr_out_switching_execpipe_output_to_io_reader
...
Switching execpipe output to io.Reader
2014-12-20 01:12:14 -08:00
Frederick F. Kautz IV
14dec50fef
Switching execpipe output to io.Reader
2014-12-20 21:57:27 +13:00
Harshavardhana
d1fe8beff5
Merge pull request #138 from harshavardhana/pr_out_add_simple_execpipe_to_be_used_with_filesystem_format_work
...
Add simple ExecPipe() to be used with filesystem format work
2014-12-19 03:33:25 -08:00
Harshavardhana
6e6f07c58b
Add simple ExecPipe() to be used with filesystem format work
2014-12-19 03:32:13 -08:00
Harshavardhana
ad998a16dd
Merge pull request #137 from harshavardhana/pr_out_rename_appname_to_commandname_for_consistency
2014-12-19 00:18:34 -08:00
Harshavardhana
b0e986c82c
Rename appname to commandname for consistency
2014-12-19 00:16:41 -08:00
Harshavardhana
a0b57edabe
Merge pull request #136 from harshavardhana/pr_out_add_minio_cli_stub_generator
2014-12-19 00:01:08 -08:00
Harshavardhana
5268a62166
Add minio-cli stub generator
2014-12-19 00:00:14 -08:00
Harshavardhana
2b3be86727
Merge pull request #135 from fkautz/pr_out_command_list
2014-12-18 23:59:48 -08:00
Frederick F. Kautz IV
ab97179323
Adding markdown manpages
2014-12-19 20:27:15 +13:00
Frederick F. Kautz IV
e219c65c9d
Command list
2014-12-19 09:27:58 +13:00
Harshavardhana
0a11b07f0c
Merge pull request #134 from fkautz/pr_out_renaming_documents
...
Renaming documents
2014-12-17 14:49:39 -08:00
Frederick F. Kautz IV
338a8bcb4d
Renaming documents
2014-12-18 09:36:04 +13:00
Harshavardhana
07cbb56221
Merge pull request #133 from harshavardhana/pr_out_add_pkgs_scsi_to_build
2014-12-17 03:36:30 -08:00
Harshavardhana
e1b6f5391f
Add pkgs/scsi to build
2014-12-17 03:36:10 -08:00
Harshavardhana
c58d22977a
Merge pull request #132 from harshavardhana/pr_out_add_linux_scsi_disk_detection_and_attribute_mapping
2014-12-17 03:31:43 -08:00
Harshavardhana
e5953acf4b
Add linux scsi disk detection and attribute mapping
2014-12-17 03:14:44 -08:00
Harshavardhana
eaf08edc5e
Merge pull request #131 from fkautz/pr_out_split_s_splitstream_now_creates_and_returns_a_read_only_channel
...
Split's SplitStream now creates and returns a read only channel.
2014-12-16 16:36:13 -08:00
Frederick F. Kautz IV
c02fa65b0f
Split's SplitStream now creates and returns a read only channel.
2014-12-17 12:34:53 +13:00
Frederick F. Kautz IV
d73c57cff9
Split's JoinStream now creates and returns a read only channel
2014-12-17 12:32:57 +13:00
Harshavardhana
fa52e8b355
Merge pull request #130 from harshavardhana/pr_out_implement_add_commit_instead_of_put_encoded
2014-12-15 21:56:29 -08:00
Harshavardhana
65e2031cfc
Implement add, commit - instead of put, encoded
2014-12-15 21:55:54 -08:00
Harshavardhana
68eb14a1f6
Merge pull request #129 from harshavardhana/pr_out_move_wrappers_at_pkgs_minio_back_to_cmd_minio_as_their_usage_is_limited
2014-12-15 21:19:01 -08:00
Harshavardhana
98119707fb
Move wrappers at pkgs/minio back to cmd/minio - as their usage is limited
2014-12-15 20:09:50 -08:00
Harshavardhana
1f8806b44f
Merge pull request #128 from harshavardhana/pr_out_implement_commands
2014-12-15 19:13:32 -08:00
Harshavardhana
3ec5c2fb8f
Implement commands
...
Commands implemented
$ minio put <objectname> <file>
$ minio encode <objectname>
$ minio get <objectname>
$ minio list
TODO :
$ minio init
$ minio verify
...
...
2014-12-15 14:59:15 -08:00
Frederick F. Kautz IV
e3af681cc1
Merge pull request #127 from fkautz/pr_out_storage_objectstorage_list_is_now_list_objectpath_string
2014-12-15 19:32:09 +13:00
Frederick F. Kautz IV
5efc0d54f8
storage.ObjectStorage List() is now List(objectPath string)
2014-12-15 19:30:08 +13:00
Frederick F. Kautz IV
16e92521bd
Merge pull request #126 from fkautz/pr_out_removing_fsstorage
2014-12-15 19:24:18 +13:00
Frederick F. Kautz IV
05baa6d9a2
Removing fsstorage
2014-12-15 19:15:22 +13:00
Harshavardhana
62797cd6a7
Merge pull request #125 from fkautz/pr_out_removing_unneeded_comment_from_storage_objectstorage
...
Removing unneeded comment from storage.ObjectStorage
2014-12-14 22:08:50 -08:00
Frederick F. Kautz IV
d7e5e0df4f
Removing unneeded comment from storage.ObjectStorage
2014-12-15 15:59:17 +13:00
Harshavardhana
9ea907e062
Merge pull request #124 from fkautz/pr_out_objectdescription_is_now_more_exact_only_exposes_necessary_details
...
ObjectDescription is now more exact, only exposes necessary details.
2014-12-14 17:37:01 -08:00
Frederick F. Kautz IV
dd7cd11c9c
ObjectDescription is now more exact, only exposes necessary details.
2014-12-15 14:26:16 +13:00
Harshavardhana
8ff7fea4ac
Was ---> is
2014-12-14 15:03:39 -08:00
Harshavardhana
baa74072b1
Merge pull request #123 from harshavardhana/pr_out_remove_stale_sha1_dir
2014-12-14 02:38:46 -08:00
Harshavardhana
2552ae450d
Remove stale sha1 dir
2014-12-14 02:38:11 -08:00
Harshavardhana
c46058282e
Merge pull request #122 from harshavardhana/pr_out_update_docs
...
Update docs
2014-12-13 21:52:54 -08:00
Harshavardhana
363930e79d
Update docs
2014-12-13 21:51:46 -08:00
Harshavardhana
f6c37b41ca
Merge pull request #121 from harshavardhana/pr_out_refactor_checksum_code_and_add_objectname_hashing_per_uploaded_objects
2014-12-13 21:44:31 -08:00
Harshavardhana
0148237e22
Refactor checksum code, and add Objectname hashing per uploaded objects
...
Add placeholder sha256, sha512 implementations from Intel with AVX, AVX2
and SSE4.1 extensions - with some boilerplate Go code.
2014-12-13 21:42:23 -08:00
Harshavardhana
b87839a1c8
Merge pull request #120 from harshavardhana/pr_out_add_license_header
2014-12-13 14:27:59 -08:00
Harshavardhana
99478c524a
Add license header
2014-12-13 14:27:06 -08:00
Harshavardhana
6c616f9bd0
Merge pull request #119 from harshavardhana/pr_out_update_documentation_for_crc32c_cpu
2014-12-13 01:39:31 -08:00
Harshavardhana
71c774f99b
Update documentation for crc32c, cpu
2014-12-13 01:39:14 -08:00
Harshavardhana
643a293d94
Merge pull request #118 from harshavardhana/pr_out_provide_new_pkgs_utils_for_generic_functions
2014-12-13 01:15:02 -08:00
Harshavardhana
20d46a1977
Provide new pkgs/utils for generic functions
2014-12-13 01:14:36 -08:00
Harshavardhana
fb1537248b
Merge pull request #117 from harshavardhana/pr_out_how_to_join_community
2014-12-12 23:57:09 -08:00
Harshavardhana
41e9084770
How to Join Community
2014-12-12 23:52:25 -08:00
Harshavardhana
73063deaa6
Merge pull request #116 from harshavardhana/pr_out_fsstorage_should_use_appendstorage_with_offsets
2014-12-12 17:18:53 -08:00
Harshavardhana
fc50291cac
fsStorage should use appendStorage with offsets
...
- also takes in 'blockSize' cli option.
- robustness fixes
2014-12-12 17:18:16 -08:00
Harshavardhana
b570b486de
Merge pull request #115 from harshavardhana/pr_out_add_list_object_support_for_erasure_and_append_storage_drivers
2014-12-12 02:56:13 -08:00
Harshavardhana
0a0e1111cd
Add list() object support for erasure and append storage drivers
...
- Reply back objects with their protectionlevel and md5sum
- // TODO hash value
- Calculate md5sum after "storeBlocks()", to make sure data is committed
2014-12-12 02:52:37 -08:00
Harshavardhana
267aa87ad7
Merge pull request #114 from harshavardhana/pr_out_provide_new_cli_options_for_put_command
...
Provide new cli options for 'put' command
2014-12-11 22:57:56 -08:00
Harshavardhana
f71b323e32
Provide new cli options for 'put' command
...
- --protection-level
- --block-size
2014-12-11 22:56:12 -08:00
Harshavardhana
429836593b
Merge pull request #113 from harshavardhana/pr_out_build_fixes_after_moving_to_go1_4
2014-12-11 20:59:13 -08:00
Harshavardhana
235949eafe
Build fixes after moving to go1.4
2014-12-11 20:57:00 -08:00
Harshavardhana
0bd2b8516e
Build fix, fixing without a PR
2014-12-11 20:44:06 -08:00
Harshavardhana
4a7d9a58f4
Merge pull request #112 from harshavardhana/pr_out_calculate_md5sum_for_each_object
2014-12-11 20:36:42 -08:00
Harshavardhana
91d1c938a8
Merge pull request #111 from harshavardhana/pr_out_implement_simple_encoded_storage_in_gateway
2014-12-11 20:36:31 -08:00
Harshavardhana
e15edbf393
Implement simple encoded storage in gateway
2014-12-11 20:29:40 -08:00
Harshavardhana
a0ec21975d
Calculate md5sum for each object
2014-12-11 19:43:58 -08:00
Harshavardhana
a56710e318
Merge pull request #110 from harshavardhana/pr_out_use_objectname_with_basename_for_the_filepath
2014-12-11 15:35:53 -08:00
Harshavardhana
dddc555491
Use objectName with basename for the filepath
2014-12-11 15:35:25 -08:00
Harshavardhana
94e11db3a8
Merge pull request #109 from harshavardhana/pr_out_add_crc_checks_for_object_get_
2014-12-11 15:10:51 -08:00
Harshavardhana
1166557da1
Add crc checks for Object Get()
2014-12-11 15:06:29 -08:00
Frederick F. Kautz IV
b3af234ee2
Merge pull request #108 from fkautz/pr_out_updates_for_cli_and_context
2014-12-11 09:28:33 -08:00
Frederick F. Kautz IV
c2290dd1d4
Updates for cli and context
...
cli adds new global options. We'll be able to specify flags that
apply to multiple options now.
context update fixes parallelWriter function, actually writes a value out now.
2014-12-11 09:19:20 -08:00
Harshavardhana
e107c3204d
Merge pull request #107 from harshavardhana/pr_out_storage_erasure_encoding_info_at_storageentry_and_also_store_crc_at_blockheader
2014-12-11 02:22:30 -08:00
Harshavardhana
803c35ee90
Storage erasure encoding info at StorageEntry and also store CRC at blockheader
2014-12-11 02:11:48 -08:00
Harshavardhana
23a3609a64
Merge pull request #106 from harshavardhana/pr_out_ignore_io_eof_for_gob_decoding
2014-12-11 01:47:49 -08:00
Harshavardhana
c0724b38b9
Ignore io.EOF for gob decoding
2014-12-11 01:45:58 -08:00
Harshavardhana
063d668133
Merge pull request #105 from fkautz/pr_out_erasure_encoded_demo
2014-12-11 00:43:02 -08:00
Frederick F. Kautz IV
568ca4e1bc
erasure encoded demo
2014-12-11 00:32:32 -08:00
Frederick F. Kautz IV
e5995f0193
Merge pull request #104 from fkautz/pr_out_erasure_demo_put_now_takes_1_or_2_arguments
2014-12-11 00:20:45 -08:00
Harshavardhana
d442988912
Merge pull request #103 from fkautz/pr_out_objectstorage_refactor_to_take_io_reader_instead_of_byte
...
ObjectStorage refactor to take io.Reader instead of []byte
2014-12-10 21:40:25 -08:00
Frederick F. Kautz IV
a1040ae657
erasure-demo put now takes 1 or 2 arguments
2014-12-10 20:59:49 -08:00
Frederick F. Kautz IV
19da9760b3
ObjectStorage refactor to take io.Reader instead of []byte
2014-12-10 20:57:59 -08:00
Frederick F. Kautz IV
c7b4e14f64
Merge pull request #102 from fkautz/pr_out_adding_list_path_to_object_storage_definition_and_accompanying_definition_to_fs
2014-12-10 19:19:26 -08:00
Frederick F. Kautz IV
cae9b288b0
Adding List(path) to object storage definition and accompanying definition to fs
2014-12-10 19:15:03 -08:00
Harshavardhana
9142de1dd5
Merge pull request #101 from fkautz/pr_out_fixing_build
...
Fixing build
2014-12-10 19:00:34 -08:00
Frederick F. Kautz IV
e670608226
Fixing build
2014-12-10 17:43:16 -08:00
Harshavardhana
9dd3dd8b53
Merge pull request #100 from harshavardhana/pr_out_add_crc32c_checksum_support_for_erasure_chunks
...
Add crc32c checksum support for erasure chunks
2014-12-10 11:53:23 -08:00
Harshavardhana
01c0e45b5c
Add crc32c checksum support for erasure chunks
...
Now upon conditions
- if checksum '.cksum' file is missing
- or corrupted chunk (mismatching chunk)
- or missing chunk entirely
Decoding works consistently
2014-12-10 00:58:15 -08:00
Harshavardhana
72ddb345a1
Merge pull request #99 from harshavardhana/pr_out_make_get_put_more_robust
2014-12-09 03:37:08 -08:00
Harshavardhana
f6a5b61b00
Make get/put more robust
...
- init folder if not present already in 'get()' call
- put.go was referencing wrong args - fix it
- if no OBJECTNAME is specified during 'get' - grab a
list of files under "~/.minio"
- Provide new API GetList() for ObjectStorage struct
2014-12-09 03:32:31 -08:00
Harshavardhana
3fe2c2af01
Merge pull request #98 from harshavardhana/pr_out_add_sha1_avx2_supported_fast_implementation_with_some_crc32c_cleanup
2014-12-09 01:01:05 -08:00
Harshavardhana
43680cd7b6
Add sha1 - AVX2 supported fast implementation, with some crc32c cleanup
2014-12-09 01:00:02 -08:00
Frederick F. Kautz IV
93e57118f8
Merge pull request #97 from fkautz/pr_out_adding_append_storage_driver
2014-12-08 23:37:54 -08:00
Frederick F. Kautz IV
1d7b3866cb
Adding append storage driver
2014-12-08 23:37:25 -08:00
Frederick F. Kautz IV
76f21c1cfc
Merge pull request #96 from fkautz/pr_out_removing_printlns_from_test_code
2014-12-08 20:40:08 -08:00
Frederick F. Kautz IV
af8eb7a55d
Removing printlns from test code
2014-12-08 20:39:30 -08:00
Frederick F. Kautz IV
a40d834a42
Merge pull request #95 from fkautz/pr_out_adding_extremely_simple_fs_based_get_and_put_in_erasure_demo
2014-12-08 20:23:02 -08:00
Frederick F. Kautz IV
d1ebf935da
Adding extremely simple fs based get and put in erasure-demo
2014-12-08 20:20:25 -08:00
Frederick F. Kautz IV
c2917f0d64
Merge pull request #94 from fkautz/pr_out_refactoring_file_storage_driver_to_fsstorage
2014-12-08 15:27:37 -08:00
Frederick F. Kautz IV
0cf80e075e
Refactoring file storage driver to fsstorage
2014-12-08 14:29:25 -08:00
Frederick F. Kautz IV
8a5b8ac90f
Merge pull request #93 from fkautz/pr_out_cpu_tests_now_actually_test_properly_on_linux_and_return_bool_instead_of_int
2014-12-08 14:25:11 -08:00
Frederick F. Kautz IV
e71998c584
CPU tests now actually test properly on linux and return bool instead of int
2014-12-08 13:10:18 -08:00
Harshavardhana
5d4c02ad4a
Merge pull request #92 from harshavardhana/pr_out_fixes_71_some_crazy_races_inside_erasure_and_simplify_the_code
2014-12-07 01:34:23 -08:00
Harshavardhana
7314b5e37d
Fixes #71 - some crazy races inside erasure and simplify the code
...
- This change also brings in changing 'unsigned char' to 'uint8_t'
for brevity
2014-12-07 01:33:49 -08:00
Harshavardhana
d57700922d
Merge pull request #91 from harshavardhana/pr_out_remove_cpufeatures_c_code_since_pkgs_crc32c_cpu_should_be_used_instead
...
Remove cpufeatures C code, since pkgs/crc32c/cpu should be used instead
2014-12-05 17:26:03 -08:00
Harshavardhana
e414d5f67b
Remove cpufeatures C code, since pkgs/crc32c/cpu should be used instead
2014-12-05 17:25:42 -08:00
Harshavardhana
f68baeaab2
Merge pull request #90 from harshavardhana/pr_out_provide_joinfileswithprefix_case_with_gochannel
...
Provide JoinFilesWithPrefix case with Gochannel
2014-12-05 17:22:19 -08:00
Harshavardhana
9a5003dd25
Provide JoinFilesWithPrefix case with Gochannel
2014-12-05 16:02:46 -08:00
Harshavardhana
d34d86c050
Merge pull request #87 from fkautz/pr_out_adding_more_unit_tests_to_strbyteconv_fixes_69
...
Adding more unit tests to strbyteconv. Fixes #69 .
2014-12-04 23:11:08 -08:00
Frederick F. Kautz IV
5abcaf4d58
Adding more unit tests to strbyteconv. Fixes #69 .
2014-12-04 23:03:21 -08:00
Harshavardhana
1b283441a4
Merge pull request #80 from harshavardhana/pr_out_add_git_and_mercurial_as_dependencies
2014-12-04 01:57:15 -08:00
Harshavardhana
c3fca84f25
Add git and mercurial as dependencies
2014-12-04 01:56:08 -08:00
Harshavardhana
69ae3b3310
Merge pull request #79 from abperiasamy/pr_out_updated_developers_guide
2014-12-04 01:22:34 -08:00
Anand Babu (AB) Periasamy
0464e2f8cc
updated developers guide
2014-12-04 01:18:26 -08:00
Harshavardhana
0a42a82df5
Merge pull request #78 from harshavardhana/pr_out_avoid_linux_to_be_hard_dependency_on_build
2014-12-04 01:13:28 -08:00
Harshavardhana
0c4dd8f5fa
Avoid linux to be hard dependency on build
2014-12-04 01:12:10 -08:00
Harshavardhana
a2e0a41f7e
Merge pull request #76 from harshavardhana/pr_out_fast_crc_implementations_ported_from_intel_s_efforts
2014-12-03 01:35:54 -08:00
Harshavardhana
fb34c5290c
Fast CRC implementations ported from Intel's efforts
...
Provides fast CRC32C with PCLMULQDQ instructions in Golang
The white papers on CRC32C calculations with PCLMULQDQ instruction can be
downloaded from:
http://www.intel.com/content/dam/www/public/us/en/documents/white-papers/crc-iscsi-polynomial-crc32-instruction-paper.pdf
http://www.intel.com/content/dam/www/public/us/en/documents/white-papers/fast-crc-computation-paper.pdf
2014-12-03 01:20:31 -08:00
Harshavardhana
d31050803b
Merge pull request #77 from harshavardhana/pr_out_change_instructions_for_go_installation_to_directory_inside_users_home_dir
...
Change instructions for go installation to directory inside users ${HOME} dir
2014-12-02 13:15:34 -08:00
Harshavardhana
18234c0666
Change instructions for go installation to directory inside users ${HOME} dir
2014-12-02 13:14:55 -08:00
Harshavardhana
39beabbbb6
Merge pull request #74 from fkautz/pr_out_adding_chunking_by_block_to_erasure_demo_via_block_size_parameter
...
Adding chunking by block to erasure-demo via --block-size parameter
2014-12-01 21:45:17 -08:00
Frederick F. Kautz IV
dac14cd7a1
Merge pull request #75 from harshavardhana/pr_out_remove_unnecessary_c_code_and_use_everything_from_golang
2014-12-01 21:10:21 -08:00
Harshavardhana
3a6cac8ada
Remove unnecessary C code and use everything from Golang
2014-12-01 21:07:55 -08:00
Frederick F. Kautz IV
46b08681a4
Adding chunking by block to erasure-demo via --block-size parameter
2014-12-01 21:06:36 -08:00
Harshavardhana
59c1197f47
Merge pull request #73 from fkautz/pr_out_adding_file_splitter_and_streaming_byte_chunker
...
Adding file splitter and streaming byte chunker
2014-12-01 20:47:25 -08:00
Frederick F. Kautz IV
89587e1391
Adding file splitter and streaming byte chunker
2014-12-01 18:51:47 -08:00
Harshavardhana
3b010d4813
Merge pull request #72 from harshavardhana/pr_out_build_restructure_top_level
...
Build restructure top level
2014-12-01 14:46:34 -08:00
Harshavardhana
6e4583ef1d
Build restructure top level
2014-12-01 14:45:50 -08:00
Harshavardhana
c2b52590ee
Merge pull request #70 from harshavardhana/pr_out_enable_cover_profile_and_test_for_split_and_strbyteconv
...
Enable cover profile and test for split and strbyteconv
2014-12-01 10:58:34 -08:00
Harshavardhana
a4676211cf
Enable cover profile and test for split and strbyteconv
2014-12-01 10:40:50 -08:00
Harshavardhana
c002f90e0b
Merge pull request #28 from harshavardhana/pr_out_enable_way_to_split_files_at_input_bytes
...
Enable way to split files at input bytes
2014-12-01 01:03:17 -08:00
Harshavardhana
3a18454f5e
Merge pull request #67 from fkautz/pr_out_refactoring_input_parsing_for_erasure_demo
...
Refactoring input parsing for erasure-demo
2014-12-01 01:03:05 -08:00
Harshavardhana
67e9a27b34
Enable way to split files at input bytes
2014-12-01 01:00:56 -08:00
Harshavardhana
2bea464c32
Merge pull request #68 from harshavardhana/pr_out_strbyteconv_stringtobytes_bytestostring_
...
strbyteconv - - StringToBytes() - BytesToString()
2014-12-01 00:37:04 -08:00
Harshavardhana
ad5b4b52e8
strbyteconv -
...
- StringToBytes()
- BytesToString()
2014-11-30 23:06:47 -08:00
Frederick F. Kautz IV
323a7b8492
Refactoring input parsing for erasure-demo
2014-11-30 22:18:23 -08:00
Harshavardhana
2a2e67cb50
Merge pull request #66 from fkautz/pr_out_fixing_build_where_cover_and_godep_cause_a_break_when_system_is_offline
...
Fixing build where cover and godep cause a break when system is offline
2014-11-30 21:59:21 -08:00
Frederick F. Kautz IV
d1f9704f25
Fixing build where cover and godep cause a break when system is offline
2014-11-30 21:58:39 -08:00
Harshavardhana
a110792ba5
Merge pull request #65 from harshavardhana/pr_out_remove_redundant_make_getdeps
...
Remove redundant 'make getdeps'
2014-11-30 16:37:27 -08:00
Harshavardhana
099d81e5c9
Remove redundant 'make getdeps'
2014-11-30 16:37:01 -08:00
Harshavardhana
653ad20c67
Merge pull request #64 from harshavardhana/pr_out_cleanup_makefile
...
Cleanup makefile
2014-11-30 15:20:32 -08:00
Harshavardhana
fa1a13a8bf
Cleanup makefile
2014-11-30 15:20:20 -08:00
Harshavardhana
08c8a029be
Merge pull request #63 from harshavardhana/pr_out_update_readme_with_clear_instructions
...
Update readme with clear instructions
2014-11-30 15:13:48 -08:00
Harshavardhana
eaf5379246
Update readme with clear instructions
2014-11-30 15:09:58 -08:00
Frederick F. Kautz IV
51022972f8
Merge pull request #61 from fkautz/pr_out_renaming_minio_demo_to_erasure_demo
2014-11-30 13:54:59 -08:00
Frederick F. Kautz IV
6382ee161e
Renaming minio-demo to erasure-demo
2014-11-30 13:54:31 -08:00
Harshavardhana
db4ad37fee
Merge pull request #62 from fkautz/pr_out_removing_groupcache_lru_dependency
...
Removing groupcache lru dependency
2014-11-30 13:50:00 -08:00
Frederick F. Kautz IV
af273c5d0f
Removing groupcache lru dependency
2014-11-30 13:49:06 -08:00
Harshavardhana
fdf860e8eb
Merge pull request #60 from fkautz/pr_out_adding_godoc_description_of_minio
...
Adding godoc description of minio
2014-11-30 13:12:41 -08:00
Frederick F. Kautz IV
b98965604c
Adding godoc description of minio
2014-11-30 13:11:37 -08:00
Harshavardhana
0c16744e2c
Merge pull request #59 from harshavardhana/pr_out_no_caps_for_stdint
...
No caps for stdint
2014-11-30 12:37:34 -08:00
Harshavardhana
7e79721452
No caps for stdint
2014-11-30 11:57:58 -08:00
Harshavardhana
7d47b72f1d
Merge pull request #58 from fkautz/pr_out_increasing_parity_from_5_to_6
...
Increasing parity from 5 to 6
2014-11-29 23:04:37 -08:00
Frederick F. Kautz IV
30687a8c66
Increasing parity from 5 to 6
2014-11-29 22:59:44 -08:00
Harshavardhana
07e1199ae1
Merge pull request #57 from fkautz/pr_out_merging_encode_and_decode_to_single_minio_demo_binary
...
Merging encode and decode to single minio-demo binary
2014-11-29 22:51:10 -08:00
Frederick F. Kautz IV
3c09029049
Merging encode and decode to single minio-demo binary
2014-11-29 22:48:37 -08:00
Harshavardhana
a37bb693ba
Merge pull request #56 from harshavardhana/pr_out_fix_function_name_change
...
Fix function name change
2014-11-29 17:30:33 -08:00
Harshavardhana
5074542636
Fix function name change
2014-11-29 17:29:38 -08:00
Harshavardhana
980ad3b373
Merge pull request #55 from harshavardhana/pr_out_decoding_bug_fixes_42
...
Decoding bug fixes #42
2014-11-29 17:24:40 -08:00
Harshavardhana
4d7c1e3b68
Decoding bug fixes #42
2014-11-29 17:22:31 -08:00
Harshavardhana
a6e75f42dd
Merge pull request #53 from fkautz/pr_out_moving_gateway_and_storage_driver_to_packages
...
Moving gateway and storage driver to packages
2014-11-29 16:33:07 -08:00
Harshavardhana
5e0c5089e6
Merge pull request #49 from fkautz/pr_out_adding_initial_documentation_to_erasure_package
...
Adding initial documentation to erasure package
2014-11-29 14:59:35 -08:00
Frederick F. Kautz IV
96e42a8443
Adding initial documentation to erasure package
2014-11-29 14:59:09 -08:00
Harshavardhana
48898d770d
Merge pull request #54 from fkautz/pr_out_exposing_newencoder_since_getencoder_is_no_longer_around
...
Exposing NewEncoder since GetEncoder is no longer around
2014-11-29 14:57:42 -08:00
Frederick F. Kautz IV
3da8e53ab8
Exposing NewEncoder since GetEncoder is no longer around
2014-11-29 14:54:21 -08:00
Frederick F. Kautz IV
03beef3afc
Moving gateway and storage driver to packages
2014-11-29 14:42:22 -08:00
Harshavardhana
dc5d7434e1
Merge pull request #52 from harshavardhana/pr_out_rename_newencoder_newencoder
...
Rename NewEncoder-->newEncoder
2014-11-29 14:42:03 -08:00
Harshavardhana
17a23876a2
Rename NewEncoder-->newEncoder
2014-11-29 14:41:20 -08:00
Harshavardhana
a86de74983
Merge pull request #50 from fkautz/pr_out_removing_lru_from_erasure_package
...
Removing LRU from erasure package
2014-11-29 14:39:07 -08:00
Harshavardhana
c2b8fee677
Merge pull request #47 from harshavardhana/pr_out_erasure_restructuring
...
Erasure restructuring
2014-11-29 14:30:01 -08:00
Harshavardhana
922e6bac0b
Erasure restructuring
2014-11-29 14:28:55 -08:00
Frederick F. Kautz IV
9ef1ef5c8c
Removing LRU from erasure package
2014-11-29 13:45:44 -08:00
Harshavardhana
d6b65f1f04
Merge pull request #46 from harshavardhana/pr_out_make_sure_to_use_simd_alignment_with_32
2014-11-27 02:59:55 -08:00
Harshavardhana
618e0e1b04
Update README.md
2014-11-27 02:49:20 -08:00
Harshavardhana
3a3d3c8d1e
Update README.md
2014-11-27 02:45:42 -08:00
Harshavardhana
f1fcd3be06
Make sure to use SIMD Alignment with 32
2014-11-27 02:15:41 -08:00
Frederick F. Kautz IV
fd52a211f4
Merge pull request #45 from fkautz/pr_out_simplifying_code
2014-11-26 23:33:57 -08:00
Frederick F. Kautz IV
becb165770
Simplifying code
2014-11-26 23:33:36 -08:00
Harshavardhana
12aae70620
Merge pull request #44 from harshavardhana/pr_out_restructure_directory_add_pkgs_folder_now
...
Restructure directory, add 'pkgs' folder now
2014-11-24 15:11:48 -08:00
Harshavardhana
3cefaf5ad8
Restructure directory, add 'pkgs' folder now
2014-11-24 15:10:25 -08:00
Harshavardhana
1707b87ddd
Merge pull request #43 from harshavardhana/pr_out_remove_overview_diagram_for_now
2014-11-24 13:11:24 -08:00
Harshavardhana
0e9764fd83
Remove overview diagram for now
2014-11-24 13:09:52 -08:00
Harshavardhana
a1aafb9648
Merge pull request #41 from harshavardhana/pr_out_make_use_of_lru_and_higher_order_erasure_functions
...
Make use of LRU and higher order erasure functions
2014-11-24 08:14:58 -08:00
Harshavardhana
6592ef3bd2
Make use of LRU and higher order erasure functions
2014-11-24 07:16:54 -08:00
Harshavardhana
95e2e472cd
Merge pull request #40 from harshavardhana/pr_out_decode_on_chunks_missing_should_build_and_continue
2014-11-24 07:03:00 -08:00
Harshavardhana
ee7ff7c436
Decode on chunks missing should build and continue
2014-11-24 06:58:27 -08:00
Harshavardhana
bd5a8c126c
Merge pull request #39 from fkautz/pr_out_adding_minio_decode
2014-11-24 06:52:08 -08:00
Harshavardhana
a98cf10920
Update README.md
2014-11-23 23:08:43 -08:00
Frederick F. Kautz IV
5a3eb47583
Adding minio-decode
2014-11-23 19:32:01 -08:00
Harshavardhana
4b97cf4b9d
Merge pull request #38 from fkautz/pr_out_adding_k_and_m_as_optional_parameters_for_encoding_in_minio_encode
...
Adding k and m as optional parameters for encoding in minio-encode
2014-11-23 17:36:32 -08:00
Frederick F. Kautz IV
cec0a0ef68
Adding k and m as optional parameters for encoding in minio-encode
2014-11-23 15:49:26 -08:00
Harshavardhana
2aa21866de
Merge pull request #37 from harshavardhana/pr_out_only_build_static_lib
...
Only build static lib
2014-11-22 21:47:55 -08:00
Harshavardhana
63d63e51ed
Only build static lib
2014-11-22 21:47:35 -08:00
Harshavardhana
5225a6b7f0
Merge pull request #36 from harshavardhana/pr_out_add_minio_overview
...
Add minio overview
2014-11-22 21:26:38 -08:00
Harshavardhana
29cc682fc0
Add minio overview
2014-11-22 21:25:07 -08:00
Harshavardhana
42ba0af9ac
Merge pull request #35 from fkautz/pr_out_adding_input_and_output_file_parameters_to_minio_encode
...
Adding input and output file parameters to minio-encode.
2014-11-22 21:02:05 -08:00
Frederick F. Kautz IV
6faa55e25a
Adding input and output file parameters to minio-encode.
2014-11-22 19:25:46 -08:00
Frederick F. Kautz IV
eb36c9e4d1
Merge pull request #34 from harshavardhana/pr_out_vandermonde_cauchy
2014-11-22 18:52:44 -08:00
Harshavardhana
3b9495d91a
Vandermonde --> Cauchy
2014-11-22 18:52:07 -08:00
Harshavardhana
6fa2ad2444
Merge pull request #31 from fkautz/pr_out_adding_minio_encode_binary
...
Adding minio-encode binary
2014-11-22 18:47:46 -08:00
Frederick F. Kautz IV
a15e935a33
Adding minio-encode binary
2014-11-22 18:39:59 -08:00
Frederick F. Kautz IV
7ca422f972
Merge pull request #33 from fkautz/pr_out_adding_cover_out_to_gitignore
2014-11-22 18:32:32 -08:00
Frederick F. Kautz IV
2e8a68a796
Adding cover.out to gitignore
2014-11-22 18:31:49 -08:00
Frederick F. Kautz IV
4c16e6e9ef
Merge pull request #32 from mattf/master
2014-11-22 18:30:46 -08:00
Matthew Farrellee
5047c055f6
stop requiring a cover directory
...
it is not automatically created. instead, store coverage output in the
current directory.
2014-11-22 21:13:56 -05:00
Matthew Farrellee
fe0e13dd6b
add missing deps
2014-11-22 21:11:40 -05:00
Harshavardhana
be17f50d17
Merge pull request #30 from fkautz/pr_out_removing_binaries
...
Removing binaries
2014-11-22 17:06:12 -08:00
Frederick F. Kautz IV
9c18624e58
Removing binaries
2014-11-22 15:16:38 -08:00
Harshavardhana
324c23f12e
Merge pull request #29 from fkautz/pr_out_fixing_isal_compile_from_root_workaround_for_https_code_google_com_p_go_issues_detail_id_7891
...
Fixing isal compile from root, workaround for https://code.google.com/p/go/issues/detail?id=7891
2014-11-22 13:14:35 -08:00
Frederick F. Kautz IV
1a6b9ccfc8
Fixing isal compile from root, workaround for https://code.google.com/p/go/issues/detail?id=7891
2014-11-22 13:09:05 -08:00
Frederick F. Kautz IV
084d029b54
Merge pull request #27 from harshavardhana/pr_out_use_deepequals_instead_of_bytes_compare
...
Use DeepEquals instead of bytes Compare
2014-11-17 01:07:45 -07:00
Harshavardhana
22853ed1d9
Use DeepEquals instead of bytes Compare
2014-11-17 00:06:42 -08:00
Harshavardhana
d476a9d87e
Merge pull request #26 from fkautz/pr_out_adding_simple_file_storage_driver_for_persistent_storage
...
Adding simple file storage driver for persistent storage
2014-11-15 04:43:27 -08:00
Harshavardhana
a468afca03
Merge pull request #25 from fkautz/pr_out_moving_minio_command_to_cmd_minio
...
Moving minio command to cmd/minio
2014-11-15 04:43:17 -08:00
Frederick F. Kautz IV
44b28166f5
Adding simple file storage driver for persistent storage
2014-11-14 18:22:50 -07:00
Frederick F. Kautz IV
f8f9f65652
Moving minio command to cmd/minio
2014-11-14 18:20:40 -07:00
Harshavardhana
2679324afe
Merge pull request #24 from harshavardhana/pr_out_add_groupcache_lru_into_godep
...
Add groupcache/lru into godep
2014-11-13 22:46:25 -08:00
Harshavardhana
3ad5f2d955
Add groupcache/lru into godep
2014-11-13 22:41:44 -08:00
Frederick F. Kautz IV
c4d2292a7d
Merge pull request #18 from fkautz/pr_out_bucket_manager_is_now_configurable_as_gatewayconfig_bucketservicedriver
2014-11-13 22:23:08 -08:00
Frederick F. Kautz IV
101ce51f78
Bucket manager is now configurable as GatewayConfig.BucketServiceDriver
2014-11-13 22:20:35 -08:00
Harshavardhana
f46fad48b0
Merge pull request #23 from harshavardhana/pr_out_squashed_contrib_signify_changes_from_a1554a4_35fe2d3
...
Squashed 'contrib/signify/' changes from a1554a4..35fe2d3
2014-11-13 21:55:39 -08:00
Harshavardhana
88ea40a2e8
Merge commit 'ef0a8b4d2e245be27cc27a782fb1b807ed0e2618'
2014-11-13 21:54:50 -08:00
Harshavardhana
ef0a8b4d2e
Squashed 'contrib/signify/' changes from a1554a4..35fe2d3
...
35fe2d3 Update upstream diff
2b7bf89 Generate shared libs
git-subtree-dir: contrib/signify
git-subtree-split: 35fe2d347b161b9ee18ca33c9d5a54bb466eecbf
2014-11-13 21:54:50 -08:00
Harshavardhana
fa13804db0
Merge pull request #22 from harshavardhana/pr_out_erasure_coding_blocks_during_encoding_were_garbage_fixes_21
...
erasure: Coding blocks during encoding were garbage, fixes #21
2014-11-13 21:49:23 -08:00
Harshavardhana
6a6dac8365
erasure: Coding blocks during encoding were garbage, fixes #21
2014-11-13 21:48:04 -08:00
Harshavardhana
c39c51328e
Merge pull request #16 from fkautz/pr_out_moving_bucket_request_channel_to_private_config_option
...
Moving bucket request channel to private config option
2014-11-13 15:21:46 -08:00
Harshavardhana
d88158b75a
Merge pull request #15 from harshavardhana/go_isal_binding
...
Add erasure coding and decoding using Intel Storage Acceleration library
2014-11-13 15:21:36 -08:00
Harshavardhana
1e7515a7df
Add erasure coding and decoding using Intel Storage Acceleration library
...
- move contrib/erasure --> contrib/isal
- bring in low level 'isal' package for Go for exposing C functions
- Implement Erasure 'encoding'
Supports - Reed Solomon Codes, Cauchy Codes
- Implement Erasure 'decoding'
Supports - Reed Solomon Codes, Cauchy Codes
- Renames Minios -> Minio at all the references
2014-11-13 15:20:18 -08:00
Anand Babu (AB) Periasamy
cbce7fd66f
Merge pull request #19 from fkautz/pr_out_in_memory_object_storage_now_uses_patricia_tree
...
In memory object storage now uses patricia tree
2014-11-08 00:11:49 -08:00
Frederick F. Kautz IV
e0b9a59ad4
In memory object storage now uses patricia tree
2014-11-07 23:34:21 -08:00
Frederick F. Kautz IV
0ed2aeb439
Merge pull request #17 from fkautz/pr_out_changing_minios_to_minio
2014-11-07 22:29:02 -08:00
Frederick F. Kautz IV
9275afa173
Changing minios to minio
2014-11-07 22:23:52 -08:00
Frederick F. Kautz IV
4e6214514f
Moving bucket request channel to private config option
2014-11-07 19:07:16 -08:00
Harshavardhana
00b5a5d380
Merge pull request #14 from fkautz/pr_out_cleaning_up_more_code_system_generated_by_config
...
Cleaning up more code, system generated by config
2014-11-06 23:23:26 -08:00
Frederick F. Kautz IV
5878dd5e50
Cleaning up more code, system generated by config
2014-11-06 22:45:27 -08:00
Harshavardhana
2011cffe8a
Merge pull request #13 from fkautz/pr_out_adding_get_and_put_methods_for_gateway
...
Adding GET and PUT methods for gateway
2014-11-06 19:33:45 -08:00
Frederick F. Kautz IV
1bf6fe1f68
Adding GET and PUT methods for gateway
2014-11-06 19:32:35 -08:00
Anand Babu (AB) Periasamy
58859a0ecc
Merge pull request #12 from fkautz/pr_out_adding_bucket_management_system_with_in_memory_storage_driver
...
Adding bucket management system with in memory storage driver
2014-11-06 18:36:13 -08:00
Frederick F. Kautz IV
0c68f52caf
Adding bucket management system with in memory storage driver
2014-11-06 21:34:46 -05:00
Anand Babu (AB) Periasamy
bd99a2185e
Merge pull request #11 from fkautz/pr_out_removing_unneeded_entries_from_gitignore
...
Removing unneeded entries from gitignore
2014-11-04 00:19:56 -08:00
Frederick F. Kautz IV
f46a32c963
Removing unneeded entries from gitignore
2014-11-04 00:19:20 -08:00
Anand Babu (AB) Periasamy
129a052fa1
Merge pull request #10 from fkautz/pr_out_moving_to_godep_for_dependency_management
...
Moving to godep for dependency management
2014-11-03 21:29:40 -08:00
Frederick F. Kautz IV
e98a77ff1a
Moving to godep for dependency management
2014-11-03 20:59:07 -08:00
Harshavardhana
74f989e3ce
Merge pull request #9 from fkautz/pr_out_adding_bucket_and_object_route_in_gateway
...
Adding bucket and object route in gateway
2014-11-02 21:55:52 -08:00
Frederick F. Kautz IV
0d4ab562af
Adding bucket and object route in gateway
2014-11-02 19:55:12 -08:00
Anand Babu (AB) Periasamy
428f012830
Merge pull request #8 from fkautz/pr_out_setting_up_initial_cli_options_and_http_handlers
...
Setting up initial cli options and http handlers
2014-11-02 18:52:31 -08:00
Frederick F. Kautz IV
fc6a2a45cb
Setting up initial cli options and http handlers
2014-11-02 17:33:40 -08:00
Anand Babu (AB) Periasamy
d02e148236
Merge pull request #7 from fkautz/pr_out_adding_framework_for_cli_options
...
Adding framework for cli options
2014-11-02 12:26:46 -08:00
Frederick F. Kautz IV
9544ed7b97
Adding framework for cli options
2014-11-02 15:06:26 -05:00
Frederick F. Kautz IV
f7efb69fac
Merge commit '2cd9d9073e6e7afa19d969269002886532492dd1' as 'third_party/src/github.com/codegangsta/cli'
2014-11-02 14:54:31 -05:00
Frederick F. Kautz IV
2cd9d9073e
Squashed 'third_party/src/github.com/codegangsta/cli/' content from commit f7ebb76
...
git-subtree-dir: third_party/src/github.com/codegangsta/cli
git-subtree-split: f7ebb761e83e21225d1d8954fde853bf8edd46c4
2014-11-02 14:54:31 -05:00
Harshavardhana
a5b7ff1175
Merge pull request #6 from fkautz/pr_out_refactoring_miniosd_and_minios_to_a_single_binary_minio
...
Refactoring miniosd and minios to a single binary minio
2014-11-02 11:35:00 -08:00
Frederick F. Kautz IV
dca2f84bc9
Refactoring miniosd and minios to a single binary minio
2014-11-02 14:33:53 -05:00
Harshavardhana
631b3bbf2e
Merge pull request #5 from fkautz/pr_out_miniosd_is_now_an_http_server_responds_with_hello_world_at_root
...
miniosd is now an http server, responds with hello world at root.
2014-11-02 02:06:11 -08:00
Frederick F. Kautz IV
e58cada88c
miniosd is now an http server, responds with hello world at root.
2014-11-01 21:55:13 -07:00
Frederick F. Kautz IV
ddefc424ce
Merge pull request #4 from fkautz/pr_out_working_build_system_builds_gopath_dynamically
2014-11-02 00:06:53 -04:00
Frederick F. Kautz IV
111b2639cd
Merge commit '4aa2e2eaa01e04e05cfbcd765afb5059ff85b8e1' as 'third_party/src/github.com/gorilla/context'
2014-11-01 21:03:30 -07:00
Frederick F. Kautz IV
4aa2e2eaa0
Squashed 'third_party/src/github.com/gorilla/context/' content from commit 14f550f
...
git-subtree-dir: third_party/src/github.com/gorilla/context
git-subtree-split: 14f550f51af52180c2eefed15e5fd18d63c0a64a
2014-11-01 21:03:30 -07:00
Frederick F. Kautz IV
a17126e581
Merge commit '00f86a28a8d65dd0be9e5f2296e654390c2def31' as 'third_party/src/github.com/gorilla/mux'
2014-11-01 21:02:55 -07:00
Frederick F. Kautz IV
00f86a28a8
Squashed 'third_party/src/github.com/gorilla/mux/' content from commit e444e69
...
git-subtree-dir: third_party/src/github.com/gorilla/mux
git-subtree-split: e444e69cbd2e2e3e0749a2f3c717cec491552bbf
2014-11-01 21:02:55 -07:00
Frederick F. Kautz IV
2c73655b7c
Working build system, builds gopath dynamically
2014-11-01 19:44:30 -07:00
Harshavardhana
1303e5aae7
Merge pull request #3 from fkautz/pr_out_squashed_third_party_src_github_com_geertjohan_go_linenoise_content_from_commit_5d5d527
...
Squashed 'third_party/src/github.com/GeertJohan/go.linenoise/' content from commit 5d5d527
2014-11-01 16:00:01 -07:00
Frederick F. Kautz IV
aeb2497bd3
Removing contrib/linenoise, since it is in third_party golang repo now
2014-11-01 15:46:51 -07:00
Frederick F. Kautz IV
d5897cc03a
Merge commit '0a65e23e3aead7caea3d3a35fa917da921257dc6' as 'third_party/src/github.com/GeertJohan/go.linenoise'
2014-11-01 15:38:01 -07:00
Frederick F. Kautz IV
0a65e23e3a
Squashed 'third_party/src/github.com/GeertJohan/go.linenoise/' content from commit 5d5d527
...
git-subtree-dir: third_party/src/github.com/GeertJohan/go.linenoise
git-subtree-split: 5d5d5277d975f40b12167b478c0d79df00bda5a8
2014-11-01 15:38:01 -07:00
Anand Babu (AB) Periasamy
999283e75d
Merge pull request #2 from fkautz/pr_out_adding_third_party_dir_support
...
Adding third_party dir support
2014-11-01 15:26:51 -07:00
Frederick F. Kautz IV
26aad4eab2
Adding third_party dir support
2014-11-01 15:20:59 -07:00
Harshavardhana
92f7a91703
Merge pull request #1 from fkautz/pr_out_adding_initial_makefile_and_main_go
...
Adding initial makefile and main.go
2014-11-01 02:58:00 -07:00
Frederick F. Kautz IV
720ccbddee
Adding initial makefile and main.go
2014-11-01 04:22:16 -04:00
Harshavardhana
2acc68d1af
Squashed 'contrib/signify/' content from commit a1554a4
...
git-subtree-dir: contrib/signify
git-subtree-split: a1554a4384c794c004da70bd006ce93101f77426
2014-10-30 22:28:37 -07:00
Harshavardhana
e91caac870
Merge commit '2acc68d1af798affc64978ed98abf0be6ebda067' as 'contrib/signify'
2014-10-30 22:28:37 -07:00
Harshavardhana
be262e7539
Merge commit '13412ca164f715e89e0eb156599ce5f6e978205c' as 'contrib/linenoise'
2014-10-30 22:26:07 -07:00
Harshavardhana
13412ca164
Squashed 'contrib/linenoise/' content from commit 5d5d527
...
git-subtree-dir: contrib/linenoise
git-subtree-split: 5d5d5277d975f40b12167b478c0d79df00bda5a8
2014-10-30 22:22:47 -07:00
Harshavardhana
397b887a87
Initial commit
2014-10-30 21:51:52 -07:00