0148237e22
Refactor checksum code, and add Objectname hashing per uploaded objects
...
Add placeholder sha256, sha512 implementations from Intel with AVX, AVX2
and SSE4.1 extensions - with some boilerplate Go code.
2014-12-13 21:42:23 -08:00
20d46a1977
Provide new pkgs/utils for generic functions
2014-12-13 01:14:36 -08:00
fc50291cac
fsStorage should use appendStorage with offsets
...
- also takes in 'blockSize' cli option.
- robustness fixes
2014-12-12 17:18:16 -08:00
0a0e1111cd
Add list() object support for erasure and append storage drivers
...
- Reply back objects with their protectionlevel and md5sum
- // TODO hash value
- Calculate md5sum after "storeBlocks()", to make sure data is committed
2014-12-12 02:52:37 -08:00
235949eafe
Build fixes after moving to go1.4
2014-12-11 20:57:00 -08:00
0bd2b8516e
Build fix, fixing without a PR
2014-12-11 20:44:06 -08:00
4a7d9a58f4
Merge pull request #112 from harshavardhana/pr_out_calculate_md5sum_for_each_object
2014-12-11 20:36:42 -08:00
e15edbf393
Implement simple encoded storage in gateway
2014-12-11 20:29:40 -08:00
a0ec21975d
Calculate md5sum for each object
2014-12-11 19:43:58 -08:00
1166557da1
Add crc checks for Object Get()
2014-12-11 15:06:29 -08:00
803c35ee90
Storage erasure encoding info at StorageEntry and also store CRC at blockheader
2014-12-11 02:11:48 -08:00
c0724b38b9
Ignore io.EOF for gob decoding
2014-12-11 01:45:58 -08:00
568ca4e1bc
erasure encoded demo
2014-12-11 00:32:32 -08:00
19da9760b3
ObjectStorage refactor to take io.Reader instead of []byte
2014-12-10 20:57:59 -08:00
cae9b288b0
Adding List(path) to object storage definition and accompanying definition to fs
2014-12-10 19:15:03 -08:00
e670608226
Fixing build
2014-12-10 17:43:16 -08:00
f6a5b61b00
Make get/put more robust
...
- init folder if not present already in 'get()' call
- put.go was referencing wrong args - fix it
- if no OBJECTNAME is specified during 'get' - grab a
list of files under "~/.minio"
- Provide new API GetList() for ObjectStorage struct
2014-12-09 03:32:31 -08:00
43680cd7b6
Add sha1 - AVX2 supported fast implementation, with some crc32c cleanup
2014-12-09 01:00:02 -08:00
1d7b3866cb
Adding append storage driver
2014-12-08 23:37:25 -08:00
af8eb7a55d
Removing printlns from test code
2014-12-08 20:39:30 -08:00
0cf80e075e
Refactoring file storage driver to fsstorage
2014-12-08 14:29:25 -08:00
e71998c584
CPU tests now actually test properly on linux and return bool instead of int
2014-12-08 13:10:18 -08:00
7314b5e37d
Fixes #71 - some crazy races inside erasure and simplify the code
...
- This change also brings in changing 'unsigned char' to 'uint8_t'
for brevity
2014-12-07 01:33:49 -08:00
e414d5f67b
Remove cpufeatures C code, since pkgs/crc32c/cpu should be used instead
2014-12-05 17:25:42 -08:00
f68baeaab2
Merge pull request #90 from harshavardhana/pr_out_provide_joinfileswithprefix_case_with_gochannel
...
Provide JoinFilesWithPrefix case with Gochannel
2014-12-05 17:22:19 -08:00
9a5003dd25
Provide JoinFilesWithPrefix case with Gochannel
2014-12-05 16:02:46 -08:00
5abcaf4d58
Adding more unit tests to strbyteconv. Fixes #69 .
2014-12-04 23:03:21 -08:00
0c4dd8f5fa
Avoid linux to be hard dependency on build
2014-12-04 01:12:10 -08:00
fb34c5290c
Fast CRC implementations ported from Intel's efforts
...
Provides fast CRC32C with PCLMULQDQ instructions in Golang
The white papers on CRC32C calculations with PCLMULQDQ instruction can be
downloaded from:
http://www.intel.com/content/dam/www/public/us/en/documents/white-papers/crc-iscsi-polynomial-crc32-instruction-paper.pdf
http://www.intel.com/content/dam/www/public/us/en/documents/white-papers/fast-crc-computation-paper.pdf
2014-12-03 01:20:31 -08:00
3a6cac8ada
Remove unnecessary C code and use everything from Golang
2014-12-01 21:07:55 -08:00
59c1197f47
Merge pull request #73 from fkautz/pr_out_adding_file_splitter_and_streaming_byte_chunker
...
Adding file splitter and streaming byte chunker
2014-12-01 20:47:25 -08:00
89587e1391
Adding file splitter and streaming byte chunker
2014-12-01 18:51:47 -08:00
6e4583ef1d
Build restructure top level
2014-12-01 14:45:50 -08:00
a4676211cf
Enable cover profile and test for split and strbyteconv
2014-12-01 10:40:50 -08:00
67e9a27b34
Enable way to split files at input bytes
2014-12-01 01:00:56 -08:00
ad5b4b52e8
strbyteconv -
...
- StringToBytes()
- BytesToString()
2014-11-30 23:06:47 -08:00
fa1a13a8bf
Cleanup makefile
2014-11-30 15:20:20 -08:00
7e79721452
No caps for stdint
2014-11-30 11:57:58 -08:00
980ad3b373
Merge pull request #55 from harshavardhana/pr_out_decoding_bug_fixes_42
...
Decoding bug fixes #42
2014-11-29 17:24:40 -08:00
4d7c1e3b68
Decoding bug fixes #42
2014-11-29 17:22:31 -08:00
a6e75f42dd
Merge pull request #53 from fkautz/pr_out_moving_gateway_and_storage_driver_to_packages
...
Moving gateway and storage driver to packages
2014-11-29 16:33:07 -08:00
96e42a8443
Adding initial documentation to erasure package
2014-11-29 14:59:09 -08:00
3da8e53ab8
Exposing NewEncoder since GetEncoder is no longer around
2014-11-29 14:54:21 -08:00
03beef3afc
Moving gateway and storage driver to packages
2014-11-29 14:42:22 -08:00
17a23876a2
Rename NewEncoder-->newEncoder
2014-11-29 14:41:20 -08:00
a86de74983
Merge pull request #50 from fkautz/pr_out_removing_lru_from_erasure_package
...
Removing LRU from erasure package
2014-11-29 14:39:07 -08:00
922e6bac0b
Erasure restructuring
2014-11-29 14:28:55 -08:00
9ef1ef5c8c
Removing LRU from erasure package
2014-11-29 13:45:44 -08:00
f1fcd3be06
Make sure to use SIMD Alignment with 32
2014-11-27 02:15:41 -08:00
3cefaf5ad8
Restructure directory, add 'pkgs' folder now
2014-11-24 15:10:25 -08:00
Harshavardhana