OpenSuse 11.2 64bit
Greetings. Some time ago I joined the Amazon S3 Cloud as an experiment. I tried the Mozilla S3fox client to access my Amazon Account, but kept getting Java errors. Recently Amazon released their own Web based client and this seems to work well. Both s3fox and the Web (Browser) based programs remind you of an old ftp client with two windows across which you can copy files to your Amazon “Buckets” or file folders. Then I came across s3fs. It allows you to mount your Amazon Service as a local file system. To me this sounded ideal. I found and downloaded s3fs-r191-1.src.rpm. It had several dependencies which I was able to provide but for the below. I am unable to find this in any opensuse repo. Any ideas on how I might solve this ?
rpmbuild --rebuild s3fs-r191-1.src.rpm
Installing s3fs-r191-1.src.rpm
warning: InstallSourcePackage at: psm.c:232: Header V4 DSA signature: NOKEY, key ID e4f1bc2d
warning: user builder does not exist - using root
warning: user builder does not exist - using root
error: Failed build dependencies:
libfuse-devel is needed by s3fs-r191-1.src
There’s a fuse-devel, perhaps that will do. You may have to edit the SPEC file. The acid test is if it builds ok. It sounds like the srpm was designed for a RPM based distro with different names for dependencies, RedHat perhaps.
Further details from the rpm. How do we go about editing a spec file ?
Name : s3fs Relocations: (not relocatable)
Version : r191 Vendor: PLD
Release : 1 Build Date: Tue 04 May 2010 21:39:04 CST
Install Date: (not installed) Build Host: src.th.pld-linux.org
Group : Applications/System Source RPM: (none)
Size : 20124 License: GPL v2
Signature : DSA/SHA1, Wed 05 May 2010 20:55:54 CST, Key ID af3f93bce4f1bc2d
Packager : PLD bug tracking system ( http://bugs.pld-linux.org/ )
URL : FuseOverAmazon - s3fs - Project Hosting on Google Code
Summary : FUSE-based file system backed by Amazon S3
Description :
s3fs is a fuse filesystem that allows you to mount an Amazon S3 bucket
as a local filesystem. It stores files natively and transparently in
S3 (i.e., you can use other programs to access the same files).
Maximum file size=5G.
s3fs is stable and is being used in number of production environments,
e.g., rsync backup to s3.
Distribution: PLD 3.0 (Th)
-rw-r–r-- 1 builder users 18485 May 4 21:39 s3fs-r191-source.tar.gz
-rw-r–r-- 1 builder users 1639 Mar 15 18:57 s3fs.spe
You “install” the srpm which will unpack it to /usr/src/packages and the spec file will go into /usr/src/packages/SPECS. You edit, then build with rpmbuild.
You might be better off building from the source tarball.
Went back to look at the tarball once again. I studied the output, it appeared that g++ could not be found.
make: g++: Command not found
After finding and installing packages to meet this dependency, I was unsuccessful until I stumbled across “cnf” or command not found.
Using cnf g++
The program ‘g++’ can be found in following packages:
The following NEW packages are going to be installed:
gcc44-c++ gcc-c++ libstdc++44-devel
3 new packages to install.
Overall download size: 6.2 MiB. After the operation, additional 28.5 MiB will be used.
Continue? [y/n/?] (y): y
Using this command g++ installed and s3fs compiled
make install
g++ -ggdb -Wall -D_FILE_OFFSET_BITS=64 -I/usr/include/fuse -pthread -lfuse -lrt -ldl -lcurl -I/usr/include/libxml2 -lxml2 -lz -lm -lcrypto s3fs.cpp -o s3fs
s3fs.cpp: In function ‘std::string calc_signature(std::string, std::string, std::string, curl_slist*, std::string)’:
s3fs.cpp:453: warning: value computed is not used
s3fs.cpp: In function ‘int put_local_fd(const char*, headers_t, int)’:
s3fs.cpp:794: warning: format ‘%llu’ expects type ‘long long unsigned int’, but argument 4 has type ‘__off_t’
s3fs.cpp: In function ‘int s3fs_readlink(const char*, char*, size_t)’:
s3fs.cpp:892: warning: comparison between signed and unsigned integer expressions
s3fs.cpp: At global scope:
s3fs.cpp:467: warning: ‘size_t readCallback(void*, size_t, size_t, void*)’ defined but not used
ok!
cp -f s3fs /usr/bin
I have not had a chance to try it yet, Here’s hoping ; )
>
>ken_yap;2181925 Wrote:
>> You “install” the srpm which will unpack it to /usr/src/packages and the
>> spec file will go into /usr/src/packages/SPECS. You edit, then build
>> with rpmbuild.
>>
>> You might be better off building from the source tarball.
>
>Went back to look at the tarball once again. I studied the output, it
>appeared that g++ could not be found.
>make: g++: Command not found
>After finding and installing packages to meet this dependency, I was
>unsuccessful until I stumbled across “cnf” or command not found.
>
>Using cnf g++
>
>
>The program ‘g++’ can be found in following packages:
>
>
>* gcc-c++ path: /usr/bin/g++, repository: zypp (repo-oss) ]
>* gcc-c++ path: /usr/bin/g++, repository: zypp (openSUSE 11.2-0) ]
>
>Try installing with:
>zypper install gcc-c++
>
>The following NEW packages are going to be installed:
>gcc44-c++ gcc-c++ libstdc++44-devel
>
>3 new packages to install.
>Overall download size: 6.2 MiB. After the operation, additional 28.5
>MiB will be used.
>Continue? [y/n/?] (y): y
>
>
>Using this command g++ installed and s3fs compiled
>
># make install
>g++ -ggdb -Wall -D_FILE_OFFSET_BITS=64 -I/usr/include/fuse -pthread
>-lfuse -lrt -ldl -lcurl -I/usr/include/libxml2 -lxml2 -lz -lm
>-lcrypto s3fs.cpp -o s3fs
>s3fs.cpp: In function ‘std::string calc_signature(std::string,
>std::string, std::string, curl_slist*, std::string)’:
>s3fs.cpp:453: warning: value computed is not used
>s3fs.cpp: In function ‘int put_local_fd(const char*, headers_t, int)’:
>s3fs.cpp:794: warning: format ‘%llu’ expects type ‘long long unsigned
>int’, but argument 4 has type ‘__off_t’
>s3fs.cpp: In function ‘int s3fs_readlink(const char*, char*, size_t)’:
>s3fs.cpp:892: warning: comparison between signed and unsigned integer
>expressions
>s3fs.cpp: At global scope:
>s3fs.cpp:467: warning: ‘size_t readCallback(void*, size_t, size_t,
>void*)’ defined but not used
>ok!
>cp -f s3fs /usr/bin
>
>I have not had a chance to try it yet, Here’s hoping ; )
Hold the presses. Are you trying to tell us that you are trying to
compile from tarballs without installing the Development pattern first?
Some people like me, prefer to install dev packages as needed instead of accepting the whole shebang which contains a lot of stuff that might be needed or never be used. As long as you know what to do on cnf, that’s fine.
>
>JosephKK;2182401 Wrote:
>> On Mon, 28 Jun 2010 22:06:02 GMT, cabernet
>> <cabernet@no-mx.forums.opensuse.org> wrote:
>>
>> >
>> >ken_yap;2181925 Wrote:
>> >> You “install” the srpm which will unpack it to /usr/src/packages and
>> the
>> >> spec file will go into /usr/src/packages/SPECS. You edit, then
>> build
>> >> with rpmbuild.
>> >>
>> >> You might be better off building from the source tarball.
>> >
>> >Went back to look at the tarball once again. I studied the output, it
>> >appeared that g++ could not be found.
>> >make: g++: Command not found
>> >After finding and installing packages to meet this dependency, I was
>> >unsuccessful until I stumbled across “cnf” or command not found.
>> >
>> >Using cnf g++
>> >
>> >
>> >The program ‘g++’ can be found in following packages:
>>
>> >
>> >
>> >* gcc-c++ path: /usr/bin/g++, repository: zypp (repo-oss) ]
>> >* gcc-c++ path: /usr/bin/g++, repository: zypp (openSUSE 11.2-0) ]
>> >
>> >Try installing with:
>> >zypper install gcc-c++
>> >
>> >The following NEW packages are going to be installed:
>> >gcc44-c++ gcc-c++ libstdc++44-devel
>> >
>> >3 new packages to install.
>> >Overall download size: 6.2 MiB. After the operation, additional 28.5
>> >MiB will be used.
>> >Continue? [y/n/?] (y): y
>> >
>> >
>> >Using this command g++ installed and s3fs compiled
>> >
>> ># make install
>> >g++ -ggdb -Wall -D_FILE_OFFSET_BITS=64 -I/usr/include/fuse -pthread
>> >-lfuse -lrt -ldl -lcurl -I/usr/include/libxml2 -lxml2 -lz -lm
>> >-lcrypto s3fs.cpp -o s3fs
>> >s3fs.cpp: In function ‘std::string calc_signature(std::string,
>> >std::string, std::string, curl_slist*, std::string)’:
>> >s3fs.cpp:453: warning: value computed is not used
>> >s3fs.cpp: In function ‘int put_local_fd(const char*, headers_t,
>> int)’:
>> >s3fs.cpp:794: warning: format ‘%llu’ expects type ‘long long
>> unsigned
>> >int’, but argument 4 has type ‘__off_t’
>> >s3fs.cpp: In function ‘int s3fs_readlink(const char*, char*,
>> size_t)’:
>> >s3fs.cpp:892: warning: comparison between signed and unsigned integer
>> >expressions
>> >s3fs.cpp: At global scope:
>> >s3fs.cpp:467: warning: ‘size_t readCallback(void*, size_t, size_t,
>> >void*)’ defined but not used
>> >ok!
>> >cp -f s3fs /usr/bin
>> >
>> >I have not had a chance to try it yet, Here’s hoping ; )
>>
>> Hold the presses. Are you trying to tell us that you are trying to
>> compile from tarballs without installing the Development pattern first?
>
>Hmmmm - quite possibly yes, due to ignorance on my part. But hey the
>program works so all is well. What methodology would you use ?
Ken has a very different approach than me, you may wish to use his
methods instead. That said; start yast, go to software management, in
the (top) dropdown listbox that normally reads ‘search’ click the down
arrow and select patterns, in the scrollable window (well below) scroll
down to Development and start with base development and add the other
pieces that you may want like RPM Build Environment or most any
combination of the several other selections that suits your fancy.
For each pattern you may modify the default selection as you wish before
clicking ‘accept’.
S3fs is now functional. What made a difference for me was installing the companion program S3cmd. Both S3fs and S3cmd are command line programs for downloading/uploading and managing files on Amazon S3. After downloading the s3cmd-0.9.9.91.tar.gz package and extracting it I ran
python setup.py install
Next step was to configure the program
Enter new values or accept defaults in brackets with Enter.
Refer to user manual for detailed description of all options.
Access key and Secret key are your identifiers for Amazon S3
Access Key: YourID
Secret Key: YourSecret
Encryption password is used to protect your files from reading
by unauthorized persons while in transfer to S3
Encryption password: gpgpass
Path to GPG program [/usr/bin/gpg]:
When using secure HTTPS protocol all communication with Amazon S3
servers is protected from 3rd party eavesdropping. This method is
slower than plain HTTP and can’t be used if you’re behind a proxy
Use HTTPS protocol [No]: Yes
New settings:
Access Key: YourID
Secret Key: YourSecret
Encryption password: gpgpass
Path to GPG program: /usr/bin/gpg
Use HTTPS protocol: True
HTTP Proxy server name:
HTTP Proxy server port: 0
Test access with supplied credentials? [Y/n] n
Save settings? [y/N] y
Configuration saved to ‘/home/user/.s3cfg’
===========================================
Using S3cmd I was now able to log into my Amazon Account and view the Buckets (folders) and files on Amazon. S3fs also did its job. I was able to mount a Bucket or folder on my local filesystem. Using Midnight Commander I could copy files to Amazon for storage and backup. The S3cmd configuration offers encryption for files stored on Amazon and during the transfer process. On Opensuse, setting this up was straight forward. Amazon charges for all files moved into and out of their storage so be shure to check costs.
I am experimenting as to how I might use the Cloud. And this procedure is part of my efforts.
I installed s3fs r191 on an opensuse 11.3 system. I was able to mount some of the buckets I created. It failed to mount buckets with capital letters and underscores in the name. Also, of the bucket it succesfully mounted, I can only see the files located at the root of the backet, no directories are listed.
Has anyone had any success in mounting buckets on their systems?
I want to use s3fs to mount buckets as windows shares in a opensuse samba server, but not being able to list directories makes this not a viable solution.