Page 1 of 3 123 LastLast
Results 1 to 10 of 21

Thread: Need script to move files and create subfolders.

  1. #1

    Lightbulb Need script to move files and create subfolders.

    I have an OWL Document management server running on OpenSuse that has been in production for several years.
    During this time they have managed to scan in over 100,000 documents to tif and pdf and word files stored in a few folders.
    Problem: Folder browsing performance sucks.
    To increase performance, the logical thing is to create subfolders based on the file creation or modified date stamp, and move files to subfolders thus decreasing the individual files count, thus increasing browsing performance.
    I need a script that will:
    Create subfolders based on month, Year, and move files to those folders.
    Any suggestions?
    Or an App that will do this in a GUI would be great.
    Any help is appreciated.

  2. #2

    Default Re: Need script to move files and create subfolders.

    Quote Originally Posted by mpaint View Post
    I have an OWL Document management server running on OpenSuse that has been in production for several years.
    During this time they have managed to scan in over 100,000 documents to tif and pdf and word files stored in a few folders.
    Problem: Folder browsing performance sucks.
    To increase performance, the logical thing is to create subfolders based on the file creation or modified date stamp, and move files to subfolders thus decreasing the individual files count, thus increasing browsing performance.
    I need a script that will:
    Create subfolders based on month, Year, and move files to those folders.
    Any suggestions?
    Or an App that will do this in a GUI would be great.
    Any help is appreciated.
    I'd be surprised if there is a GUI-based automation tool for this kind of task but you never know . Otherwise it looks like you've set yourself a nice project in running bash scripts revolving around the use of mkdir and date (or date +"%y.%m.%d"). I must confess I couldn't do this entirely in Linux shell script (branch points I have scripted but not loops!), but I'm sure there are enough bash gurus around to help. Have you considered using a coding language that offers convenient string handling (e.g. Python)?

  3. #3
    dd NNTP User

    Default Re: Need script to move files and create subfolders.

    > Document management server running on OpenSuse that has
    > been in production for several years.


    -=Welcome=- new poster!

    i wonder what version of openSUSE you are running, so therefore ask you
    to please show us the terminal output from

    Code:
    cat /etc/SuSE-release
    and i wonder whether or not it is connected to a network, internal or
    external..

    --
    dd

  4. #4

    Default Re: Need script to move files and create subfolders.

    Thanks for looking folks!
    openSUSE 12.2 Mantis i586
    Running on a Poweredge 2850 with about 400GB of Raid 5 and dual dual core Xeons at 3GHZ or so and 4GB of ECC.
    This will be its only task.
    And is currently connected to LAN and Internet.
    Yes, I have seen some bash scripts, not that deep into shell programming.
    But something like that.

  5. #5
    dd NNTP User

    Default Re: Need script to move files and create subfolders.

    On 11/28/2012 11:46 PM, mpaint wrote:
    > openSUSE 12.2 Mantis i586


    ok, but your "running on OpenSuse that has been in production for
    several years." had me expecting more like openSUSE 11.x or even 10.x

    --
    dd

  6. #6

    Default Re: Need script to move files and create subfolders.

    The original server we are replacing is on SUSE enterprise 10.1 i believe...this is the new server. ;-)

  7. #7
    Join Date
    Feb 2009
    Location
    Spain
    Posts
    25,547

    Default Re: Need script to move files and create subfolders.

    On 2012-11-28 20:46, mpaint wrote:

    > I need a script that will:
    > Create subfolders based on month, Year, and move files to those
    > folders.
    > Any suggestions?


    You have to determine a criteria for detecting which files to save
    where. Perhaps date of the files themselves?

    Also are all the files in the same folder, or distributed?

    > Or an App that will do this in a GUI would be great.
    > Any help is appreciated.


    It doesn't look complicated so far, just busy coding.

    You could use 'find' to generate a list of files between two dates, and
    move or copy to a selected destination.

    --
    Cheers / Saludos,

    Carlos E. R.
    (from 12.1 x86_64 "Asparagus" at Telcontar)

  8. #8

    Default Re: Need script to move files and create subfolders.

    Quote Originally Posted by robin_listas View Post
    It doesn't look complicated so far, just busy coding.
    ?

    Code:
    #! /bin/bash
    
    src_dir=/tmp/srcdir
    dst_base=/tmp/destdir
    
    cd $src_dir
    
    for f in *; do
            dst_dir=${dst_base}/$(stat -c "%y" $f | awk '{ gsub(/-/,"",$1) ; print $1 }')
            [ -d  $dst_dir ] || mkdir -p $dst_dir
            [ -d  $dst_dir ] && mv $f $dst_dir
    done
    You'll have to replace /tmp/srcdir and /tmp/destdir in this example.
    Although it's quite simple, I recommend you understand this code before using it.
    There are many different ways to do this in bash.

  9. #9
    dd NNTP User

    Default Re: Need script to move files and create subfolders.

    On 11/29/2012 12:36 AM, mpaint wrote:
    >
    > The original server we are replacing is on SUSE enterprise 10.1 i
    > believe...this is the new server. ;-)



    do you understand that there is quite a difference between SUSE
    Enterprise Linux and openSUSE??

    for one thing: one is a LONG term (SLES 10 SP1 is still supported),
    stable, dependable Enterprise class operating system while the current
    offering of openSUSE (12.2) will not be supported past Jan 2014 (site:
    https://en.opensuse.org/Lifetime) and continues to have
    sufficient growing pains that i won't use it on my daily driver..

    so, think about changing operating system version in the next 12 months,
    and every 12 to 18 months thereafter..

    and, before turning openSUSE loose with that RAID setup, i would
    *strongly* suggest you consider using an operating system which is
    certified by Dell for use on that hardware! i do NOT believe you will
    find any of Ubuntu, Fedora or openSUSE certified for _that_ hardware..

    on the other hand you might (probably will) find SUSE Linux Enterprise
    Server 10 or 11 listed!

    i'll leave it to you to find the spec sheet delivered by Dell with that
    hardware...but, i think you are on a dangerous (to your data) path.

    the forums for SUSE Enterprise Linux are at http://forums.suse.com/ and
    the ID/Pass you used here will work there also....additionally, you will
    find many SLES Admins there...some of which probably have the kind of
    script you are wanting (whereas here we are mostly desktop users helping
    other desktop users)..

    ymmv

    --
    dd

  10. #10

    Default Re: Need script to move files and create subfolders.

    mpaint wrote:
    > I have an OWL Document management server running on OpenSuse that has
    > been in production for several years.
    > During this time they have managed to scan in over 100,000 documents to
    > tif and pdf and word files stored in a few folders.
    > Problem: Folder browsing performance sucks.
    > To increase performance, the logical thing is to create subfolders
    > based on the file creation or modified date stamp, and move files to
    > subfolders thus decreasing the individual files count, thus increasing
    > browsing performance.


    I disagree. Having folders with subfolders creates an annoying interface
    for the users, IME & HO. The sucky performance is most likely down to
    the filesystem you have chosen. Personally, I use Reiser because it
    handles things like this well. But given the cloud hanging over it, you
    might want to look at xfs or even btrfs (keep good backups!).

    Alternatively, change the requirement. What app do they use for browsing
    that gives them sucky performance? Why are they browsing the whole set
    of files instead of searching to get a short list etc etc? e.g. is there
    a role for a database here or even one of the dreaded filesystem
    scanning bots.

Page 1 of 3 123 LastLast

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •