Is it possible to read the lines of a file directly into an array without a while read line; do … done<file;
I mead something more direct.
BashFAQ - Greg’s Wiki bookmark here it’ll answer most from my brief look yes but probably not good to do.
Though it says while, still read it, it has another suggestion.
At first I thought, nah, it isn’t possible, but turns out it is with some sneaky use of IFS, and the -a and -d options of declare and read.
declare -a FOO
IFS='
' read -d^D -a FOO < inputfile
echo ${FOO[li]}[/li]```
We change the Internal Field Separator to newline and the delimiter character to control-D (any other character that doesn't appear in the file is ok, note that isn't typed in as caret + D, it's typed in as control-V control-D) and read in all words, i.e. all lines into the array FOO. There is one flaw in the method, leading whitespace on the line is trimmed by read, maybe you can fix that, or it doesn't matter to you. The declare is actually optional since -a of the read assumes an array.
Voila!
Amazing. I still learn something every day.Then I believe you could also use awk perhaps an example of that as I only found snippets 
On the faq they do it as so…
oIFS=$IFS IFS=$'
' arr=($(< myfile)) IFS=$oIFS
# Warning: breaks if lines contain "*" or similar
I find there a very good answer!
Thank you for the advise!
Thanks both of you!Both solutions are very interesting.
The second one I found it too on the wiki page.
Thanks again!
Actually it’s not true that leading whitespace is stripped, it was just the way I expanded ${FOO$i]}. I should have put that in ""s. So everything is cool.
NP with a bit of trial and error this would seem to be the awk solution not going to get involved whether it is bash or not. I just think many would class it as a tool of bash scripting.
IFS='
'
arr=(`awk 'BEGIN { FS="
"}{print $1 "
"}' this.txt`)
Perhaps Ken has a better understanding why? I usually use the construct $() but in this instance it doesn’t work I had to backtick. Using the other I only ended up with a var rather than an array.
Works for me.
IFS='
' arr=($(awk 'BEGIN { FS="
"}{print $1 "
"}' inputfile))
Thankyou Ken
I see now what I did I’ve got used to the single ( and forgot I needed the substitution inside brackets.
On Fri, 2009-07-24 at 12:06 +0000, ionpetrache wrote:
> Is it possible to read the lines of a file directly into an array
> without a while read line; do … done<file;
> I mead something more direct.
>
>
Yes and no… if the data size isn’t too big, for speed you might:
Assume the file is test.txt…
I took a guess at escaping certain things in the first sed.
I escaped doube quotes, dollar signs and backticks… there may be
others that need to be escaped.
sed -e '/^:blank:]]*$/d' -e 's/\("$`]\)/\\\1/g' test.txt |
cat -n | sed 's/^:blank:]]*\(^ ]*\).\(.*\)/line\1]="\2"/'
You could capture that output…
mylines=$( … the sed thing from above …)
eval “$mylines”
No warranty… but that would be fast as long as you don’t blow out the
environment space with the size of the variable mylines. Alternatively,
capture to a temp file and source the file into your script… that
would keep from blowing out the variable space.
Since this is our playground I have a solution with awk which automatically will put the entire contents of the input file into an array without any loop. here is an input file testfile.txt
aa aaa aab john 3
bb bbb bbc julia 2
cc ccc ccd george 3
aa aaa aab john 5
bb sds sss george 4
and here is the awk script awk_test:
#! /usr/bin/gawk -f
{
myarray$0]++;
}
END
{
print “Here is the content of the file:”;
for (field in myarray)
{
print field ; #, myarray[field];
}
print “That’s All Folks!!!”;
}
and after running the command(this can be done in a single line but i was lazy to look for the exact syntax):
awk -f awk_test < testfile.txt
here is the result:
"Here is the contents of the file:
cc ccc ccd george 3
aa aaa aab john 3
bb bbb bbc julia 2
aa aaa aab john 5
bb sds sss george 4
That’s All Folks!!! "
What I don’t understand is why would you need to put everything into an array when you could do the selection of the fields you want at the first run and then just print the tweaked results. You will need to output I assume only parts of the file otherwise why would you load it into an array?
I’m not sure what would be the max size the file awk could store it but you can try a large file and see if is processed or not.
if you want just specific fields to be loaded that’s very easy and instead of $0 load the field/fields you want. You can select multiple ones (ex. myarray$1,$3,$4]) and you automatically have a count of the number of occurences for the specific string in the field that you want.(myarray[field])
for example if change the array to myarray$2] the result would look like this
“Here is the content of the file:
sds 1 # of occurences of this string(sds) in field 2
aaa 2
bbb 1
ccc 1
That’s All Folks!!!”
cheers!
take note that doing it this way, the lines will not be the same (ie jumbled) when printing it out using for loop. the more "correct " way to preserve line numbers is
myarray++f] = $0
or
myarray[NR]=$0
I’m not sure what would be the max size the file awk could store it but you can try a large file and see if is processed or not.
it will be as much as what the memory on the system can take.