Page 1 of 2 12 LastLast
Results 1 to 10 of 11

Thread: How do I browse a large file

  1. #1
    Join Date
    Jan 2018
    Location
    Annandale, VA
    Posts
    123

    Default How do I browse a large file

    Normally when I want to browse a file I use a text editor. However, many text editors load the entire file into memory and I need to browse a 2 GB file. Specifically, I need to capture a few screens of text after the second occurrence of the string TRAP.

    A secondary issue is that the file is likely to contain multiple occurrences of zero bytes (it's an OS/2 dump and the room my OS/2 system is in currently has no power.)

  2. #2
    Join Date
    Sep 2008
    Posts
    2,997

    Default Re: How do I browse a large file

    have you considered using a hex editor like Okteta (KDE) or GHex (Gnome)
    Hex editors usually load files from disk and support large files and as your files are not text files but dumps a hex editor seams like a better choice
    you could also try mc's (Midnight Commander) build in viewer/editor as it's a lot faster then kate/kedit/gedit or use a console editor like vim
    install mc run it then select the file and hit F3 or F4 mc is not installed by default but it's in the main repos

  3. #3
    Join Date
    Jun 2008
    Location
    Netherlands
    Posts
    24,973

    Default Re: How do I browse a large file

    I do not know if this covers your problem, but reading large text file in Unix/Linux there are tools like
    Code:
    more
    others prefer
    Code:
    less
    Henk van Velden

  4. #4

    Default Re: How do I browse a large file

    well, 2G is not that big. I take it you already tried grepping the file and it came back as binary, right?

    Unless you want to split the file, I suggest using vi. It does a fairly good job at very large files and it will not modify them when it comes to line endings and special chars.

    Just for the fun of it I pointed it at a 10G file. It took about a minute to load but then was perfectly browsable while using about 4G of RAM.

  5. #5
    Join Date
    Aug 2010
    Location
    Chicago suburbs
    Posts
    12,480
    Blog Entries
    3

    Default Re: How do I browse a large file

    I normally use "less" for that kind of task.

    Bytes that are binary zero, usually show up as "^@" (without the quotes) when using "less".
    openSUSE Leap 15.1; KDE Plasma 5;
    testing Leap 15.2Alpha

  6. #6
    Join Date
    Jun 2008
    Location
    San Diego, Ca, USA
    Posts
    11,109
    Blog Entries
    2

    Default Re: How do I browse a large file

    *NIX utilities usually stream instead of loading the entire file.
    In fact, last week I ran into a Microsoft KB article that required reading the contents of an extremely large text fileon MSWindows. Guess what tool they wanted you to use? VIM.

    TSU
    Beginner Wiki Quickstart - https://en.opensuse.org/User:Tsu2/Quickstart_Wiki
    Solved a problem recently? Create a wiki page for future personal reference!
    Learn something new?
    Attended a computing event?
    Post and Share!

  7. #7
    Join Date
    Dec 2008
    Location
    FL, USA
    Posts
    1,625

    Default Re: How do I browse a large file

    Quote Originally Posted by shmuelmetz View Post
    Normally when I want to browse a file I use a text editor. However, many text editors load the entire file into memory and I need to browse a 2 GB file. Specifically, I need to capture a few screens of text after the second occurrence of the string TRAP.
    Use the F3 and F7 keys in MC the same way you would use the them in FC/2. All my file browsing and most of my editing on OS/2 is with FC/2, while on Linux it's a relatively even split between MC and FC/L.

    A secondary issue is that the file is likely to contain multiple occurrences of zero bytes (it's an OS/2 dump and the room my OS/2 system is in currently has no power.)
    Given power to the OS/2 PC or HD, I'd probably copy the dump to another filename, then try F4 instead of F3. If it refuses to edit, then I'd F3, switch to another vtty, and either copy the content of /dev/vcs# to a file, then put newlines in that saved file where needed, or try a screencap of the F3 screen or window. If the "screens" of the dump literally take more than one screen, I'd configure Xorg with a (panning) virtual desktop 3 or more screens tall to fit the needed dump portion, then get a screenshot of a window using the full screen height. For the general idea e.g. http://fm.no-ip.com/SS/brunosFontDefaults1406-180L2.png 2880 px tall.
    Reg. Linux User #211409 *** multibooting since 1992
    Primary: 42.3,TW,15.0 & 13.1 on Haswell w/ RAID
    Secondary: eComStation (OS/2)&42.3 on 965P/Radeon
    Tertiary: TW,15.0,42.3,Fedora,Debian,more on Kaby Lake,Q45,Q43,G41,G3X,965G,Cedar,Caicos,Oland,GT218&&&

  8. #8
    Join Date
    Jun 2008
    Location
    San Diego, Ca, USA
    Posts
    11,109
    Blog Entries
    2

    Default Re: How do I browse a large file

    Quote Originally Posted by shmuelmetz View Post
    Normally when I want to browse a file I use a text editor. However, many text editors load the entire file into memory and I need to browse a 2 GB file. Specifically, I need to capture a few screens of text after the second occurrence of the string TRAP.

    A secondary issue is that the file is likely to contain multiple occurrences of zero bytes (it's an OS/2 dump and the room my OS/2 system is in currently has no power.)
    For your specific situation,

    If you use VIM,
    Open a windowed text console and re-size however you wish, anticipating the size of the text you want to capture in your screenshot.
    Code:
    vi filename
    Then you can skip to the next occurrence of any text string by typing a forward slash followed by your string
    Code:
    / text_string 
    Take your screenshot.
    Repeat the text search followed by another screenshot.
    Type the following to exit (assuming you've made no changes to the text)
    Code:
    :q
    Don't know what you might mean about your zero byte issue... Are you saying that it's captured somehow or reported in the text?

    TSU
    Beginner Wiki Quickstart - https://en.opensuse.org/User:Tsu2/Quickstart_Wiki
    Solved a problem recently? Create a wiki page for future personal reference!
    Learn something new?
    Attended a computing event?
    Post and Share!

  9. #9
    Join Date
    Jan 2018
    Location
    Annandale, VA
    Posts
    123

    Default Re: How do I browse a large file

    Quote Originally Posted by I_A View Post
    have you considered using a hex editor like Okteta (KDE) or GHex (Gnome)
    Hex editors usually load files from disk and support large files and as your files are not text files but dumps a hex editor seams like a better choice
    you could also try mc's (Midnight Commander) build in viewer/editor as it's a lot faster then kate/kedit/gedit or use a console editor like vim
    install mc run it then select the file and hit F3 or F4 mc is not installed by default but it's in the main repos
    I installed Okteta, but it crashed. MC worked once I googled how to select text for a copy. Thanks.

  10. #10
    Join Date
    Jan 2018
    Location
    Annandale, VA
    Posts
    123

    Default Re: How do I browse a large file

    Quote Originally Posted by hcvv View Post
    I do not know if this covers your problem, but reading large text file in Unix/Linux there are tools like
    Code:
    more
    others prefer
    Code:
    less
    I tried less, but It seemed to have problems with the file. Midnight Commander worked fine. Thanks.

Page 1 of 2 12 LastLast

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •