ksh shell (run via bash) - script/path execution question

[oldcpu]

@karlmistelberger

From “man bash”:
**
ARGUMENTS**
If arguments remain after option processing, and neither the **-c **nor the **-s **option has been supplied, the first argument is assumed to be the name of a file containing shell commands. If **bash **is invoked in this fashion, **$0 **is set to the name of the file, and the positional parameters are set to the remaining arguments.

I presume #!/bin/bash and #!/bin/ksh are ignored in this case.

Wow.

I read that 5 times, and still don’t understand it. …

I don’t mind using the command line, but I think >90% of it often goes over my head. < sigh >

Anyway, problem solved ( running “ksh setup” ) works to get past the initial problem.

[oldcpu]

@arvidjaar

The problem is exactly due to bash and how it invokes external programs. The problem goes away if you use ksh to invoke this script. The script had exactly the same problem if it had /bin/bash shebang because it does not depend on which shell is invoked (except of course shell should be capable to interpret script), but how it is invoked.

Its all educational for me … until recently (ie a couple of years ago I retired and I now have more time on my hands) I avoided running command line scripts. Yes, using GNU/Linux since 1998 and I have religiously avoided scripts most of the time. … Strange you may think from someone who recommends running " /usr/sbin/alsa-info.sh " to help with audio, but I had an alsa developer teach me (to have that script run) back on an IRC channel over a decade ago. Other than the odd example like that, I have avoided scripts.

Hence when I saw the first line of the script was: " #!/bin/ksh" I had no clue as to what that meant initially, other than ksh rang a bell from something I read over 20 years ago about a ‘korn shell’. But 2 decades ago is a long time, and I was lost with other aspects … other tidbits in the problem - and spinning my wheels. So I decided it was time to post, take/receive ‘punches’ from those who know more, and hopefully make progress. It was ONLY after reading your first post in this thread questioning why I mentioned ‘ksh’ (which I did mention twice in my first post, as it was partially puzzling me) that the fact a script might have different behavour for a different shell occurred to me, but even then, it took me longer to realize that two different shells may not have been coded to the same spec (ie no common spec such that the commands must follow precisely an identical standard and thus scripts might run differently in two different shells).

I see this (expecting a 100% followed standard between shells) a lack of experience (and maybe niaivity) on my side . Others might see it as a lack of logic, but I don’t see it that way.

I do appreciate the help as my knowledge has increased.
.

[karlmistelberger]

@oldcpu

I read that 5 times, and still don’t understand it. … I don’t mind using the command line, but I think >90% of it often goes over my head. < sigh >

Examples:

Command Line Arguments in Linux Shell Scripts - TecAdmin

Bash - Hello World – TecAdmin

[tckosvic]

oldcpu, I am a has been engineer hacking in retirement in linux. I’ve been unix and linux user since 60s. Sounds like we have some commonality.

I appreciate your finding this app, looks like fun to play with. With some tweaking, I have gotten through steps including “./upscale – help” basically following steps in reference document.

I have ksh on my machine and ignored changing the “setup script” code. I ran “bash setup” to run setup. Couple of gliches in installs. There is a misplaced use of just python instead of python3 in the setup file. I modified that to python3 so as not to use python2. I needed also to pip3-install the “six” module which was not on my system but was required. The “six” module allows some kind of compatibility between python2 and python3 code. I was also surprised that many basic python packages such as scipy were reinstalled in the app directory as they exist as both system and local files already on my machine. Apparently the existing ones were not found…

Anyway, with paying no attention to /bin/ksh, in setup I successfully got to the last command in the installation steps “./upscale --help”.

Trying to use this might be another matter but looks like fun to try. Have to find a source of some old pictures to try to upgrade.

tom kosvic

[oldcpu]

@tckosvic

I appreciate your finding this app, looks like fun to play with. With some tweaking, I have gotten through steps including “./upscale – help” basically following steps in reference document.

Thanks for your post. Knowing someone else is ‘hacking at it’ gave me some encouragement.

@tckosvic

There is a misplaced use of just python instead of python3 in the setup file. I modified that to python3 so as not to use python2.

Thanks for pointing that out. I edited the ‘setup’ script there.

@tckosvic

I was also surprised that many basic python packages such as scipy were reinstalled in the app directory as they exist as both system and local files already on my machine. Apparently the existing ones were not found…

I also experienced this.

I currently have it running (this time on openSUSE-LEAP-15.3 on my new laptop) … and hopefully the app is not in an infinite loop, as its been running for a very long time now (over an hour). I do thou see its hidden output file is being populated with image files (it produces one new frame every 30-seconds), which I assume when complete will be recombined to make a video (although I don’t know if there are ‘multiple’ passes of frame generation). This test of mine is on a low 16-second 160x120 resolution @ 15 fps video.

Its a pretty basic video and the processing is incredibly slow on my core-i7 1165G7 CPU. I can see a higher resolution video on my PC could take a week or more. So if this app does not produce amazing quality, I am unlikely to use it.

The command I used (in ksm running in bash) on my ~16 second video:

source pytorch/bin/activate
./upscale inputvideo.mp4 --scene 00:00:00-00:00:16

I found I had to use that ‘scene’ specification to work around what I believe to be a bug in the script (thanks to an observation by arvidjaar ).

Anyway, I am way off topic now - as this is now openSUSE LEAP-15.3 (and not 15.2), the original problem is solved, and I am trying to get app running.
I am working on some other apps to improve resolution and frame rate, … wanting to ask for help, but at this stage don’t know enough to ask the right questions lol!

That thread is here:
https://forums.opensuse.org/showthread.php/563869-Looking-for-Deep-Machine-Learning-video-image-resolution-frame-rate-applications-for-upscaling

I will post more on that thread, after I make some progress with this upscale app.

[arnon_weinberg]

I am the author of the lousy script.

In my defense, the academic project that this script was written for is much worse. I’m actually quite pleased that you were able to get it working with so few tweaks, that’s far less work than I had to do to get it working. Anyway, feel free to submit issues via GitHub, I’m happy to help further if needed.

@hcvv

My impression is that the script is lousy. When the programmer thinks it is needed that the working ditectory should be the same directory where the script is, one of the first lines could have been (works in ksh)

cd $(dirname ${0})

Great idea, the script works much better now, thank you for that!

@arvidjaar

It means opt.scene is undefined (or does not exist). Briefly looking at upscale code - opt.scene refers to value of argument --scene and it does not check whether this argument is defined before using it.

Indeed. Thanks for tracking that down.

@tckosvic

oldcpu, I am a has been engineer hacking in retirement in linux. I’ve been unix and linux user since 60s. Sounds like we have some commonality.

Something like that here too.

@tckosvic

There is a misplaced use of just python instead of python3 in the setup file.

Ugh, yeah, that’s embarrassing, thanks for that.

@tckosvic

I needed also to pip3-install the “six” module which was not on my system but was required. The “six” module allows some kind of compatibility between python2 and python3 code.

Interesting. This seems to be installed by default with Python on my system.

@tckosvic

I was also surprised that many basic python packages such as scipy were reinstalled in the app directory as they exist as both system and local files already on my machine. Apparently the existing ones were not found…

The source project is not forward compatible with newer versions of scipy, so an older version is typically installed. Likely similar future forward-compatibility issues will break the script at some point.

@oldcpu

I currently have it running (this time on openSUSE-LEAP-15.3 on my new laptop) … and hopefully the app is not in an infinite loop, as its been running for a very long time now (over an hour). I do thou see its hidden output file is being populated with image files (it produces one new frame every 30-seconds)

Yeah, for me too, without a GPU, it’s painfully slow.

@oldcpu

which I assume when complete will be recombined to make a video

I’m afraid not - the source project only works on images. I added some features for extracting frames via ffmpeg that were useful for my testing. I’ve posted a question about a generic way to generate output video, but no answers so far.

@oldcpu

Its a pretty basic video and the processing is incredibly slow on my core-i7 1165G7 CPU. I can see a higher resolution video on my PC could take a week or more. So if this app does not produce amazing quality, I am unlikely to use it.

My thoughts exactly. Mostly just fun to play with.

@oldcpu

That thread is here:
https://forums.opensuse.org/showthread.php/563869-Looking-for-Deep-Machine-Learning-video-image-resolution-frame-rate-applications-for-upscaling

Thanks so much for posting this, I look forward to seeing updates!

[oldcpu]

@arnon_weinberg

I am the author of the lousy script.

In my defense, the academic project that this script was written for is much worse. I’m actually quite pleased that you were able to get it working with so few tweaks, that’s far less work than I had to do to get it working. Anyway, feel free to submit issues via GitHub, I’m happy to help further if needed.

Many thanks for creating it.

After the help on this thread, I have the script running on my Lenovo X1 Carbon Generation-9 laptop (intel-core-i7 1165G7 TigerLake CPU) running openSUSE LEAP-15.3 (with a slightly newer xorg and kernel than what comes with LEAP-15.3). I was not able to get it running on my older desktop with a core-i7-4770 (Haswell CPU) with a stock openSUSE LEAP-15.2 - BUT I have not tried that desktop yet with the fixes I learned from this thread, so probably I should try again.

I took an old video at 160x120 resolution and 15 fps, and upscaled it to 640x480 @ 15 fps, and then I used RIFE to increase frame rate to 30 fps, … and finally used ffmpeg to stabilize. I then did the same 160x120 @ 15fps => 640x480 @ 15fps => 640x480 @ 30fps ONLY with ffmpeg (using ffmpeg billinear for the resolution increase). I also stabilized the ffmpeg processed version (with ffmpeg).

Then I compared the two:

//youtu.be/IGtRqz8kdtA

There was a massive time difference in processing time, where ffmpeg took less than a minute, while super-resolution using STARnet together with RIFE (actually rife_ncnn_vulkan as I don’t have a powerful GPU on my computers) took greater than 150 minutes processing time !! … of course my Lenovo laptop has no high speed GPU.

At first, I liked the smoother ffmpeg on the LEFT, … but with time I began to prefer the sharper STARnet/RIFE combination on the RIGHT.

I wish I knew how to upscale a video ‘keeping’ the original pixelization (bicubic doesn’t do that) ie take 1 pixel and make it a 4x4 pixel with NO interpolation), but I don’t know how to do that. If I could do that I could better show the MASSIVE improvement the upscaling can do to the very old primitive digital videos (such as what this has done to this 160x120 => 640x480 video) .

[hcvv]

@arnon_weinberg

I am the author of the lousy script.

Hello and welcome to the openSUSE forums.

Maybe I was a bit harsh, but I wanted to express that the script was to be taken as not to well designed/written. A message for those who might think that a script offered with a product is always a good one. Sometimes people with less knowledge about shell scripts think, because they do not understand it, that it is good magic by definition.

I am very glad you found this thread and looked into it with the intention to gather the positive bits and pieces.

Wishing all success in making the product better.

[oldcpu]

@arnon_weinberg

Thanks so much for posting this, I look forward to seeing updates!

So to avoid spamming this thread any longer, I posted a comparison of the original video resized from 160x120 to 640x480 keeping original pixelization (best that I could) : https://forums.opensuse.org/showthread.php/563869-Looking-for-Deep-Machine-Learning-video-image-resolution-frame-rate-applications-for-upscaling?p=3095044#post3095044

This is the code I used to attempt to keep original poor quality when resizing to 640x480 … (where I wanted to do this for comparison purposes):

ffmpeg -i MVI_1002.AVI -s 640x480 -sws_flags neighbor -sws_dither none -vcodec rawvideo resized_video.avi

where “MVI_1002.AVI” was the name of my original video … and “resized_video” was the name of the original video resized to 640x480, with an effort to keep the original pixelization (ie 1 pixel in the 160x120 changed to 4x4 pixels (with no interpretation) in the 640x480). I don’t think I fully succeeded but I believe it comes close. The ouput MUST be an “avi” file for that command to work … try doing it with ‘mp4’ and it will fail (I believe).

Youtube thou reprocessed the videos , … so some of the quality demonstration aspects will have been lost by youtube changing the videos.

I showed 3 short 16-second videos:

• original (resized) vs STARnet upscale to 640x480
• original (resized) vs STARnet upscale (plus ffmpeg stabilization)
• ffmpeg upscale to 640x480 vs STARnet upscale to 640x480 (both videos stabilized).

Its been fun … although I might take a short break for some days and move on to another (video processing) hobby, and then return to this later.