problem automating scp with cron

Hey Everyone,

I’ve STTF, but I can’t seem to find the cause of a problem I’m having. I’m trying to automate the following scp command vi cron.

This is a bone stock install of Opensuse, with only the keys having been generated.

Here’s my command (with private info changed):
/usr/bin/scp -i /home/myusername/.ssh/id_rsa -P 1234 myftpname@mydestination.com:/home/myftpname/Back1tUp/* /home/myusername/backups/

This works great from command line, great from my script, but when I put it in my crontab (using crontab -e), It fails. I have tried putting it in the root crontab and doing:
su - myusername -c /home/me/myscript.sh >>logfile
That fails too.

When I put my scp command in verbose logging mode, Here’s the difference between the good version (run from command line), and the automated version (cron)

GOOD:
ebug1: Authentications that can continue: publickey,password,keyboard-interactive

debug3: start over, passed a different list publickey,password,keyboard-interactive

debug3: preferred publickey,keyboard-interactive,password

debug3: authmethod_lookup publickey

debug3: remaining preferred: keyboard-interactive,password

debug3: authmethod_is_enabled publickey

debug1: Next authentication method: publickey

debug1: Offering public key:

debug3: send_pubkey_test

debug2: we sent a publickey packet, wait for reply

debug1: Server accepts key: pkalg ssh-rsa blen 149

debug2: input_userauth_pk_ok: fp 40:<redacted>:fb

debug3: sign_and_send_pubkey

debug1: Authentication succeeded (publickey).

debug2: fd 4 setting O_NONBLOCK

debug2: fd 5 setting O_NONBLOCK

debug2: fd 6 setting O_NONBLOCK

debug1: channel 0: new [client-session]

debug3: ssh_session2_open: channel_new: 0

debug2: channel 0: send open

debug1: Entering interactive session.

debug2: callback start

debug2: client_session2_setup: id 0

BAD:
debug1: Authentications that can continue: publickey,password,keyboard-interactive

debug3: start over, passed a different list publickey,password,keyboard-interactive

debug3: preferred publickey,keyboard-interactive,password

debug3: authmethod_lookup publickey

debug3: remaining preferred: keyboard-interactive,password

debug3: authmethod_is_enabled publickey

debug1: Next authentication method: publickey

debug1: Trying private key: /home/myusername/.ssh/id_rsa

debug1: PEM_read_PrivateKey failed

debug1: read PEM private key done: type <unknown>

debug1: read_passphrase: can’t open /dev/tty: No such device or address

debug2: no passphrase given, try next key

debug1: Trying private key: /home/myusername/.ssh/id_dsa

debug3: no such identity: /home/myusername/.ssh/id_dsa

debug2: we did not send a packet, disable method

debug3: authmethod_lookup keyboard-interactive

debug3: remaining preferred: password

debug3: authmethod_is_enabled keyboard-interactive

debug1: Next authentication method: keyboard-interactive

debug2: userauth_kbdint

debug2: we sent a keyboard-interactive packet, wait for reply

Here are some things I have added to help correct the problem:
export HOME="/home/myusername"

. /home/myusername/.profile
. /home/myusername/.env

Sorry for the long post, but thanks in advance for your help!

Instead of putting it in root’s crontab (I assume you’re referring to the tabs in /var/spool/cron/tabs), create a file with your username and put it in that. See if that works.

I run scp and rsync both from crontabs all the time, by the way. Just got to figure out why it’s not finding the correct key. Try what I said.

k, made some changes. Here’s my script:

#!/usr/bin/sh

/usr/bin/scp -vvv -i /home/myusername/.ssh/id_rsa -P 2222 ftpusername@mydomain.com:/home/ftpusername/Back1tUp/g* /home/myusername/mydomain_backups/

exit 0

Here’s how it was run:
21 23 * * * /home/myusername/backup_mydomain.sh >> backup_asnick.log 2>&1

And here’s the output to the log:

Executing: program /usr/bin/ssh host mydomain.com, user ftpusername, command scp -v -f /home/ftpusername/Back1tUp/g*
OpenSSH_5.0p1, OpenSSL 0.9.8g 19 Oct 2007
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Applying options for *
debug2: ssh_connect: needpriv 0
debug1: Connecting to mydomain.com [myipaddress port 2222.
debug1: Connection established.
debug3: Not a RSA1 key file /home/myusername/.ssh/id_rsa.
debug2: key_type_from_name: unknown key type ‘-----BEGIN’
debug3: key_read: missing keytype
debug2: key_type_from_name: unknown key type ‘Proc-Type:’
debug3: key_read: missing keytype
debug2: key_type_from_name: unknown key type ‘DEK-Info:’
debug3: key_read: missing keytype
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug2: key_type_from_name: unknown key type ‘-----END’
debug3: key_read: missing keytype
debug1: identity file /home/myusername/.ssh/id_rsa type -1
debug1: Remote protocol version 2.0, remote software version OpenSSH_4.3
debug1: match: OpenSSH_4.3 pat OpenSSH*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_5.0
debug2: fd 3 setting O_NONBLOCK
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug2: kex_parse_kexinit: diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
debug2: kex_parse_kexinit: ssh-rsa,ssh-dss
debug2: kex_parse_kexinit: aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,arcfour128,arcfour256,arcfour,aes192-cbc,aes256-cbc,rijndael-cbc@lysator.liu.se,aes128-ctr,aes192-ctr,aes256-ctr
debug2: kex_parse_kexinit: aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,arcfour128,arcfour256,arcfour,aes192-cbc,aes256-cbc,rijndael-cbc@lysator.liu.se,aes128-ctr,aes192-ctr,aes256-ctr
debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: none,zlib@openssh.com,zlib
debug2: kex_parse_kexinit: none,zlib@openssh.com,zlib
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit: first_kex_follows 0
debug2: kex_parse_kexinit: reserved 0
debug2: kex_parse_kexinit: diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
debug2: kex_parse_kexinit: ssh-rsa,ssh-dss
debug2: kex_parse_kexinit: aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,arcfour128,arcfour256,arcfour,aes192-cbc,aes256-cbc,rijndael-cbc@lysator.liu.se,aes128-ctr,aes192-ctr,aes256-ctr
debug2: kex_parse_kexinit: aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,arcfour128,arcfour256,arcfour,aes192-cbc,aes256-cbc,rijndael-cbc@lysator.liu.se,aes128-ctr,aes192-ctr,aes256-ctr
debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: none,zlib@openssh.com
debug2: kex_parse_kexinit: none,zlib@openssh.com
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit: first_kex_follows 0
debug2: kex_parse_kexinit: reserved 0
debug2: mac_setup: found hmac-md5
debug1: kex: server->client aes128-cbc hmac-md5 none
debug2: mac_setup: found hmac-md5
debug1: kex: client->server aes128-cbc hmac-md5 none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
debug2: dh_gen_key: priv key bits set: 119/256
debug2: bits set: 494/1024
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
debug3: put_host_port: [myipaddress]:2222
debug3: put_host_port: [mydomain.com]:2222
debug3: check_host_in_hostfile: filename /home/myusername/.ssh/known_hosts
debug3: check_host_in_hostfile: match line 1
debug3: check_host_in_hostfile: filename /home/myusername/.ssh/known_hosts
debug3: check_host_in_hostfile: match line 1
debug1: Host ‘[mydomain.com]:2222’ is known and matches the RSA host key.
debug1: Found key in /home/myusername/.ssh/known_hosts:1
debug2: bits set: 520/1024
debug1: ssh_rsa_verify: signature correct
debug2: kex_derive_keys
debug2: set_newkeys: mode 1
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug2: set_newkeys: mode 0
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug2: key: /home/myusername/.ssh/id_rsa ((nil))
debug1: Authentications that can continue: publickey,password,keyboard-interactive
debug3: start over, passed a different list publickey,password,keyboard-interactive
debug3: preferred publickey,keyboard-interactive,password
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Trying private key: /home/myusername/.ssh/id_rsa
debug1: PEM_read_PrivateKey failed
debug1: read PEM private key done: type <unknown>
debug1: read_passphrase: can’t open /dev/tty: No such device or address
debug2: no passphrase given, try next key
debug2: we did not send a packet, disable method
debug3: authmethod_lookup keyboard-interactive
debug3: remaining preferred: password
debug3: authmethod_is_enabled keyboard-interactive
debug1: Next authentication method: keyboard-interactive
debug2: userauth_kbdint
debug2: we sent a keyboard-interactive packet, wait for reply
debug1: Authentications that can continue: publickey,password,keyboard-interactive
debug3: userauth_kbdint: disable: no info_req_seen
debug2: we did not send a packet, disable method
debug3: authmethod_lookup password
debug3: remaining preferred:
debug3: authmethod_is_enabled password
debug1: Next authentication method: password
debug1: read_passphrase: can’t open /dev/tty: No such device or address
debug3: packet_send2: adding 64 (len 53 padlen 11 extra_pad 64)
debug2: we sent a password packet, wait for reply
debug1: Authentications that can continue: publickey,password,keyboard-interactive
Permission denied, please try again.
debug1: read_passphrase: can’t open /dev/tty: No such device or address
debug3: packet_send2: adding 64 (len 53 padlen 11 extra_pad 64)
debug2: we sent a password packet, wait for reply

Permission denied, please try again.
debug1: read_passphrase: can’t open /dev/tty: No such device or address
debug3: packet_send2: adding 64 (len 53 padlen 11 extra_pad 64)
debug2: we sent a password packet, wait for reply
Received disconnect from myipaddress: 2: Too many authentication failures for ftpusername

doesn’t it seem like it can’t read the key? I just use the command crontab -e to create cron entry… i’m presuming suse is fine with this approach…

Thanks for looking at this… i’m stumped.

Don’t slap me for asking this, but it looks like you’re using different usernames and/or home directories when you run it from the CLI, as opposed to from inside the cron tab. If not (if I’m just reading it wrong), let me repeat my suggestion.

Try it by inserting a plan text crontab file directly into /var/spool/cron/tabs. Name this file with your username – “joe,” or “sally.” (Edited) - do it with a standard text editor. Don’t use “crontab -e.”

In that file, put your cron job. Ex., run it once a day at 10 minutes past midnight, use

10 0 * * * /full/path/to/script --args

A tab inserted in the /var/spool area doesn’t need the username as part of the cron job, because the filename explicitly implies the user.

I don’t think I’m running them from different locations.

In my cron I was specifying /home/myusername/script.sh
from the CLI was in /home/myusername and running script.sh.

I opened the myusername file in /var/spool/cron/tabs and saw that my cron was already in there. I took it out and put it back in.
19 18 * * * /home/myusername/backup.sh >> backup.log 2>&1

Unfortunately i’m seeing the same output. Looks like it just can’t read the key. I even inserted a cd /home/myusername in the script to make sure I head back to my home directory (for whatever it’s worth).

Any other suggestions? I really don’t want to default back to regular ftp or anything insecure like that.

Maybe you should work out why it’s complaining about the id_rsa file. AFAIK it should have only a RSA key and none of the other lines it complained about. It should look like this:

-----BEGIN RSA PRIVATE KEY-----
about a dozen lines of key
-----END RSA PRIVATE KEY-----

Did you generate that file somewhere else?