Results 1 to 10 of 10

Thread: 11.0: Stopping iSCSI target service: failed

  1. #1

    Default 11.0: Stopping iSCSI target service: failed

    On OpenSUSE 11.0 x86_64,
    kernel 2.6.25.5-1.1-default
    yast2-iscsi-server-2.16.1-26.1
    iscsitarget-0.4.15-63.1
    iscsitarget-kmp-default-0.4.15_2.6.25.5_1.1-63.1

    I cannot shut down iscsitarget child processes. This is preventing stopping and restarting iscsi, and is apparently holding open the associated storage devices.

    It appears that simply starting iscsitarget with the most recent kernel update prevents ever stopping the "[istd1]" and eight "[istiod1]" processes started for the first exported iSCSI device, the 9 istd2/istiod2 processes for the second, and so on. rciscsitarget start is ok, then iscsitarget stop fails due to the insmod -d failing. "In use"...by what?

    When heartbeat is in control of iscsitarget, this failure causes heartbeat to reboot the system by default whenever shutting down iscsitarget, instead of proceeding in an orderly shutdown.

    When the init script is patched so that the failure to insmod -d properly is masked, and ietd's presence is used to determine if it is running, and DRBD devices backed by LVs are used instead of LVs themselves, the DRBD devices cannot be unmounted and DRBD cannot be changed to secondary mode. This also causes heartbeat to reboot.

    lsof shows nothing using the drbd devices and the orphaned (? ietd is gone ?) processes with no open files

    ietd.conf:
    IncomingUser <deleted> <deleted>
    Target iqn.2008-07.com.koozlefrep:a277.target.sta
    Lun 0 Path=/dev/mapper/system-iscsi,Type=fileio
    IncomingUser <deleted> <deleted>

    /dev/mapper/system-iscsi is a raw unmounted LV.

    No matter how simple a configuration, I cannot seem to make iscsitarget work with heartbeat and drbd.

  2. #2
    Join Date
    Jun 2008
    Location
    The Netherlands
    Posts
    1,273

    Default Re: 11.0: Stopping iSCSI target service: failed

    Quote Originally Posted by normmackey View Post
    ...
    No matter how simple a configuration, I cannot seem to make iscsitarget work with heartbeat and drbd.
    I haven't used this configuration before & you are way ahead of me in this, but just thinking out loud.. could it be you need to create start and stop scripts that include the correct stop/start talking all these services in account (e.g. stop drdb, heartbeat and the iscsitarget and visa versa)?
    If you are stopping the itarget service itself it should not be an issue to have to stop heartbeat and drdb as well?

    Have you found a sequence that does work when doing it manually?

    -Wj
    Have a lot of fun!

  3. #3

    Default Re: 11.0: Stopping iSCSI target service: failed

    I haven't used this configuration before & you are way ahead of me in this, but just thinking out loud.. could it be you need to create start and stop scripts that include the correct stop/start talking all these services in account (e.g. stop drdb, heartbeat and the iscsitarget and visa versa)?
    If you are stopping the itarget service itself it should not be an issue to have to stop heartbeat and drdb as well?


    The presumed sequence: shut down iscsi first, before the backing DRBD storage is switched to secondary. heartbeat should keep running to handle the resource items in the opposite order as when started, and as a monitor afterwards.

    Have you found a sequence that does work when doing it manually?

    No, however I have tried. The basic problem seems to be that the iSCSI will not shut down, something stays attached to the devices' backing storage. My guess would be the impossible-to-terminate processes.

    Rather simple to reproduce. configure some iscsi disks in ietd.conf. rciscsitarget start; rciscsitarget stop, watch it fail. ps axf, see the processes.

  4. #4
    Join Date
    Jun 2008
    Location
    The Netherlands
    Posts
    1,273

    Default Re: 11.0: Stopping iSCSI target service: failed

    Quote Originally Posted by normmackey View Post
    [I]
    The presumed sequence: shut down iscsi first, before the backing DRBD storage is switched to secondary. heartbeat should keep running to handle the resource items in the opposite order as when started, and as a monitor afterwards.
    And if you switch the DRBD storage to secondary manually before iscsi? Is that possible?

    Can the lsof command assist in finding what file/process is holding the lock?
    Have a lot of fun!

  5. #5

    Default Re: 11.0: Stopping iSCSI target service: failed

    "drbdadm secondary all" fails. Reports devices in use.

    lsof shows no connections to the devices. I am not sure if I believe it, but this would presumably be a clue.

  6. #6
    pechal2 NNTP User

    Default Re: 11.0: Stopping iSCSI target service: failed

    Hi, I have the same problem - I have recently switched from opensuse 10.3 to opensuse 11 (because of better yast etc.) and my configuration of drbd and iscsi just don't work.
    Using latest Xen kernel (2.6.25.11-0.1-xen), I have some LVM logical volumes, which I am backing up through drbd and also making the drbd devices public through iscsi. It worked in 10.3, but 11 is not working for me. DRBD alone is working, when I start iscsitarget, it works too. But I can't shut down iscsitarget - it always fails.

    I tried to follow the iscsitarget init script to find the problem, and it definitely the kernel module iscsi_trgt. It can be unloaded only through rmmod -f iscsi_trgt and it leaves some locks behind, because drbd cannot be switched to secondary mode and it shouts, that something is still using it...

    Do you have any new information about this problem? It critical to me, I don't want to switch back to opensuse 10.3.

    Thanks, Luke

  7. #7
    Join Date
    Jun 2008
    Location
    The Netherlands
    Posts
    1,273

    Default Re: 11.0: Stopping iSCSI target service: failed

    It's probably your best bet to open a bug report for this as it's is not a setup combination you will find much (yet).

    You have a good chance of getting a bug squashed!

    Cheers,
    Wj
    Have a lot of fun!

  8. #8
    pechal2 NNTP User

    Default Re: 11.0: Stopping iSCSI target service: failed

    Thanks for an advice, I have posted it just now (I hope that I have filled in the form right, it is my first bug report ) )
    https://bugzilla.novell.com/show_bug.cgi?id=417286

  9. #9
    Join Date
    Jun 2008
    Location
    The Netherlands
    Posts
    1,273

    Default Re: 11.0: Stopping iSCSI target service: failed

    Hi Luke,

    I think the devs wished everyone would fill in a bug report like that!!
    As it is vacation time I don't know how quick you'll get response but the information you gave should be a good starting point.

    Wishing you luck,
    Wj
    Have a lot of fun!

  10. #10

    Default Re: 11.0: Stopping iSCSI target service: failed

    I rebuilt the latest iscsitarget-0.4.16 package from Sourceforge and drbd-8.2.6 from www.linbit.com: LINBIT Information Technologies and the 2.6.25.11-0.1 kernel source, installed the modules in the new kernel's /lib/modules/, then renamed the /etc/init.d/iscsi-target to /etc/init.d/iscsitarget, and the problems went away.

    But the new start/stop init script does not report status, etc, i am not certain about heartbeat's monitoring of it.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •