Page 1 of 2 12 LastLast
Results 1 to 10 of 15

Thread: DNS is not case sensitive; caching, or lack thereof; python3 socket.getaddrinfo()

  1. #1

    Default DNS is not case sensitive; caching, or lack thereof; python3 socket.getaddrinfo()

    DNS is not case sensitive. m3047.net is equivalent to M3047.NeT, etc. That's the standard, and it's the way that nameservers work. Regarding the behavior described below: is this intentional, or is it just evolving randomly via bitrot and pain? What is the objective of caching?

    Where do I go with this line of inquiry? I'm not assuming a SuSE forum is going to answer a question this fundamental to network architecture and which spans all Linux distributions. However as we'll see, different distributions / releases do things slightly differently.

    Using python3 socket.getaddrinfo() for testing on SuSE Leap 15.0; although 15.1 might be different in some regards, unless this issue has been thoughtfully addressed I'll assume it persists in some form or another. Let's assume a wildcarded domain (like m3047.net) for the following (but please use your own domain, don't code that into your automated testing) and a caching resolver (BIND).

    Assuming that the caching resolver hasn't seen it before an initial request for SuSE.m3047.net will be resolved, cached and returned (in the answer section) with the oname SuSE.m3047.net. A subsequent request for suse.m3047.net will be resolved from cache and returned with the oname SuSE.m3047.net. This is all expected and correct behavior, there are RFCs so "I disagree" is not on the table. :-)

    SuSE Leap 15.0

    SCENARIO:

    We make a request e.g. socket.getaddrinfo('mARble.m3047.net',6379), which the wildcarded domain answers with fidelity as to fqdn case; the nameserver caches it preserving case fidelity; the DNS answer is returned to the client (over the wire) with the question fqdn matching the case of the answer oname, i.e. mARble.m3047.net. socket.getaddrinfo() completes successfully and correctly (with the correct address).

    In all cases, socket.getaddrinfo() returns successfully and correctly.

    Now if we issue a second call socket.getaddrinfo('mARble.m3047.net',6379), there is no DNS request/response, a correct answer is returned from some local cache.

    However, if we issue a third call socket.getaddrinfo('marble.m3047.net',6379), a DNS request is generated and answered by the caching resolver from its cache; the DNS answer is returned to the client (over the wire) with the question fqdn matching the request, but the answer oname matching the case of the original cached answer i.e. mARble.m3047.net.

    If we issue a fourth call socket.getaddrinfo('mARble.m3047.net',6379), there is no DNS request/response, a correct answer is returned from local cache.

    If we issue a fifth call socket.getaddrinfo('mARble.m3047.net',6379), there is no DNS request/response, a correct answer is returned from local cache.

    CONCLUSIONS:

    SuSE Leap has a local cache, although it is case sensitive.

    Debian 9.8 32 bit (in a VM)

    SCENARIO:

    We make a request e.g. socket.getaddrinfo('ballOOn.m3047.net',6379), and again the authoritative nameserver replies to the caching nameserver with fidelity, and again the caching nameserver caches it with fidelity.

    We make a second request socket.getaddrinfo('ballOOn.m3047.net',6379) and a DNS request is issued and answered by the caching nameserver from cache, with fidelity.

    We make a third request socket.getaddrinfo('balloon.m3047.net',6379) and a DNS request is issued and answered by the caching nameserver from cache, with the oname ballOOn.m3047.net. As noted previously socket.getaddrinfo() returns with the correct answer.

    We make a fourth request socket.getaddrinfo('balloon.m3047.net',6379) and a DNS request is issued and answered by the caching nameserver from cache.

    We make a fifth request socket.getaddrinfo('ballOOn.m3047.net',6379) and a DNS request is issued and answered by the caching nameserver from cache.

    CONCLUSIONS:

    Debian 9.8 does no local caching whatsoever. At least 32 bit in a VM. ;-)



    Who are the brilliant minds behind all of this? I suppose in the best of all possible worlds local caching does no harm, as long as you always get it right; which is why I'm slightly concerned that the cache can't see mARble and marble as equivalent.

    I was a little more concerned a few weeks ago, when Leap failed to resolve a mixed-case FQDN unless the case of the DNS reply exactly matched the query. However, that problem seems to have fixed itself after an OS update. But that's what drew my attention. Since VirtualBox for Leap only support 32 bit OS and Leap doesn't have 32 bit it's hard to go all out on testing without hardware lying around or a credit card.

    My next test iteration would involve DNS replies with serious brokenness.

  2. #2
    Join Date
    Aug 2010
    Location
    Chicago suburbs
    Posts
    12,503
    Blog Entries
    3

    Default Re: DNS is not case sensitive; caching, or lack thereof; python3 socket.getaddrinfo()

    As far as I know, "nscd" handles the caching. It caches for DNS. But it also caches for "/etc/hosts" which traditionally is taken as case sensitive. You can try stopping the "nscd" service to see if that solves your problem. But it might slow network lookups. (I just stopped it here, and have not noticed any resulting problems)

    I have not tried repeating your tests.

    I've been using KVM for virtualization, and that does support 64-bit. I have not used virtualbox, so I was not aware of that 32-bit limitation for virtualbox on Leap.
    openSUSE Leap 15.1; KDE Plasma 5;
    testing Leap 15.2Alpha

  3. #3
    Join Date
    Jun 2008
    Location
    Belleville, Ontario, Canada
    Posts
    439

    Default Re: DNS is not case sensitive; caching, or lack thereof; python3 socket.getaddrinfo()

    Since VirtualBox for Leap only support 32 bit OS and Leap doesn't have 32 bit it's hard to go all out on testing without hardware lying around or a credit card.


    And I've been using 64-bit guests on virtualbox on openSUSE since there have been 64-bit guests. Not sure where you got your info from...
    "Making rich people richer doesn't make the rest of us richer."
    Ha-Joon Chang

    openSUSE 15.0 4.12.14-lp150.12.45-default x64

  4. #4

    Default Re: DNS is not case sensitive; caching, or lack thereof; python3 socket.getaddrinfo()

    Quote Originally Posted by kerijan2003 View Post


    And I've been using 64-bit guests on virtualbox on openSUSE since there have been 64-bit guests. Not sure where you got your info from...


    That was fun. Couldn't do a screen grab with the menu showing, had to grab a camera.

    --

    Fred Morris

    "Showing people things doesn't make them see"

  5. #5

    Default Re: DNS is not case sensitive; caching, or lack thereof; python3 socket.getaddrinfo()

    Thanks nrickert.

    NSCD: Yes, you are correct. NSCD is performing the caching (verified). I assume it's case sensitive, haven't conducted additional testing or review to verify. Since it's just a caching subsystem, the downside is that it misses cache when it doesn't have to, which is not fatal.

    /etc/hosts: This is NOT case sensitive:
    sophia:/etc # tail -n4 hosts
    10.0.0.224 sophia.m3047 sophia
    127.0.0.22 TWENTYTWO


    sophia:/etc # ping twentytwo
    PING TWENTYTWO (127.0.0.22) 56(84) bytes of data.
    64 bytes from TWENTYTWO (127.0.0.22): icmp_seq=1 ttl=64 time=0.040 ms
    ^C
    --- TWENTYTWO ping statistics ---
    1 packets transmitted, 1 received, 0% packet loss, time 0ms
    rtt min/avg/max/mdev = 0.040/0.040/0.040/0.000 ms
    sophia:/etc # ping TWENTYTWO
    PING TWENTYTWO (127.0.0.22) 56(84) bytes of data.
    64 bytes from TWENTYTWO (127.0.0.22): icmp_seq=1 ttl=64 time=0.039 ms
    ^C
    --- TWENTYTWO ping statistics ---
    1 packets transmitted, 1 received, 0% packet loss, time 0ms
    rtt min/avg/max/mdev = 0.039/0.039/0.039/0.000 ms

    (Works with Python3 socket.getaddrinfo() as well.) Note that I very assiduously used ping not dig, guaranteeing that local name resolution was used (because files is listed first in nsswitch.conf).

    There is nothing saying that upper or mixed case is forbidden, but there are internet best practices captured in RFCs which suggest that hostnames should be lower case, plus you can review man -S5 hostname:

    however, it is recommended that it consists only of 7-bit ASCII lower-case characters

    In spite of best practices, things end up in zonefiles in either upper or lower case depending on the tools you use. That's why name resolution should not be case sensitive. If you want to understand the hell that is case sensitivity in name resolution, look for some of the threads regarding NIS. NIS, ironically brought to us by Scott "the network is the computer" McNeely, is the only directory scheme (not to be confused with "directory" as in a file name) in any real use which is case sensitive that I know of.




    I'm investigating KVM. If it works I'll go to that. This requires more environment than Docker.

    In the old days we used to set the entries in the various files that make name resolution work by hand. Now we use GUIs. Since I've been reviewing it I'm rather shocked at how bad it's gotten, including entriess in nsswitch for MDNS which I didn't ask for (or use), no FQDN in hosts, multiple entries in hosts, DHCP updating manually configured resolvers in resolv.conf by default (if I wanted to use Gmrgle I'd set them myself, don't need my fat fingered broadband provider doing it for me).

    I hadn't really paid close attention for a few years, until suddenly in June it started being case sensitive with DNS resolutions, which just as suddenly cleared up after an OS update. At this point if it comes back I'm probably somewhat prepared to investigate it further.

    At the time I simply "routed around the damage" as the saying goes, since I was using python and trying to resolve the address of a Redis server I just used dnspython. The hack I came up with is in this file, you can find the relevant bits by searching for USE_DNSPYTHON in https://github.com/m3047/shodohflo/b.../pcap_agent.py


  6. #6
    Join Date
    Jun 2008
    Location
    San Diego, Ca, USA
    Posts
    11,143
    Blog Entries
    2

    Default Re: DNS is not case sensitive; caching, or lack thereof; python3 socket.getaddrinfo()

    Firstly,
    The most common reason for lack of 64-bit virtualization support is that you don't have CPU virtualization extensions enabled in your BIOS. You should enable in your BIOS, then re-install Virtualbox if necessary.

    As for case sensitive name resolution issues,
    I just did a small test and I don't see anything has changed since forever...
    Linux fundamentally is a system that enforces case sensitivity, but specific applications (and of course app components) might over-ride and support camel case.

    Try this test using wget...
    Code:
    wget www.google.com/index.html
    Your result should be a successful download of the Google search page (code 200)

    Because the download was successful, of course the act of the operation should be to place a fresh entry of the successful name resolution in your cache.

    Now, immediately so that if necessary your cache would be utilized (and of course executing the new query if your cache fails)
    Run the same command with various(at least one) camel case mixed in...
    eg
    Code:
    wget wWw.GoOgLe.CoM/inDex.html
    You'll find that the operation fails (error 404 File not found)

    So, conclusion...
    Both Linux as a system when doing name resolution and its name resolution cache are case sensitive as expected, but an application might not comply and support camel case.

    If you further want to test this using other apps, you'll find for instance that PING isn't case sensitive and web browsers generally aren't either.

    HTH,
    TSU
    Beginner Wiki Quickstart - https://en.opensuse.org/User:Tsu2/Quickstart_Wiki
    Solved a problem recently? Create a wiki page for future personal reference!
    Learn something new?
    Attended a computing event?
    Post and Share!

  7. #7
    Join Date
    Jun 2008
    Location
    San Diego, Ca, USA
    Posts
    11,143
    Blog Entries
    2

    Default Re: DNS is not case sensitive; caching, or lack thereof; python3 socket.getaddrinfo()

    Hmmmm...

    After thinkinga about my test further, I questioned the "file not found" error instead of a name resolution error.
    So, I repeated my test with "index.html" all in lower case,
    And found that results in a success.

    So indeed,
    The FQDN part of the URL is not case sensitive,
    And only the actual file name is case sensitive.

    So,
    It does look like there was a big change sometime in how name resolution works, not what is in the name resolution cache..

    Warrants further testing with various apps or a deeper inspection of how name resolution works today.

    TSU
    Beginner Wiki Quickstart - https://en.opensuse.org/User:Tsu2/Quickstart_Wiki
    Solved a problem recently? Create a wiki page for future personal reference!
    Learn something new?
    Attended a computing event?
    Post and Share!

  8. #8
    Join Date
    Jun 2008
    Location
    San Diego, Ca, USA
    Posts
    11,143
    Blog Entries
    2

    Default Re: DNS is not case sensitive; caching, or lack thereof; python3 socket.getaddrinfo()

    The following is a 2006 RFC that declares case insensitive names when using the host name resolution system (DNS)

    https://tools.ietf.org/html/rfc4343

    Skimming various Internet search hits,
    It looks like in general Linux complies with the standard, although there can be some special situations where there might exist an artificial network object, like a network overlay and since such a thing isn't the actual network might conform to different requirements.

    TSU
    Beginner Wiki Quickstart - https://en.opensuse.org/User:Tsu2/Quickstart_Wiki
    Solved a problem recently? Create a wiki page for future personal reference!
    Learn something new?
    Attended a computing event?
    Post and Share!

  9. #9

    Default Re: DNS is not case sensitive; caching, or lack thereof; python3 socket.getaddrinfo()

    Quote Originally Posted by tsu2 View Post
    Hmmmm...

    After thinking about my test further, I questioned the "file not found" error instead of a name resolution error.
    So, I repeated my test with "index.html" all in lower case,
    And found that results in a success.

    So indeed,
    The FQDN part of the URL is not case sensitive,
    And only the actual file name is case sensitive.
    Correct!

    Don't feel bad, when I tried your example I was in a readonly directory and got "Cannot write to ‘index.html’ (Permission denied)." LOL!

    [...]
    It does look like there was a big change sometime in how name resolution works, not what is in the name resolution cache..
    No, it's been that way as long as I've been around, which is before Linux. :-p

    My suspicion in regards to NSCD is that it's preserving compatibility with NIS. Probably hasn't changed, either.

    Warrants further testing with various apps or a deeper inspection of how name resolution works today.
    Yes, I agree. I'm concerned based on my experience in June that maybe people are starting to strip some gears on the whole notion. Maybe it would be useful to have case-sensitive hostnames, I mean the file system is case sensitive after all. I'm kidding! I admit, I hadn't even noticed the other bitrot in the config files until I started looking.

    It's a can of worms which is going to turn out to be otherworldly large. I concur, further testing or at least vigilance is called for. If I had one ask, it's this: don't offhand dismiss someone as crazy if they report that wWw.gooGLe.cOM doesn't resolve. (Makes me wonder if I could register it as a punycode domain.)

    For my part, I've paid more attention to the actual mechanics of name resolution... outside of the DNS... than I have in at least five years.

    --

    Fred

  10. #10
    Join Date
    Jan 2017
    Location
    Nürnberg, Germany
    Posts
    176

    Default Re: DNS is not case sensitive; caching, or lack thereof; python3 socket.getaddrinfo()

    Quote Originally Posted by m3047 View Post


    That was fun. Couldn't do a screen grab with the menu showing, had to grab a camera.
    Spectacle, the KDE screengrabbing tool, has a capture mode with adjustable time delay.
    After setting this delay to a few seconds and choosing »Take a New Screenshot«, it gives you time to change to the »Create Virtual Machine« window, then open the multiselect/pull-down/pop-up menu, then point at the 32bit entry and just wait. Spectacle should be able to then capture everything including mouse pointer.

    As for the DNS/DHCP/KVM/NSCD stuff, I've found this thread a very interesting read. Cheers!

Page 1 of 2 12 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •