Your local linux box wont have a limit.
Web based ones will put a limit on to stop one user (/bot) from making zillions of requests and chewing up the service provider's data allowance/bandwidth/cpu/whatever other resources they are charged by.
whois [ -h HOST ] [ -p PORT ] [ -aCFHlLMmrRSVx ] [ -g SOURCE:FIRST-LAST ] [ -i ATTR ] [ -S SOURCE ] [ -T TYPE ] object
whois -t TYPE
whois -v TYPE
whois -q keyword
Step 2
"wh" ) whois $1@whois.tucows.com;;"wh-ripe" ) whois $1@whois.ripe.net;;"wh-apnic" ) whois $1@whois.apnic.net;;"wh-cw" ) whois $1@whois.cw.net;;* ) echo "Usage: `basename $0` [domain-name]";;esacexit $?
Your local linux box wont have a limit.
Web based ones will put a limit on to stop one user (/bot) from making zillions of requests and chewing up the service provider's data allowance/bandwidth/cpu/whatever other resources they are charged by.
Every registrar has its own output format. If you query a registrar who does not provide this information, it cannot be obtained (at least not using the whois protocol).
For your particular example of google.com
this
information is maintained by the top-level registrar at
whois.internic.net
, though. Apparently your
particular whois
client is not displaying the output
from whois.internic.net
. From mine, I get a lengthy
result from whois.internic.net
(including
information about registry entries beginning with
google.com.
which there are several dozen, some of
them rather embarrassing) before the information for this
particular domain from its registrar.
I'd recommend writing a simple whois
client of your
own if you really need to see exactly what is going on.
telnet whois.internic.net 43
and type in
=google.com
for a good start.
for DOMAIN in newcastle.co.uk guinness.co.uk ;do
echo "";
echo $DOMAIN;
whois $DOMAIN | awk -F: '/Registrar:/ && $0 != "" { getline; REGISTRAR=$0 } END { print REGISTRAR }';
whois $DOMAIN | awk -F: '/Expiry date:/ && $0 != "" { EXPDATE=$0 } END { print EXPDATE }';
done
Output:
newcastle.co.uk
Corporation Service Company (UK) Limited [Tag = CSC-CORP-DOMAINS]
Expiry date: 30-Mar-2014guinness.co.uk
Melbourne IT t/a Internet Names Worldwide [Tag = MELBOURNE-IT]
Expiry date: 06-Jan-2014
Why not just use the whois
command?
whois -h whois.internic.net =google.com > whois.txt
Though in this case I get a better answer from
whois =google.com > whois.txt
Addendum
Telnet is great for interactive exploration of arbitrary text-based TCP protocols (such as SMTP, WHOIS etc) but it isn't really good for scripting
try netcat
instead
$ echo =google.com | nc whois.internic.net 43 > whois.txt
$ head whois.txt
Whois Server Version 2.0
Domain names in the .com and .net domains can now be registered
with many different competing registrars. Go to http://www.internic.net
for detailed information.
Server Name: GOOGLE.COM.ZZZZZZZZZZZZZZZZZZZZZZZZZZZ.LOVE.AND.TOLERANCE.THE-WONDERBOLTS.COM
IP Address: 50.62.130.9
Registrar: GODADDY.COM, LLC
$ tail whois.txt
use electronic processes that are automated and high-volume to access or
query the Whois database except as reasonably necessary to register
domain names or modify existing registrations. VeriSign reserves the right
to restrict your access to the Whois database in its sole discretion to ensure
operational stability. VeriSign may restrict or terminate your access to the
Whois database for failure to abide by these terms of use. VeriSign
reserves the right to modify these terms at any time.
The Registry database contains ONLY .COM, .NET, .EDU domains and
Registrars.
$ grep -i status whois.txt
Status: clientDeleteProhibited
Status: clientTransferProhibited
Status: clientUpdateProhibited
Status: serverDeleteProhibited
Status: serverTransferProhibited
Status: serverUpdateProhibited
I get same output from netcat as I do from script + telnet. netcat is a lot easier
$ grep 'Name Server' whois.telnet | dos2unix | tee a
Name Server: NS1.GOOGLE.COM
Name Server: NS2.GOOGLE.COM
Name Server: NS3.GOOGLE.COM
Name Server: NS4.GOOGLE.COM
$ grep 'Name Server' whois.netcat | tee b
Name Server: NS1.GOOGLE.COM
Name Server: NS2.GOOGLE.COM
Name Server: NS3.GOOGLE.COM
Name Server: NS4.GOOGLE.COM
$ diff -s a b
Files a and b are identical
Other ideas
The canonical tool for scripting telnet is expect
.
The canonical tool for capturing terminal output is
script
.
In this case I think these would not suit your task (but you may disagree)
That's not too hard.
Check the list of domains domains.txt
, and add the
domain to available.txt
once it's found (by looking
at the return value of grep
, which is stored in
$?
).
Then, remove the found domains from domains.txt
with
sed
in-place editing.
#!/bin/bash
AVAILABLE=~/available.txt
DOMAINS=~/domains.txt
lockfile whois-script.lock
while read -r domain; do
whois $domain | grep -qci "No match"
if [ $? -ne 0 ]; then
# found
echo $domain >> $AVAILABLE
fi
done < $DOMAINS
while read -r domain; do
sed -i "/$domain/d" $DOMAINS
done < $AVAILABLE
rm -f whois-script.lock
Note: On BSD sed
, you want to use the following
command:
sed -i "" "/$domain/d" $DOMAINS
You can save this script and call it from your
crontab
. Enter
crontab -e
and then add a line like this:
*/2 * * * * /path/to/script.sh > /dev/null
This will run the script every two minutes (*/2
).
Make sure to adjust the paths to the domain before.
I have the same thing happening and it's almost certainly because one of the central whois servers has blocked you for too many queries. I suspect it's Verisign's .com server that is being touchy as I can get names lik
I haven't yet found out a way to remove myself from the block list but am working on it and would appreciate hearing anything you find out!
To test this, try a whois of flabbalabbaz.info - you'll get an immediate response; then try a whois of flabbalabbaz.com or .net - you'll time out.
The most usual cause of getting blacklisted is too many queries from some sort of automated system. We have a whois script built into one of our helpdesk operators' tools, but it doesn't actually issue many requests so I'm rather disappointed they've seen fit to block us for what would only have been a relatively small number of queries.
whois searches for an object in a RFC 3912 database.
This version of the whois client tries to guess the right server to ask for the specified object. If no guess can be made it will connect to whois.networksolutions.com for NIC handles or whois.arin.net for IPv4 addresses and network names.
--verbose
Be verbose.
Other options are flags understood by RIPE-like servers.
LANG
When querying whois.nic.ad.jp and whois.jprs.jp english text is requested unless the LANG or LC_MESSAGES environment variables specify a Japanese locale.
WHOIS_OPTIONS
A list of options which will be evalued before the ones specified on the command line.
WHOIS_SERVER
This server will be queried if the program cannot guess where some kind of objects are located. If the variable does not exist then whois.arin.net will be queried.
/etc/whois.conf
Please remember that whois.networksolutions.com by default will only search in the domains database. If you want to search for NIC handles you have to prepend a ! character. When you do this, the default server becomes whois.networksolutions.com.
When querying whois.arin.net for IPv4 or IPv6 networks, the CIDR netmask length will be automatically removed from the query string.
When querying whois.nic.ad.jp for AS numbers, the program will automatically convert the request in the appropriate format, inserting a space after the string AS.
When querying whois.denic.de for domain names and no other flags have been specified, the program will automatically add the flag -T dn.
When querying whois.dk-hostmaster.dk for domain names and no other flags have been specified, the program will automatically add the flag --show-handles.
RIPE-specific command line options are ignored when querying non-RIPE servers. This may or may not be the behaviour intended by the user. When querying a non-standard server, command line options which are not to be interpreted by the client should always follow the -- separator (which marks the beginning of the query string).
If the /etc/whois.conf config file exists, it will be consulted to find a server before applying the normal rules. Each line of the file should contain a regular expression to be matched against the query text and the whois server to use, separated by white space. IDN domains must use the ACE format.
The whois protocol does not specify an encoding for characters which cannot be represented by ASCII and implementations vary wildly. If the program knows that a specific server uses a certain encoding, if needed it will transcode the server output to the encoding specified by the current system locale.
Command line arguments will always be interpreted accordingly to the current system locale and converted to the IDN ASCII Compatible Encoding.
The program may have buffer overflows in the command line parser: be sure to not pass untrusted data to it. It should be rewritten to use a dynamics strings library.
This program closely tracks the user interface of the whois client developed at RIPE by Ambrose Magee and others on the base of the original BSD client. I also added support for the protocol extensions developed by David Kessens of QWest for the 6bone server.
RFC 3912: WHOIS Protocol Specification
RIPE-223: RIPE NCC Database Documentation
Detailed help on available flags can be found in RIPE-223 or in the help file which can be obtained with the command:
whois -h whois.ripe.net HELP
Whois and this man page were written by Marco d’Itri <md[:at:]linux[:dot:]it> and are licensed under the terms of the GNU General Public License, version 2 or higher.