Table of Contents
1 Introduction
MapResources - A Whois Resource Discovery Tool.
There a number of reasons why an organization may wish to build an inventory of all routing resources (ASNs, network blocks) that it holds. While much of this information can be found in Whois registries, the process of building such a list and the maintenance of such a list over time is often non-trivial. This is because organizations, through mergers and splits, may change in form and composition over time. The Whois database itself may become stale from not receiving timely updates, or could simply become fractured enough over time such that no one person has full knowledge of the organizational routing resources.
Clearly, an automated interface to Whois is required.
ARIN, the Regional Internet Registry for the North America region, offers access to its Whois database through a RESTful API. While ARIN does not itself provide a tool to automate Whois access, the RESTful API provides the necessary building block to implement such capability. The MapResources package is an implementation of this capability.
A starting point in the form of a known POC handle, organization handle,
net handle, or ASN number is assumed. Using information contained within
the
The main driver utility program for this package is
Even though most interfaces in the
This document is prepared under Contract Number HSHQDC-14-C-B0035 for DHS S&T CSD
2 MapResources Installation
The MapResources software may be installed using the instructions given in this section. Prior to installing the package, ensure that the packages in the dependencies list are already installed.
2.1 Dependencies
The MapResources package has the following external dependencies. These packages must be installed prior to installing the MapResources package.
- pymongo
- pprint
- urllib
- requests
- xmltodict
- pygraphviz
- networkx
- matplotlib
- json2html
- sqlite3
- ipaddress
2.2 Installation
The package can be installed by running the
$ python setup.py install
2.3 MongoDB Setup
In order to use MongoDB as the persistent data store, follow directions given at http://docs.mongodb.org/manual/installation/ in order to install MongoDB.
2.4 Route Views Database Setup
When trying to compare against Route Views data, the scripts in this package expect the Route Views data to be stored in a database that has the following schema.
- Routeadv Table
Column Type advId integer primary key autoincrement at integer prefix varchar(255) prefixStart int prefixEnd int sourceAddr varchar(255) sourceAS integer originated int nexthop varchar(255) lastAS integer asciiPath varchar(4096) - Path Table
Column Type pathId integer primary key autoincrement advId integer REFERENCES routeadv(advId) fromAS integer toAS integer count integer
3 MapResources Examples
The following examples are for illustration purposes only. The resources listed below were found at a particular instance in time. For a more accurate representation of organizational resources the ARIN whois database should be queried directly.- Initiate Search from an Organizational Handle
Suppose we wish to obtain the graph of resources assigned to ARIN. We know that one of the organizational handles for ARIN is 'ARIN', thus we can use this handle as one of our starting points:$ python map_whois.py -X -o ARIN -g ARIN.svg
The
-o option specifies the organizational handle of 'ARIN', while the-g option specifies the name of the file for the resource graph.The
-X option in the above command specifies that no caching is to be performed during query lookups. If the-H option were used, the script would use a hash store as the caching store.The hash data store allows scripts that wrap around the
map_resources module to make use of cached values in subsequent calls to the graph-generation routine. However, the hash data is still non-persistent. In order to use a persistent data store the above script should be called with the '-D host:port ' option. The-D option is the default.The graph produced by the above command is shown below:
We note the following:
- Only a single network block is shown. For ARIN, this likely represents only one of many network blocks that are assigned to it.
- The ARIN block is grayed out. This indicates that the number of resource
dependencies from the ARIN block exceeded some threshold. The hover text that
appears over the grayed block indicates that the number of resources for
http://whois.arin.net/rest/org/ARIN/nets is 51, which is clearly above our default threshold of 25.
- Initiate Search from an Organizational Handle, with a Larger Threshold
If we re-run the above command (this time, with a local DB store) with the-t 51 option, we get a different output:$ python map_whois.py -o ARIN -t 51 -g ARIN.svg
- Initiate search from multiple handles
A search for 'ARIN' onhttp://whois.arin.net indicates that the AS10745 and AS394018 are also assigned to ARIN. Since they were not found using the ARIN organization handle as the starting point, we can renew the search using multiple starting handles. Since we are making use of a local persistent data store, resources that were fetched previously are not re-queried.The command that we use and the image that is produced are as follows:
$ python map_whois.py -o ARIN -a AS10745 -a AS394018 -t 51 -g ARIN.svg
- Comparison Against Route Views
The-R option enablesmap_whois.py to find new resource handles from a comparison with Route Views data.NOTE: The Route Views data has to be ingested into a database in order to be able to use this option.
New resources can be found in two ways:
- ASNs that were previously unknown, but which originated known prefixes.
- Prefixes that were previously unknown, but which are originated by known ASNs.
In addition, the
-R option also checks whether any known prefixes were originated by any unknown ASNs.For example, given an SQLite file
rib.sqlite containing the Route Views RIB data, the following command lists a number of "unknown" ASNs:$ python map_whois.py -o ARIN -a AS10745 -a AS394018 -t 51 -g ARIN.svg -R rib.sqlite
However, most (if not all) such ASNs correspond to different clusters of resources that belong to other organizations that are recipients of network resources from ARIN.
- Generation of a Report
Determining a complete list of starting handles is an iterative process. Once a list of handles has been obtained, a report listing out the different resources can be generated using the-r option:$ python map_whois.py -o ARIN -a AS10745 -a AS394018 -t 51 -g ARIN.svg -r report.html
The report formatting is highly primitive and is a work in progress.
4 map_whois.py Usage
The
Usage:
map_whois [-h] [-v] [-a ASN] [-p POC] [-o ORG] [-n NET] [-c CIDR] [-i IP] [-u URL] [-t THRESHOLD] [-l] [-s] [-f {png,svg}] [-R RVDB] [-L RESOURCELIST] [-j JSONFILE] [-g GRAPHFILE] [-r REPORTFILE] [-X | -H | -D DBSTORE] optional arguments: -h, --help show this help message and exit -v, --verbose increase output verbosity -a ASN, --asn ASN Start from the given ASN handle -p POC, --poc POC Start from the given POC handle -o ORG, --org ORG Start from the given Org handle -n NET, --net NET Start from the given Net handle -c CIDR, --cidr CIDR Start from the given CIDR block -i IP, --ip IP Start from the given IP address -u URL, --url URL Start from the given domain -t THRESHOLD, --threshold THRESHOLD Maximum number of dependencies to follow -l, --longform Dsplay detailed information -s, --showgraph Dsplay the graph -f {png,svg}, --format {png,svg} Graphviz file format to use -R RVDB, --rvdb RVDB Check against given Route Views Database file -L RESOURCELIST, --resourcelist RESOURCELIST Extract resource handles from the given file. Each line of the file should be formatted as: , where the different supported types are 'asn', 'poc', 'org', 'net', 'cidr', 'ip' and 'url'. -j JSONFILE, --jsonfile JSONFILE Output resource information in json format -g GRAPHFILE, --graphfile GRAPHFILE Output graph image -r REPORTFILE, --reportfile REPORTFILE Output report -X, --nostore Do not use any data store -H, --hashstore Use an indexed hash store -D DBSTORE, --dbstore DBSTORE Use a DB store and issue queries if needed
5 map_resources Package
The
This package exports the following classes:
Data Stores:
- GenericStore: No caching data store (base class)
- HashStore: Data store with hash back-end
- DBStore: Data store with MongoDB as the back-end
- WhoisCollection: Base class
- POCCollection: Point of Contact Collection
- URLCollection: URL Collection (Ephemeral Class)
- OrgCollection: Organization Collection
- NetCollection: Network resource collection
- IPCollection: IP address collection (Ephemeral Class)
- CIDRCollection: CIDR block Collection (Ephemeral Class)
- ASNCollection: Autonomous System Number Collection
- POCCollection: Point of Contact Collection
- WhoisAnalyzer: Cluster analyzer
- ResourceReporter: formats cluster information for reporting
- AnalyzeOptParser: Parse command line options
- RVFetcher: Fetch Route Views data from local DB
- RVComparator: Compare whois and Route Views data
- RVComparatorRes: Container for
RVComparator results
5.1 map_resources Module
The
Attributes:
verbose: (boolean) Turns on verbosity of log messages.
5.1.1 map_resources.analyze.AnalyzeOptParser Class
This class is used to parse options related to any
Constructor:
Constructor Arguments:
prog : (string) Name of the program.
5.1.1.1 add_stores()
Synopsis:
Register the options associated with different data-store types.
5.1.1.2 get_help()
Synopsis:
Return the formatted help text.
Return Values:
5.1.1.3 host_port()
Synopsis:
A simple type for a
Throws
Arguments:
s : (string) A string of the formhostname:port .
Return Values:
5.1.1.4 parse()
Synopsis:
Parse the list of options.
Arguments:
argv : (list) A list of arguments.
Return Values:
5.1.1.5 parse_objs()
Synopsis:
Extract object handles and types from the provided options. The object handle could come from a file or from the relevant option associated with each resource type.
Arguments:
p : (Namespace) Contains various command-line parameters.
Return Values:
5.1.1.6 parse_objs_from_file()
Synopsis:
Extract resource handles from the given file. Each line of the file should
be formatted as "
Arguments:
rsrcfile : (file handle) The file to be parsed.
Return Values:
5.1.1.7 parse_store()
Synopsis:
Extract the store type from the provided options.
Arguments:
p : (Namespace) Contains various command-line parameters.
Return Values:
5.1.2 map_resources.analyze.ResourceReporter Class
This class formats cluster information for reporting.
Constructor:
Constructor Arguments:
analyzer - TheWhoisAnalyzer object.resources - The cluster information to report about.links - The links associated with the clusters.
5.1.2.1 display_graph()
Synopsis:
Display the resource graph using
Return Values:
5.1.2.2 get_agraph()
Synopsis:
Generate a Graphviz Agraph from graph information. Produce the
Agraph from the
Arguments:
- G(Graph): A networkx graph object.
- o(dict): A
dict of object collections indexed by the origin handle.
Return Values:
5.1.2.3 get_asninfo()
Synopsis:
Generate ASN information. Build a
Arguments:
loc : (string) The object identifier.
Return Values:
5.1.2.4 get_clusterinfo()
Synopsis:
Return cluster information. Build a list of cluster information along with relevant network information for reporting.
Arguments:
terse : (boolean) If True, produce terse details.rvf : (RouteViewsFetcher) If not None, fetch information from a RouteViews database to augment reported cluster information.
Return Values:
5.1.2.5 get_netinfo()
Synopsis:
Generate network information. Build a
Arguments:
loc : (string) The object identifier.rvf : (RouteViewsFetcher) If not None, fetch information such as routednetblock s and origination AS in order to determine if any network resources are originated by resources in a different cluster.
Return Values:
5.1.2.6 get_orginfo()
Synopsis:
Build a
Arguments:
loc : (string) The object identifier.
Return Values:
5.1.2.7 get_pocinfo()
Synopsis:
Build a
Arguments:
loc : (string) The object identifier.
Return Values:
5.1.2.8 plot_graph()
Synopsis:
Build a graph of whois resources.
Use
Arguments:
outputfile(file handle): If not None, write graph to this file.
Return Values:
5.1.2.9 plot_resources()
Synopsis:
Scatter Plot the number of URLs and IPs in each cluster.
Produce a Scatter Plot number of URLs and IPs in each cluster.
Finally annotate the top 10 clusters in terms of resource density.
Return Values:
5.1.2.10 write_report()
Synopsis:
Write cluster information.
Arguments:
rh : (file handle) If not None, the target file to write the HTML report into.clusterplot : (boolean) If True, generate and write cluster plot information to report.clustergraph : (boolean) If True, generate and write network resource information to report.terse : (boolean) If True, produce terse details.rvf : (RouteViewsFetcher) If not None, use this object to augment report with RouteViews-derived information.
Return Values:
5.1.3 map_resources.analyze.WhoisAnalyzer Class
Define a class for analyzing a list of collection objects.
Constructor:
Constructor Arguments:
threshold : (int) If the number of node dependencies exceed this limit the dependencies are not followed.store : (GenericStore ) The store to use for fetching data.
5.1.3.1 analyze()
Synopsis:
Analyze a list of handles. Each handle in the list is used as
a starting point for the collection of resources through the
Arguments:
objlist : (dict ) Adict of handles->type mappings.
5.1.3.2 append_message()
Synopsis:
Append a new message to the
Return Values:
5.1.3.3 fetch()
Synopsis:
Fetch the data corresponding to the given
Arguments:
ctype : (string) Collection type.loc : (string) The ID string.
5.1.3.4 generate_clusters()
Synopsis:
Group resources according to their subgraphs.
Return Values:
5.1.3.5 generate_results()
Synopsis:
Generate various result components. The different result components can
include a report file, a JSON structure with cluster information, a Graphviz
graph, and a plot of the graph using
Arguments:
reportfile : (file handle) If not None, the target file to write the HTML report into.jsonfile : (file handle) If not None, the target file to write JSONized cluster information into.showgraph : (boolean) If True, display network graph plot usingmatplotlib .outputfile : (filehandle) If True, writegraphviz image to this file.terse : (boolean) If True, produce terse details.rvf : (RouteViewsFetcher) If not None, use this object to augment report with RouteViews-derived information.
5.1.3.6 get_resobj()
Synopsis:
Get the collection container object.
Return Values:
5.1.3.7 pack()
Synopsis:
Pack resources and links into a
Arguments:
resources : (dict ) Adict of resources indexed by type.links : (dict ) Adict of links.
Return Values:
5.1.3.8 process_new_collection()
Synopsis:
Create a new collection object with the given type. Reuse the cache and
the data store if available, but don't link to any other collection object.
Finally, subsume the resulting object into the main collection container
object within
Arguments:
t : (string) Collection type.h : (string) The collection handle.
5.1.3.9 unpack()
Synopsis:
Unpack a
Arguments:
clustobj : (dict ) Adict that has 'resources ' and 'links ' as two of its keys.
Return Values:
5.2 map_resources.fetch_whois Module
Query ARIN whois objects. This module provides the functionality to query the ARIN RESTful API for all resource dependencies starting from the given ASN, POC, Org or Net handle.
Attributes:
verbose : (boolean) Turns on verbosity of log messages.
5.2.1 map_resources.fetch_whois.ASNCollection Class
ASN Resource Class
Constructor:
Constructor Arguments:
origin_handle : (string) The handle that identifies the container.origin : (WhoisCollection ) The parent container object.store : (GenericStore ) The store associated with this collection object.cache : (dict ) Any pre-cached values.tt : (string) An initial tool-tip.
Bases:
5.2.1.1 slurp()
Synopsis:
Look for all objects that can be reached from this ASN container.
ASN collections may be comprised of POC and Org collections.
Arguments:
base : (string) The base URL for lookups.threshold : (int) If the number of node dependencies exceeds this limit the dependencies are not followed.
Return Values:
5.2.1.2 slurp_set()
Synopsis:
Find the ASN handle and slurp data. ASN resources may be found in XML
elements named
Arguments:
base : (string) The base URL for lookups.threshold : (int) If the number of node dependencies exceeds this limit the dependencies are not followed.
Return Values:
5.2.2 map_resources.fetch_whois.CIDRCollection Class
IP container class (ephemeral).
Constructor:
Constructor Arguments:
origin_handle : (string) The handle that identifies the container.origin : (WhoisCollection ) The parent container object.store : (GenericStore ) The store associated with this collection object.cache : (dict ) Any pre-cached values.tt : (string) An initial tool-tip.
Bases:
5.2.2.1 slurp()
Synopsis:
Look for all objects that can be reached from this CIDR block. Look for the
associated
Arguments:
handle : (string) The origin handle for the slurp operation.threshold : (int) If the number of node dependencies exceeds this limit the dependencies are not followed.
Return Values:
5.2.3 map_resources.fetch_whois.DBStore Class
Wrapper around a MongoDB data store.
Constructor:
Bases:
5.2.3.1 fetch()
Synopsis:
Fetch data for the given ID string and collection type. First look at the MongoDB store for any matching data. If found, return that data; if not, fetch new data but only if we are not limiting lookups to already-cached values.
Arguments:
ctype : The collection type.idstr : The location reference for the whois object.
Return Values:
5.2.3.2 find_collection()
Synopsis:
Find the DB collection associated with the given object type.
Arguments:
ctype : (string) The collection object type.
Return Values:
5.2.4 map_resources.fetch_whois.GenericStore Class
Base class for all data stores with no caching support.
Constructor:
5.2.4.1 fetch()
Synopsis:
Fetch data for the given ID string and collection type. No caching is done in this routine. Just query and return.
Arguments:
ctype : (string) The collection type.idstr : (string) The ID string.
5.2.4.2 fetchAssociated()
Synopsis:
Fetch an object's associated data, given an ID string.
Arguments:
obj : (string) The object for which we are seeking associated data.idstr : (string) The ID string.
5.2.4.3 get_idstr()
Synopsis:
Determine the set of IDs for the given type and handle. This is the default way of constructing an ID.
Arguments:
typepfx : (string) The collection type.handle : (string) The resource handle.
5.2.4.4 query()
Synopsis:
Query the data store for the given ID string. This is the common query method for all types of data stores.
Arguments:
idstr : (string) The ID string.
5.2.5 map_resources.fetch_whois.HashStore Class
Implementation of a simple hash data store (non-persistent).
Constructor:
Bases:
5.2.5.1 fetch()
Synopsis:
Fetch data for the given ID string and collection type. First look at the hash for any matching data. If found return that data; if not fetch new data.
Arguments:
ctype : (string) The collection type.idstr : (string) The ID string.
5.2.6 map_resources.fetch_whois.IPCollection Class
IP container class (ephemeral).
Constructor:
Constructor Arguments:
origin_handle : (string) The handle that identifies the container.origin : (WhoisCollection ) The parent container object.store : (GenericStore ) The store associated with this collection object.cache : (dict ) Any pre-cached values.tt : (string) An initial tool-tip.
Bases:
5.2.6.1 slurp()
Synopsis:
Look for all objects that can be reached from this IP address. Look for the
associated
Arguments:
handle : (string) The origin handle for the slurp operation.threshold : (int) If the number of node dependencies exceeds this limit the dependencies are not followed.
Return Values:
5.2.7 map_resources.fetch_whois.NetCollection Class
Net resource class.
Constructor:
Constructor Arguments:
origin_handle : (string) The handle that identifies the container.origin : (WhoisCollection ) The parent container object.store : (GenericStore ) The store associated with this collection object.cache : (dict ) Any pre-cached values.tt : (string) An initial tool-tip.
Bases:
5.2.7.1 slurp()
Synopsis:
Look for all objects that can be reached from this Net container. Net collections may be comprised of POC or Org collections.
Arguments:
base : (string) The base URL for lookups.threshold : (int) If the number of node dependencies exceeds this limit the dependencies are not followed.
Return Values:
5.2.7.2 slurp_set()
Synopsis:
Find the Net handle and slurp data. Net resources may be found in XML
elements named
Arguments:
base : (string) The base URL for lookups.threshold : (int) If the number of node dependencies exceeds this limit the dependencies are not followed.
Return Values:
5.2.8 map_resources.fetch_whois.OrgCollection Class
Organization Resource Class
Constructor:
Constructor Arguments:
origin_handle : (string) The handle that identifies the container.origin : (WhoisCollection ) The parent container object.store : (GenericStore ) The store associated with this collection object.cache : (dict ) Any pre-cached values.tt : (string) An initial tool-tip.
Bases:
5.2.8.1 slurp()
Synopsis:
Look for all objects that can be reached from this Org container. Org collections may be comprised of POC, ASN and Net collections.
Arguments:
base : (string) The base URL for lookups.threshold : (int) If the number of node dependencies exceeds this limit the dependencies are not followed.
Return Values:
5.2.8.2 slurp_set()
Synopsis:
Find the Org handle and slurp data. Orgs may be found in XML elements named
Arguments:
base : (string) The base URL for lookups.threshold : (int) If the number of node dependencies exceeds this limit the dependencies are not followed.
Return Values:
5.2.9 map_resources.fetch_whois.POCCollection Class
Point of Contact Resource Class.
Constructor:
Constructor Arguments:
origin_handle : (string) The handle that identifies the container.origin : (WhoisCollection ) The parent container object.store : (GenericStore ) The store associated with this collection object.cache : (dict ) Any pre-cached values.tt : (string) An initial tool-tip.
Bases:
5.2.9.1 slurp()
Synopsis:
Look for all objects that can be reached from this POC container. POC collections may be comprised of Org, ASN and Net collections.
Arguments:
handle : (string) The origin handle for the slurpthreshold : (int) If the number of node dependencies exceeds this limit the dependencies are not followed.
Return Values:
5.2.9.2 slurp_set()
Synopsis:
Find the POC handle and slurp data. POCs may be found in XML elements
named
Arguments:
base : (string) The base URL for lookups.threshold : (int) If the number of node dependencies exceeds this limit the dependencies are not followed.
Return Values:
5.2.10 map_resources.fetch_whois.URLCollection Class
Constructor:
Constructor Arguments:
origin_handle : (string) The handle that identifies the container.origin : (WhoisCollection ) The parent container object.store : (GenericStore ) The store associated with this collection object.cache : (dict ) Any pre-cached values.tt : (string) An initial tool-tip.
Bases:
5.2.10.1 slurp()
Synopsis:
Look for all objects that can be reached from this URL. Look for
the associated
Arguments:
handle : (string) The origin handle for the slurpthreshold : (int) If the number of node dependencies exceeds this limit the dependencies are not followed.
5.2.11 map_resources.fetch_whois.WhoisCollection Class
Base class for all
Constructor:
Constructor Arguments:
origin_handle : (string) The handle that identifies the container.origin : (WhoisCollection ) The parent container object.store : (GenericStore ) The store associated with this collection object.cache : (dict ) Any pre-cached values.tt : (string) An initial tool-tip.
5.2.11.1 add_collection()
Synopsis:
Add a new associated collection to the current object. Collections are indexed by the origin handle. There can be multiple collections per origin handle. As part of adding the collection also update the cache from the to-be-added collection to the main (parent) collection holder.
Arguments:
col : (WhoisCollection ) The collection object to be added.
Return Values:
5.2.11.2 add_link()
Synopsis:
Create a link between the current object and the given handle.
Arguments:
handle : (string) The handle of the collection to link with.
Return Values:
5.2.11.3 add_tooltip()
Synopsis:
Append a new tool-tip to the list indexed by the given handle.
Arguments:
handle : (string) The handle to associate the tool-tip with.msg : (string) The tool-tip message.
Return Values:
5.2.11.4 do_slurp()
Synopsis:
Entry point for looking up resource objects.
Arguments:
threshold : (int) If the number of node dependencies exceeds this limit, the dependencies are not followed.
Return Values:
5.2.11.5 fetchAssociatedObj()
Synopsis:
Get associated data for given idstr. This method asks the data store for any associated objects.
Arguments:
idstr : (string) Object ID string.
Return Values:
5.2.11.6 fetchObj()
Synopsis:
Get data corresponding to the given handle and collection type. This method
first constructs the ID string(s) corresponding to the handle and then calls
Arguments:
typepfx : (string) Collection type.handle : (string) Resource handle.cache : (boolean) If true, any data returned byget_data() is cached.
Return Values:
5.2.11.7 get_cache()
Synopsis:
Return the cache structure for the collection object.
Return Values:
5.2.11.8 get_collections()
Synopsis:
Get list of collections associated with the given object. Return all objects below and including the current one grouped by their handle.
Arguments:
recurse : (boolean) Return collection objects by traversing all collections that are linked through previous calls toadd_collection() .
Return Values:
5.2.11.9 get_data()
Synopsis:
Get data corresponding to the given ID string and collection type. First check if the data exists in the cache. If it doesn't then look for data in the data store.
Arguments:
ctype : (string) Collection type.idstr : (string) ID string.
Return Values:
5.2.11.10 get_draw_attribs()
Synopsis:
Get the different attributes of this collection.
Return Values:
5.2.11.11 get_links()
Synopsis:
Get the list of links associated with the given object. For each object below and including the current node return the list of connected nodes.
Arguments:
recurse : (boolean) Return links by traversing all collections that are linked through previous calls toadd_collection() .
Return Values:
5.2.11.12 get_parent()
Synopsis:
Return the parent object for the object.
Return Values:
5.2.11.13 get_parent_handle()
Synopsis:
Return the origin (parent) handle for the object.
Return Values:
5.2.11.14 get_resources()
Synopsis:
Get the list of resources associated with the given object.
Arguments:
recurse : (boolean) Return resources by traversing all collections that are linked through previous calls toadd_collection() .
Return Values:
5.2.11.15 get_store()
Synopsis:
Get the store associated with this object.
Return Values:
5.2.11.16 get_tooltip()
Synopsis:
Get the list of tool-tips associated with the given object.
Arguments:
recurse : (boolean) Return tool-tips by traversing all collections that are linked through previous calls toadd_collection() .
Return Values:
5.2.11.17 get_type()
Synopsis:
Get the collection type for the given collection object.
Return Values:
5.2.11.18 set_limit_exceeded()
Synopsis:
Set the object state to indicate too many dependencies.
Arguments:
handle : (string) The handle against which the limit-exceeded state is to be associated.idstr : (string) The ID string that triggered the limit-exceeded event.lim : (int) The number of dependencies that were detected.
Return Values:
5.2.11.19 slurp_common()
Synopsis:
Common convenience function for data slurping. If we're presented with a
Arguments:
p : (string) The link reference to process.idstr : (string) Object ID string.threshold : (int) If the number of node dependencies exceed this limit, the dependencies are not followed.
Return Values:
5.2.11.20 subsume()
Synopsis:
Subsume the new collection object. Merge various pertinent data associated with the new collection object into the current collection object.
Arguments:
col : (WhoisCollection ) The collection object that to be subsumed.
5.3 map_resources.whois_rv_cmp Module
This module serves as the interface between the
This module provides three classes:
-
RVComparator : Does some comparisons between a set of ASNs and network block data and theRouteViews DB. -
RVComparatorRes : Class for encapsulating the comparator result. -
RVFetcher : Provides a simple interface to fetchnetblocks from the database.
Attributes:
verbose : (boolean) Turns on verbosity of log messages.
5.3.1 map_resources.whois_rv_cmp.RVComparator Class
Comparison between
Constructor:
5.3.1.1 add_asn()
Synopsis:
Add the ASN to the SQL query.
Arguments:
asn : (string) The ASN handle.
Return Values:
5.3.1.2 add_net()
Synopsis:
Add the network block to the SQL query. IPv6 address blocks are ignored.
Arguments:
startAddr : (string) The start address in the block.endAddr : (string) The end address in the block.oaslist : (list) A list of origin ASNs associated with thisnetblock .
Return Values:
5.3.1.3 compare()
Synopsis:
Compare known resources against Route Views data. Issue the SQL query, check which prefixes and ASNs are unknown.
Return Values:
5.3.1.4 compare_resources()
Synopsis:
Register known resources and compare against Route Views.
Register all ASNs and Net objects from the
Arguments:
resob : (WhoisAnalyzer ) Object that holds the list of ASN and Net resources we are interested in analyzing.
Return Values:
5.3.1.5 is_pfx_known()
Synopsis:
Check if the given network block is one that we know about.
Arguments:
start : (string) Network block start.end : (string) Network block end.
Return Values:
False if not.
5.3.2 map_resources.whois_rv_cmp.RVComparatorRes Class
Class for encapsulating the comparator result.
Constructor:
5.3.2.1 add_unknown_asn()
Synopsis:
Register the previously unknown ASN that was found to originate a known prefix.
Arguments:
asn : The (unknown) ASN originating the prefix.pfx : The prefix from a known block that was originated.
Return Values:
5.3.2.2 add_unknown_oasn()
Synopsis:
Register the ASN originating a known prefix but was not configured as the originating AS in whois.
Arguments:
asn : The (known) ASN originating the prefix.pfx : The prefix from a known block that was originated.
Return Values:
5.3.2.3 add_unknown_pfx()
Synopsis:
Register the previously unknown prefix that was originated from a known ASN.
Arguments:
asn: The known ASN originating the prefix. pfx : The unknown prefix that was originated.
Return Values:
5.3.2.4 get_unknown_asn()
Synopsis:
Return the list of unknown ASNs.
Return Values:
5.3.2.5 get_unknown_oasn()
Synopsis:
Return the list of unknown originating ASNs.
Return Values:
5.3.2.6 get_unknown_pfx()
Synopsis:
Return the list of unknown prefixes.
Return Values:
5.3.2.7 process_unknown_resources()
Synopsis:
Process all unknown dependencies. Feed all unknown dependencies that were
detected in Routeviews through the
Arguments:
analyzer : (WhoisAnalyzer ) The analyzer object.
5.3.3 map_resources.whois_rv_cmp.RVFetcher Class
Class to fetch objects from route views.
Constructor:
5.3.3.1 find_netblocks()
Synopsis:
Look up all
Arguments:
start : (string) Start prefix.end : (string) End prefix.
Return Values:
6 Full Table of Contents
- Table of Contents
- 1 Introduction
- 2 MapResources Installation
- 3 MapResources Examples
- 4
map_whois.py Usage
- 5
map_resources Package
- 5.1
map_resources Module
- 5.2
map_resources.fetch_whois Module
- 5.3
map_resources.whois_rv_cmp Module
- 5.1
- 6 Full Table of Contents
Copyright (c) 2015, Parsons, Inc
All rights reserved