TOPIC
This article contains the Lookupd Release Notes.
DISCUSSION
Lookupd Whenever a program needs certain kinds of information, it sends a request to a program named lookupd, which acts as an information clearing-house for your computer. lookupd is responsible for coordinating information about user accounts, printers, e-mail aliases and distribution lists, computer names, Internet addresses, and several other kinds of information. lookupd has two main functions: Beginning with MacOS X Server Release 1.0, X.500-style databases that implement the schema described in RFC 2307 may be accessed using the "LDAP" protocol. lookupd restarts in response to a SIGHUP signal. This may be used to reset lookupd's state. When it restarts, it will re-read it's configuration options and will start with an empty cache. Lookup Strategy Internally, lookupd uses a set of software "agents" to get information. There are agents for NetInfo, NIS, DNS, the files in "/etc (also known as the "Flat Files"), an "LDAP" agent, and an agent which manages the internal cache. There is also a special agent (the NILAgent ) which returns "negative" entries. When lookupd searches for information about an item, it queries agents in a specific order until the item is found or until all sources of information have been consulted without finding the desired item. By default, lookupd first queries its cache agent, and then NetInfo. If the item is a host or network, lookupd will query the DNS agent last. In some cases, lookupd creates lists of all the information available about some sort of entity. For example, all printers or all NFS mounts. In these cases lookupd queries each agent in turn and concatenates all retrieved information into a single list. The lookup order is configurable. For example, you might specify that lookupd queries its internal cache, then NetInfo, then the Flat Files, then NIS. You may also change the lookup order for a particular category of item. The known categories are users , groups , hosts , networks , services , protocols , rpcs , mounts , printers , bootparams , bootp , aliases , and netgroups . You can set the lookup order (and other configuration options) for all categories, and override them for individual categories. Details on configuring lookupd are found in the Configuration section below. Some agents may have their own configuration options. Details on configuring individual agents are found in the Agents section below. Caching There are caches for all categories of lookups. The caches are unlimited in capacity (although you can set a maximum size, see below). The default lookup order starts with the Cache agent for each lookup category. Caching may be disabled for all categories or for specific categories by removing the Cache agent from the lookup order. Validation When it retrieves an item from cache, lookupd validates the cached entry before returning it. If the entry is invalid, a fresh copy is fetched to replace the stale one in cache. The idea is that in many cases it is possible for lookupd to quickly determine that a cached entry is still valid (by checking a timestamp or a database sequence number, for example). When cache validation is enabled, performance is enhanced because many items can be stored in lookupd's cache, while at the same time clients can be certain that any data they get from lookupd is as up-to-date as possible. If cache validation is disabled, items are returned from cache without any checks. In this case it is possible that the information is out-of-date. You can place limits on how "stale" an item might be by setting the cache TimeToLive (see below). You can also get the best of both worlds by having cache validation enabled and adjusting the ValidationLatency . The idea is that if an agent has just fetched a database version number or read a timestamp in order to validate one item in the cache, then it can use that value for a few seconds to validate other entries in the cache with only a small risk that those entries have become out-of date. Setting ValidationLatency to 0 seconds causes validation on every fetch from cache. Setting the value to a larger number means that lookupd will avoid re-checking timestamps, sequence numbers, version numbers or other validation indicator for the indicated number of seconds. This allows you to say, for example, that you are willing to let the cache return an item that might be no more than a few seconds out-of-date in order to reduce network traffic. The default value for ValidationLatency is 15 seconds. Validation is enabled by default, but may be disabled for all categories, or for specific categories. For example, since computer names and addresses very rarely change, you might want to turn off cache validation for the host cache to save time and reduce network traffic. This is especially useful on slow network lines. If network access is fast, cache validation is inexpensive, and ensures that all data is up-to-date. The cache validation strategy in the NetInfo, NIS, and Flat File agents makes validation as fast and inexpensive as possible. At present, there is no cache validation for DNS. That means that objects originally obtained from DNS will only be available from the cache if validation is disabled for hosts. The NILAgent returns a "negative" record for any lookup. This stops further search for an item. The NILAgent should always appear last in any LookupOrder specification (see below). The value of these negative records is that they will added to the cache (if caching is enabled). Thus if a lookup for a particular item fails (after a long, slow search of all available information sources), then subseqent lookups for the same item will fetch the negative record from the cache ± avoiding another long search. Like all other cached records, negative records are validated. By default, negative records are valid for 60 seconds. You can override the default by setting the TimeToLive option in the NILAgent's configuration. See the Agent Configuration section below. When an object is placed in a cache, it is given a time to live. The default is 12 hours. Making things expire more quickly will cause the cache to stay smaller, but will result in more network traffic. Whenever an object is validated, it's time to live is reset. Agents may over-ride the default time-to-live value. For example, the DNS agent uses the time-to-live value returned by a DNS server for a host entry. Configuration All the configuration parameters are located in NetInfo under the /locations/lookupd directory, stored as values for various keys. There may also be subdirectories for each lookup category (and for each agent ±Äsee below). For example: /locations/lookupd/hosts , /locations/lookupd/users , /locations/lookupd/printers , and so on. lookupd will climb the NetInfo hierarchy, starting at the local domain, to find this directory. If it doesn't find it, it will use a default configuration (see below). The keys and their allowed values are: LogFile Name of a log file that contains a copy of all messages sent to syslog. There is no default (i.e. no log file is kept). MaxThreads Maximum number of threads in the query dispatcher. The default is16. Under very heavy load, only 5 or 6 threads are used, so 16 is more than enough. MaxIdleThreads When a thread finishes servicing a query, it will usually go back to the message queue to wait for another query. This setting limits the maximum number of idle threads waiting on the queue. If a thread finishes servicing a query and MaxIdleThreads are already waiting on the queue, the thread will exit. The default value is 16. MaxIdleServers The dispatcher uses a "server" object to actually answer a client lookup. One server is required for each active thread. The dispatcher keeps a pool of servers so that they can be re-used. This setting limits the maximum number of servers in the pool, waiting for a query to answer. The default value is 16. ValidateCache This boolean value determines whether cache validation is enabled for all cache categories. The default is "YES". Use "NO" to disable validation. The setting of this value may be over-ridden for specific cache categories (see below). ValidationLatency If Cache validation is enabled, this integer value specifies the number of seconds that may elapse between successive validation checks for a particular agent. The default is 15 seconds. This value applies to specific agents rather than to the cache. The setting of this value may be over-ridden for specific agents (see below). CacheCapacity Maximum number of objects in the cache for each category (e.g. this many users, this many hosts, ...). Least-recently-used objects are removed when space is required. By default, there is no limit to the cache size. TimeToLive Time to live (measured in seconds) in cache. The default is 43200 seconds (12 hours). This is the default mechanism used to limit the growth of the cache. LookupOrder Sets the lookup order for all categories, although you may override this for specific categories. This key takes multiple values. The default for all categories except host and network lookups is CacheAgent then NIAgent . For hosts and networks, the default lookup order is CacheAgent , NIAgent , then DNSAgent . Configuration options for specific categories reside in the directories: /locations/lookupd/users , .../groups , .../hosts , .../networks , .../services , .../protocols , .../rpcs , .../mounts , .../printers , .../bootparams , .../bootp , .../aliases , and .../netgroups . Options that can be set per category are ValidateCache, CacheCapacity, TimeToLive, and LookupOrder. Agents Configuration parameters for agents are located in NetInfo under the /locations/lookupd/agents/ AgentName directory. For example: /locations/lookupd/agents/NIAgent , /locations/lookupd/agents/DNSAgent , and so on. CacheAgent The operation and configuration of the Cache agent are described in detail in the sections above. The configuration options for the Cache agent are ValidateCache, CacheCapacity, and TimeToLive. These options may be set gloablly and/or for specific categories. Options set for a specific category will over-ride the global setting. NIAgent The NIAgent is lookupd's NetInfo client. It supports the ValidationLatency and DomainOrder options. ValidationLatency is described above in the Caching section. NIAgent validates cached entries by checking the NetInfo server's database checksum. The NetInfo checksum changes whenever the database changes. Thus, any time a NetInfo domain is updated, all cached entries from that domain will be invalidated. The DomainOrder option allows you to specify a list of NetInfo domains (or specific NetInfo servers) that should be queried for information. By default, the NetInfo agent starts with a computer's local domain, then climbs the NetInfo hierarchy until reaching the root domain. In very rare cases, you might find that you can solve a difficult configuration problem by altering the default lookup order. Using this option can make your information systems configuration very confusing and spaghetti-like, so exercise great caution in its use! The domain search order may be set globally (for all categories of lookups), and/or for specific categories. A domain order for a specific category will over-ride the global order for for lookups of that type (e.g. for user lookups). The DomainOrder option may have multiple values. Each value specifies a domain or a specific NetInfo server. Domain names may be absolute paths starting at the root domain (e.g. /sales, /sol/jupiter), or a path relative to the local domain (e.g. ../zippy, ../../marketing). A "." stands for the local domain. You may also specify a domain relative to a remote computer by using a value of the form "nidomain:path@address". For example, If you used the value "nidomain:/central@192.42.172.1", NIAgent would connect to the computer with Internet address 192.42.172.1 and locate the domain named "/central" relative to that computer. You can also include values of the form "niserver:tag@address" to connect to a specific NetInfo server. For example, "niserver:network@192.42.172.5" would contact the server for the database tagged "network" at the given address. You may set the domain order for a particular lookup category by creating a NetInfo directory with the category name as a subdirectory of /locations/lookupd/agents/NIAgent. For example, /locations/lookupd/agents/NIAgent/users or /locations/lookupd/agents/NIAgent/printers. YPAgent The YPAgent is lookupd's NIS client. The NIS domain name must be set before lookupd starts, or this agent will not be able to connect to a server (in which case it does nothing). The NIS domain name is usually set during system startup using the value of the NISDOMAIN variable in the file /etc/hostconfig. For example: NISDOMAIN=quinta The only configuration option which can be set for the YPAgent is ValidationLatency. YPAgent validates entries by checking the map order numbers. Note that lookupd has separate agents for the Flat Files (see FFAgent below) and NIS. NIS and the files are viewed as independent information systems. You may use either agent or both in any order. YPAgent reads the following maps. Note that some of these maps are extensions to the standard set of maps created by most YP servers.
LDAPAgent Records retrieved by the LDAP agent must have their information organized following the schema described by RFC 2307. In addition to ValidationLatency, LDAPAgent supports the following configuration options:
DNSAgent The DNSAgent is lookupd's DNS client. Since DNS does not have a fast mechanism that would allow for validation of cached entries, the agent does not support cache validation. DNSAgent is only used for host name/address and network name/address lookups. FFAgent The FFAgent reads the "Flat Files" in your computer's /etc directory. Specifically, it reads the files:
NILAgent The NILAgent always returns a result for any query, so it stops any search. However, it returns a "negative" record, which means "the item you requested does not exist". The use of "negative" entries in a cache is controversial, so lookupd does not include the NILAgent in its default lookup order. However, adding NILAgent at the end of the LookupOrder can result in significant performance improvements in some cases. If there are many network information sources being searched it can take a long time for lookupd to try them all when you ask for something that doesn't exist. By including NILAgent at the end of the search, lookup will cache a negative record. The next time lookupd gets a request for that item, it will find the negative record quickly in the cache, and avoid a long (and useless) search. It is always possible that lookupd may fail to find an item ± and cache a negative record created by NILAgent ± just before someone adds that item to one of your information systems. In that case, the negative record will then be incorrect, and should be removed from cache. Unfortunately, there's no way for lookupd to know that without doing the same expensive search all over again. As a compromise, negative records only are allowed to remain in the cache for a short time. The NILAgent assigns all negative records a short time-to-live value. By default, the time-to-live is 60 seconds. You may change this by setting the TimeToLive option for the NILAgent. Example Configuration Here's a sample configuration as it might appear in the output of the "nidump" utility program. # nidump -r /locations/lookupd / name = lookupd; LogFile = /var/log/lookupd.log; LookupOrder = (CacheAgent, NIAgent); CHILDREN = ({ name = users; LookupOrder = (CacheAgent, NIAgent, YPAgent, LDAPAgent, FFAgent); }, { name = hosts; LookupOrder = (CacheAgent, NIAgent, DNSAgent, NILAgent); ValidateCache = NO; }, { name = agents; CHILDREN = ({ name = NIAgent; ValidationLatency = 60; }, { name = NILAgent; TimeToLive = 120; }, { name = LDAPAgent; suffix = mydomain.com; host = myservername; deref = always; scope = sub; suffix = "DC=base,DC=mydomain,DC=com"; timelimit = 60; }); }); Performance Tuning and Troubleshooting lookupd may be run in an interactive mode useful for testing and troubleshooting configuration problems. When you use the interactive mode, you start a second copy of the lookupd program from a command line with a "-d" option: mycomputer> lookupd -d lookupd version 123 Debug mode Enter command name, "help", or "quit" to exit > This second copy of lookupd runs independently of the system's "main" lookupd, and does not provide information to other programs running on your computer. This allows you to try queries and test configuration options without disturbing the normal operation of your computer. The second copy of lookupd will attempt to read its configuration options from a NetInfo directory named /locations/lookupd_debug. If it can't find /locations/lookupd_debug, it will attempt to use /locations/lookupd instead. This means that it will then run with the same settings as the main version of the program. The interactive mode "command line" supports escape completion for commands, so you can type a character or two then press the Escape key. lookupd will complete as much of the command as it can (sometimes there are several command that start with the same characters). To see all possible completions, press Control-d. To see all possible commands, press Control-d before you type in any characters at all. For on-line help, use the "help" command. > help Enter command name, "help" for general help, or "quit" to exit help help> help This is lookupd's interactive query and testing facility. ... There are interactive commands for all standard queries, such as userWithName , hostWithInternetAddress , and so on. When you enter a query, lookupd will print the result that it located or "nil" if the item was not found. lookupd also keeps track of the information source for each item and a number of other useful pieces of information that can help you track lookupd's activities. For example: > userWithName: jru Dictionary: "NIAgent: user jru" _lookup_NI_checksum: 68850661 _lookup_NI_domain: / _lookup_NI_index: 2 _lookup_NI_server: pacific/network _lookup_info_system: NetInfo _writers_passwd: jru change: 0 expire: 0 gid: 114 home: /Network/Servers/fiji/Users/jru name: jru netgroups: programmer passwd: 2YEsFfX2fmC8. realname: Jane Random User shell: /bin/csh uid: 1664 + Category: user + Time to live: 43200 + Age: 0 (expires in 43200 seconds) + Negative: No + Cache hits: 3 + Retain count: 6 When you enter a query, lookupd follows its normal lookup order to obtain an answer. If you wish to query a specific agent, you can use the agent command. This stops the normal lookup and will direct all further queries to the agent you specify. You can use the agent command again to switch to a different agent, or use the normalLookupOrder command to resume normal lookups. > agent: NI > hostWithName: fiji Dictionary: "NIAgent: host fiji" _lookup_NI_checksum: 68850661 _lookup_NI_domain: / _lookup_NI_index: 1 _lookup_NI_server: pacific/network _lookup_info_system: NetInfo bootfile: mach bootparams: en_address: 0:5:2:fe:ef:4b ip_address: 192.42.172.168 name: fiji netgroups: island serves: fiji/local > agent: DNS > hostWithName: fiji Dictionary: "D-0x6d470" _lookup_DNS_time_to_live: 28800 _lookup_DNS_timestamp: 912796168 _lookup_domain: mydomain.com _lookup_info_system: DNS ip_address: 192.42.172.168 name: fiji.mydomain.com fiji > normalLookupOrder Using normal lookup order You can get timing and usage statistics for all types of lookups using the statistics command. Timing measurments can help you determine what might be causing slowdowns or problems on your network. Statistics for individual agents available using the statisticsForAgent command includes information such as current server bindings. You can examine all items in memory using the memory and showMemoryObject commands. This includes stored configuration settings, statistical records, and cached information. |
| Document Information | |
| Product Area: | Mac OS System Software |
| Category: | Mac OS X Server |
| Sub Category: | General Topics |
Copyright © 2000 Apple Computer, Inc. All rights reserved.