INTERNET DRAFT Vishal Goenka
Novell, Inc.
Expires in six months from 1 June 1999
Intended Category: Standards Track
Complex Directory Lookup using
Java Based LDAP Query Extension
<draft-goenka-ldapext-java-complex-search-01.txt>
1. Status of this memo
This document is an Internet-Draft and is in full conformance with
all provisions of Section 10 of RFC2026. Internet-Drafts are
working documents of the Internet Engineering Task Force
(IETF), its areas, and its working groups. Note that other
groups may also distribute working documents as
Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six
months and may be updated, replaced, or obsoleted by other
documents at any time. It is inappropriate to use
Internet-Drafts as reference material or to cite them other
than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt
To view the list Internet-Draft Shadow Directories, see
http://www.ietf.org/shadow.html.
2. Abstract
LDAP provides simple primitives to construct search filters for
directory lookups. Complex LDAP search queries such as those
involving joins or functional evaluation of simple query
results require client side processing of the intermediate
search results. The network overhead of retrieving
intermediate results can be substantial for large databases.
This document defines an LDAP extension for supporting
server side evaluation of complex search queries.
Goenka INTERNET DRAFT [Page 1]
<draft-goenka-ldapext-java-complex-search-01.txt> June 1999
2.1 Specification Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL
NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as
described in RFC 2119.
3. Motivation
"Directory" is at the heart of many powerful and distributed
applications today. With the widespread use of directory for
variety of applications, the lookup queries would no longer
be simple exact/partial matches of a subset of object
attributes. Search queries involving joins or complex
functional evaluation of simple query results would be quite
common. An example of a search involving joins is : "Get me
all the names and phone numbers of all printer
administrators", which currently would require one query to
get the administrator attribute of all printer objects and
then separate queries to get the phone number for each
administrator. A query based on functional evaluation would
be one, where the search criteria is based on a function,
f(x) of a simple query result x, rather than on x itself. An
example is a X.509 Certificate parsing function, which
returns a particular attribute (say, key usage) of a BER
encoded X.509 Certificate. Assuming that a user's public key
certificates are stored in the directory as simple
attributes (byte array) rather than as objects, a search for
the Root CA (Certification Authority) of a user's data
encryption public key certificate would require retrieving
all the certificates of the user, parsing them at the client
side to find the one corresponding to the data encryption
public key, and issuing another set of search queries to
find each certificate's signer (CA), until the Root CA's
self signed certificates are found.
LDAP [1] is the most widely used protocol to access a directory.
LDAP search queries can be simple enough to be built from
rather simple primitives or could be quite complex to
require special logic for evaluation. An LDAP client would
resolve a complex query by fetching many directory entries
from the server and evaluating each of them at the client
side, issuing further queries based on intermediate results
till the final desired result is found. Unless the initial
search space can be reduced at the server end, using some
exact/partial matches, this would be prohibitively expensive
for large directory databases.
This document outlines an efficient alternative for such enhanced
directory search operations.
Goenka INTERNET DRAFT [Page 2]
<draft-goenka-ldapext-java-complex-search-01.txt> June 1999
4 Introduction
LDAP provides simple primitives to construct search filters.
Specific implementations of LDAP Server may provide custom
matching rules (called extensibleMatch), but their
usefulness is limited because they are server specific and
cannot be updated dynamically. To be useful to a large
variety of directory based applications, the search
mechanism should allow LDAP clients to specify application
specific filters based on complex logic, (such as X509
Certificate parsing logic to retrieve specific attributes of
a BER encoded certificate, or a logic to determine a prime
number) in a secure manner. The variety of processing logic
required to construct complex search filters can only be
adequately specified using a programming/scripting language.
The choice for the right language is a difficult one and
subject to debate. We follow an elimination process on the
following non-exhaustive list of requirements of a suitable
language.
1. Wide acceptance as a programming language.
2. Availability on wide range of server and client platforms.
3. Availability of a rich and standard set of useful libraries to
facilitate writing complex search queries.
4. Security features to address security implications of running
client code on the server.
5. Good integration with the programming language used to write
LDAP client applications, so that the search query can be
adequately constructed by the client application.
Scripting languages such as Perl, JavaScript, Visual Basic Script
etc. do not meet several of the above requirements. In
particular, requirements of availability on a wide variety
of platforms, suitability for general purpose application
programming, support for security features and availability
of a rich set of standard libraries and protocol
implementations. Among programming languages, Java is the
only language that meets all the above requirements with its
write-once-run-anywhere feature and technologies for secure
object migration, namely, serialization and reflection, and
security features designed for remote execution in web
browsers across the Internet. With wider acceptance of Java
as the Internet Programming Language, many business
application are being written in Java, including directory
based applications. A rich set of standard libraries is
available on all Java platforms as part of Java Development
Kit (JDK) or standard extensions to it. Non-standard
Goenka INTERNET DRAFT [Page 3]
<draft-goenka-ldapext-java-complex-search-01.txt> June 1999
libraries are easy to integrate since classes can be
dynamically retrieved and loaded as required.
4.1 Proposal Overview
The key idea is to define an LDAP extension for sending complex
search queries as serialized java objects and the required
class definitions in a java archive (jar), which may
optionally be signed. The serialized java object contains
state information such as query parameters, while the class
definition contains the evaluation logic. The server
instantiates the java object in a Java Virtual Machine (JVM)
and initializes it with an LDAP connection (over a socket)
to itself with the same associated privileges as the client
that originated the search. The java object MUST implement a
known interface (discussed later)to enable the LDAP server
invoke its initialization routine. The java object executing
at the server end acts as a "client-proxy" and makes local
LDAP search queries to the server. It evaluates the search
results using the embedded custom logic and may perform
several simple directory lookups to arrive at the final
result. The client-proxy passes only the final result back
to the LDAP server, which returns them back to the
originating client.
The client-proxy interacts with the LDAP server using LDAP calls
over the LDAP connection that the server initialized it
with. This allow the client-proxy to perform additional
intermediate queries without requiring any API change to the
LDAP server. The server enforces the client's access
restrictions by granting the LDAP connection with the
client-proxy, the same privileges as the originating client
connection. The client-proxy SHOULD NOT be granted java
permissions to open any new socket or do any file I/O on the
host machine for security reasons. The JVM running the
client-proxy code MAY execute on a different physical
machine than the LDAP server, since the client-proxy
communicates only with the LDAP server and over an LDAP
socket that it is bootstrapped with. This flexibility allows
various options for deployment of JVM to run client-proxies,
including a dedicated pool of Java Application Servers,
possibly connected to the host running the LDAP server over
a high speed LAN. Java applications that use LDAP can also
reuse a good chunk of their existing search filter code by
sending them across to LDAP servers that support the
proposed extension, and executing it locally otherwise (the
default choice).
4.2 Goals and Requirements
The requirements of this specification are :
Goenka INTERNET DRAFT [Page 4]
<draft-goenka-ldapext-java-complex-search-01.txt> June 1999
- to provide a uniform and extendible interface and data format
for securely executing client-proxy code on the server and
communicating the results back to the client
- to provide a simple way of indicating time-based resource
requirements for executing client code on the server side
- to provide a light-weight way of communicating the client proxy
code to the server, including all java objects and required
class definitions
The goals of this document are :
- to show the way in which the notion of a client-proxy is used
to extend the capability of LDAP search operations
- to describe the data structure of the LDAP extended operation
for communicating java objects encapsulating search queries
- to indicate an appropriate design for implementing the extension
handler at the LDAP server, requiring minimum changes to the
server code
- to address the security concerns of executing client code at
server side
4. Security Considerations
Secure data exchange is one of the foremost issues associated with
any client-server communication. The privacy and/or
integrity of data exchanges is not available to LDAP, unless
security at the transport layer (e.g., Secure Socket Layer)
or at network layer (e.g., IPSEC or VPN) is used. Thus the
mechanism used to protect all data exchanges between the
LDAP client and the server, applies to the proposed
extension as well, for all known attacks in this category
(e.g., server spoofing, data sniffing, man-in-the-middle
attack, session hijacking).
There are two key aspects of security associated with the
execution of client-proxy at the server end. The first one
concerns the security of data and resources at the server,
that hosts the client-proxy. This aspect of security is
analogous to the security considerations of running a
downloaded Java Applet in a client's web browser. The Java
Security Architecture allows association of permissions with
java code, based on where the code came from (known as code
source) and who signed it. LDAP server implementations MAY
define their own permission policies, and thereby achieve a
sandbox environment for the execution of the client-proxy.
Goenka INTERNET DRAFT [Page 5]
<draft-goenka-ldapext-java-complex-search-01.txt> June 1999
LDAP servers MAY require that the jar files containing the
client-proxy be digitally signed in order to place trust on
the non-malicious nature of the code.
The second aspect is the fairness of execution. Its possible for
the client-code (malicious, bug-ridden or otherwise) to take
more than its fair share of server processing time. The
client could use the lease model [6] to negotiate an
approximate processing time at the server end. The server
would use the client authentication information, along with
current load to grant a lease for appropriate time. The java
thread executing the client-proxy could be killed if the
lease expires, unless the lease is renewed. The details of
the leasing model are discussed in the following section.
5. Lease of Server Resources
The lease model is used to negotiate the resource requirements at
the server-side. The server resources to be considered for
lease are execution time, memory usage, I/O resources (e.g.,
reading from or writing to files) and network resources such
as sockets.
- Execution Time -- The client indicates the estimated and maximum
values of execution time (estimatedDuration and maxDuration)
that might be required by the client-proxy. The server
should either completely deny the lease or grant for atleast
estimatedDuration up to a maximum of maxDuration. If the
leased duration is elapsed and the client-proxy is still
executing, the server can either extend the lease (depending
upon server load conditions) or terminate the client-proxy
thread.
- Memory Usage -- The client indicates the estimated and maximum
values of runtime memory (estimatedMemory and maxMemory)
that might be required by the client-proxy. The server
should either completely deny the lease or grant for atleast
estimatedMemory up to a maximum of maxMemory. Memory
requests beyond the granted lease MAY fail, unless the
server relaxes or extends the lease, and the client-proxy
code SHOULD be prepared to handle such situations. Server
implementations MAY choose a relaxed memory lease policy, if
a strict policy is difficult to implement.
- I/O and Network Resources -- It is not expected of the
client-proxy to use the I/O or network resources at the
server end. An LDAP socket would be provided to the
client-proxy by the bootstrap code at the server to
communicate with the LDAP server locally. Java Permissions
(as defined by RequestedPermissions) are to be used to
Goenka INTERNET DRAFT [Page 6]
<draft-goenka-ldapext-java-complex-search-01.txt> June 1999
restrict or selectively allow usage of specific I/O or
network resources.
The lease response contains the lease granted to the client.
Server implementations MAY implement their own algorithms to
decide the lease grant for time-duration and memory usage
depending upon the client's rights and server load
conditions.
5.1 Lease Renewal
A client-proxy might need to execute for a much longer duration to
complete, though it might be able to generate partial
results in much shorter time. A server might be unable to
grant lease for the entire requested duration (maxDuration)
at first negotiation, but might be fine with renewing the
grant for another chunk of time, depending upon load
conditions. If the client-proxy hasn't finished executing
completely till the granted lease expires, the server can
choose to renew the lease up to a maximum of maxDuration
minus the previous grant. Once a granted lease expires, the
server must notify the client about either the lease renewal
or the termination of the client-proxy. (The lease grant on
resources other than execution time would have to be handled
in an analogous way.)
6. Architecture for JVM activation and client-proxy execution
The current architecture assumes no change to the existing LDAP
server other than an extension handler for "Java based
Search Query Extension". The extension handler parses the
extended operation request, and determines whether the
requirements to invoke a JVM for client-proxy activation are
met. The basic requirements include :
- Lease Grant : Does the server have enough resources to grant the
requested lease for server resources?
- Java LDAP implementation : Does the server have an
implementation of LDAP in Java, conforming to the interface
that the client-proxy is written to? There are a few
possible standards emerging, namely, JNDI (Java Naming and
Directory Interface) with LDAP binding, and java LDAP API
[3]. It is expected that the server may already have the
java class files corresponding to standard LDAP interfaces,
in which case these classes need not be sent as part of the
query. For clients which are written to proprietary java
LDAP interfaces, the implementation for the same must be
supplied in a jar, along with the extended request.
Goenka INTERNET DRAFT [Page 7]
<draft-goenka-ldapext-java-complex-search-01.txt> June 1999
- Java Permissions : Does the server policy allow granting the
requested java permissions to the client-proxy code,
possibly based on the client's authentication credentials.
An appropriate response is sent to the client, whether or not the
requirements for activation are met. If the requirements are
met, the server launches a JVM (or communicates with an
existing JVM), and schedules the activation and execution of
the client-proxy. The java extension handler thread creates
a class loader, to read the required classes from the jar
accompanying the request. Appropriate permissions are
associated with the protection domain defined by these
classes. Next, it opens a socket with the LDAP server, and
associates the permissions of the requesting client with the
socket, using the internal APIs of the LDAP server. Finally,
the java object encapsulating the query is de-serialized,
using the readObject() method of the object's class. The
LDAP socket is passed as an argument to the startup function
(run(socket)) and a new thread starts executing the
client-proxy.
The client-proxy code does a simple bind to the LDAP server over
the given LDAP socket, and makes further LDAP search queries
to the server. Separate methods are provided to return
(partial) results, and to query the progress of the
client-proxy. The client-proxy MUST be allowed client's
privileges using simple LDAP Bind.
The interface definition for the client-proxy is as follows:
public interface org.ietf.ldap.client.LDAPClientProxy
implements java.io.Serializable {
/**
* The startup function, which queries the LDAP server using
* the given LDAP Socket. The partial results are stored
* internally such that they can be retrieved at any time,
* using getPartialResults().
*/
void run (java.net.Socket socket);
/**
* Returns the (partial) results (if available). The results
* are formatted as a byte [], which is interpreted by the
* client. null is returned, if there are no partial results
* available.
*/
synchronized byte [] getPartialResults();
/**
* Returns the progress in terms of percentage (0-100). This
* helps the monitor thread to gauge the progress, and take
Goenka INTERNET DRAFT [Page 8]
<draft-goenka-ldapext-java-complex-search-01.txt> June 1999
* the lease renewal decisions. The main thread (executing
* run()) has the responsibility to increment the progress
* appropriately.
*/
synchronized int progress();
}
The client-proxy thread executes through the run() method,
incrementing the progress variable as well as generating
partial results (formatted as byte[]) during its execution.
A monitor thread MAY be started by the java extension
handler to monitor the progress of activity, as well as
conformance to lease agreement. Periodically, the monitor
thread reads the partial results (if any) and communicates
them back to the LDAP server (after adding other fields of
the response) for returning as part results to the client.
The frequency at which partial results should be sent can be
controlled by the client-proxy by way of generating the
results. client-proxy code SHOULD catch all java exceptions
that might occur during the execution of the run() method,
and translate exceptions into results, to be returned back
to the client. Server implementations SHOULD return stack-
trace for any thrown exceptions back to the client, with
appropriate error code.
7. ASN.1 Definition of Search Query Extension
Comments on the design and utility of these definitions are
specifically invited from all reviewers.
The ASN.1 definition of extended operation as per LDAP (v3)
specification [2] is as follows :
ExtendedRequest ::= [APPLICATION 23] SEQUENCE {
requestName [0] LDAPOID,
requestValue [1] OCTET STRING OPTIONAL }
ExtendedResponse ::= [APPLICATION 24] SEQUENCE {
COMPONENTS OF LDAPResult,
responseName [10] LDAPOID OPTIONAL,
response [11] OCTET STRING OPTIONAL }
where, LDAPOID is a dotted-decimal representation of the OBJECT
IDENTIFIER corresponding to the request/response, and
LDAPResult corresponds to the final status (success/failure
indications) of the protocol operation request.
In the following section, the ASN.1 definition of request and
response values are presented.
requestValue ::= SEQUENCE {
leaseRequest LEASERequest,
Goenka INTERNET DRAFT [Page 9]
<draft-goenka-ldapext-java-complex-search-01.txt> June 1999
ldap_sdk ENUMERATED {
JNDI (0),
LDAP_JavaAPI (1),
-- 2 to 5 reserved --
Proprietary (6) }
interface ClassName,
serializedObject OCTET STRING,
classes JavaARchive,
classURL [0] LDAP_URL OPTIONAL,
permissions [1] RequiredPermissions OPTIONAL }
LEASERequest ::= SEQUENCE {
estimatedDuration INTEGER (0 .. maxInt), -- in seconds --
maxDuration INTEGER (0 .. maxInt),
estimatedMemory INTEGER (0 .. maxInt), -- in KB --
maxMemory INTEGER (0 .. maxInt) }
RequiredPermissions ::= SEQUENCE {
permType ClassName,
permName OCTET STRING,
critical BOOLEAN DEFAULT TRUE,
actions [0] OCTET STRING OPTIONAL }
ClassName ::= OCTET STRING
JavaARchive ::= OCTET STRING
- estimatedDuration -- duration in seconds, for which the
client-proxy must execute to generate any useful results.
- maxDuration -- the duration in seconds, which is the maximum
time the client-proxy would need to execute completely.
- estimatedMemory -- the memory requirement in KB, which the
client-proxy must have to execute.
- maxMemory -- the maximum amount of memory that the client-proxy
would ever need.
- ldap_sdk -- the java LDAP API SDK to which the serializedObject
(client-proxy) is written. There are two possible standards
emerging, namely, JNDI (Java Naming and Directory Interface)
with LDAP binding, and java LDAP API [3]. It is expected
that the server may already have the class files
corresponding to standard APIs, in which case these classes
need not be sent as part of the query. For clients which are
written to proprietary java LDAP implementation, the client
MUST send the entire LDAP implementation in a (optionally
signed) jar, as part of the query, at the expense of the
overhead involved. ASN.1 OIDs might be a better alternative
for specifying the ldap_sdk than the current method via
Goenka INTERNET DRAFT [Page 10]
<draft-goenka-ldapext-java-complex-search-01.txt> June 1999
enumeration. Comments.
- interface -- the full name {package.class} of the
class/interface corresponding to the serialized object. This
is required by the server in order to appropriately
deserialize the object and invoke its methods.
- serializedObject -- the serialized java object which
encapsulates the state and methods to execute the search
query on the server. There is only one root object
(implementing the given interface), which would be the entry
point for execution. The root object may have references to
other objects (deep copy of non-transient references) which
are preserved during serialization.
- classes -- the supporting class definitions (java byte code) for
all the classes required to de-serialize and execute the
client-proxy. This may include java implementations of LDAP,
if the server doesn't support the ldap_sdk to which the
client-proxy is written. The classes are packaged in a JAR
(java archive), in the standard JAR format.
- classURL -- specifies (optionally) the URL reference for the
class definitions that might be required for executing the
client-proxy. This would include class definitions for
proprietary java LDAP implementations. The Server MAY NOT
support downloading classes from a URL, in which case an
appropriate error is returned.
- permissions -- these correspond to java permissions that might
be required by a client-proxy to execute. By default, the
client-proxy may have a minimal set of permissions grant
(similar to a restricted applet). Any specific permission
must be approved by the server. This model assumes the
jdk1.2 security architecture with respect to permissions.
The permission requests have the following structure :
- permType -- the java permission class governing the required
permission. e.g., java.lang.RuntimePermission,
java.io.FilePermission etc.
- permName -- the name of the permission or first argument to the
appropriate permission class, such as host name (for
java.net.SocketPermission), createClassLoader (for
java.lang.RuntimePermission)
- critical -- determines whether this permission is a must for the
client-proxy to execute. All critical permissions must be
granted by the server, in order to execute the client-proxy.
- actions -- second argument that some permissions take. e.g., for
java.io.FilePermission the allowed actions are read, write,
Goenka INTERNET DRAFT [Page 11]
<draft-goenka-ldapext-java-complex-search-01.txt> June 1999
execute, delete. Some permissions do not take a third
argument (such as java.lang.RuntimePermission), and hence
this is an optional argument.
The ASN.1 definition for the extended response value follows :
response ::= SEQUENCE {
COMPONENTS OF LDAPExtendedResult,
leaseGrant LEASEGrant,
ldap_sdk_supported [0] ENUMERATED {
JNDI (0),
LDAP_JavaAPI (1),
-- 2 to 5 reserved --
} OPTIONAL
resultValue [1] OCTET STRING OPTIONAL
resultDone BOOLEAN DEFAULT FALSE }
LEASEGrant ::= SEQUENCE {
duration INTEGER (0 .. maxInt), -- in seconds --
memory INTEGER (0 .. maxInt) } -- in KB --
LDAPExtendedResult ::= SEQUENCE {
resultCode ENUMERATED {
success (0),
leaseExpired (1),
resourceUseExceeded (2),
securityViolation (3),
noActivity (4),
permissionDenied (5),
dataError (6),
serverBusy (7),
leaseRenewed (8),
leaseGranted (9),
URL_ReferenceNotSupported (10),
-- 11-30 unused --
other (31) }
operationAbort BOOLEAN DEFAULT TRUE,
description OCTET STRING OPTIONAL }
- LDAPExtendedResult -- these return codes are specific to this
extension and are in addition to the more general LDAPResult
codes. They are used to communicate errors specific to the
java execution of the client-proxy. The above enumerated
list is clearly non-exhaustive, and have obvious meanings.
If an error results in the abortion of client-proxy, the
operationAbort flag is set true. description provides
additional textual information about the error, such as the
stack trace in case an exception is thrown in the run(
socket) method.
- LEASEGrant -- the duration (in seconds) and the memory (in KB)
for which the lease is granted by the server. In case of a
Goenka INTERNET DRAFT [Page 12]
<draft-goenka-ldapext-java-complex-search-01.txt> June 1999
lease renewal, these values are in addition to the values or
previous lease grants.
- ldap_sdk_supported -- the LDAP APIs supported by the server. A
server may use this to communicate all SDKs that it
supports. If the ldap_sdk that the client-proxy is written
to, is not supported by the server, the client MUST send a
new extended request, along with the class files for the
ldap_sdk.
- resultValue -- the result value is generated by the
client-proxy, to be consumed only by the client, and hence
can have any proprietary format, including but not limited
to existing LDAP response formats, or even java object(s)
etc.
- resultDone -- when false, denotes that this is a partial result,
and more partial results would follow. When true, denotes
that this is the last and final part of the result. This
allows the client- proxy to generate partial results, as
appropriate, and communicate them to the client.
8. Implementation Considerations
To make use of this proposal, the LDAP server should be capable of
invoking a JVM, and should implement the proposed LDAP
extension. Java LDAP APIs should be standardized and the
client should be written to them. Until standard Java APIs
for LDAP are available, clients MUST send all the java
classes required for LDAP communication over a given socket.
The LDAP clients should be enabled to send a search query
encapsulated as a serialized java object using the LDAP
extension API. Java Security Architecture version 1.2 is
assumed on the server end for associating specific
permissions with client-proxy code.
9. Acknowledgements
Thanks are due to Brian Jarvis and Vipul Modi for their
contributions to this work.
10. References
[1] M. Wahl, T. Howes, S. Kille, "Lightweight Directory Access
Protocol", RFC 2251.
[2] M. Wahl, T. Howes, S. Kille, "Lightweight Directory Access
Protocol (v3)", Internet Draft
draft-ietf-asid-ldapv3-protocol-04.txt, March 1997.
Goenka INTERNET DRAFT [Page 13]
<draft-goenka-ldapext-java-complex-search-01.txt> June 1999
[3] R. Weltman, T. Howes, M. Smith, Christine Ho, "The Java LDAP
Application Program Interface", Internet Draft
draft-ietf-ldapext-ldap-java-api-02.txt, July 1998.
[4] T. Howes, "A String Representation of LDAP Search Filters,"
RFC 1960, June 1996.
[5] David Boreham, Chris Weider, "LDAP Extensions for Scrolling
View Browsing of Search Results", Internet Draft,
draft-ietf-ldapext-ldapv3-vlv-01.txt, March, 1998
[6] Sun Microsystems, Inc. "Distributed Leasing Specification",
Whitepaper (JINI), lease.pdf, Revision 1.0 Beta, July, 1998
11. Author's Address
Vishal Goenka
Novell, Inc.
122 East, 1700 South
Provo, UT 84606
USA
Phone: +1 801 861 4573
Fax: +1 801 861 2522
Email: vgoenka@novell.com