Internet DRAFT - draft-peterson-scim-cursor-pagination
draft-peterson-scim-cursor-pagination
SCIM M. Peterson
Internet-Draft One Identity
Updates: 7643, 7644 (if approved) D. Zollner
Intended status: Informational Microsoft
Expires: 16 June 2023 13 December 2022
Cursor-based Pagination of SCIM Resources
draft-peterson-scim-cursor-pagination-02
Abstract
This document defines additional SCIM query parameters and result
attributes to allow use of cursor-based pagination in SCIM
implementations that are implemented with existing code bases,
databases, or APIs where cursor-based pagination is already well-
established.
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on 16 June 2023.
Copyright Notice
Copyright (c) 2022 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents (https://trustee.ietf.org/
license-info) in effect on the date of publication of this document.
Please review these documents carefully, as they describe your rights
and restrictions with respect to this document. Code Components
extracted from this document must include Revised BSD License text as
described in Section 4.e of the Trust Legal Provisions and are
provided without warranty as described in the Revised BSD License.
Peterson & Zollner Expires 16 June 2023 [Page 1]
Internet-Draft SCIM Cursor Pagination December 2022
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
1.1. Notational Conventions . . . . . . . . . . . . . . . . . 2
2. Query Parameters and Response Attributes . . . . . . . . . . 3
2.1. Pagination errors . . . . . . . . . . . . . . . . . . . . 5
2.2. Sorting . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.3. Cursors as the Only Pagination Method . . . . . . . . . . 6
3. Querying Resources Using HTTP POST . . . . . . . . . . . . . 7
4. Service Provider Configuration . . . . . . . . . . . . . . . 8
5. Security Considerations . . . . . . . . . . . . . . . . . . . 9
6. Normative References . . . . . . . . . . . . . . . . . . . . 9
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 10
1. Introduction
The two common patterns for result pagination in HTTP-based protocols
are index-based pagination and cursor-based pagination. Rather than
attempt to compare and contrast the advantages and disadvantages of
competing pagination patterns, this document simply recognizes that
SCIM service providers are commonly implemented as an
interoperability layer on top of already existing application
codebases, databases, and/or APIs that already have a well-
established pagination pattern.
Translating from an underlying cursor-based pagination pattern to the
index-based pagination defined in Section 3.4.2.4 of [RFC7644]
ultimately requires the SCIM service provider to fully iterate the
underlying cursor, store the results, and then serve indexed pages
from the stored results. This task of "pagination translation"
dramatically increases complexity and memory requirements for
implementing a SCIM Service Provider, and may be an impediment to
SCIM adoption for some applications and identity systems.
This document defines a simple addition to the SCIM protocol that
allows SCIM service providers to reuse underlying cursors without
expensive translation. Support for cursor-based pagination in SCIM
encourages broader cross-application identity management
interoperability by encouraging SCIM service provider implementations
for applications and identity systems where cursor-based pagination
is already well-established.
1.1. Notational Conventions
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in [RFC2119] .
Peterson & Zollner Expires 16 June 2023 [Page 2]
Internet-Draft SCIM Cursor Pagination December 2022
2. Query Parameters and Response Attributes
The following table describes the URL pagination parameters requests
for using cursor-based pagination:
+===========+===================================================+
| Parameter | Description |
+===========+===================================================+
| cursor | The string value of the nextCursor attribute from |
| | a previous result page. The cursor value MUST be |
| | empty or omitted for the first request of a |
| | cursor-paginated query. |
+-----------+---------------------------------------------------+
| count | A positive integer. Specifies the desired |
| | maximum number of query results per page, e.g., |
| | count=10. When specified, the service provider |
| | MUST NOT return more results than specified, |
| | although it MAY return fewer results. If count |
| | is not specified in the query, the maximum number |
| | of results is set by the service provider. |
+-----------+---------------------------------------------------+
Table 1
The following table describes cursor-based pagination attributes
returned in a paged query response:
Peterson & Zollner Expires 16 June 2023 [Page 3]
Internet-Draft SCIM Cursor Pagination December 2022
+================+===============================================+
| Element | Description |
+================+===============================================+
| nextCursor | A cursor value string that MAY be used in a |
| | subsequent request to obtain the next page of |
| | results. Service providers supporting |
| | cursor-based pagination MUST include |
| | nextCursor in all paged query responses |
| | except when returning the last page. |
| | nextCursor is omitted from a response only to |
| | indicate that there are no more result pages. |
+----------------+-----------------------------------------------+
| previousCursor | A cursor value string that MAY be used in a |
| | subsequent request to obtain the previous |
| | page of results. Use of previousCursor is |
| | OPTIONAL. Service Providers that are unable |
| | to support a previousCursor MAY omit |
| | previousCursor when sending paged query |
| | responses. |
+----------------+-----------------------------------------------+
Table 2
The SCIM client MUST consider cursor to be opaque and make no
assumptions about cursor values. When the client wants to retrieve
another result page for a query, it should query the same Service
Provider endpoint with all query parameters and values being
identical to the initial query with the exception of the cursor value
which should be set to a nextCursor (or previousCursor) value that
was returned by Service Provider in a previous response.
For example, to retrieve the first 10 Users with username starting
with "J", use an empty cursor and set the count to 10:
GET /Users?filter=username%20sw%20J&cursor&count=10
Host: example.com
Accept: application/scim+json
Authorization: Bearer U8YJcYYRMjbGeepD
The response to the query above returns metadata regarding pagination
similar to the following example (actual resources removed for
brevity):
Peterson & Zollner Expires 16 June 2023 [Page 4]
Internet-Draft SCIM Cursor Pagination December 2022
{
"totalResults":100,
"itemsPerPage":10,
"nextCursor":"VZUTiyhEQJ94IR",
"schemas":["urn:ietf:params:scim:api:messages:2.0:ListResponse"],
"Rsesources":[{
...
}]
}
Given the example above, to request the next page or results, use the
same query parameters and values except set the cursor to the value
of nextCursor ("VZUTiyhEQJ94IR"):
GET /Users?filter=username%20sw%20J&cursor=VZUTiyhEQJ94IR&count=10
Host: example.com
Accept: application/scim+json
Authorization: Bearer U8YJcYYRMjbGeepD
{
"totalResults":100,
"itemsPerPage":10,
"previousCursor: "ze7L30kMiiLX6x"
"nextCursor":"YkU3OF86Pz0rGv",
"schemas":["urn:ietf:params:scim:api:messages:2.0:ListResponse"],
"Rsesources":[{
...
}]
}
In the example above, the response includes the OPTIONAL
previousCursor indicating that the Service Provider supports forward
and reverse traversal of result pages.
As described in Section 3.4.1 of [RFC7644] Service Providers SHOULD
return an accurate value for totalResults which is the total number
of resources for all pages. Service Providers implementing cursor
pagination that are unable to estimate totalResults MAY return a
response with totalResults set to zero (0).
2.1. Pagination errors
If a Service Provider encounters an invalid pagination query
parameters (invalid cursor value, count value, etc), or other error
condition, the Service Provider SHOULD return the appropriate HTTP
response status code and detailed JSON error response as defined in
Section 3.12 of [RFC7644]. Most pagination error conditions would
generate HTTP response with status code 400. Since many pagination
Peterson & Zollner Expires 16 June 2023 [Page 5]
Internet-Draft SCIM Cursor Pagination December 2022
error conditions are not user recoverable, error messages SHOULD
focus on communicating error details to the SCIM client developer.
For example, cursor pagination implementations SHOULD anticipate the
following error conditions:
+==========+========================================================+
| Status | Message |
+==========+========================================================+
| 400 (Bad | Cursor value is invalid. Cursor value |
| Request) | should be empty to request the first page |
| | and set to the nextCursor or previousCursor |
| | value for subsequent queries. |
+----------+--------------------------------------------------------+
| 400 (Bad | Cursor has expired. Do not wait longer |
| Request) | than cursorTimeout (600 sec) to request |
| | additional pages. |
+----------+--------------------------------------------------------+
| 400 (Bad | Count value is invalid. Count value must |
| Request) | be between 1 - and maximumPageSize (500) |
+----------+--------------------------------------------------------+
Table 3
2.2. Sorting
If sorting is implemented as described Section 3.4.2.3 of [RFC7644] ,
then cursor-paged results SHOULD be sorted.
2.3. Cursors as the Only Pagination Method
A SCIM Service Provider MAY require cursor-based pagination to
retrieve all results for a query by including a "nextCursor" value in
the response even when the query does not include the "cursor"
parameter.
For example:
GET /Users
Host: example.com
Accept: application/scim+json
The SCIM Service Provider may respond to the above query with a page
containing defaultPageSize results and a "nextCursor" value as shown
in the below example (Resources omitted for brevity):
Peterson & Zollner Expires 16 June 2023 [Page 6]
Internet-Draft SCIM Cursor Pagination December 2022
{
"totalResults":5000,
"itemsPerPage":100,
"nextCursor":"HPq72Pax3JUaNa",
"schemas":["urn:ietf:params:scim:api:messages:2.0:ListResponse"],
"Resources":[{
...
}]
}
3. Querying Resources Using HTTP POST
Section 3.4.2.4 of [RFC7644] defines how clients MAY execute the HTTP
POST verb combined with the "/.search" path extension to issue
execute queries without passing parameters on the URL. When using
"/.search", the client would pass the parameters defined in Section 2
POST /User.search
Host: example.com
Accept: application/scim+json
Authorization: Bearer U8YJcYYRMjbGeepD
{
"schemas": [
"urn:ietf:params:scim:api:messages:2.0:SearchRequest"],
"attributes": ["displayName", "userName"],
"filter":
"displayName sw \"smith\"",
"cursor": "",
"count": 10
}
Which would return a result containing a "nextCursor" value which may
be used by the client in a subsequent call to return the next page of
resources
{
"totalResults":100,
"itemsPerPage":10,
"nextCursor":"VZUTiyhEQJ94IR",
"schemas":["urn:ietf:params:scim:api:messages:2.0:ListResponse"],
"Resources":[{
...
}]
}
Peterson & Zollner Expires 16 June 2023 [Page 7]
Internet-Draft SCIM Cursor Pagination December 2022
4. Service Provider Configuration
The /ServiceProviderConfig resource defined in Section 4 of [RFC7644]
facilitates discovery of SCIM service provider features. A SCIM
Service provider implementing cursor-based pagination SHOULD include
the following additional attribute in JSON document returned by the
/ServiceProviderConfig endpoint:
pagination A complex type that indicates pagination configuration
options. OPTIONAL.
cursor A Boolean value specifying support of cursor-based
paginations. REQUIRED.
index A Boolean value specifying support of index-based
pagination. REQUIRED.
defaultPageSize Non-negative integer value specifying the default
number of results returned in a page when a count is not
specified in the query. OPTIONAL.
maxPageSize Non-negative integer specifying the maximum number of
results returned in a page regardless of what is specified for
the count in a query. OPTIONAL.
cursorTimeout Non-negative integer specifying the maximum number
seconds that a cursor is valid between page requests. Clients
waiting too long between cursor pagination requests may receive
an invalid cursor error response. OPTIONAL.
Before using cursor-based pagination, a SCIM client MAY fetch the
Service Provider Configuration document from the SCIM service
provider and verify that cursor-based pagination is supported.
For example:
GET /ServiceProviderConfig
Host: example.com
Accept: application/scim+json
A service provider supporting both cursor-based pagination and index-
based pagination would return a document similar to the following
(full ServiceProviderConfig schema defined in Section 5 of [RFC7643]
has been omitted for brevity):
Peterson & Zollner Expires 16 June 2023 [Page 8]
Internet-Draft SCIM Cursor Pagination December 2022
{
"schemas": [
"urn:ietf:params:scim:schemas:core:2.0:ServiceProviderConfig"],
...
"pagination": {
"cursor": true,
"index": true
},
...
}
5. Security Considerations
Service Provider implementors SHOULD ensure that misuse of pagination
by a SCIM client does not deplete Service Provider resources or
prevent valid requests from other clients being handled. Defenses
for a SCIM Service Provider are similar those used to protect other
Web API services -- including the use of a "Web API gateway" layer,
to provide authentication, rate limiting, IP allow/block lists,
logging and monitoring, response caching, etc.
For example, an obvious protection against abuse is for the Service
Provider to require client authentication in order to retrieve large
result sets and enforce an overriding totalResults limit for non-
authenticated clients. Another example, would be for a Service
Provider that implements cursor pagination to restrict number of
cursors that can be allocated by a client or enforce cursor lifetime.
6. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>.
[RFC7643] Hunt, P., Ed., Grizzle, K., Wahlstroem, E., and C.
Mortimore, "System for Cross-domain Identity Management:
Core Schema", RFC 7643, DOI 10.17487/RFC7643, September
2015, <https://www.rfc-editor.org/info/rfc7643>.
[RFC7644] Hunt, P., Ed., Grizzle, K., Ansari, M., Wahlstroem, E.,
and C. Mortimore, "System for Cross-domain Identity
Management: Protocol", RFC 7644, DOI 10.17487/RFC7644,
September 2015, <https://www.rfc-editor.org/info/rfc7644>.
Peterson & Zollner Expires 16 June 2023 [Page 9]
Internet-Draft SCIM Cursor Pagination December 2022
Authors' Addresses
Matt Peterson
One Identity
Email: matt.peterson@oneidentity.com
Danny Zollner
Microsoft
Email: danny.zollner@microsoft.com
Peterson & Zollner Expires 16 June 2023 [Page 10]
