Part 4: How to configure paging for REST data

In the earlier tutorials of this series, we showed you how to do the following using the Autonomous REST Connector:

• Part 1: How to query REST data with JDBC applications
• Part 2: How to query data from multiple REST endpoints
• Part 4: How to edit auto-generated schema for REST endpoints

In the final tutorial, we will discuss how to configure paging in the Autonomous REST Connector to allow you to return multiple pages in your response. Most REST services implement paging to reduce the loads on the server and improve performance when accessing large amounts of data. However, when querying the Yelp API in earlier tutorials in this series, you were only seeing data from the first page of the API response. This was fine for demonstration purposes, but, in a production environment, you would likely need to access the data beyond the first page.

• Paging mechanisms and properties
• Configuring paging for the Yelp REST API
• Conclusion

The Autonomous REST Connector supports the following paging mechanisms:

• Row offset paging
• Page number paging
• Next page token

If we take a look at the Yelp documentation on how to page results, you will notice the use offset paging parameters:

To configure row offset paging for Yelp, we will need to set the following Autonomous REST Connector paging properties using the Autonomous REST Composer:

Note that the Autonomous REST Connector supports a number of paging properties to configure different paging mechanisms and implementations. For more information on configuring supported paging mechanisms, refer to Paging in the Progress DataDirect Autonomous REST Connector for JDBC User's Guide.

In this section, we are going to configure the Autonomous REST Connector to page results returned from Yelp. Since Yelp uses offset paging parameters, you will need to configure a subset of the driver's row offset parameters to control paging. In the following steps, we will demonstrate how to configure paging for the Events endpoint.

1. If you have not done so already, open the Autonomous REST Composer, import your Model file, and configure your authentication. If you need help importing a Model file, see Part 3: How to edit auto-generated schema for REST endpoints.
2. Select the Configure Endpoints tab to begin editing your paging. 
3. In the left pane, select the endpoint for which you want to configure pagination. For our example, we will configure pagination for the events endpoint:

   GET https://api.yelp.com/v3/events/events

4. In the customization pane on the right, expand the Pagination section.
5. In the Template field, select Row Offset Paging; then, provide the following values in the corresponding field:
   • MaximumPageSize: 50
   • PageSizeParameter: limit
   • FirstRowNumber: 0
   • RowOffsetParameter: offset
   In this configuration, MaximumPageSize is set to 50 because Yelp returns a maximum of 50 results per page, while the value of PageSizeParameter is set according to the instructions in the Yelp documentation.
6. When prompted, click Refresh to apply your changes to the Model file.
7. Move your Model file to a location that is accessible by the driver. When configuring your connection string or data source, you will need to specify this file and location using the Config connection property to apply your updates.
8. Open your database or SQL query tool, such as Dbeaver, and execute the following query:

   SELECT * FROM EVENTS

   You should receive over 200 records, instead of the 20 records you saw with the default paging configuration.

   

You have reached the end of this tutorial series. By completing this series, you have learned how to:

• Connect to a single REST endpoint
• Query data from multiple endpoints
• Edit the schema for your REST endpoints
• Configure paging for your REST data

To learn more, refer to the Tutorials or Documentation for the Autonomous REST Connector.