Model Storing with Concepts Server: Streamlined Access, Easy Retrieval and Better Collaboration

Progress Semaphore Concepts Server, a new modern store for published models, allows organizations to more easily store and share semantic models across the enterprise at scale. See for yourself and learn how Semaphore can help.

The Concepts Server, a new feature available with the Semaphore 5.8. release, presents a powerful, scalable and flexible store for published models. This feature, like the Semantic Enhancement Server (SES), provides a RESTful set of APIs to query your model information that is friendly to a wide variety of contexts, from command-line tools to JavaScript browser apps. This service, backed by Progress MarkLogic rather than SOLR, provides a wide array of API improvements, especially for larger models.

Semantic Enhancement Server vs Concepts Server

First things first though: let’s quickly do a round of introductions.

The Semantic Enhancement Server (SES) is a high-speed index that enables you to search for information in the model in real-time. It is used as a single source of truth for downstream systems that need model information and in metadata authoring workflows to double-check to see that “approved” values are used. With the help of SES, users can navigate their content effectively, helping them to refine search results or offer related concepts from the model. These results can be leveraged to suggest related areas of interest and show associated documents, containing the concepts, people, places and organizations that match search criteria.

In Semaphore 5.8, we introduce the Concepts Server that has substantial functional overlap with SES including features like Tree Viewer, Concept Mapping, SAYT and Visualizer. In our efforts to provide better integration with MarkLogic, the new feature uses MarkLogic Server as a backend. It is not a replacement to SES, but an additional feature available to Semaphore and MarkLogic users. This new feature is designed to guide users at query time towards concepts that are used during classification time. With Semaphore 5.8, we have created a new model that has a lot of functional overlap with SES – this is based on the MarkLogic database and we took the opportunity to refresh the API that the data returned is much more flexible and better represents the data of the model itself.

The tool can be used as a server for downstream systems, allowing users to get large chunks of the model out of the read-only version with one call. It has been tuned so that data is streamed to calling applications, which means it has much lower memory footprint and it is designed to support these large-scale queries, while serving demanding clients at API level for elsewhere in the enterprise.

5 Reasons to Use the Concepts Server

With its new capabilities, the Concepts Server revolutionizes how businesses manage and store their semantic models.

Faster Response Time: The Concepts Server supports faster time to the first byte in responses irrespective of result size. This means that the time between the initial request and the server’s response is significantly reduced to accelerate time to insight.

Multiple Languages at Once: The Concepts Server is designed to return labels and properties of all or a selected list of language tags in the same response. With Concept Server, if you want data for multiple languages this can be achieved in a single request – just present the list of languages for which you want data as part of the request with the “lang” parameter.

More Compact Data: The Concepts Server enables clients to specify which data they want to return and reduces the amount of movable data.

Full Paging: Response data includes total response size along with limit and offset parameters. This means that the client fully controls the amount of data returned for each request.

More Comprehensive Filtering: The Concepts Server offers numerous ways to filter data for responses. This enables more precise and contextually relevant information retrieval, enabling users to navigate and filter vast amounts of data to find the most relevant information to their business use case.

Getting Started with Concepts Server

If you want to use the Concepts Server as part of your Semaphore 5.8 installation, here is how to get started. Imagine that you have published a simple model “Test” to Concepts Server and want to query for all the concepts in the model, but only want the GUIDs and preferred labels in all languages to be returned. You can construct this query using RESTful get request as follows:

http://[SERVER]:5092/models/Test/concepts?extracts=id,prefLabels

You can page through the results if desired using limit and offset (zero-based) parameters:

http://[SERVER]:5092/models/Test/concepts?extracts=id,prefLabels&limit=100&offset=0

Responses by default are in JSON format, and are tabular in appearance:

Publishing a model to Concepts Server requires you configure a MarkLogic Server backend in the Concepts Server Service configuration file “marklogic.properties.” If the specified MarkLogic database does not exist, Concepts Service will create it:

Concepts/conf/marklogic.properties:

## Marklogic server configuration

marklogic.host=localhost

marklogic.port=8000

marklogic.monitoring.port=8002

marklogic.admin.user=admin

marklogic.admin.password=myadminpassword

# The name of the database where the documents will be stored.

# If the database does not exist, it will be created.

marklogic.database=Concepts

# The name of the forest which will be attached to the database.

# If the forest does not exist, it will be created.

marklogic.forest=semaphore-concepts-forest

After you have configured Concepts Server, you need to configure Concepts Service inside a Semaphore Studio environment:

Studio KMM provides a default publisher configuration file for publishing to Concepts Service services:

Once the publish completes, you can use a browser to fetch data. I know you’d like to see how all of this translates into the real world of day-to-day tasks; check out this video to learn more about this exciting feature.

Conclusion

The new feature in Semaphore 5.8 further increases the productivity of Semaphore users tasked with the management and dissemination of semantic models. Contact us to migrate to Semaphore 5.8 and experience this new feature.