How to Use Continuous Integration to Automate Kinvey Flex Services

Repeatedly deploying new versions of a Kinvey Flex Service you’re developing can be a chore. Learn how to automate the process with Continuous Integration.

Progress Kinvey is a high productivity development platform for rapidly building modern mobile apps and experiences at consumer-scale. Developer productivity is dramatically accelerated using open-source frontend frameworks integrated with a low-code backend that enables out-of-the-box integrations with enterprise and legacy systems. 

Kinvey FlexServices are low code, lightweight NodeJS microservices that are used for data integrations and functional business logic. Flex Services utilize the Flex SDK and can consist of FlexData for data integrations, FlexFunctions for trigger-based data pre/post hooks or custom endpoints and FlexAuth for custom authentication through Mobile Identity Connect (MIC).

If you are not yet a Kinvey customer, head to our Kinvey product page to learn about the platform and sign up for a free account. In addition, this tutorial assumes some knowledge about SOA (Service Oriented Architecture) and automation (Continuous Integration/Deployment).

When a FlexService is in active development, it may become a chore to constantly deploy new versions for testing or production purposes. Luckily, you can use a Continuous Integration (CI) tool like Travis CI to fully automate the process—from triggering a build on each repository commit, to deploying the FlexService on FSR (FlexService Runtime—cloud-based infrastructure managed by Kinvey). 

This document goes through all the steps, including setting up a repository, Travis CI configuration, and a local Flex Service, as well as making sure that the necessary service is set up in the Kinvey Console.

Travis CI is a hosted, distributed CI service used to build and test software projects hosted on GitHub. It is completely free for public repositories while using it with private repositories entails charges.

You need the following to successfully complete the instructions in this article:

• GitHub account

• Travis CI account (you can log in with your GitHub account)

• Kinvey account with two-factor authentication (2FA)—Single-factor authentication accounts can still benefit from this tutorial although some steps will require changes

The GitHub repository used in this tutorial is available at: https://github.com/bilger-progress/flex-automation-article

Kinvey Console

Start by creating a Kinvey app and a Flex Service using the Kinvey Console. First, log in to the Kinvey console using your Kinvey account and create a new application. We will call our app flex-automation-article. Then navigate to Service Catalog from the upper left corner and add a new service. It needs to be a Flex Service with Flex Services Runtime. You need to give it a name such as flex-automation-article-service and choose the app you just created to allow it to access the service.

GitHub Repository

You will need a public GitHub repository where Travis CI will access your project. For the purposes of this tutorial, we will call both the repository and the NodeJS project flex-automation-article. After you clone the repository on your development machine, create a subfolder with the same name which will be the actual project folder. This convention helps keep your folder structure clear and in separating your files. Next, create a .gitignore file inside the repo folder (not inside the project directory) and add the following entries in it. The files they represent will be kept away from source control. Don’t forget to replace "flex-automation-article" with the actual folder name you’ve chosen.

 

The .kinvey file, which stores information about configured Flex Services, is excluded from version control because the automation process reads that information from environment variables on the Travis CI platform which we will set up further down the tutorial. 

It is important to also create a README.md file on the same folder level as .gitignore. We will need it later when we display the build status.

NodeJS Project

At this point, we have the basic structure needed for our Flex Service and it’s time to initialize it. Navigate to the project directory and initialize a simple NodeJS project using NPM. When asked for the entry point, specify src/index.js.  For the test command, specify ./node_modules/mocha/bin/mocha. Having done that, it is probably a good idea to commit the current state to GitHub.

Next up are dependencies. Add the following entries inside package.json and then run npm install to install them. 

The purpose of each dependency will be explained later. Package versions are up-to-date as of this writing.

Flex Service Code

Create a couple of directories inside the project directory (not the parent directory!) and call them src and test. In them, create index.js and index.test.js files respectively. These files will serve as the entry point for the Flex Services’ code.

Add the following to your index.js source file:

The code snippet does basically two things:

• Initializes the Kinvey Flex Service

• Registers a Kinvey Flex Function

Next, add the following code to your index.test.js test file:

The test has nothing to do with your function's code. We add it just for the sake of having some tests during the automation process. Once you start using Kinvey Flex Services, you should create proper tests for your source code. 

Remember it’s always a good idea to include tests in your project. 

Run the following command to ensure that the test passes:

Following that, you’ll want to test if the microservice works properly locally.

This will start a local service that will listen on port 10001 (which you can see in the console output). Then, using a REST client (like Postman), you can make a simple POST request to http://localhost:10001/_flexFunctions/testFunction and see if you get the expected response. You should see an output similar to the following:

To stop the local service, please press CTRL + C.

This is a good point to make your next commit.

Kinvey CLI

Cool! At this point, we have everything we need to run a Flex Service. All we need to do is to tie together the NodeJS project and the Flex Service we created using Kinvey Console. This is done using the Kinvey CLI command-line utility that we will install next. Open a terminal window and type the following:

After it finishes installing, open a terminal window inside the project directory (not the parent directory!) and run the following:

Use the same credentials that you use to log in to the Kinvey Console when prompted. You also need to name the profile that Kinvey CLI will create for you after it is done. We’ll call our profile development because it's on our local machine. If you are a Kinvey customer, you might have a dedicated Kinvey instance. If that's the case, please enter the correct instance ID. If you do not have a dedicated Kinvey instance, you can leave this option empty. 

Having done that, let's now tell the Kinvey CLI that we would like to be using that specific development profile, which we just created. To do so, please run the following:

Next, we want to connect the NodeJS project with the service from the Kinvey Console. To do so, run the following from the project folder:

The command prompts you to choose the application to use the service with, as well as the service you created. It used the provided information to create a Flex profile which it stores in a .kinvey file inside the project folder. 

If you’ve reached this point and everything is alright, you can go ahead and deploy the service. Again, it’s important to run the command from the project folder.

 

The output should contain a message to the effect that a deploy job has been initiated. Uploading the code and provisioning server resources to your service may take a minute or two, that’s why you need to monitor the progress before you move forward. You can monitor the progress of the job via running the following:

The status will start at NEW, then switch to UPDATING and finish at ONLINE. There will be no automatic notifications when the deploy is finished. So, you will need to manually check the service deployment status via running the command given above (note: you might need to run it couple of times until you see the ONLINE status).

Testing the Flex Service

Once the deploy job has been finished, you can go to the Kinvey Console, navigate to the Service Catalog, choose your correct service, and after you click the FlexFunction Handlers tab you can verify that you see the testFunction handler defined in our code. This means our service is online.

To quickly test the service, we will setup a custom endpoint. We do that by navigating to the Apps section of the Kinvey Console (top left corner), selecting the correct app (click the environment name that you are using for this tutorial), and then going to Custom Endpoints in the left-hand side navigation. Then we create a MicroService-type endpoint, preferably with the same name as the function - testFunction. From the next window, we select the Flex Service we created followed by the handler that we’ve registered.

To call our custom endpoint, we navigate to API Console in the left-hand side navigation. Select the POST request type and then from the drop-down menu select the endpoint (endpoints appear at the bottom of the drop-down menu). Sending the POST request should give you the expected response.

 

Travis CI Transcript

Having reached this far, we are ready to start playing with the real automation. Let's now add a transcript file for Travis CI. This transcript file will tell the automation engine what steps to follow in order to deploy the Flex Service. We do that by creating a .travis.yml file inside the parent folder (not the project directory!). Inside the file, put the following contents:

 

Don’t forget to change the folder name on line five if you used a different name. 

To a large extent, the transcript repeats the steps you took when you manually initialized and deployed the Flex Service. In short, it installs the needed packages, runs the tests, and if they are successful, runs the Kinvey-specific commands that result in a service deployment job.

Notice that on line 9 we run a JS script. It is there to take care of authentication in the event of the Kinvey 2FA authentication token expiring.

Paste the script code inside a file called buildProcessAuth.js and place it inside the project directory (not the parent repository folder).

 

The script code creates a session with Kinvey using the environment variables for email, password and user secret (we will come to this in a bit). Creating a session with Kinvey requires a Two-Factor Authentication token, which explains the use of One-Time Password manager (otp.js).

As this concludes all local edits that we need to make, go ahead and make the final git commit for the project.

Travis CI UI

After creating and committing a transcript file, we can continue setting up Travis CI. Navigate to https://travis-ci.org/ and sign in (preferably with your GitHub account as this will automatically connect the two accounts). On the Settings page you will see your public repositories (you will need the paid version of Travis CI to use it with private repos). You need to turn on Travis CI for your repository. After you do this, open the repository settings page. At least for testing purposes, I recommend only leaving the build-pushed-branches setting enabled. On the auto-cancellation menu, leave everything unchanged.

Next, we start adding the environment variables. They are as follows:

• KINVEY_USER_EMAIL

• KINVEY_USER_PASSWORD

• KINVEY_SERVICE_ID

• KINVEY_SERVICE_PROFILE

• KINVEY_USER_SECRET

For email and password, enter your Kinvey credentials.

You can get the service ID from the .kinvey file created earlier by kinvey flex init. 

For the service profile, I would specify testing, since we already have a development profile on our local machine.

The user secret is random string that uniquely identifies you when you enable 2FA (make sure to enable it from your Kinvey Console profile settings). Take these steps to get your user secret:

• From the Kinvey Console, click the profile icon in the upper-right corner and select Profile

• Open your web browser’s developer’s console and start tracking the network requests

• Navigate to Two-Factor Authentication in the left-hand navigation and press Change device, then when prompted for your Kinvey password and a 2FA token, enter them

• In the developer’s console, you will see a POST request going out to https://console.kinvey.com/_api/v2/user/generate-two-factor-auth-key—copy the response to this request in a simple text editor

• In the request string, find the word secret and followed by %3D—copy everything starting after this character sequence up to, but excluding, the next % character

• Fill in the Travis CI environment variable with the obtained secret

• Using your smartphone, scan the barcode that appears in the Kinvey Console and enter the 2FA code that you get

• You will be redirected to the Kinvey Console login page, but Do NOT login—the reason for that being that we want to make sure that login through buildProcessAuth.js works fine (after you see a build passing successfully, you can login to the Console again)

Congratulations, we are finally there! We can now test our automation setup. Open the source file of the NodeJS project in your favorite IDE or text editor. In the return statement towards the end, bump the version to 2.0.0:

to

At the same time, we need to make a similar change inside the package.json file. If we don’t bump the version there, Kinvey CLI will refuse to deploy a new version of the service. Bump that one to 2.0.0 as well.

Because we have Travis CI set up to track our GitHub repo, a push to its master branch should result in triggering of an automation job in Travis CI. 

If you look at the logs, you will see Travis CI setting the environment variables at the beginning, even though their values are masked. If all the steps from the Travis CI transcript have exited with 0, then the automation job is successful. This should result in a deploy of the Flex Service. 

Now you can log in back to the Kinvey Console and run the same POST request we ran earlier. The response should now show the updated version number.

One last thing to do: attach a build status icon to the repository for easier tracking. Get the icon URL from the Travis CI UI by clicking the icon next to your repository name and copying it. Now you can paste it at the top of your repo’s README.md file in the following format:

 

Do not forget to bump the version on the two places mentioned above.

 

Once you push your changes, Travis CI needs to follow the automation transcript. If that passes well, there will be a new deployment job initiated. Please be aware that even if the Travis CI log messages say that everything is "green," this still does not mean that your Kinvey Service is successfully deployed. It just shows that a new deployment job has been initiated. So, it might take another five minutes until your Kinvey Service is up and running with the new changes applied (the time for the actual deploy process). You can follow the deploy process manually from your machine by running the status command as mentioned above.

 

One Last Step

As you have seen, we used our Kinvey Console User e-mail address and password for the automation process. This is indeed required to be able to successfully deploy the service on Kinvey Flex Services Runtime (FSR). To make the process more robust, it would be great if you create a secondary Kinvey Console User and use it for the above-described purpose. You will just need to add that user as a collaborator to your Kinvey Application. The process is quite straightforward, you just need to send an invitation to that user's e-mail address, using the Kinvey Application's settings from the Web Console. Once you have the secondary user set-up as a collaborator, you can go ahead and modify the environment variables from the Travis CI project settings. There you will need to add the new user's e-mail address, password, and user secret (follow the same steps as above to get that secret). Having done that, you will have more (better) control on this automation process, and once you no longer wish to have this process automated - you can just go ahead and revoke the secondary user's collaboration access rights.

 

Any comments and questions are welcome in the comments below! Or feel free to reach out directly if you have questions about Kinvey, and if you haven’t yet, don’t forget to check out the Kinvey homepage or Dev Center to learn more.