How to Quickly Configure Azure's Application Gateway with a Hybrid Data Pipeline Installation

Accessing on-premises data from the cloud often brings with it many security and availability headaches, but with DataDirect Hybrid Data Pipeline it is now possible to securely access data behind any firewall while still leveraging the benefits of advanced cloud load balancing.

In this tutorial, you’ll learn how to quickly configure Azure’s Application Gateway to achieve both the scalability and reliability of a clustered Hybrid Data Pipeline installation by leveraging our support of the WebSockets protocol.

Important: This tutorial may contain outdated content. For the latest details on configuring Azure Application Gateway, please refer to Application Gateway configuration overview in the Azure Application Gateway documentation.

This document assumes you are already familiar with installing a Hybrid Data Pipeline cluster on 2 or more Azure instances which meet the minimum requirements – including use of an external configuration database and shared storage. It also assumes familiarity with configuring VPCs, Networking and basic administration within the Azure environment.

General Information about installing a single node in Azure Cloud can be found here, while details on a cluster installation can be found in our Installation Guide.

Note that there are a few changes with the DataDirect Hybrid Data Pipeline installation specific to supporting cloud load balancers:

• There is a new option for Load Balancers. Be sure to choose option 3 if using a WebSockets based load balancer such as the Application Load Balancer.

  

• Make sure to load your PEM file (SSL cert) on the HDP server as well as on the Google Load Balancer. This ensure correct the redistributable files are created to support installation of the On-Premises Connector and hybrid ODBC/JDBC drivers.
• Ensure your “Load Balancing Host Name” matches the hostname in your SSL cert.

1. Create a new Application Gateway
   1. It needs to have access to your HDP Nodes on ports 8080, 11280 and 40501 via the internal private IP
   2. Basics
      1. Name: HDPGateway
   3. Settings
      1. Put into a Virtual Network or Create one
      2. Public IP
      3. Choose Public IP or create existing one
      4. Set Idle Timeout to 5 min
      5. Listener – HTTP on Port 80
      6. Leave rest of settings as default
   4. Wait until Gateway is fully created before continuing.

1. Create Listener (Basic)
   1. Name: HTTPS
   2. Frontend IP Config: appGatewayFrontendIP
   3. Frontend port:
      1. Name: HTTPS
      2. Port: 443
   4. Protocol: HTTPS
   5. Certificate:
      1. Load in PFX Cert
      2. Name: <domain name>
      3. Password: <password used when PFX was created>
   6. Wait until listener is fully created before proceeding

1. Create Backend Pools
   1. Name: HDPServerPool
      1. Add all HDP Nodes to this pool
   2. Name: HDPNode1
      1. Add 1^st HDP node to this pool
   3. Name: HDPNode2
      1. Add 2^nd HDP node to this pool
   4. Add additional nodes for each HDP instance

       

   

2. Configure Health Probe
   1. Name: HDPHealthProbe
   2. Protocol: HTTP
   3. Select “Pick host name from backend settings”
   4. Path: /api/healthcheck
   5. Interval: 30
   6. Timeout: 30
   7. Unhealthy: 3

       

   

3. Configure HTTP Settings
   1. Edit existing “appGatewayBackendHttpSettings”
      1. Port: 8080
      2. Protocol: HTTP
      3. Cookie based affinity: Enabled
      4. Connection draining: Disabled
      5. Request timeout: 30
   2.  Add “OPA”
      1. Port 40501
      2. Protocol: HTTP
      3. Cookie based affinity: Disabled
      4. Connection draining: Disabled
      5. Request timeout: 20
   3. Add “Notification”
      1. Port 11280
      2. Protocol: HTTP
      3. Cookie based affinity: Disabled
      4. Connection draining: Disabled
      5. Request timeout: 20

1. Configure Rules
   1. Create Path-Based Rule
   2. Name: HDPRules
   3. Listener: HTTPS
   4. Default backend pool: HDPServerPool
   5. Default HTTP Settings: appGatewayBackendHttpSettings
   6. Add Configuration:
      1. HDPNode1 (Create one of these rules for EVERY node)
         1. Name: HDPNode1
         2. Path: /connect/opa_<hostname provided during HDP install>_40501
            1. BE SURE TO REPLACE dots with underscores in hostname
         3. Backend pool: HDPNode1
         4. HTTP Settings: OPA_Node1
      2. Notification
         1. Name: Notification
         2. Path: /connect/X_DataDirect_Notification_Server
         3. HTTP Settings: Notification_Pool

   

2. Delete default listener
   1. appGatewayHttpListener on port 80 can be removed.
3. Delete default backend pool
   1. appGatewayBackendPool
4. Add the front-end IP address of the Application Gateway to DNS as an A-Record. This will vary depending on where your DNS records are managed.

We hope this tutorial assisted in creating a cloud-based solution to OData enable both your on-premises and cloud data sources using Progress Hybrid Data Pipeline and Azure’s Application Gateway. Now you can have security, scalability and reliability all together in a single data access solution which lets you bring sources such as Oracle, Postgres, MySQL, DB2 and SQL Server out from behind the firewall. If you have any question, please feel free to contact us.