Integration Baby Steps with Flowmon REST API and Python Requests Library

One of the ways Flowmon integrates with 3rd party solutions is by using REST API. In this article, we provide examples and show how easy it is to use the REST API to get the data from Flowmon.

Let’s firstly check simple example using curl (command line tool and library for transferring data) and then we get straight to the Python.

Login and acquiring the token with curl:

curl https://flowmon.invea.com/resources/oauth/token -k -d "grant_type=password" -d "client_id=invea-tech" -d "username=demo" -d "password=demo"

Response:

{"access_token":"b32269b7615f6dfac79ff839d99f27a4a123682e6222ea9bf411808bd9dc6b6d"
,"expires_in":604800,"token_type":"bearer",
"refresh_token":"f0cf23f074cb0d500a0b07086d48ab02c8aed1e508608ee91af1acf75cefe4ce"}

Curl is great for simple queries, however with more advanced queries it gets unreadable fast. Here is an example of how to get a list of flows with a few parameters (from, to, profile, channels, etc):

curl "https://flowmon.invea.com/rest/fmc/analysis/flows​?search={"from​":"2017-03-01 10:​00","to":​"2017-03-17 14:00","profile": "mail","channel​s":["pop3", "spop3", "smtp" ]​,"listing": {"aggregateBy​": [ "srcip", "proto" ],"sortBy​": "tstart"},"filter​": "any"}&​output=["ts","td"​,"pr","​sa","pr", "pkt","byt","fl"]" -k -H 'Authorization: bearer b32269b7615f6dfac79ff839d99f27a4a123682e6222ea9bf411808bd9dc6b6d'

Knowing this we'll use Python from now on. There is popular HTTP library Requests which makes things easier for us. Login and acquiring the token in Python with Requests library:

import requests  data = [
  ('grant_type', 'password'),
  ('client_id', 'invea-tech'),
  ('username', 'demo'),
  ('password', 'demo'),
]

response = requests.post('https://flowmon.invea.com/resources/oauth/token', data = data)

print(response.text)

We're using POST HTTP method. You can use GET method as well, however there is a limit for the length of the URL (2048 characters). You can learn more about HTTP methods at W3Schools.

{"access_token":"b32269b7615f6dfac79ff839d99f27a4a123682e6222ea9bf411808bd9dc6b6d",
"expires_in":604800,
"token_type":"bearer",
"refresh_token":"f0cf23f074cb0d500a0b07086d48ab02c8aed1e508608ee91af1acf75cefe4ce"}

Let's store the token in a variable and use it for subsequent queries:

import json
responsejson = json.loads(response.text)
token = responsejson["access_token"]
headers = {
'Authorization': 'bearer ' + token,
}

Now, we can try basic query such as getting channels in the "mail" profile:

params = (
    ('profileId', 'mail'),
)
response = requests.get('https://flowmon.invea.com/rest/fmc/profiles/channels',
headers=headers, params=params)
print(response.text)

And the response:

[{"id":"mail\/smtps","name":"smtps*"},{"id":"mail\/pop3","name":"pop3"},
{"id":"mail\/spop3","name":"spop3"},{"id":"mail\/imap","name":"imap"},
{"id":"mail\/imaps","name":"imaps"},{"id":"mail\/smtp","name":"smtp"}]

It's a bit hard to read. We can get more readable output with json.dumps:

responsejson = json.loads(response.text)
print(json.dumps(responsejson, sort_keys=True, indent=4))

And the reponse:

[
{
  "id":"mail/smtps",
  "name":"smtps*"
},
{
  "id":"mail/pop3",
  "name":"pop3"
},
{
  "id":"mail/spop3",
  "name":"spop3"
},
{
  "id":"mail/imap",
  "name":"imap"
},
{
  "id":"mail/imaps",
  "name":"imaps"
},
{
  "id":"mail/smtp",
  "name":"smtp"
}
]

Here is our entire Python script so far:

import requests

import json

# acquiring the token #

data = [
('grant_type', 'password'),
('client_id', 'invea-tech'),
('username', 'demo'),
('password', 'demo'),
]

response = requests.post('https://flowmon.invea.com/resources/oauth/token', data = data)
responsejson = json.loads(response.text)
token = responsejson["access_token"]

headers = {
'Authorization': 'bearer ' + token,
}
#mail profile channels #
params = (
('profileId', 'mail'),
)

response = requests.get('https://flowmon.invea.com/rest/fmc/profiles/channels', headers=headers, params=params)
responsejson = json.loads(response.text)
print(json.dumps(responsejson, sort_keys=True, indent=4))

Now, we've warmed up and are finally ready to tackle the task of taking the long and unreadable curl example (getting a list of flows with parameters) and convert it to Python. There is actually a website which can do this for us.

After a few manual changes:

params = (
('search',
    """{"from":"2018-08-28 10:00",
    "to":"2018-08-29 10:00",
    "profile": "mail",
    "channels":["pop3", "spop3", "smtp" ],
    "listing": {
    "aggregateBy": [ "srcip", "proto" ],
    "sortBy": "tstart"
    },
    "filter": "any"
    }"""
),
('output', '["ts","td","pr","sa","pr", "pkt","byt","fl"]'),
)

response = requests.get('https://flowmon.invea.com/rest/fmc/analysis/flows', headers=headers, params=params)
responsejson = json.loads(response.text)
print(json.dumps(responsejson, sort_keys=True, indent=4))

This returns a result id:

{
"id": 7621
}

We'll store this id and use it to access the results:

id = responsejson["id"]
response = requests.get('https://flowmon.invea.com/rest/fmc/analysis/results/' + str(id), headers=headers)
responsejson = json.loads(response.text)
print(json.dumps(responsejson, sort_keys=True, indent=4))

It's possible we'll get this response:

{
"code": 202,
"message": "{_\"Requested result is not available yet.\"}",
"status": "error"
}

This is expected since Flowmon needs a second or two to process our request before providing us with a results page. We'll just call sleep function for 5 seconds in our simple example and then try to access the results.Add import:

import time

Add sleep function:

id = responsejson["id"]
time.sleep(5)
response = requests.get('https://flowmon.invea.com/rest/fmc/analysis/results/' + str(id), headers=headers)
responsejson = json.loads(response.text)
print(json.dumps(responsejson, sort_keys=True, indent=4))

And here is our response (it was abbreviated for readability):

[
{
  "byt": "190.1 K",
  "fl": "34",
  "pkt": "4.4 K",
  "pr": "TCP",
  "srcip": "192.168.0.22",
  "td": "85446.877",

  "ts": "2018-08-28 10:00:18.533"
},
{
  "byt": "444.4 K",
  "fl": "30",
  "pkt": "1.2 K",
  "pr": "TCP",
  "srcip": "192.168.0.32",
  "td": "84600.884",
  "ts": "2018-08-28 10:05:45.568"
}
]

We can use Flowmon REST API for various things, for example to get the list of perspectives in Flowmon ADS:

# list of perspectives in ADS #

response = requests.get('https://flowmon.invea.com/rest/ads/perspectives', headers=headers)
responsejson = json.loads(response.text)
print(json.dumps(responsejson, sort_keys=True, indent=4))

Abbreviated response:

[
{
  "id": 2,
    "name": "Operational issues",
    "priorities": [
        {
            "filterIn": null,
            "filterOut": null,
            "methodCode": "VPN",
            "nfSource": null,
            "substring": null,
            "value": 1
        },
        {
            "filterIn": null,
            "filterOut": null,
            "methodCode": "VOIP",
            "nfSource": null,
            "substring": null,
            "value": 1
        },
      ]
{
    "id": 1,
    "name": "Security issues",
    "priorities": [
    ]
}
]

We can get all events for a particular perspective:

params = (
('search',
    """{"from":"2018-08-28 10:00",
    "to":"2018-08-29 10:00",
    "perspective": "1"}"""),
)

response = requests.get('https://flowmon.invea.com/rest/ads/events', headers=headers, params=params)
responsejson = json.loads(response.text)
print(json.dumps(responsejson, sort_keys=True, indent=4))

And the response (abbreviated):

[
{
    "batch": 1535529300,
    "comments": [],
    "detail": "The 249 services were provided on the network (prediction: 62.15). This device has provided 61 services (increase up to 290%).",
    "flowStamp": "2018-08-29 09:55:01",
    "id": 4510401,
    "interest": 0.24,
    "nfSource": {
        "id": 6,
        "name": "LAN/live/10-0-0-22_p3000",
        "virtual": 1
    },
    "perspectives": [
        {
            "id": 1,
            "name": "perspective_1",
            "priority": 3
        }
    ],
    "priority": 3,
    "source": {
        "blacklisted": 0,
        "country": "LAN",
        "ip": "192.168.0.16",
        "resolved": ""
    },
    "targets": [
        {
            "blacklisted": 0,
            "country": "LAN",
            "ip": "192.168.0.1",
            "resolved": ""
        },
        {
            "blacklisted": 0,
            "country": "LAN",
            "ip": "192.168.0.252",
            "resolved": ""
        }
    ],
    "time": "2018-08-29 09:55:01",
    "type": "ANOMALY"
}

Each event has its own id. We can access a particular event by providing its id in the URL:

response = requests.get('https://flowmon.invea.com/rest/ads/event/4510401, headers=headers)

As can see there are many ways we can leverage Flowmon REST API. This article was meant to show you the very basics and illustrate how easy it's to access data provided by Flowmon Solution.

Go ahead, request the Python script and experiment with it. Take a look at Flowmon REST API Developer Guides page on our support portal for API reference as well as Flowmon REST API sample client to show how to build custom web application using Flowmon API.