Using with InfluxDB¶
InfluxDB is a great open-source time-series database that works very well for storing metrics. It is scalable, simple to use, and has an active developer community. Plus, open-source dashboarding software such as Graphana includes plugins for InfluxDB, making it very easy to create dashboards from your Kadabra metrics.
This section of the docs will help you set up Kadabra with a locally-running instance of InfluxDB from scratch.
Install Kadabra and Redis¶
First, make sure you have installed Kadabra and have a locally running Redis server by following the directions from Getting Started. Make sure you can connect to Redis using the CLI:
=> redis-cli
127.0.0.1:6379> ping
PONG
Install and Run InfluxDB¶
Now you need to install the InfluxDB server by following the directions here. Make sure the InfluxDB server is running and that you can connect to it using the CLI:
⇒ influx
Visit https://enterprise.influxdata.com to register for updates, InfluxDB
server management, and monitoring.
Connected to http://localhost:8086 version 0.9.6.1
InfluxDB shell 0.9.6.1
>
Note that your InfluxDB might be a later version.
Create the Metrics Database¶
You will create the database that Kadabra will use to store metrics. The easiest way to do this is through the command line:
> create database kadabra;
>
Make sure the database exists by switching to it (we will use this to see the metrics we send shortly):
> use kadabra;
Using database kadabra
>
Configure and Run the Agent¶
Save the following into a file called run_agent.py:
import logging, sys, kadabra
handler = logging.StreamHandler(sys.stdout)
agent_logger = logging.getLogger("kadabra.agent")
agent_logger.setLevel("INFO")
agent_logger.addHandler(handler)
publisher_logger = logging.getLogger("kadabra.publisher")
publisher_logger.setLevel("INFO")
publisher_logger.addHandler(handler)
agent = kadabra.Agent(configuration={"AGENT_PUBLISHER_TYPE": "influxdb"})
agent.start()
The default arguments for the InfluxDBPublisher
target the InfluxDB server running locally on port 8086, using the kadabra
database.
Run the agent in its own terminal window:
python run_agent.py
Send Some Metrics¶
Let’s write a simple program that calculates the Nth fibonacci number, and records some metrics. We’ll call it fib.py:
import kadabra, sys, datetime
client = kadabra.Kadabra()
metrics = client.metrics()
metrics.set_dimension("program", "fibonacci")
n = int(sys.argv[1])
start = datetime.datetime.utcnow()
a, b = 0, 1
for i in range(n):
metrics.add_count("iterations", 1)
a, b = b, a + b
end = datetime.datetime.utcnow()
metrics.set_timer("runTime", end - start, kadabra.Units.MILLISECONDS)
client.send(metrics.close())
print a
This program will take a single command line integer argument, calculate that fibonacci number, and print it. But, it will also time how long this takes and count the number of loop iterations (admittedly a silly metric, since it will always be equal to N), and send these to InfluxDB.
Make sure the agent is running in a seperate terminal, and run the fibonacci program with a reasonable number (like 30):
=> python fib.py 30
832040
See the Metrics in InfluxDB¶
Now let’s take a look at what ended up in InfluxDB. Using the CLI, let’s view what measurements are available in our kadabra database:
> show measurements;
name: measurements
------------------
name
iterations
runTime
There are two measurements available, iterations and runTime corresponding to the counter and timer we set in our application.
Let’s look at runTime:
> select * from runTime;
name: runTime
----------
time program unit value
1479333981920492032 fibonacci milliseconds 0.828
time is the Unix epoch timestamp when the metric was created. program is the dimension we set. It’s actually an indexed tag in InfluxDB, meaning we could efficiently query for all the metrics that share the same program. The unit tells us how to interpret the value: on my machine the fibonacci program calculated the value in 0.828 milliseconds.
Now let’s take a look at iterations:
> select * from iterations;
name: iterations
----------------
time program value
1479333981919803904 fibonacci 30
This counter looks mostly the same as our timer, although there is only a value field which is, as expected, equal to the value we passed in for N.
Next Steps¶
You’ve now used Kadabra to publish metrics into a real database suitable for storing and querying! But the database is running locally, which isn’t particularly helpful as your infrastructure starts to grow and incorporate additional hosts. For deployment in a real production environment you’ll want to host the InfluxDB server separately from your application hosts. It’s easy to use any InfluxDB host with Kadabra; you just need to change the “host” argument to the IP or DNS of the remote InfluxDB host.