A Short Python Tutorial¶

This tutorial explains by example how to process a dict using Python’s list comprehension facilities.

Suppose we’re interested in the total number or order failures.

First, we need to query the appropriate endpoint to get the data, and call the json() method.

http('http://www.example.com/foo/bar/data.json').json()

This endpoint returns JSON data that is structured as follows (with much of the data omitted):

{
    ...
    "itr-http04_orderfails": [1, 0],
    "itr-http05_addtocart": [0.05, 0.0875],
    "http17_addtocart": [0.075, 0.066667],
    "http27_requests": [14.666667, 12.195833],
    "http13_orderfails": [null, 2],
    ...
}

The parsed object will therefore be a dict mapping strings to lists of numbers, which may contain None values.

We need to find all entries ending in _orderfails. In Python, we can transform a dict in a list of tuples (key, value) using the items() method:
```
http(...).json().items()
```
We now need to filter this list to include only order failure information. Using a loop and an if statement, this could be accomplished like this:
```
result = []
for key, value in http(...).json().items():
    if key.endswith('_orderfails'):
        result.append(value)
```
(Note how the tuples in the list returned by items() are automatically “unpacked”, their elements being assigned to key and value, respectively.)

Since the check command needs to be a single expression, not a series of statements, this is unfortunately not an option. Fortunately, Python provides a feature called list comprehension, which allows us to express the code above as follows:
```
[value for key, value in http(...).json().items() if key.endswith('_orderfails')]
```
That is, code of the form
```
result = []
for ELEMENT in LIST:
    if CONDITION:
        result.append(RESULT_ELEMENT)
```
becomes
```
[RESULT_ELEMENT for ELEMENT in LIST if CONDITION]
```
(The if CONDITION part is optional.)

We now have a list of lists [[1, 0], [None, 2]].
In order to sum the list, we’d need to flatten it first, so that it has the form [1, 0, None, 2]. This can be accomplished with the chain() function. Given one or more iterable objects (such as lists), chain() returns a new iterable object produced by concatenating the given objects. That is
```
chain([1, 0], [None, 2])
```
would return
```
[1, 0, None, 2]
```
Unfortunately, the lists we want to chain are themselves elements of a list, and calling chain([[1, 0], [None, 2]]) would just concatenate the list with nothing and return the it unchanged. We therefore need to tell Python to unpack the list, so that each of its elements becomes a new argument for the invocation of chain().

This can be accomplished by the * operator:
```
chain(*[[1, 0], [None, 2]])
```
That is, out expression is now
```
chain(*[value for key, value in http(...).json().items() if key.endswith('_orderfails')])
```
Now we need to remove that pesky None from the list. This could be accomplished with another list comprehension:
```
[value for value in chain(...) if value is not None]
```
For didactic reasons, we shall use the filter() function instead. filter() takes two arguments: a function that is called for each element in the filtered list and indicates whether that element should be in the resulting list, and the list that is to be filtered itself. We can create an anonymous function for this purpose using a lambda expression:
```
filter(lambda element: element is not None, chain(...))
```
In this case, we can use a somewhat obscure shortcut, though. If the function given to filter() is None, the identity function is used. Therefore, objects will be included in the resulting list if and only if they are “truthy”, which None isn’t. The integer 0 isn’t truthy either, but this isn’t a problem in this case since the presence or absence of zeros does not affect the sum. Therefore, we can use the expression
```
filter(None, chain(*[value for key, value in http(...).json().items() if key.endswith('_orderfails')]))
```
Finally, we need to sum the elements of the list. For that, we can just use the sum() function, so that the expression is now
```
sum(filter(None, chain(*[value for key, value in http(...).json().items() if key.endswith('_orderfails')])))
```

Python Recipes¶

Merging Data Into One Result

You can merge heterogeneous data into a single result object:

{
    'http_data': http(...).json()[...],
    'jmx_data':  jmx().query(...).results()[...],
    'sql_data':  sql().execute(...)[...],
}

Mapping SQL Results by ID

The SQL results() methods returns a list of maps ([{'id': 1, 'data': 1000}, {'id': 2, 'data': 2000}]). You can convert this to a single map ({1: 1000, 2: 2000}) like this:

{ row['id']: row['data'] for row in sql().execute(...).results() }

Using Multiple Captures

If you have a alert condition such as

FOO > 10 or BAR > 10

adding capures is a bit tricky. If you use

capture(foo=FOO) > 10 or capture(bar=BAR) > 10

and both FOO and BAR are greater than 10, only foo will be captured because the or uses short-circuit evaluation (True or X is true for all X, so X doesn’t need to be evaluated). Instead, you can use

any([capture(foo=FOO) > 10, capture(bar=BAR) > 10])

which will always evaluate both comparisons and thus capture both values.

Defining Temporary Variables

You aren’t supposed to be able to do define variables, but you can work around this restriction as follows:

(lambda x:
    # Some complex operation using x multiple times
)(
    x = sql().execute(...)  # Some complex or expensive query
)

Defining Functions

Since you can define variables with the trick above, you can also define functions:

(lambda f:
    # Some complex operation calling f multiple times
)(
    f = lambda a, b, c: sql().execute(...)  # Some code using the arguments a, b, and c
)