Plotting a histogram

Exercise

PDAL doesn’t provide every possible analysis option, but it strives to make it convenient to link PDAL to other places with substantial functionality. One of those is the Python/Numpy universe, which is accessed through PDAL’s Python bindings and the filters.python filter. These tools allow you to manipulate point cloud data with convenient Python tools rather than constructing substantial C/C++ software to achieve simple tasks, compute simple statistics, or investigate data quality issues.

This exercise uses PDAL to create a histogram plot of all of the dimensions of a file. matplotlib is a Python package for plotting graphs and figures, and we can use it in combination with the Python bindings for PDAL to create a nice histogram. These histograms can be useful diagnostics in an analysis pipeline. We will combine a Python script to make a histogram plot with a pipeline.

Note

Python allows you to enhance and build functionality that you can use in the context of other Pipeline operations.

PDAL Pipeline

We’re going to create a PDAL Pipeline to tell PDAL to run our Python script in a filters.python stage.

{
    "pipeline": [
        {
            "filename": "./exercises/python/athletic-fields.laz"
        },
        {
            "type": "filters.python",
            "function": "make_plot",
            "module": "anything",
            "pdalargs": "{\"filename\":\"./exercises/python/histogram.png\"}",
            "script": "./exercises/python/histogram.py"
        },
        {
            "type": "writers.null"
        }
    ]
}

Note

This pipeline is available in your workshop materials in the ./exercises/python/histogram.json file.

Python script

The following Python script will do the actual work of creating the histogram plot with matplotlib. Store it as histogram.py next to the histogram.json Pipeline file above. The script is mostly regular Python except for the ins and outs arguments to the function – those are special arguments that PDAL expects to be a dictionary of Numpy dictionaries.

Note

This Python file is available in your workshop materials in the ./exercises/python/histogram.py file.

# import numpy
import numpy as np

# import matplotlib stuff and make sure to use the
# AGG renderer.
import matplotlib
matplotlib.use('Agg')
import matplotlib.pyplot as plt
import matplotlib.mlab as mlab

# This only works for Python 3. Use
# StringIO for Python 2.
from io import BytesIO

# The make_plot function will do all of our work. The
# filters.programmable filter expects a function name in the
# module that has at least two arguments -- "ins" which
# are numpy arrays for each dimension, and the "outs" which
# the script can alter/set/adjust to have them updated for
# further processing.
def make_plot(ins, outs):

    # figure position and row will increment
    figure_position = 1
    row = 1

    fig = plt.figure(figure_position, figsize=(6, 8.5), dpi=300)

    for key in ins:
        dimension = ins[key]
        ax = fig.add_subplot(len(ins.keys()), 1, row)

        # histogram the current dimension with 30 bins
        n, bins, patches = ax.hist( dimension, 30,
                                    density=0,
                                    facecolor='grey',
                                    alpha=0.75,
                                    align='mid',
                                    histtype='stepfilled',
                                    linewidth=None)

        # Set plot particulars
        ax.set_ylabel(key, size=10, rotation='horizontal')
        ax.get_xaxis().set_visible(False)
        ax.set_yticklabels('')
        ax.set_yticks((),)
        ax.set_xlim(min(dimension), max(dimension))
        ax.set_ylim(min(n), max(n))

        # increment plot position
        row = row + 1
        figure_position = figure_position + 1

    # We will save the PNG bytes to a BytesIO instance
    # and the nwrite that to a file.
    output = BytesIO()
    plt.savefig(output,format="PNG")

    # a module global variable, called 'pdalargs' is available
    # to filters.programmable and filters.predicate modules that contains
    # a dictionary of arguments that can be explicitly passed into
    # the module by the user. We passed in a filename arg in our `pdal pipeline` call
    filename = pdalargs['filename'] if 'filename' in pdalargs else 'histogram.png'
    
    # open up the filename and write out the
    # bytes of the PNG stored in the BytesIO instance
    with open(filename, 'wb') as o:
        o.write(output.getvalue()) 


    # filters.programmable scripts need to
    # return True to tell the filter it was successful.
    return True

Run pdal pipeline

$ pdal pipeline ./exercises/python/histogram.json
anything:47: UserWarning: Attempting to set identical low and high xlims makes transformation singular; automatically expanding.

Output

Notes

1. writers.null simply swallows the output of the pipeline. We don’t need to write any data.

2. The pdalargs JSON needs to be escaped because a valid Python dictionary entry isn’t always valid JSON.