Scripts

Overview

You can deploy Python scripts and notebooks to Datapane in order to create reports dynamically.

Scripts on Datapane are exposed to users through web forms which can be run in the browser to generate reports. This allows other people to self-serve on your reports without worrying about code, notebooks, or setting up a Python environment. Scripts can take parameters from users which are passed into your Python code at runtime.

Writing a script

To make your existing Python script or notebook compatible with Datapane, you only need to add a render method to it which returns the components which you want in your report.

Creating a report from a script is much like how you create a regular report, but instead of calling Report.create , you return a list of components from your render method.

When a user runs your script, your code is executed in its entirety, and Datapane subsequently generates a report, which is then presented to the user.

Although adding a render method to your notebook or script is the recommend approach, as it keeps your existing code portable and does not pollute the namespace, you can alternatively add components to a report object in your code which Datapane will pick out.

report = [dp.Plot.create(plot), dp.Table.create(df)]

For example, the script below creates a report with a single table component containing a DataFrame between 1 and 20k rows.

analysis.ipynb
import pandas as pd
import datapane as dp
tickers = ["GOOG"]
dfs = []
for t in tickers:
t_df = pd.read_csv(f'https://query1.finance.yahoo.com/v7/finance/download/{t}?period1=1553600505&period2=1585222905&interval=1d&events=history')
t_df['ticker'] = t
dfs.append(t_df)
stock_data = pd.concat(dfs)
stock_data['pct_change'] = stock_data.groupby('ticker')['Close'].pct_change()
stock_data['cum_prod'] = (1 + stock_data['pct_change']).cumprod()
def render():
return dp.Table.create(df)

To deploy your script, use Datapane's CLI.

$ datapane script upload analysis.ipynb --title=simple_script
[-] Script created: https://datapane.com/scripts/deadbeef

This makes your script available on datapane.com (or your private instance).

When a user runs your script, they will receive the following Report, which contains a single Table.

Configuration

In the above examples, we are deploying a single notebook and providing configuration -- such as title -- through command-line arguments. It's often useful to create a more structured project which contains the config for your script. You can put your configuration in a special file called datapane.yaml. When you run a command such as a upload, Datapane looks for this file.

You can create a sample project structure with the script init command, which creates a sample configuration file and a simple script.

~/C/d/d/my-new-proj> datapane script init
Created script 'my-new-proj', edit as needed and upload
~/C/d/d/my-new-proj> ls
datapane.yaml template.py

Inside your datapane.yaml, you can set the following fields.

Field

Description

public

Whether or not your script is public (boolean, default false)

title

The user-facing title of your script

parameters

A list of parameters for your script (see below)

If there is a config file, when you upload your script, you don't need to provide the path of your analysis. Instead, your script lives in template.py or template.ipynb by default.

~/C/d/d/my-new-proj> datapane script upload
16:39:39 [INFO ] Uploading template.py
Uploaded template.py to https://datapane.com/scripts/deadbeef/

Running & Parameters

As well as simply running and generating a report, scripts can take input parameters, which users enter through a web form and are passed into your code at runtime. These are defined in your datapane.yaml. Parameters are accessible on a params object, which is available in the global namespace. For instance, in the example we used above, you could allow users could provide the stock tickers they are interested in.

analysis.ipynb
import pandas as pd
import datapane as dp
tickers = params.tickers
dfs = []
for t in tickers:
t_df = pd.read_csv(f'https://query1.finance.yahoo.com/v7/finance/download/{t}?period1=1553600505&period2=1585222905&interval=1d&events=history')
t_df['ticker'] = t
dfs.append(t_df)
stock_data = pd.concat(dfs)
stock_data['pct_change'] = stock_data.groupby('ticker')['Close'].pct_change()
stock_data['cum_prod'] = (1 + stock_data['pct_change']).cumprod()
def render():
return dp.Table.create(df)

Defining parameters

If you want to take input parameters, you provide a list of parametersin your datapane.yaml . These define the fields in your form. For instance, in the following report, we may want to take two dates and a list of stock tickers to plot.

We would define the following datapane.yaml

datapane.yaml
# Script title
title: Stock plotter
# Script parameter definitions - see docs.data
# see xyz for configuration docs
parameters:
- name: start_date
type: date
description: "Start period for the plot"
- name: end_date
type: date
description: "End period for the plot"
- name: tickers
type: list
description: "Tickers to plot"

When uploaded, Datapane would use this configuration to generate the following form.

Parameters Options

When you define your list of parameters, all can include the following options.

Field

Required

Description

name

True

The name of the parameter. This must be a combination of lower case letters, numbers, and dashes and must be unique to your script.

type

True

The type of the parameter (see options below)

description

False

A description which is presented to the user when they run your form.

required

False

Whether your parameter is required or optional. Defaults to true.

Parameter Form Fields

The type of your parameter and other settings dictates how it is presented in your form. Datapane supports the following form fields.

Form Field

type

Extra options

Default value

Text input

string

””

Slider

integer

Both min andmaxproperties set.

min value

Integer input

integer

One of min/max not set

undefined

Float input

float

undefined

Boolean input

boolean

false

Dropdown

enum

The first choice supplied

Date picker

date

The current date

Time picker

time

The current time

Date and time picker

datetime

The current date and time

List Input

list

If provides with no choices

[]

Multi Select Input

list

If provided with choices: an array of strings and numbers.

[]

An example with all implemented form fields can be found here, and is available on GitHub here.

Querying Databases and APIs

Scripts often pull data from third-party data sources, such as your data warehouse or an internal API. Datapane does not provide proprietary integrations into other platforms, instead allowing you to query your chosen platform using Python.

For instance, if we want to create a script which queries BigQuery based on some user input, we could deploy the following:

from datapane import Table, Text
from google.cloud import bigquery
client = bigquery.Client()
form_query = params.query;
query_job = client.query(f'''
SELECT
CONCAT(
'https://stackoverflow.com/questions/',
CAST(id as STRING)) as url,
view_count
FROM `bigquery-public-data.stackoverflow.posts_questions`
WHERE tags like '%{form_query}%'
ORDER BY view_count DESC
LIMIT 1000''')
results = query_job.result()
def render():
report = []
if results.length > 0:
report += Markdown(f"Woohoo! We got {results.length} results :)!")
report += Table.create(results)
else:
report += Markdown(f"Oh no! No results for {form_query} :(!")
return report